[Bro-Dev] Per item expiration for tables

[Jan Grashöfer]

Hi,

I have a few things I am planning to add to the intel-framework. One of
them is expiration for intelligence items. To achieve per item
expiration in a table there is a little hack that is used in the
notice-framework and in the new netcontrol-framework: By setting
&create_expire=0 and returning the intended timeout for each item in the
corresponding expire_func, one can achieve per item expiration (see e.g.
scripts/base/frameworks/netcontrol/catch-and-release.bro).

This approach however does not work for &read_expire and &write_expire,
because accessing the item resets the expiration timeout based on the
&read/write_expire attribute of the table (in this case 0) instead of
the value that was previously returned by the expire_func. The following
script demonstrates this effect:
https://gist.github.com/J-Gras/061983dac59224a03d3bfad4476a1dd9

The straight-forward solution would be to allow each item to hold its
own expiration timeout. Talking to Seth about this, we came up with two
possible approaches to achieve this:
1) Use the return value of the expire_func to set this value.
2) Use a bif or language feature (e.g. expire 10sec { tbl[idx] }; ) to
set this value.

I would prefer the second approach, as the intention of the expire_func
return value is to provide a delay for a single expiration event. This
would e.g. allow to set an individual expire timeout of e.g. 1 hour for
a single item. Once the expire_func is called one could set a delay of
e.g. 10min. In case the item is accessed, the timeout would be reset to
the originally intended 1 hour instead of 10min.

What are your opinions on that? Which approach would you prefer or do
you think per item expiration is a bad idea in general?

Best regards,
Jan