[Zeek] Deprecation of &persistent

[David Hoelzer]

Hello all!

TLDR:
I'd like to ask that there be some thought given to the deprecation and
eventual removal of the &persistent option in favor of Broker data
stores.  IMHO, there are uses cases where the &persistent attribute is
much more attractive and lower overhead than the data store approach.

Longer:
As you are likely aware, &persistent is now marked deprecated and we
expect it to disappear in the next version or two.  The recommendation
for replacement is the much more robust, SQLite backed, Broker data store.

The data store solution is very elegant, though it does seem to require
more fiddling than it ought to to get a data store set up.  In the long
term and when dealing with large amounts of data that must be persistent

and synchronized across nodes, this really is a wonderful solution.

That said, there seem to me to be some use cases where that is a massive

hammer to swing at some very small problems.  For example, we have one
analysis script that is tracking successful external DNS resolutions. 
Specifically, it is keeping track of all IPv4 and IPv6 addresses
resolved in the last 7 days (&read_expire 7 days) in a set.  For all
outbound connection attempts, this script generates a notice when the
connection involves an external host that never appeared in a DNS answer

record.  This is quite handy when it comes to locating unauthorized

outbound scanning, some C2 behaviors that do not rely on DNS/fast flux
sorts of things, fragile configurations of enterprise services, etc. 
This has been performing quite well for several years now in more than
one relatively decent sized networks (100,000+ hosts).

For this problem (and others that I can imagine that would take a
similar tack - i.e., only storing a set, vector, or other single
primitive, rather than a massive record in a table or a table of
tables), the &persistent is perfectly "sized."

Am I alone in thinking that this feature should be retained *along side
of* Broker data stores and potentially documented as recommended for
simple primitive data persistence?

Thanks!

-- 
----
David Hoelzer
Chief of Operations
Enclave Forensics, Inc.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: pEpkey.asc
Type: application/pgp-keys
Size: 1808 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20190420/75b88f51/attachment.bin