[Bro] - recommended DB for Bro logs

[Clark Gaylord]

I have done some proof of concept work with PostgreSQL (mostly in AWS RDS)
and have been very happy with the results so far. Of course the rub is you
need to set up the schema, but it is pretty straightforward to ingest after
that from the JSON.

What I've done is load JSON into a text field of a temp table, then cast
that as JSON on insert (there was a little trick to getting this right that
I don't recall off the top of my head). My load process is currently out of
service but I can try to look up my code for this if you need it.

Anyway, works like a champ since PG has not only JSON but inet and cidr
data types!
https://www.postgresql.org/docs/11.1/datatype-net-types.html

You could do a document database that would handle the JSON gracefully, but
then you're constantly paying the parse tax. Works great if you don't
actually want to use your data, though. :-)

If you use standard bro text files you've got more parsing to do but it's
certainly doable. I like having JSON bro output to avoid that heavy lifting.

Cheers
Clark

-- 

--
Clark Gaylord
cgaylord at vt.edu
... Autocorrect may have improved this message ...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181209/0a5eb7b6/attachment.html