[Bro-Dev] Update on log management

Seth Hall seth at icir.org
Wed Aug 31 08:29:36 PDT 2011

On Aug 31, 2011, at 10:52 AM, Martin Holste wrote:

> Can you guys please lay out what the basic goals, roadmap, and
> timeline are for all Bro output?  Please be clear about things you
> would *like* to do versus things which will actually be production
> ready in the next few months.

Right now we have the logging framework's writer plugin API ready to go.  We don't have any concrete plans for when or what writer plugins we are going to write but personally I'm very interested in a PostgreSQL plugin and you've gotten me interested in getting a syslog writer created if TCP syslog does in fact allow for extremely long messages.

To be honest it pained me a little bit when I saw your code to insert the ascii logs into a database (however I can't fault anyone for doing things out of operational necessity).  The benefits to outputting logs directly to a database from Bro is that you get to take advantage of logging framework features.  You can filter out log lines that you don't want to go to certain outputs or remove certain fields.  For instance, if we had a PostgreSQL writer plugin and you only wanted SSL sessions that didn't successfully validate put into the database, you could do this (keeping in mind that due to the lack of a postgresql writer, this code very much does not work)....

event bro_init()
	Log::add_filter(SSL::SSL, [$name="failed_to_pg", 
		$path="ssl_invalid_cert_session",   # This equates to the table name in postgresql
		$pred(rec: SSL::Info) = { return (rec$validation_status != "ok"); }]);

All of the SSL logs would still be written to the normal (ascii by default) logs but all of the sessions using invalid certificates would also be sent to postgresql.

In terms of future plans we're really just at the point where we need more writer plugins, most of the rest of the code is finished.  Does that answer your questions?


Seth Hall
International Computer Science Institute
(Bro) because everyone has a network

More information about the bro-dev mailing list