[Bro] Bro Web Frontend

[Gilbert Clark]

Hi Martin:

This is cool!

A few notes:

*) There's a ticket open (http://tracker.bro-ids.org/bro/ticket/558) 
which is looking at revising the ASCII log header 
(topic/gilbert/ascii-header is a branch containing what's described in 
that ticket) to offer more information about the log fields themselves, 
and to support better classification of log files after they've been 
rotated / compressed.  The idea there is to use that type information to 
automate the table creation process and / or do some simple type conversion.

*) topic/gilbert/log-util is (what sounds like, at least) a similar log 
wrapper I'm building in Python.  The library's in a holding pattern 
while the new log header format gets pushed out, but should find itself 
in reasonable shape after that.

If you're interested, I'd love to migrate over to bro-dev and chat about 
stuff that's useful to have in this kind of a library :)

--Gilbert Clark

On 8/30/2011 4:53 PM, Martin Holste wrote:
> In order to really drill down on some SSL stuff, I decided it was time
> to finally put a lightweight web framework on top of my Bro logs.  So,
> unless the server shoots it down, attached is a tarball that contains
> an alpha-quality framework for reading Bro logs and writing them to
> syslog or a database or both.  If written to a database, they can be
> accessed through a very light web frontend included which can be run
> right from the Bro server, or from a traditional web server, if you
> prefer.  The web frontend solely consists of a query bar for searching
> on terms contained in the tab separated Bro logs.  So you can do
> things like this:
>
> subject:CN=mail.google.com* -validation_status:ok
>
> This would look for any connections to Gmail with an invalid certificate.
>
> Or you could do
>
> id.resp_h:74.125.*
>
> Which would find any traffic destined for Google's 74.125/16, or just
> 74.125.*
> which would do the same thing, but for either originator or responder.
>
> At this point, there's not much else you can do, as I mainly wanted to
> get a quick web frontend up so I could more easily explore the data
> that Bro outputs.  The key part of the framework is that it doesn't
> care what the fields are.  It will take whatever is at the top of the
> log files and use that as the field names.  If it can't find the field
> list, then it won't use the file.  The database doesn't have a
> standard table structure, rather it uses a key-value store, so the
> schema is very flexible.  It's been tested on MySQL, but it should
> work on almost any database.  It would be very easy to create entirely
> ad-hoc reporting using this structure.
>
> Aside from cosmetics, the big to-do is database table rollover, but if
> you just want syslog output from Bro, this will get you going very
> quickly.  See the INSTALL doc for a quick how-to, or email if you have
> any questions; feedback is appreciated!  If there is any interest, I
> will create a project page for the files for future downloading.
>
> Thanks,
>
> Martin