[Bro] Bro Log ingestion

[Paul Halliday]

I am using logstash.

I have Bro 2.3 running on a sensor and the logs are sent to a collector via
syslog-ng. There, they are written to disk where they are read by logstash
and sent to elasticsearch. I use logrotate to gzip these files once they
get close to about a gig and keep them just in case ES craps out or I need
to process them in other ways. I use squert (www.squertproject.org) to
browse them once in ES but kibana would probably be a more versatile tool.

I process anywhere from 1800-2500 entries/second on a 8core box with 96GB
ram running FreeBSD.

If you want to quickly PoC something take a look at securityonion  (
http://blog.securityonion.net/).


On Tue, Sep 16, 2014 at 10:54 PM, Jonathon Wright <
jonathon.s.wright at gmail.com> wrote:

> Hello,
>
> Requirement:
> I'm trying to find the most efficient way to ingest all of Bro's
> logs, where Bro is running on multiple servers, and get a
> single server/point of query/mining/reporting, etc.  Servers are
> running Red Hat 6.5 and Bro 2.3 built from source with file extraction
> enabled (HTTP protocol for exe files). All Bro logs and extracted files
> seem to be by default owned by root:root, but I'd like to have them
> available to a non-root group once on the single server/point/interface to
> the analyst.
>
>
> (My apologies if this has been covered, but I do not know where to search
> other than just ask or google it. )
>
> Current setup
> Red Hat is running fine, Bro 2.3 with file extraction is working fine. So
> no worries, I just need the best methodology to implement for ingesting all
> the Bro logs (and extracted files) to a single point for
> analysis/mining/querying/reporting etc.
>
> Research
> Looking around and doing some reading, I've found two possible solutions
> ELSA and LOGSTASH although I don't know them very well and / or what their
> capabilities are either. But I'd like to know if they are viable,
> especially given my scenario, or if there is something better. Also, a
> how-to so I can set it up.
>
> I look forward to your reply, thanks!
>
> JW
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Paul Halliday
http://www.pintumbler.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140917/60b09f54/attachment.html