[Bro] elastic search / bro questions

[Seth Hall]

> On Nov 6, 2014, at 7:54 PM, Joe Blow <blackhole.em at gmail.com> wrote:
> 
> Long story short, we're having some problems keeping up with this feed.  Does anyone have any performance tuning with this module?  I've played a lot with rsyslog batch sizes with elasticsearch and was hoping there would be some simple directive i could try and apply to BRO.
> 
> Does anyone have this experience here?  Does this module batch anything?

There is a solution that has been in development for some time.  We've done some work with having Bro write directly to NSQ (a disk backed http based queuing daemon) and there is another tool that pulls from NSQ and inserts into Elasticsearch.  So far it seems that this can keep up with quite high volume networks.

Thanks for reporting to the list.  More people showing problems like this can certainly prompt development on features like this. ;)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/