[Bro] elastic search / bro questions

Seth Hall seth at icir.org
Mon Nov 10 19:00:46 PST 2014

> On Nov 10, 2014, at 9:19 PM, Joe Blow <blackhole.em at gmail.com> wrote:
> One more thing i wanted to share... In 'bro/share/bro/base/frameworks/logging/writers/elasticsearch.bro' it says:
> ##! There is one known memory issue.  If your elasticsearch server is
> ##! running slowly and taking too long to return from bulk insert
> ##! requests, the message queue to the writer thread will continue
> ##! growing larger and larger giving the appearance of a memory leak.
> Interesting to see this queuing graphed out on a box with 96gb of ram....  It ran into swap pretty quickly... :)

Yeah, unfortunately ES frequently is having a hard time keeping up for people.  This is where having logs go to an external queueing system first can be beneficial.  


Seth Hall
International Computer Science Institute
(Bro) because everyone has a network

More information about the Bro mailing list