[Bro] elastic search / bro questions

[Seth Hall]

> On Nov 10, 2014, at 9:19 PM, Joe Blow <blackhole.em at gmail.com> wrote:
> 
> One more thing i wanted to share... In 'bro/share/bro/base/frameworks/logging/writers/elasticsearch.bro' it says:
> ##! There is one known memory issue.  If your elasticsearch server is
> ##! running slowly and taking too long to return from bulk insert
> ##! requests, the message queue to the writer thread will continue
> ##! growing larger and larger giving the appearance of a memory leak.
> 
> Interesting to see this queuing graphed out on a box with 96gb of ram....  It ran into swap pretty quickly... :)

Yeah, unfortunately ES frequently is having a hard time keeping up for people.  This is where having logs go to an external queueing system first can be beneficial.  

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/