[Bro] Elastic Search

Vlad Grigorescu vladg at cmu.edu
Mon Jul 29 12:48:17 PDT 2013


On Jul 29, 2013, at 1:23 PM, Bob Probert <bruisebrotherprobert at gmail.com> wrote:

> Seth,
> 
> I read on the security-onion list that Bro's Elastic Search writer doesn't write the correct metadata indexes for use with Kibana. Can you elaborate?
> 
> https://groups.google.com/forum/#!topic/security-onion/mmxZfVfO0D0

Previous versions of Kibana were designed to only work with Logstash data, which introduces certain assumptions about the way the data is stored. At one point, I had a fork of Kibana that worked with Bro, because it allowed extra configuration. I haven't updated it with the release of Kibana3, and don't intend to. Kibana3 might be flexible enough to support Bro's data, but I haven't checked yet.

If you are interested in using Kibana, I would advise you to be *very* careful about the risks introduced by allowing a user's browser to directly query ElasticSearch. ElasticSearch has no security protections, and all it takes is a curl -XDELETE es.local:9200 to delete all your log data. Kibana does come with a sample nginx config, so I would definitely look at that at least.

> Also, I read in the comments of "elasticsearch.bro" that there is some sort of memory leak either in the elastic search writer or elastic search itself? Can you also elaborate on this?
> 
> http://www.bro-project.org/sphinx-git/_downloads/elasticsearch.bro

Bro queues undelivered log messages in memory. If Bro is logging faster than ElasticSearch can accept logs, more and more memory is used. There's no memory leak - if your ElasticSearch server can't keep up, the logs simply have nowhere to go, and instead of being dropped, they just queue up on the manager.

  --Vlad



More information about the Bro mailing list