[Bro] Logstash vs. Bro ElasticSearch Writer

[Seth Hall]

On Jan 24, 2014, at 11:48 AM, "Rosebraugh, Connar" <connar.rosebraugh at egov.com> wrote:

> Hi all!
>  
> I am trying to decide if I should use the Bro ElasticSearch writer or Logstash for generating ElasticSearch logs. I found this great write-up on using Logstash for creating ElasticSearch logs. What are the differences between the two? I know that Kibana has difficulty making sense of the ElasticSearch logs that Bro writes, especially with respect to the time that events occur.

Kibana is actually fine with it you just have to make sure and configure your dashboard to use the "ts" field as time instead of @timestamp.  I do believe there are some other minor issues too because I know when I was playing with Kibana I made some small patches to Bro (I don't recall how necessary they were though).  We are hoping soon to come back around to Bro+Elasticsearch and make that much higher performance and more reliable though.  Right now there are some issues with it under extremely high load and we still don't feel completely comfortable marking it as production ready.

I will say that I don't particularly like the way that people use logstash to push logs into elasticsearch either.  Hopefully we'll have better guidance and support for this soon.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140127/93b43919/attachment.bin