[Bro] Elasticsearch Writer vs logstash
jlay at slave-tothe-box.net
Fri Jan 30 05:41:47 PST 2015
On Thu, 2015-01-29 at 23:48 -0700, Luis Miguel Silva wrote:
> Dear all,
> I'm interested in dumping my bro logs into an elastic search instance
> and, based on what I was able to learn thus far, it seems I have two
> different options:
> - use the elasticsearch writer (which the documentation says should
> not be used in production as it doesn't have any error checking)
> - or use logstash to read info directly from the bro logs and
> externally dump it into elasticsearch
> It seems to me the logstash route is better, given that I should be
> able to massage the data into more "user friendly" fields that can be
> easily queried with elasticsearch.
> So my question is, based on your experience, what is the best option?
> And, if you do use logstash, can you share your logstash config?
> Thanks in advance,
> Bro mailing list
> bro at bro-ids.org
I've used bro and logstash with good success...one setup is everything
is on one machine, the other is remote using rsyslog to get the data to
logstash. I tried going direct bro->elasticsearch, but logstash creates
logstash-* shards, and bro creates bro-* shards, and kibana had a hard
time seeing both. I'm currently just piping conn.log, but here's my
An interesting gotcha is the fact that the above doesn't see sizes as
values but strings, so I had to add a mutate to get that to work:
convert => [ "resp_bytes", "integer" ]
convert => [ "resp_ip_bytes", "integer" ]
convert => [ "orig_bytes", "integer" ]
convert => [ "orig_ip_bytes", "integer" ]
Hope that helps...feel free to ping me off list if you need any help.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Bro