[Bro] Couple elasticsearch questions

[M K]

Bro converts the data to json and then writes that to elasticsearch using
ES's bulk interface. But it does a "fire and forget" so doesn't confirm
that the data was actually accepted.

I wrote an AMQPRiver writer a while back that allows you to leverage an
ElasticSearch River, it provided for a higher level of reliability of data
ingestion, but I haven't touched it since I wrote it a few months back.


On Wed, Jul 23, 2014 at 12:15 PM, James Lay <jlay at slave-tothe-box.net>
wrote:

> On 2014-07-23 10:08, Seth Hall wrote:
> > On Jul 23, 2014, at 11:50 AM, James Lay <jlay at slave-tothe-box.net>
> > wrote:
> >
> >> I'm guessing I'm going to have to create something like the above
> >> grok
> >> for each bro log file....which...is going to be a hoot ;)
> >
> > Are you saying that you're going to have to do this because you don't
> > want Bro to write directly to ElasticSearch?
> >
> >   .Seth
> >
> > --
> > Seth Hall
> > International Computer Science Institute
> > (Bro) because everyone has a network
> > http://www.bro.org/
>
> Negative.  In order to get Logstash/Kibana to identify fields, the grok
> patterns are what is used.  I guess that's the question for me....does
> Bro dump the data raw into elasticsearch?  If it does then I'll need to
> include a grok line in my logstash config to parse out the data of each
> type of log that bro generates.  I hope that makes sense..thanks Seth.
>
> James
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140723/9000c44b/attachment.html