[Bro] Couple elasticsearch questions

[James Lay]

On Wed, 2014-07-23 at 12:27 -0400, M K wrote:
> Bro converts the data to json and then writes that to elasticsearch
> using ES's bulk interface. But it does a "fire and forget" so doesn't
> confirm that the data was actually accepted. 
> 
> I wrote an AMQPRiver writer a while back that allows you to leverage
> an ElasticSearch River, it provided for a higher level of reliability
> of data ingestion, but I haven't touched it since I wrote it a few
> months back.
> 
> 
> 
> On Wed, Jul 23, 2014 at 12:15 PM, James Lay <jlay at slave-tothe-box.net>
> wrote:
>         On 2014-07-23 10:08, Seth Hall wrote:
>         > On Jul 23, 2014, at 11:50 AM, James Lay
>         <jlay at slave-tothe-box.net>
>         > wrote:
>         >
>         >> I'm guessing I'm going to have to create something like the
>         above
>         >> grok
>         >> for each bro log file....which...is going to be a hoot ;)
>         >
>         > Are you saying that you're going to have to do this because
>         you don't
>         > want Bro to write directly to ElasticSearch?
>         >
>         >   .Seth
>         >
>         > --
>         > Seth Hall
>         > International Computer Science Institute
>         > (Bro) because everyone has a network
>         > http://www.bro.org/
>         
>         
>         Negative.  In order to get Logstash/Kibana to identify fields,
>         the grok
>         patterns are what is used.  I guess that's the question for
>         me....does
>         Bro dump the data raw into elasticsearch?  If it does then
>         I'll need to
>         include a grok line in my logstash config to parse out the
>         data of each
>         type of log that bro generates.  I hope that makes
>         sense..thanks Seth.
>         
>         James
>         _______________________________________________
>         Bro mailing list
>         bro at bro-ids.org
>         http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>         
> 
> 

Thanks MK...that does help...this has been an interesting day of
discovery.

James