[Bro] ElasticSearch plugin

[Gary Faulkner]

I believe supporting dots in field names again is on the fix list for 
Elastic Stack v5 which is currently in development, so that part might 
at least get fixed on the Elastic end. Technically I believe that fix is 
a plus even for folks using another plugin such as Kafka as those folks 
still potentially had to do something to rewrite the field names. I 
can't speak for anything else that might be broken in the plugin.

Here is the reference to that bit in the Elastic Blog: 
https://www.elastic.co/blog/elasticsearch-5-0-0-alpha3-released

I could still see cases where someone could have a low volume Bro + 
local ES where it might not be desirable or necessary to run a bunch of 
other stuff in between Bro and ES, but maybe it just isn't too big a 
deal then to just let Logstash do the work of reading local log files 
assuming that one is OK with still writing normal Bro log output in 
addition to ES. An example might be if I wanted to do something akin to 
a stand-alone Security Onion node, but with Bro and Elastic instead of 
Bro and ELSA.

I'm not using that functionality currently and will probably look at 
something like Kafka, but I already have a log volume where the overhead 
of running something like Kafka probably makes sense.

~Gary


On 6/13/16 10:44 AM, Seth Hall wrote:
> Is there anyone here relying on the elasticsearch writer plugin in the bro-plugins repository?  It doesn't appear to work with current versions of elasticsearch anymore and it has always had trouble at sites with high rates of logging.
>
> If we don't get much of a response on this we will be deprecating and/or removing the elasticsearch writer.  There should be more reliable mechanisms available soon anyway by either writing to a Kafka server and then forwarding to ElasticSearch or writing files as JSON and the forwarding to ElasticSearch.
>
> Thanks,
>    .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro