[Bro] Logging to Accumolo?

[Kurt Grutzmacher]

Hi Ben,

We do use Flume for applications we can't extend with Kafka. There are
pluses and minuses for either route but with Bro we wanted to queue
directly from the output process code to avoid the additional IO of disk
read/writes. Adding another processor like Flume also increases the
complexity and potential failure points.

You do give up some reliability of message delivery but it's an acceptable
risk for us at the moment. A good Kafka/Zookeeper environment is
over-engineered anyway and Bro already is a single point of failure.




--
 Kurt Grutzmacher -=- grutz at jingojango.net


On Mon, Jun 30, 2014 at 6:40 AM, <ben.bt.wood at gmail.com> wrote:

>  You may try experimenting with Flume. It's part of that larger group of
> Hadoop related Apache projects, and included in Cloudera's distro.
>
> It's quite a capable log management tool, and can provide a lot of
> flexibility.
>
> http://flume.apache.org/
>
>  ------------------------------
> From: Kurt Grutzmacher <grutz at jingojango.net>
> Sent: ‎6/‎27/‎2014 6:29 PM
> To: Donaldson, John <donaldson8 at llnl.gov>
> Cc: bro at bro.org List <bro at bro.org>
> Subject: Re: [Bro] Logging to Accumolo?
>
> Logging is going through an update so we'll have to see what comes out at
> the end.
>
> I have code working for Kafka that was fairly easy to integrate in 2.2
> (haven't tried 2.3 yet). Should be fairly simple to integrate others.
> On Jun 27, 2014 6:04 PM, "Donaldson, John" <donaldson8 at llnl.gov> wrote:
>
>> I know that there is support for logging into ElasticSearch, but has
>> anyone looked into sending logs into Accumulo?
>>
>> John Donaldson
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140630/ce5259b0/attachment.html