[Bro] send all logs to kafka

Azoff, Justin S jazoff at illinois.edu
Mon Apr 3 07:29:47 PDT 2017

> On Apr 3, 2017, at 3:09 AM, tkg_cangkul <yuza.rasfar at gmail.com> wrote:
> hi,
> i'm trying to using bro kafka plugin to send the bro logs into kafka. 
> i've a problem to send all the logs type to kafka.
> i've set this into my local.bro :
> @load Bro/Kafka/logs-to-kafka.bro
> redef Kafka::logs_to_send = set(HTTP::LOG, DNS::LOG, CONN::LOG, Known::SERVICES_LOG, Weird::LOG, Notice::LOG);
> but when i check on kafka topic. there are only http, conn, & dns. 
> i've check in my bro logs dir and there are so many types of log.

http,dns,conn are all high volume log files compared to known services, weird, and notice.

Based on your file sizes it looks like you only had a few notice and known services log entries, so is it possible that you just missed them among the large volume of conn and dns log entries?

Also, your weird log looks to be very large, you should do a

    cat weird.log |bro-cut  name|sort|uniq  -c|sort -nr|head -n 10

to see why you have so many weird entries.

- Justin Azoff

More information about the Bro mailing list