[Bro] send all logs to kafka

[tkg_cangkul]

Hi Azoff,

I've running bro with that config about 2 days and the picture that i 
sent before is just a current log dir. This is the result of the command :

cat weird.log |bro-cut  name|sort|uniq  -c|sort -nr|head -n 10




On 03/04/17 21:29, Azoff, Justin S wrote:
>> On Apr 3, 2017, at 3:09 AM, tkg_cangkul <yuza.rasfar at gmail.com> wrote:
>>
>> hi,
>>
>> i'm trying to using bro kafka plugin to send the bro logs into kafka.
>> i've a problem to send all the logs type to kafka.
>>
>> i've set this into my local.bro :
>>
>> @load Bro/Kafka/logs-to-kafka.bro
>> redef Kafka::logs_to_send = set(HTTP::LOG, DNS::LOG, CONN::LOG, Known::SERVICES_LOG, Weird::LOG, Notice::LOG);
>>
>> but when i check on kafka topic. there are only http, conn, & dns.
>> i've check in my bro logs dir and there are so many types of log.
>
> http,dns,conn are all high volume log files compared to known services, weird, and notice.
>
> Based on your file sizes it looks like you only had a few notice and known services log entries, so is it possible that you just missed them among the large volume of conn and dns log entries?
>
> Also, your weird log looks to be very large, you should do a
>
>      cat weird.log |bro-cut  name|sort|uniq  -c|sort -nr|head -n 10
>
> to see why you have so many weird entries.
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170403/016bf108/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot from 2017-04-03 21:33:21.png
Type: image/png
Size: 9641 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170403/016bf108/attachment-0001.bin