[Bro] bro_json-logs

[Tim Desrochers]

I am using the following to output bro logs to JSON:

#!bin/bash
## Configure bro to write JSON logs
mkdir -p /opt/bro/share/bro/site/scripts
sudo tee /opt/bro/share/bro/site/scripts/json-logs.bro << EOF
@load tuning/json-logs

redef LogAscii::json_timestamps = JSON::TS_ISO8601;
redef LogAscii::use_json = T;
EOF

sudo tee -a /opt/bro/share/bro/site/local.bro << EOF

# Load policy for JSON output
@load scripts/json-logs
EOF

I ship my logs with logstash into my ELK stack.  In logstash I am using the
date match ts ISO8601.  This is reading my dates correctly the issue is, I
think, the bro to json function of bro is not parsing the date correctly.

I begin to get records in logs with dates all over the map.  Some dates are
in the past, 1970, some are in the future, 2024, and some are correct.  I
have debugged my ELK stack down to the output of bro logs on the sensor and
it appears that dates printed in bro logs are getting printed incorrectly
and causing strange indices down stream in my ELK stack.

Has anyone had this issue?  Is there a bug in the way bro is reading UNIX
and translating to ISO8601?  Any thoughts on correcting this.  I don't mind
having TS in UNIX and having Logstash do the translating for me, but how
would I do that with bro.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151117/4cee3f93/attachment.html