[Bro] Worker System Memory Exhaustion

[Hovsep Levi]

This was a battle we endured for many many moons (12+ months), look to the
archives for the pain and suffering.

Final solution :  Enable multiple loggers (now part of Bro), disable
writing logs to disk and stream logs to Kafka.  (Thank you KafkaLogger
author)

Reasoning  :  At some point Bro's log writing cannot keep up with the
volume.  Believed to be a bottleneck with the the default architecture
using a single "Logger" node.

Possible alternative  :  Enable multiple loggers, but when writing to disk
you might have a possible race condition with filenames and dates.  Also
you'll have multiple logs for each rotation interval (ex: 4 loggers means 4
conn.log, 4 http.log, 4 ssh.log, etc...)


^ Hovsep
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180406/c5a262e8/attachment.html