[Bro] Bro cluster requirements and manager logging backlog bug
hovsep.sanjay.levi at gmail.com
Fri Jan 6 13:58:13 PST 2017
I'm using four loggers and the memory usage remains stable. When I
re-enable writing logs to disk there's a difference since logs/current is a
symlink to the first logger, spool/logger-1; the other loggers write into
their own spool directories (ex: "spool/logger-3"). I think you mentioned
For some reason logger-1 and logger-3 are doing all of the work, there are
no logs in logger-2 and logger-4 and the communication.log files for each
doesn't show any worker communications. At startup there was "peer sent
worker-1-1" but nothing afterwards. I'm not sure yet if this happens when
Kafka only logging is enabled. The cluster-layout.bro looks correct and
shows the 4 loggers are distributed among the workers correctly, so it's
When I reduced the number of loggers to 2 it's the same phenomenon,
logger-1 is working OK but logger-2 seems to be stalled. Only one worker
has sent data and it's very low volume.
Overall the multiple logger setup shows promise for fixing the issue but
there's a few more things to discover and tune. It seems the reason the
cluster is stable is because only half of the logs are being received when
using multiple loggers.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Bro