[Bro] Bro cluster requirements and manager logging backlog bug

[Hovsep Levi]

I'm using four loggers and the memory usage remains stable.  When I
re-enable writing logs to disk there's a difference since logs/current is a
symlink to the first logger, spool/logger-1; the other loggers write into
their own spool directories (ex: "spool/logger-3").  I think you mentioned
this before.

For some reason logger-1 and logger-3 are doing all of the work, there are
no logs in logger-2 and logger-4 and the communication.log files for each
doesn't show any worker communications.  At startup there was "peer sent
worker-1-1" but nothing afterwards.  I'm not sure yet if this happens when
Kafka only logging is enabled.  The cluster-layout.bro looks correct and
shows the 4 loggers are distributed among the workers correctly, so it's
not that.

When I reduced the number of loggers to 2 it's the same phenomenon,
logger-1 is working OK but logger-2 seems to be stalled.  Only one worker
has sent data and it's very low volume.

Overall the multiple logger setup shows promise for fixing the issue but
there's a few more things to discover and tune.  It seems the reason the
cluster is stable is because only half of the logs are being received when
using multiple loggers.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170106/dee21468/attachment.html