[Bro] Bro cluster requirements and manager logging backlog bug

[Azoff, Justin S]

> On Dec 19, 2016, at 4:26 PM, Hovsep Levi <hovsep.sanjay.levi at gmail.com> wrote:
> 
> Hello all,
> 
> 
> We are still having a problem with our Bro cluster and logging.  During peak times the manager will slowly consume all available memory while the logs sent to disk are delayed by an hour or more.


You're saying "the manager" but do you mean "the manager node" or "the manager process"?

With the added logger process the manager process does not have anything to do with logs.

The last time you mentioned these issues the logger node capability did not exist yet.  A lot has changed since then but the logs you show are from 4 months ago.

We need to see what this command outputs when your cluster is having log issues:

broctl top manager logger

Also, you've never mentioned the actual rate of logs you are seeing at these peak times

Running this in your log directory would help:

du -ms;cat *|wc -l;sleep 60;du -ms;cat *|wc -l


-- 
- Justin Azoff