[Bro] cluster manager crash

[Johanna Amann]

Hello,

>     I have an issue about cluster manager crash when lots of log event send
> to it.
>     I set up a bro cluster  on my server, the cluster have 32 workers and 1
> proxy and handle about 5Gb/s. After run about one and a half hour, the
> cluster no longer produces logs, but workers still extracts files. So it
> seems that the manager was crashed.
>     Is there any possibility that the manager doesn't work anymore when
> workers send lots of log event? If so, what`s the limit of the log event?
> Or maybe the issue won`t happen if I run a real cluster on several servers?

yes, it is possible to kill a manager by sending too many data too it,
though that is usually caused by event traffic and not by logs. There is
no definitive limit, that depends a bit on your hardware and traffic.

Generally, if your manager really crashes, it should be restarted by the
broctl cron process. If you have a lot of logging, starting with bro 2.5
(currently in Beta), you also can separate logging from the manager and
move it into a logger node. To enable this on 2.5, put the following into
your node.cfg (this is also part of the example configuration):

[logger]
type=logger
host=localhost

>     By the way, if I want to handle 10Gb/s, how much memory should I leave
> for each worker ? If I do memory usage restrictions, will it affect
> the performance of the cluster?

The amount of memory depends on your traffic mix and is a bit difficult to
predict (I will let others chime in what their experiences are). If you
put in memory usage restrictions, it will kill the processes if they need
more memory than they are allowed to allocate.

I hope this helps,
 Johanna