[Bro] OOM-killer & Bro
gary at doit.wisc.edu
Mon Feb 3 18:31:45 PST 2014
I'm not sure. I'm running Bro 2.2 (release) with the default scripts and
most of the known memory leak issues don't seem to apply to me. Other
than some limited testing I haven't been using any custom scripts, and
none that depend on the input framework.
I've been thinking I may need more than 64G of RAM per node (16 core /
3-5G traffic, & 12 workers each). I seem to run with 100% of the RAM
allocated, but 20-30% of my RAM cached before something happens to cause
a sudden drop in cached memory (as seen on Orca graphs) resulting in
OOM-killer dropping one or more Bro processes.
I've been reading a bit about OOM-killer and some high performance
situations seem to call for disabling it, so I'm investigating whether
it makes sense to tweak vm-overcommit settings to disallow allocating
more than the total physical RAM + SWAP, but I don't know if this is
advisable for Bro or not, hence the query.
On 2/3/2014 7:06 PM, Alex Waher wrote:
> Are you chasing a memory leak? `broctl top` will generally report >500MB
> of reserved memory (90% of the time even >256M) per worker in a 40 worker
> cluster capable of handling spikes of 10Gb. Each worker has ~3GB RAM to it.
> I recall the log rotation process is a separate cron-style job that
> shouldn't really be bring down the cluster workers.
More information about the Bro