[Bro] Bro's limitations with high worker count and memory exhaustion

[Siwek, Jon]

> On Jul 2, 2015, at 5:09 PM, Baxter Milliwew <baxter.milliwew at gmail.com> wrote:
> 
> This is something I checked before and it seemed a non-issue.  I'm using a ram backed fs and rotating logs every 5 minutes.  With prof.log the pending counter is always 0/0.  Is there something else I should be looking for ?

No, I recall that being the thing to watch and the prof.log snippet looked sane to me if it is from a time period when memory usage was high.  You might also graph memory usage over time to see if it’s spiky or just a steady increase.  The later may suggest a leak or script-layer state (e.g. tables/sets) not having an appropriate expiration interval.  I think prof.log also has info about memory usage of script-layer tables that might be worth looking for.  Other than trying to get things running under a memory profiler, sorry I don’t have much other ideas at the moment.

- Jon