[Bro] Version: 2.0-907 -- Bro manager memory exhaustion
Siwek, Jonathan Luke
jsiwek at illinois.edu
Mon Aug 13 13:37:16 PDT 2012
> I have the following in my local.bro file:
> redef SMTP::generate_md5 += /image.*/;
> redef HTTP::generate_md5 += /image.*/;
> Using broctl's top and a little trial and error, I can see that these
> lines are the cause of my high CPU usage. It also causes higher
> memory usage as well, but memory usage always climbs and never gets
> smaller. I don't know if these lines are responsible for just higher
> memory usage in general, or whether they are also responsible gradual
> climb in memory. It appears that memory gradually climbs even without
> these lines, but I haven't had enough time to test that idea.
In general, the digest BiFs don't look like they leak, but if there is not a md5_hash_finish() for each corresponding md5_hash_init(), that could lead to growth of some internal state over time. The base scripts all attempt to clean up any md5_hash_init()'s with a corresponding md5_hash_finish(), but I'm not confident all edge cases are covered.
If you have any other local changes, you might see if there's a difference running with them rather than just the vanilla bro scripts -- it can be easy to add something which causes too much state to accumulate over time. Another quick check is to look for any errors in reporter.log -- currently interpreter exceptions due to scripting errors will not abort bro, but do cause a memory leak. Otherwise, it might be easiest for you to start looking into using a memory profiling tool (e.g. valgrind, gperftools) to try to locate the problem more definitely.
More information about the Bro