[Bro-Dev] Memory leaks on git eds2245
gregor at icir.org
Thu Sep 8 08:41:21 PDT 2011
On 9/7/11 22:37 , Seth Hall wrote:
> On Sep 8, 2011, at 1:25 AM, Gregor Maier wrote:
>> (BTW, the memory problems I have/had weren't "real" leaks. One a SSL
>> connection was done Bro would free the memory for it again. The problem
>> is that many SSL connections can live for days and thus they ultimately
>> consume memory like a "real" leak would).
> I implemented the code yesterday to stop analyzing connections with the skip_further_processing bif and it caused Bro to peak using more memory on the tracefile I was using it with than not stopping analysis of connections. One thing the SSL scripts are currently doing that I probably need to change is after logging the SSL log, I should probably do "delete c$ssl". The certificate and certificate chain are stored in there. Actually, as I think about it more that's probably most of the problem.
skip_further_processing() actually just sets the skip flag which means
that no further data is delivered to the analyzers, but the analyzers
The disable_analyzer() bif will actually remove the analyzer, however it
needs an analyzer_id, so the SSL analyzer would somehow need to add it's
analyzer_id to one of it's events.
> We may want to look into the real traffic implications of calling the skip_further_processing bif eventually too though. I was pretty disheartened to see more memory used from calling that than not calling it. Perhaps it results in more memory use to remember which connections to ignore? I suppose I wasn't checking completion time which is probably where the savings should mostly come from.
How much difference in memory usage did you see? And how much memory
usage do you see in general? My Bros usually need about 100-300MB and
with SSL I will eventually get to 1GB or more (not on all nodes and it
will often take hours until it starts get there).
(Disabling SSL altogether will reduce the "baseline" usage by about
However, it still puzzles me that skip_further_processing didn't help.
In my case memory consumption was ramping up over several minutes so
either data is still delivered to the analyzer during this ramp-up or
there are additional reasons for the memory consumption?
<gregor at icir.org> <gregor at icsi.berkeley.edu>
Int. Computer Science Institute (ICSI)
1947 Center St., Ste. 600
Berkeley, CA 94704, USA
More information about the bro-dev