[Bro] Memory leak?

[Hosom, Stephen M]

No problem at all!

Errors in script land DO cause memory leaks :)



-----Original Message-----
From: Joe Blow [blackhole.em at gmail.com<mailto:blackhole.em at gmail.com>]
Sent: Monday, September 22, 2014 04:27 PM Eastern Standard Time
To: Hosom, Stephen M
Cc: bro at bro-ids.org List
Subject: Re: [Bro] Memory leak?

Fantastic advise.  The reporter.log showed a bug for an empty variable inside a when statement.  Fixed it with an if - check and i'm not seeing any errors.  Hopefully this will quell the memory issues.  Thanks for the help.

Cheers,

JB

On Mon, Sep 22, 2014 at 10:24 AM, Hosom, Stephen M <hosom at battelle.org<mailto:hosom at battelle.org>> wrote:
At a glance… with only the information available… It looks more like Bro is legitimately filling up the system memory and the kernel is OOM killing your Bro worker. How much memory is in this worker, what kind of connection is it monitoring? Are you willing to provide a copy of your reporter.log?

From: bro-bounces at bro.org<mailto:bro-bounces at bro.org> [mailto:bro-bounces at bro.org<mailto:bro-bounces at bro.org>] On Behalf Of Joe Blow
Sent: Monday, September 22, 2014 10:19 AM
To: bro at bro-ids.org<mailto:bro at bro-ids.org> List
Subject: [Bro] Memory leak?

Hey guys,
I'm trying to figure out if there is a memory leak in Bro that's causing me issues.  I wonder if it's a memory leak because every few hours i'll have a worker die like this:
<snip>
If you want to help us debug this problem, then please forward this mail to reports at bro.org<mailto:reports at bro.org>
Bro 2.3.1
Linux 2.6.32-358.11.1.el6.x86_64
==== No reporter.log
==== stderr.log
listening on bond1, capture length 8192 bytes
1411357505.663118 processing suspended
1411357505.663118 processing continued
1411392324.017784 received termination signal
1411392324.017784 105126164 packets received on interface bond1, 0 dropped
==== stdout.log
max memory size         (kbytes, -m) unlimited
data seg size           (kbytes, -d) unlimited
virtual memory          (kbytes, -v) unlimited
core file size          (blocks, -c) unlimited
==== .cmdline
-i bond1 -U .status -p broctl -p broctl-live -p local -p worker-0-3 local.bro broctl base/frameworks/cluster local-worker.bro broctl/auto
 ==== .env_vars
PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/sbin:/bin:/usr/sbin:/usr/bin
BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site
CLUSTER_NODE=worker-0-3
 ==== .status
TERMINATED [atexit]
 ==== No prof.log
 ==== No packet_filter.log
 ==== No loaded_scripts.log
--
[Automatically generated.]
</snip>
The picture of the system's memory looks like this:

[Inline image 1]
This sort of corresponds with the crashing of Bro.  Any ideas?  Where would you guys start digging if the workers just started dying (seemingly) randomly?
Cheers,

JB

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140922/4d6b7f67/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 19357 bytes
Desc: image002.jpg
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20140922/4d6b7f67/attachment.jpg