[Bro] Bro Memory Consumtion

[Powell, Scott]

Bro Gurus,

I am having an issue with Bro and memory exhaustion. Currently I'm using click on a system with 8 x CPU cores to break up a network tap into three virtual interfaces (tap0, tap1 and tap2).  I'm then running my Bro cluster on the same machine with a three workers operating on different CPU cores and virtual interfaces. The system has 16G of physical RAM. After running for about 24 hours or so all of the physical RAM is exhausted and Bro being to go after swap. I increased swap to 8GB but this is a never ending battle as Bro will eventually eat everything it can find and crash the system.

How do I go about diagnosing which scripts/policies are causing this, or if it is an internal memory leak somewhere? I have seen references to reduce-memory.bro and profile.bro in some of the Wiki and or mailing list searches but these don't appear to be in the current 1.5.1 release.

I am running a large number of scripts from Seth Hall's script repository in addition to the ones that are enabled by default. Below are the policies I'm loading in local.bro:

@load alarm
@load notice
@load weird

@load dpd
@load detect-protocols
@load detect-protocols-http
@load dyn-disable
@load inactivity

@load dns
@load dns-lookup
@load finger
@load frag
@load ftp
@load icmp
@load hot
@load http-request
#@load http-reply
@load ident
@load irc
@load irc-bot
@load login
@load ntp
@load pop3
@load portmapper
@load scan
@load smtp
@load ssh
@load ssl
@load synflood
@load tcp
@load tftp
@load udp
@load worm

# Seth Hall Scripts
@load dns-passive-replication
@load http-identified-files
redef HTTP::ignored_urls = /^http:\/\/(www\.download\.windowsupdate\.com)|(download\.windowsupdate\.com)|(au\.download\.windowsupdate\.com)|(download\.microsoft\.com)|(office\.microsoft\.com)\//;
@load known-hosts
@load known-services
@load logging.dns-ext
@load logging.ftp-ext
@load logging.http-ext
@load logging.smtp-ext
@load logging.ssh-ext
@load smtp-ext-count-rejects
@load ssh-ext
@load ssl-ext
redef SSH::authentication_data_size = 4000;

Thanks,

Scott Powell
Unix Systems Engineer / Information Security Analyst
Office of the CIO - Information Systems (OCIO-IS)
Medical University of South Carolina
powellsm at musc.edu
(843) 792-6651

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100222/317952cd/attachment.html