[Bro-Dev] [JIRA] (BIT-1103) Memory leak in Bro Intel framework

Andrew Hoying (JIRA) jira at bro-tracker.atlassian.net
Wed Dec 4 20:16:45 PST 2013


    [ https://bro-tracker.atlassian.net/browse/BIT-1103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14907#comment-14907 ] 

Andrew Hoying commented on BIT-1103:
------------------------------------

I ran some more checks and the memory leak is only based on the size and frequency of the files loaded through Intel::read_files, the size and content of the trace files tested doesn't seem to make much difference in the final number of lost objects.

> Memory leak in Bro Intel framework
> ----------------------------------
>
>                 Key: BIT-1103
>                 URL: https://bro-tracker.atlassian.net/browse/BIT-1103
>             Project: Bro Issue Tracker
>          Issue Type: Problem
>          Components: Bro
>    Affects Versions: 2.2
>         Environment: Red Hat Enterprise Linux Server release 6.5
>            Reporter: Andrew Hoying
>            Priority: High
>              Labels: intel, leak
>
> The policy/frameworks/intel/seen bro scripts have a memory leak. On my moderately busy Bro installation I am leaking about a gig of memory a day per worker process with the Intel framework enabled. I can replicate by adding the following to the local.bro default script and then running through a small PCAP with primarily dns, dhcp and syslog traffic.
> {{
> @load policy/frameworks/intel/seen
> redef Intel::read_files += {
>     "/usr/local/bro/spool/domain_suspicious.txt",
> };
> }}
> The intel file is in the following format, here's a few sample lines. It is generated automatically by CIF:
> {{
> #fields indicator       indicator_type  meta.source     meta.desc       meta.url        meta.cif_impact meta.cif_severity       meta.cif_confidence
> mete-tools.biz  Intel::DOMAIN   CIF - need-to-know      spammed domain  http://www.spamhaus.org/query/dbl?domain=mete-tools.biz (public)        -       -       95
> rttvxygkmwlqmq.net      Intel::DOMAIN   CIF - need-to-know      spammed domain  http://www.spamhaus.org/query/dbl?domain=rttvxygkmwlqmq.net (public)    -       -       95
> podserveruho.com        Intel::DOMAIN   CIF - need-to-know      spammed domain  http://www.spamhaus.org/query/dbl?domain=podserveruho.com (public)      -       -       95
> wwfcogdgntlxw.biz       Intel::DOMAIN   CIF - need-to-know      spammed domain  http://www.spamhaus.org/query/dbl?domain=wwfcogdgntlxw.biz (public)     -       -       95
> }}
> I compiled bro with gperftool debug support and followed the instructions here: http://www.bro.org/development/howtos/leaks.html. (Note, the instructions are wrong on the flags for ./configure, you need to add --enable-perftools-debug to get the -m option for bro)
> Here's the output from pprof top after running a PCAP trace with 10,000 packets. Running traces with more packets show a greater number of lost objects in the same code locations.
> {{
> # pprof bin/bro "/tmp/bro.24541.net_run-end.heap" --inuse_objects --lines --heapcheck  --edgefraction=1e-10 --nodefraction=1e-10
> Using local file bin/bro.
> Using local file /tmp/bro.24541.net_run-end.heap.
> Welcome to pprof!  For help, type 'help'.
> (pprof) top
> Total: 4295 objects
>     2150  50.1%  50.1%     2150  50.1% AsciiFormatter::ParseValue /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:186
>     2141  49.8%  99.9%     2141  49.8% copy_string /usr/src/bro-2.2/src/util.cc:155
>        2   0.0% 100.0%        2   0.0% re_alloc /usr/src/bro-2.2/build/src/re-scan.cc:2287
>        1   0.0% 100.0%        1   0.0% RE_parse /usr/src/bro-2.2/build/src/re-parse.y:110
>        1   0.0% 100.0%        1   0.0% RE_parse /usr/src/bro-2.2/build/src/re-parse.y:133
>        0   0.0% 100.0%     2141  49.8% AsciiFormatter::ParseValue /usr/src/bro-2.2/src/threading/AsciiFormatter.cc:195
>        0   0.0% 100.0%        4   0.1% Connection::NextPacket /usr/src/bro-2.2/src/Conn.cc:259
>        0   0.0% 100.0%        4   0.1% NetSessions::DispatchPacket /usr/src/bro-2.2/src/Sessions.cc:189
>        0   0.0% 100.0%        4   0.1% NetSessions::DoNextPacket /usr/src/bro-2.2/src/Sessions.cc:709
>        0   0.0% 100.0%        4   0.1% NetSessions::NextPacket /usr/src/bro-2.2/src/Sessions.cc:247
> }}



--
This message was sent by Atlassian JIRA
(v6.2-OD-03#6206)


More information about the bro-dev mailing list