[Bro] Manager memory requirements for the intel framework

[Brian OBerry]

Oh, right, thanks for pointing that out.  If the decision is to stay with 2.4.x on that
system, we'll definitely look to install 2.4.2.

-----Original Message-----
From: Michael Shirk <shirkdog.bsd at gmail.com>
Date: Tuesday, May 15, 2018 at 1:11 PM
To: Drew Dixon <dwdixon at umich.edu>
Cc: Brian OBerry <brian.oberry at bluvector.io>, "bro at bro.org" <bro at bro.org>
Subject: EXT: Re: [Bro] Manager memory requirements for the intel framework

    You should update to at least 2.4.2 due to this vulnerability:
    
    http://blog.bro.org/2017/10/bro-252-242-release-security-update.html
    
    
    
    On Tue, May 15, 2018 at 12:34 PM, Drew Dixon <dwdixon at umich.edu> wrote:
    > It may be worth upgrading your production system either way since I do not
    > believe 2.4.x is technically supported any longer being that it's a release
    > from 2015.  Plus, lots of improvements since then...
    >
    > I'm not sure if this will help your specific problem but I'd be curious to
    > know if you're also running the default scan detection script on this box?
    > In your local.bro are you loading "misc/scan"?  If so, that script is known
    > to be a huge memory hog and could be indirectly contributing to your high
    > memory usage...just wanted to mention that, since you mentioned you
    > commented out the conditional that invokes “event Intel::new_item(item)” the
    > problem may reside more directly with the Intel Framework.
    >
    > How many indicators of each indicator type are you loading into the Intel
    > Framework?
    >
    > -Drew
    >
    > On Tue, May 15, 2018 at 9:13 AM, Brian OBerry <brian.oberry at bluvector.io>
    > wrote:
    >>
    >> Hello, All,
    >>
    >>
    >>
    >> We’re trying to understand manager memory requirements when the intel
    >> framework is in use, after experiencing multiple manager crashes per day
    >> when using the framework on a low-bandwidth (less than 1Gbps) CentOS 6
    >> machine running a production Bro 2.4.1 cluster.  These are happening because
    >> the manager is exhausting its tcmalloc heap limit of 16G, as reported in its
    >> stderr.log.  We removed the heap limit on an idle (no network traffic) Bro
    >> 2.4.1 test system, and found the parent VSize reported by “broctl top
    >> manager” went to 27G for an intel input file of 18K unique Intel::DOMAIN
    >> items.  It remained at 27G after many cycles of replacing the input file
    >> with 18K new unique items.
    >>
    >> Restoring the heap limit and attaching gdb to the manager on the test
    >> system shows a malloc failure backtrace that comes out of
    >> RemoteSerializer::SendCall ().  We commented the conditional that invokes
    >> “event Intel::new_item(item)” in base/frameworks/intel/main.bro to disable
    >> remote synchronization with the workers, and the huge VSize disappeared.
    >>
    >> We then built bro from master (version 2.5-569) and retested.  The manager
    >> VSize is much lower, but is still about 15G.
    >>
    >> Any advice on how to proceed with further diagnostics to hopefully reign
    >> in the manager memory requirements for intel?  It doesn’t appear at first
    >> blush that upgrading Bro will fix it, at least not entirely, and we’re
    >> reluctant to upgrade the production system without fully understanding the
    >> problem.
    >>
    >> Thanks,
    >>
    >> Brian
    >>
    >>
    >> _______________________________________________
    >> Bro mailing list
    >> bro at bro-ids.org
    >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
    >
    >
    >
    > _______________________________________________
    > Bro mailing list
    > bro at bro-ids.org
    > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
    
    
    
    -- 
    Michael Shirk
    Daemon Security, Inc.
    https://www.daemon-security.com