[Bro] Worker System Memory Exhaustion

Aashish Sharma asharma at lbl.gov
Fri Apr 6 14:08:13 PDT 2018


[ sorry to chime late - still catching up on the thread ] 

Greg, 

I generally shunt (disable) sumstats on all my clusters. back in the day, while
early adapting I had memory issues in cluster and since then I have disabled it
every where for years now. 

Basically: in /usr/local/bro/share/bro/base/frameworks/sumstats/main.bro

to completely make sumstats ineffective - Add a return at the top of "function observe"


--- /home/bro/master/share/bro/base/frameworks/sumstats/main.bro        2018-04-06 14:02:05.131016000 -0700
+++ /home/bro/master/share/bro/base/frameworks/sumstats/main.bro.dis    2018-04-06 14:01:54.384697000 -0700
@@ -402,6 +402,9 @@ function create(ss: SumStat)
 function observe(id: string, key: Key, obs: Observation)
        {

+       ### this retun disables the sumstats
+       return;
+
        if ( id !in reducer_store )
                return;



As of scan-detection: 

Justin did us a favor by releasing bro-simple-scan - its simpler and effective
and flags >90-95% of scans flagged by scan-NG. 

scan-NG needs configurations - see : scan-config.bro in there. 

We've been using scan-NG for > 2years and is our work horse for dynamic firewall
blocking. very minimal FPs compared to old-scan. 

I have never used stock misc/scan  because it was only flagged < 10% of scanners
for us when I ran comparisons many many years ago and memory issues. 

may be start with bro-simple-scan and move to scan-NG if you really find stuff
missing. Send me pcaps if scan-NG is missing something. I'd be curious about
those kinds of scans. 

> Aashish is closer to you if you need someone to blame if it breaks :-)

Totally! :) 

Aashish 

On Fri, Apr 06, 2018 at 08:45:50PM +0000, Azoff, Justin S wrote:
> 
> > On Apr 6, 2018, at 4:31 PM, Greg Grasmehr <greg.grasmehr at caltech.edu> wrote:
> > 
> > I think Justin hit the nail on the head, we monitor two full /16, 3 /24
> > and 2 partial /16, in front of any local FW devices; similar to LBL.
> > Commenting out misc/scan did the trick, memory is now being freed as one
> > would expect.
> > 
> > We already know we have TONS of scanners traversing the network, so
> > probably don't need this at all although I am interested in hearing of
> > good alternatives.
> > 
> > Thanks again everyone, greatly appreciate the help.
> > 
> > Greg
> 
> https://github.com/ncsa/bro-simple-scan
> 
> https://github.com/initconf/scan-NG
> 
> both are available in bro-pkg.  I'm obviously partial to simple-scan, but 
> Aashish is closer to you if you need someone to blame if it breaks :-)
> 
> 
>> Justin Azoff
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


More information about the Bro mailing list