[Bro] Worker System Memory Exhaustion
Aashish Sharma
asharma at lbl.gov
Fri Apr 6 14:08:13 PDT 2018
[ sorry to chime late - still catching up on the thread ]
Greg,
I generally shunt (disable) sumstats on all my clusters. back in the day, while
early adapting I had memory issues in cluster and since then I have disabled it
every where for years now.
Basically: in /usr/local/bro/share/bro/base/frameworks/sumstats/main.bro
to completely make sumstats ineffective - Add a return at the top of "function observe"
--- /home/bro/master/share/bro/base/frameworks/sumstats/main.bro 2018-04-06 14:02:05.131016000 -0700
+++ /home/bro/master/share/bro/base/frameworks/sumstats/main.bro.dis 2018-04-06 14:01:54.384697000 -0700
@@ -402,6 +402,9 @@ function create(ss: SumStat)
function observe(id: string, key: Key, obs: Observation)
{
+ ### this retun disables the sumstats
+ return;
+
if ( id !in reducer_store )
return;
As of scan-detection:
Justin did us a favor by releasing bro-simple-scan - its simpler and effective
and flags >90-95% of scans flagged by scan-NG.
scan-NG needs configurations - see : scan-config.bro in there.
We've been using scan-NG for > 2years and is our work horse for dynamic firewall
blocking. very minimal FPs compared to old-scan.
I have never used stock misc/scan because it was only flagged < 10% of scanners
for us when I ran comparisons many many years ago and memory issues.
may be start with bro-simple-scan and move to scan-NG if you really find stuff
missing. Send me pcaps if scan-NG is missing something. I'd be curious about
those kinds of scans.
> Aashish is closer to you if you need someone to blame if it breaks :-)
Totally! :)
Aashish
On Fri, Apr 06, 2018 at 08:45:50PM +0000, Azoff, Justin S wrote:
>
> > On Apr 6, 2018, at 4:31 PM, Greg Grasmehr <greg.grasmehr at caltech.edu> wrote:
> >
> > I think Justin hit the nail on the head, we monitor two full /16, 3 /24
> > and 2 partial /16, in front of any local FW devices; similar to LBL.
> > Commenting out misc/scan did the trick, memory is now being freed as one
> > would expect.
> >
> > We already know we have TONS of scanners traversing the network, so
> > probably don't need this at all although I am interested in hearing of
> > good alternatives.
> >
> > Thanks again everyone, greatly appreciate the help.
> >
> > Greg
>
> https://github.com/ncsa/bro-simple-scan
>
> https://github.com/initconf/scan-NG
>
> both are available in bro-pkg. I'm obviously partial to simple-scan, but
> Aashish is closer to you if you need someone to blame if it breaks :-)
>
>
> —
> Justin Azoff
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
More information about the Bro
mailing list