[Bro] Possible system tweak for reducing memory usage

Michael Shirk shirkdog.bsd at gmail.com
Tue Jan 31 12:09:40 PST 2017


This was always a RHEL6/CentOS6 requirement for applications like you
stated.

Which OS are you noticing the issue and the performance gains on?

--
Michael Shirk
Daemon Security, Inc.
http://www.daemon-security.com

On Jan 31, 2017 3:04 PM, "Azoff, Justin S" <jazoff at illinois.edu> wrote:

> TL;DR: It's possible that transparent huge pages and bro do not get along,
> try doing a
>
>     # on all nodes
>     echo never > /sys/kernel/mm/transparent_hugepage/enabled
>     # then
>     broctl restart
>
> There are ways to make that permanent if it helps.
>
>
> I've been doing some research to try to figure out why some people have
> more memory issues than others.  I think the kernel feature Transparent
> Huge Pages (THP) and bro may not get along well.  It's supposed to help
> performance for memory allocations, but many services recommend disabling
> it (mongodb, redis, mysql).  For example:
>
> > Transparent Huge Pages (THP) is a Linux memory management system that
> reduces the overhead of Translation Lookaside Buffer (TLB) lookups on
> machines with large amounts of memory by using larger memory pages.
> >
> > However, database workloads often perform poorly with THP, because they
> tend to have sparse rather than contiguous memory access patterns. You
> should disable THP on Linux machines to ensure best performance with
> MongoDB.
>
> Bro memory allocations can best described as unpredictable, especially on
> 'custer in a box' deployments.
>
> On our systems, disabling it drops bro worker memory usage by 20% and
> manager/logger usage by even more, but since we only have one of those it's
> harder to compare.  For workers I disabled THP on half the nodes, and the
> post bro restart memory usage is consistently lower.
>
> --
> - Justin Azoff
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170131/99cc6e78/attachment.html 


More information about the Bro mailing list