[Bro] Bro & malloc implementations

[Clark, Gilbert]

Hi Jim:

Re: #3, here's what I did:

First, I ran './configure'

After successfully running configure, I opened up build/CMakeCache.txt and
found:
CMAKE_EXE_LINKER_FLAGS:STRING=' '

I modified that line to read:
CMAKE_EXE_LINKER_FLAGS:STRING=-L/path/to/jemalloc/lib/dir -ljemalloc

Then I did make clean / make / make install.

To verify bro was linked against jemalloc, I ran otool -L /path/to/bro and
verified that jemalloc was in the list of libraries I found there.

Also, other stuff I found to be useful:

* Exporting VERBOSE=1 before executing make will tell Cmake to dump the
complete compilation string to the console.  This can be used to manually
verify that -ljemalloc is included in the flags when the bro binary is
being linked.

* The above will link all executables output during the build process with
the linker flags specified in CMAKE_EXE_LINKER_FLAGS.  Since some of the
utilities generated earlier in the build are used later in the process,
the runtime library path needs to include libjemalloc before make is run.


Hope that helps,
Gilbert Clark

From:  Jim Mellander <jmellander at lbl.gov>
Date:  Mon, 19 Nov 2012 14:18:15 -0500
To:  "bro at bro-ids.org List" <bro at bro-ids.org>
Subject:  [Bro] Bro & malloc implementations


I wanted to share my experiences with bro and various malloc
implementations.  These are all running various versions of bro on SL 6.2
- a recompile of RedHat Enterprise Linux from Fermi Lab -
https://www.scientificlinux.org/
1. We are running a legacy Bro 1.5 installation currently supporting our
old Instrumented SSHd infrastructure -
http://code.google.com/p/auditing-sshd/ - we found that the standard
malloc uses more memory than either of tcmalloc or jemalloc.  I build bro
with --enable-perftools to test tcmalloc, with a smaller memory footprint.
 To use jemalloc I just set LIBS=-ljemalloc  -  at this point we are using
jemalloc, as it seems even a bit more memory thrifty than tcmalloc

2. As part of the Instrumented SSHd infrastructure, we also have a perl
script (ssllogmux) that runs a select loop accepting connections from all
the Instrumented SSHd clients - typically several thousand at once.  This
is also in the Instrumented SSHd distribution.  Perl is compiled to not
use its own malloc, but to use the system.  However, using the system
malloc, this program would freeze after a day or so of operation.  By
using LD_PRELOAD, we forced it to use alternate malloc libraries - under
both tcmalloc and jemalloc, it runs reliably.

3. On another system, we ran Bro 2.0 (now running 2.1), and bro was
(un)reliably freezing after a day or two - running but capturing no data
when compiled with --enable-perftools - when compiled with
--disable-perftools, it has been rock solid.  I don't know enough yet
about Cmake to have it use another malloc implementation easily -
hopefully someone else knows how to do that - I want to test jemalloc

So thats our current state of play - I would be interested in other folks
experience.