[Bro] Version: 2.0-907 -- Bro manager memory exhaustion

Martin Holste mcholste at gmail.com
Wed Aug 1 14:39:44 PDT 2012


It ought to be easy enough to tell if it is indeed disk logging IO
that's the bottleneck.  I would think simply disabling all logs or
sending all logs to /dev/null via symlink or even putting the logging
directory on /dev/shm for a bit would all be ways of seeing if the raw
IO is the problem.

On Wed, Aug 1, 2012 at 4:34 PM, Vlad Grigorescu <vladg at cmu.edu> wrote:
> This sounds like an interesting experiment. The main difference that I see
> is that when writing raw PCAP, you're writing to mostly-contiguous blocks.
> With the text logs, you're writing to 20 files, scattered who-knows-how,
> and at least 3 or 4 of those files are heavily used. The SATA layer
> *should* try to reorder the writes to minimize seeks, but who knows how
> efficient that is.
>
> If this actually was the issue, it'd be nice if Bro would detect that
> $sourceRate > $sinkRate, and log something to reporter, that there's a
> bottleneck in the log writer. That's a pretty daunting task, however.
>
>   --Vlad
>
> On 8/1/12 5:09 PM, "Martin Holste" <mcholste at gmail.com> wrote:
>
>> You might be right.  I had considered disk I/O and ran the manager on a
>> decent raid-10 array (the elsa server), disk i/o did not appear to be the
>> problem so much as CPU and memory.  That observation carried over to
>> development builds with a threaded manager, but at an accelerated rate;
>> right now I'm watching the disk I/O under threshold and the manager is
>> consuming 100M of memory per second until memory exhaustion.
>
> How much I/O from the manager have you seen thus far?  I'm not yet
> convinced that writing raw text files is the bottleneck.  When you
> think about writing raw pcap, it's orders of magnitude more MB/sec to
> disk than logging.
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



More information about the Bro mailing list