[Bro-Dev] [Bro] Version: 2.0-907 -- Bro manager memory exhaustion

[Vlad Grigorescu]

This seems to me to just be an issue of my ElasticSearch server not
keeping up with the load.

I ran master with just the Ascii logs for a few hours, and saw no evidence
of a memleak. Valgrind also came back cleanly (and actually, insertion
rate into ElasticSearch was about the same with/without valgrind - I was
expecting more of a performance hit).

  --Vlad

On 7/31/12 12:59 PM, "Vlad Grigorescu" <vladg at cmu.edu> wrote:

I've been running 2.0-905 for ~25-26 hours. The manager's memory usage has
slowly crept up to 13 GB.

One thing of note - I'm using the ElasticSearch log writer. I see 3
possible scenarios for this memleak:

1) There is indeed a leak in master, potentially only triggered by
specific traffic,
2) There is a leak in the ElasticSearch log writer,
3) My ElasticSearch server can't keep up with the load, and the manager is
receiving logs faster than it can send them to the writer, so they just
queue up.

Has anyone else tried the current code over an extended period on live
traffic? Also, if anyone has any ideas to try to figure out where this
leak is occurring, please let me know. I'm going to switch back to ASCII
logs for a bit, and see what that's like.

  --Vlad

On 7/31/12 12:26 PM, "Robin Sommer" <robin at icir.org> wrote:

(Taking to bro-dev).

On Tue, Jul 31, 2012 at 08:11 -0700, I wrote:

> That's not a good sign for the manager ... It's possible that we have
> a memory leak in there.

I just reran our leak tests and they didn't report anything (which is
good, but doesn't completely rule out any leaks).

I did see this though from valgrind:

    Object at 0x94e3410 of 68 bytes from an IgnoreObject() has disappeared

Does anybody know what valgrind it trying to tell me with that? Is it
a problem?

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org
_______________________________________________
bro-dev mailing list
bro-dev at bro-ids.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev


_______________________________________________
bro-dev mailing list
bro-dev at bro-ids.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev