[Bro] Version: 2.0-907 -- Bro manager memory exhaustion

[Chris Crawford]

I have the following in my local.bro file:

redef SMTP::generate_md5 += /image.*/;
redef HTTP::generate_md5 += /image.*/;
redef SMTP::generate_md5 += /text.*/;
redef HTTP::generate_md5 += /text.*/;
redef SMTP::generate_md5 += /application.*/;
redef HTTP::generate_md5 += /application.*/;
redef SMTP::generate_md5 += /audio.*/;
redef HTTP::generate_md5 += /audio.*/;
redef SMTP::generate_md5 += /video.*/;
redef HTTP::generate_md5 += /video.*/;


Using broctl's top and a little trial and error, I can see that these
lines are the cause of my high CPU usage.  It also causes higher
memory usage as well, but memory usage always climbs and never gets
smaller.  I don't know if these lines are responsible for just higher
memory usage in general, or whether they are also responsible gradual
climb in memory.  It appears that memory gradually climbs even without
these lines, but I haven't had enough time to test that idea.  I
believe that the climbing memory eventually leads to a crash,
typically when Reassem.cc attempts to allocate some new memory and an
unhandled exception is triggered.  The broctl cron command restarts
bro for me.

-Chris

On Wed, Aug 8, 2012 at 11:48 AM, Tritium Cat <tritium.cat at gmail.com> wrote:
> On Thu, Aug 2, 2012 at 1:45 PM, Tritium Cat <tritium.cat at gmail.com> wrote:
>>
>> On Wed, Aug 1, 2012 at 7:27 PM, Chris Crawford
>> <christopher.p.crawford at gmail.com> wrote:
>>>
>>> Have you seen any of my threads from earlier this year?
>>>
>>> http://bit.ly/JJQVVf
>>> http://bit.ly/N2l4yT
>>>
>>> Your issue sounds similar to what I was experiencing.
>>>
>>> Bro 2.0 is routinely uses up all available memory and then crashes for
>>> me.
>>>
>
> Someone mentioned it's likely due to the traffic on the network; they had a
> similar problem that involved certain SSL traffic.  The idea is to disable
> features until finding the problem and then devise a workaround.  That's the
> plan for now.
>
> --TC
>
>