[Bro] crash with std::bad_alloc

Peter Wurzinger pw at iseclab.org
Mon Nov 24 09:29:44 PST 2008


Hi!

I have applied suggestion (a) and set sig_max_group_size = 10. That
greatly improved the situation. I have been running Bro with my
signature set for a week now without a problem concerning memory
(before, it used to crash after hours), and from the prof.log it looks
like there is still a lot of headroom.

Thanks a bunch!

Peter.


Robin Sommer wrote:
> On Thu, Nov 06, 2008 at 12:23 +0100, you wrote:
> 
>> Memory: total=3126520K total_adj=3116888K malloced: 2878549K
> 
> Yeah, that's a lot ... 
> 
>> ".*byte_seq1.*byte_seq2.*byte_seq3.*"
> 
> I'm guessing that these are indeed the problem, assuming there's no
> leak somewhere.  Having lots of such patterns is essentially the
> worst case for a DFA-based pattern matcher (recall that Bro
> internally combines many of these into a *one* regexp, which will
> let the number of states explode). 
> 
> Three things you could try:
> 
> (a) there is a tuning option for the signature engine which tells
> Bro how many regexps to combine internally into Big Ones. It's
> called sig_max_group_size and the default is 50. It might help to
> reduce this quite a bit (e.g., 10 or 20). 
> 
> (b) you could split each signature into several, one for each
> component of the regexp (byte_seq1, byte_seq2, ...), and then either
> chain these signatures with requires_signature condititions, or
> raise an event for each one individually and correlate the matches
> on the script-level to find out when all have matched. Both
> approahces have the disadvantage that they don't consider the order
> in which the subpatterns appear. 
> 
> (c) this one is kind of scary. :) There's a configure option
> --expire-dfa-states which enables some internal code to limit the
> size of the DFAs Bro builds (by expiring less frequently used states
> and recalculating them later if necessary). Enabling this has quite
> a performance impact on the matching process but even more worse is
> the fact that this option has most likely not been used by anybody
> for >5 years ... I'd almost bet it's broken in some way but you can
> still give it a try ... 
> 
> Robin
> 



More information about the Bro mailing list