[Bro] A more parallel Bro

[dalbrech at illinois.edu]

Robin Sommer wrote:
> On Mon, Mar 02, 2009 at 10:25 -0600, dalbrech at illinois.edu wrote:
>
>> real system.  So I built a simple tool I call "dns-packet-gen", which 
generates 
>> pathological DNS packets constructed for their difficulty in parsing.  The 
tool 
>> emits pcap traces containing DNS traffic with many records (50+), all of 
which 
>> feature deeply-recursive DNS names. 
>
> Neat. Is that tool available? 
Making it available is on my todo, but it's toward the bottom.  My group 
currently doesn't have a good web presence, but we're planning to fix that one 
of these days as I think it does a lot for a research group's visibility.

>> I'm wondering how you plan to address cache behavior in your multi-
threaded 
>> implementation of Bro.  Obviously, this is a really hard problem, and I'm not 
>> sure how well-studied it is in the literature, especially in the context of IDS.  
>
> That's an excellent question and I don't think it has been studied
> in the IDS context yet. Our main strategy is to keep related state
> together, e.g., by running all analysis involving the same
> connection on the same core. It's hard to predict though how cache
> effects will eventually turn out to hurt us, and we expect to do a
> number of measurements in this regard once the multi-threaded Bro is
> in a shape that we can actually measure something. Going from there,
> I'm sure there'll be lots of further optimization potential.

I'm going to start analyzing Bro's memory use and performance; do you have 
any recommendations about which policy I use?  I'm looking for a reasonably 
"representative" workload (if such a thing even exists), something that will test 
a lot of code paths through the system.

Thanks again,
DA

-- 
David R. Albrecht
Graduate Assistant, Hatswitch Group
U. Illinois Urbana-Champaign
(312) 445-0883