[Bro] A few questions

Tyler Schoenke Tyler.Schoenke at colorado.edu
Thu Feb 4 08:06:15 PST 2010


Tim,

I figured that I would chime in since I'm running with the BPF filter.
I have had decent performance using the BPF filter.  I am attempting to
process 1 Gbps on a two-processor, quad-core box with 24 GB of RAM.
Right now, it is dropping about 40-50% of the traffic.  I think Seth is
running with a hardware frontend that distributes the traffic to eight
worker computers with dual or quad cores.  That gives 16 or 32 CPU cores
to process traffic.  Since I don't have a hardware frontend, I am
limited to processing traffic on a single computer, which means up to
eight workers (one per core).

Here is the formula that Robin gave me:
((ip[12:4]+ip[16:4])-((ip[12:4]+ip[16:4])/N)*N) = J

N is the total number of processes you have (ideally a prime), and J
is a different value out of {0...N-1} for each process.

The mod operation broken down:
((A)-((A)/N)*N) = J
A=ip[12:4]+ip[16:4]
ip[12:4] = source IP
ip[16:4] = destination IP

It will probably work as-is, but the offset of 12 and 16 for may vary
depending on how much of the ethernet header is on the packet.  You can
use dumpcap/Wireshark to grab packets and figure out where the header is
for your interface. Wireshark shows my source IP offset at 26, which is
what Justin had in his Click configuration, but 12 and 16 seem to be
working for me.  The ethernet header appears to be stripped off by the
time it gets processed by Bro.

--- Example node.cfg ----
[worker-1]
type=worker
host=_dns_name_of_worker_
interface=igb1
aux_scripts=worker1.bro


--- Example worker1.bro ---
redef restrict_filters += { ["mod source and dest pairs over multiple
procs "] = "((ip[14:2]+ip[18:2])-((ip[14:2]+ip[18:2])/8)*8) == 0" };

I was using 14:2 and 18:2 to look at the last two octets of the src and
dst IP since we have a class B subnet.  I don't know that it is
computationally less expensive, but it seems to give the same results.

I have all 8 workers listening to the same ethernet interface igb1, so
all eight workers have to filter the full 1Gbps of traffic down to 1/8th
the amount.  Click! should have better performance because it splits the
1Gbps eight ways first, then sends 1/8th to each worker.

It looks like Click is available for FreeBSD.  I'd like to test that to
see if I can gain some performance.  I am having difficulty compiling it
on FreeBSD 7.1 amd64 however.  If anyone has Click/FreeBSD working,
please let me know.

The Bro wiki mentions that Click! is limited to 2Gbps in tests.  I
wonder if that is still true?  I was thinking about the possibility of
installing a 10 Gig card in the current server as well as some
additional 1 Gig ports.  Then using Click to split the traffic to some
workers on this box and send the rest out the additional 1 Gig ports to
some additional workers.  That way I could use this server as a frontend
plus workers, but expand the cluster to additional computers.

Tyler

--
Tyler Schoenke
Network Security Analyst
IT Security Office
University of Colorado - Boulder

On 02/04/2010 06:41 AM, Tim Rupp wrote:
> For the sake of future reference (and my own curiosity) can I request
> the filter be sent to the list?
> 
> Thanks,
> Tim
> 
> On 02/03/2010 11:03 AM, Robin Sommer wrote:
>>
>> On Tue, Feb 02, 2010 at 13:53 -0500, Powell, Scott wrote:
>>
>>> Given our current setup, how would I go about these BPF tricks to
>>> leverage multiple cores on a single machine?
>>
>> The click setup already mentioned is probably the better solution,
>> but when using BPF, you would give each worker a different BPF
>> filter ignoring all but its slice of the traffic. One can express
>> the hash "(src+dst) mod n" in BPF (let me know if you want the exact
>> filter).
>>
>> Robin
>>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 



More information about the Bro mailing list