[Bro] Bro performance issues

[William Jones]

Don't you need more foo to get PF_RING to load balance  it looks like you have to bind a bro instances to a cpu?


From: Tomer Teller [mailto:djteller at gmail.com]
Sent: Thursday, November 03, 2011 10:17 AM
To: Martin Holste
Cc: William Jones; bro at bro-ids.org
Subject: Re: [Bro] Bro performance issues

PF_RING Version     : 5.0.0 ($Revision: exported$)
Ring slots          : 4096
Slot version        : 13
Capture TX          : Yes [RX+TX]
IP Defragment       : No
Socket Mode         : Standard
Transparent mode    : Yes (mode 0)
Total rings         : 0
Total plugins       : 0

[BroControl] > status
Name       Type       Host       Status        Pid    Peers  Started
manager    manager    localhost  running       13328  3      03 Nov 19:08:41
proxy-1    proxy      localhost  running       13359  3      03 Nov 19:08:43
worker-1   worker     localhost  running       13403  2      03 Nov 19:08:45
worker-2   worker     localhost  running       13402  2      03 Nov 19:08:45


[BroControl] > netstats
  worker-1: 1320362018.486131 recvd=669511 dropped=286 link=669797
  worker-2: 1320362018.686087 recvd=669650 dropped=147 link=669797



On Thu, Nov 3, 2011 at 2:47 PM, Martin Holste <mcholste at gmail.com<mailto:mcholste at gmail.com>> wrote:
What's the output of cat /proc/net/pf_ring/* ?  There should be a file
per Bro worker in there when Bro is running.

On Thu, Nov 3, 2011 at 6:47 AM, Tomer Teller <djteller at gmail.com<mailto:djteller at gmail.com>> wrote:
> Thanks for the tip William,
> This trick doesn't work on Bro2.0
> I tried both aux_scripts and creating the worker-1.local.bro file
> I installed the new policy stop/start and again the workers are not
> load-balancing traffic
> tcpreplay  -i ethX --mbps=1000.0 680000_packets.pcap
> [BroControl] > netstats
>   worker-1: 1320349321.810072 recvd=639496 dropped=30101 link=669597
>   worker-2: 1320349322.010123 recvd=639549 dropped=30048 link=669597
> This is Out of the box Bro2.0 with PF_RING and libpcap compiles with
> PF_RING.
>
>
> On Wed, Nov 2, 2011 at 9:41 PM, William Jones <jones at tacc.utexas.edu<mailto:jones at tacc.utexas.edu>> wrote:
>>
>> Try the following pcap filtering trick:
>>
>> '
>>
>> [manager]
>>
>> type=manager
>>
>> host=localhost
>>
>>
>>
>> [proxy-1]
>>
>> type=proxy
>>
>> host=localhost
>>
>>
>>
>> [worker-1]
>>
>> type=worker
>>
>> host=localhost
>>
>> interface=em0
>>
>> aux_scripts=q1of2
>>
>>
>>
>> [worker-2]
>>
>> type=worker
>>
>> host=localhost
>>
>> aux_scripts=q2of2
>>
>>
>>
>> Add the following scripts in  <bro>share/bro/site
>>
>>
>>
>> q1of2.bro:
>>
>> redef restrict_filters += { ["q0"] = "(ip[12:4] + ip[16:4]) & 1 == 0" };
>>
>>
>>
>> q2of2.bro:
>>
>>
>>
>> redef restrict_filters += { ["q0"] = "(ip[12:4] + ip[16:4]) & 1 == 1" };
>>
>>
>>
>>
>>
>>
>>
>> This will split input between the two bro workers.
>>
>> ~
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> From: bro-bounces at bro-ids.org<mailto:bro-bounces at bro-ids.org> [mailto:bro-bounces at bro-ids.org<mailto:bro-bounces at bro-ids.org>] On Behalf
>> Of Tomer Teller
>> Sent: Wednesday, November 02, 2011 10:46 AM
>> To: Seth Hall
>> Cc: bro at bro-ids.org<mailto:bro at bro-ids.org>
>> Subject: Re: [Bro] Bro performance issues
>>
>>
>>
>> This is my node.cfg config file:
>>
>>
>>
>> [manager]
>>
>> type=manager
>>
>> host=localhost
>>
>>
>>
>> [proxy-1]
>>
>> type=proxy
>>
>> host=localhost
>>
>>
>>
>> [worker-1]
>>
>> type=worker
>>
>> host=localhost
>>
>> interface=em0
>>
>>
>>
>> [worker-2]
>>
>> type=worker
>>
>> host=localhost
>>
>> interface=em0
>>
>>
>>
>> I am running on Bro 2.0 Beta.
>>
>>
>>
>> I am replying a 680000 packets pcap file to the machine.
>>
>>
>>
>> When running broctl's netstats, this is what i see:
>>
>>
>>
>> worker-1: 1320276618.514073 recvd=669576 dropped=0 link=669576
>>
>> worker-2: 1320276618.714115 recvd=669576 dropped=0 link=669576
>>
>>
>>
>> I expect to see load-balance between worker-1 and worker-2 but they are
>> getting the same traffic.
>>
>>
>>
>> I decided to patch Bro in order to "support" that load-balancing.
>>
>>
>>
>> I edited PktSrc::Process() (PktSrc.cc) and added my own code to
>> distinguish between processes (i.e. worker-1, worker-2).
>>
>>
>>
>> Then looked at the data variable, extracted the ip src & dst and checked
>> for ->  (ipSRC ^ ipDST) % 2 == 0
>>
>>
>>
>> Worker-1 gets all the Even Result, Worker-2 gets all the Odd Results
>>
>>
>>
>> Also had to play around with (++stats.received) to reflect the new
>> changes.
>>
>>
>>
>> This small patch dramatically improved my performance.
>>
>>
>>
>>
>>
>>
>>
>> On Tue, Nov 1, 2011 at 4:08 PM, Seth Hall <seth at icir.org<mailto:seth at icir.org>> wrote:
>>
>> On Nov 1, 2011, at 4:26 AM, Tomer Teller wrote:
>>
>> > I configured the node.cfg and added:
>> > 1 manager
>> > 1 proxy
>> > 2 workers  - sniffing the same interface
>>
>> Could you send the content of your node.cfg file?  I noticed in your
>> broctl session you sent, there was a reference to worker-3 which isn't
>> represented in your description.
>>
>> Thanks,
>>
>>  .Seth
>>
>> --
>> Seth Hall
>> International Computer Science Institute
>> (Bro) because everyone has a network
>> http://www.bro-ids.org/
>>
>>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org<mailto:bro at bro-ids.org>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20111103/04a74ec7/attachment.html