[Bro] Bro 10Gb Performance
ed.sealing at sealingtech.org
Thu May 11 13:08:02 PDT 2017
I've done some research on the use of commodity hardware and the Intel X710
and XL710 cards (Amazon shows ~$475 or so). They are the only non-capture
card that allows SR-IOV in promiscuous mode (with on-card vlan popping),
which was a requirement for us. AFPacket seems to work well with it (i40e
drivers) and passes Justin's fanout checks on a default RHEL/CentOS 7.3
Did some analysis of the Suricata performance tuning paper WRT bro, and
it's almost universally applicable. Here are some quick recommendations
based on some of our analysis:
- Isolate CPUs in grub (e.g. isolcpus=1-7,9-15)
- # works = CPU Cores - 4. (This leaves 2 cores for the kernel, 1 for
proxy, 1 for manager.
- Lower the Ring Descriptors on the card to ~256 (ethtool -G eth0 rx 256)
- Reduce "Combined RSS" to 1 (ethtool -L eth0 combined 1)
In the Michael's paper, he suggests turning off "irqbalance", but I haven't
seen this make much of a difference, and sometimes hurts more than helps
(but I'm also using a single chip system).
Anyway, hope this helps.
On Thu, May 11, 2017 at 3:36 PM, Edgmand, Craig <craig.edgmand at okstate.edu>
> We are currently running Bro with 1 Gb intel cards and vanilla PF_RING and
> we have acceptable packet loss after filtering (1 – 3 percent), but we need
> to move up to 10 Gb sensors.
> Is there anyone that is using commodity hardware and Intel X520 network
> cards with Bro to process 10GB of traffic using AF_PACKET, vanilla PF_RING
> or PF_RING ZC?
> In the paper 100G Intrusion Detection, they utilized Myricom 10 Gb cards,
> with the sniffer software and were only running 10 workers per node or up
> to 1 Gb per worker. Is this possible on with Intel X520 using AF_PACKET or
> PF_RING? It is my understanding that AF_PACKET is broken in some kernels (I
> have used Justin’s fanout tool) and requires a driver update.
> Is there a diminishing return for number of workers per server?
> Michael Purzynski published a great paper on Suricata performance tuning
> to achieve 20 Gb throughput on commodity hardware using AF_PACKET. Is
> there a corresponding Bro document?
> Craig Edgmand
> IT Security
> Bro mailing list
> bro at bro-ids.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Bro