[Bro] Bro 10Gb Performance

[Ed Sealing]

I've done some research on the use of commodity hardware and the Intel X710
and XL710 cards (Amazon shows ~$475 or so). They are the only non-capture
card that allows SR-IOV in promiscuous mode (with on-card vlan popping),
which was a requirement for us. AFPacket seems to work well with it (i40e
drivers) and passes Justin's fanout checks on a default RHEL/CentOS 7.3
build).

Did some analysis of the Suricata performance tuning paper WRT bro, and
it's almost universally applicable. Here are some quick recommendations
based on some of our analysis:

- Isolate CPUs in grub (e.g. isolcpus=1-7,9-15)
- # works = CPU Cores - 4. (This leaves 2 cores for the kernel, 1 for
proxy, 1 for manager.
- Lower the Ring Descriptors on the card to ~256 (ethtool -G eth0 rx 256)
- Reduce "Combined RSS" to 1 (ethtool -L eth0 combined 1)

In the Michael's paper, he suggests turning off "irqbalance", but I haven't
seen this make much of a difference, and sometimes hurts more than helps
(but I'm also using a single chip system).

Anyway, hope this helps.

~Ed

On Thu, May 11, 2017 at 3:36 PM, Edgmand, Craig <craig.edgmand at okstate.edu>
wrote:

> We are currently running Bro with 1 Gb intel cards and vanilla PF_RING and
> we have acceptable packet loss after filtering (1 – 3 percent), but we need
> to move up to 10 Gb sensors.
>
>
>
> Is there anyone that is using commodity hardware and Intel X520 network
> cards with Bro to process 10GB of traffic using AF_PACKET, vanilla PF_RING
> or PF_RING ZC?
>
>
>
> In the paper 100G Intrusion Detection, they utilized Myricom 10 Gb cards,
> with the sniffer software and were only running 10 workers per node or up
> to 1 Gb per worker.  Is this possible on with Intel X520 using AF_PACKET or
> PF_RING? It is my understanding that AF_PACKET is broken in some kernels (I
> have used Justin’s fanout tool) and requires a driver update.
>
>
>
> Is there a diminishing return for number of workers per server?
>
>
>
> Michael Purzynski published a great paper on Suricata performance tuning
> to achieve 20 Gb throughput on commodity hardware using AF_PACKET.  Is
> there a corresponding Bro document?
>
>
>
> Thanks,
>
>
>
> Craig Edgmand
>
> IT Security
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170511/cb2f4857/attachment.html