[Bro] bro and pf_ring zc configuration success stories

[radek]

Hi!
Thank you for your reply.

In 'full zerocopy' mode:

zbalance_ipc cluster-27.conf:

https://gist.github.com/radoslawc/afa7293fde9ba5bc9f51640d5fc63005

node.cfg:

https://gist.github.com/radoslawc/c7406452f01c14caa43c729c164d701b

bro doctor output for above setup:

https://gist.github.com/radoslawc/bb3e608dfa7ceca97378c26e98520fae

Bro doctor states that bro binary is not linked against pfring (which is
correct, as configure doesn't give this option) instead I've used pf_ring
plugin from aux:

Bro-PF_RING.linux-x86_64.so
user at u1604:/opt/bro/lib/bro/plugins/Bro_PF_RING/lib$ ldd
Bro-PF_RING.linux-x86_64.so
        linux-vdso.so.1 =>  (0x00007ffdd37f1000)
        libpfring.so => /usr/local/lib/libpfring.so (0x00007f85dbd5e000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007f85db9dc000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007f85db7c6000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f85db3fc000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f85db1df000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f85dafd7000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f85dadd3000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f85daaca000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f85dc1dc000)

I'll rebuild bro with gperftools only, thank you for pointing that out.

Best regard
Rado


On 27 September 2017 at 23:53, Azoff, Justin S <jazoff at illinois.edu> wrote:

>
> > On Sep 26, 2017, at 6:07 PM, radek <radoslawc at gmail.com> wrote:
> >
> > Hi!
> >
> > Anyone care to share bro + pfring success story?
> >
> > What's the speed, what NIC, what's the configuration.
> >
> > I'm running bro 2.5.1 built with jemalloc and gperftools and against
> pf_ring 6.6.0 with ixgbe_zc on CentOS 7.2.
>
> You can't be using both jemalloc and gperftools(tcmalloc).. they are both
> malloc implementations.
>
> > In ZeroCopy mode with zbalance_ipc dividing NIC to 20 application rings
> (-n 20) I'm getting each CPU core loaded at 100% and around 50% packet drop
> (reported by netstats in broctl).
>
> Sounds like the load balancing is not working right and you are just
> analyzing all of your traffic 20 times.  What does your node.cfg contain?
>
> > When redirecting from zc to 20 dummy interfaces (zbalance_ipc -r
> 0:dummy0 and so on) I'm getting around 50% load on each core and a lot less
> of packet drop (10% - 15%).
> >
> > This is with traffic around 700 - 800 Mbit/s
>
> A few workers should be able to handle this load, not to mention 20..
>
> > All input will be highly appreciated.
> >
>
> Can you install bro-pkg (http://bro-package-manager.
> readthedocs.io/en/stable/quickstart.html) and then do
>
> bro-pkg install bro-doctor --version 1.16.1
> broctl doctor.bro
>
> And share the results.
>
> —
> Justin Azoff
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170928/d4f5b7e7/attachment.html