[Bro] bro and pf_ring zc configuration success stories

[Azoff, Justin S]

Do you have the pf_ring plugin installed.  Do you see this output?

$ bro -N | grep -v built-in
Bro::PF_RING - Packet acquisition via PF_RING (dynamic, version 1.0)


— 
Justin Azoff

> On Sep 28, 2017, at 9:43 AM, radek <radoslawc at gmail.com> wrote:
> 
> Hi!
> 
> I've rebuilt bro with gperftools only.
> 
> With worker defined like this:
> 
> [worker-1]
> type=worker
> host=localhost
> interface=pf_ring::zc:27
> lb_method=pf_ring
> lb_procs=20
> 
> all worker threads fail with below message:
> ==== stderr.log
> 
> fatal error: problem with interface pf_ring::zc:27 (No such device) 
> 
> with zbalance_ipc stopped and using NIC device:
> 
> [worker-1]
> type=worker
> host=localhost
> interface=pf_ring::zc:enp5s0f0
> lb_method=pf_ring
> lb_procs=20
> 
> only one worker thread starts:
> 
> [BroControl] > status
> Name         Type    Host             Status    Pid    Started
> logger       logger  localhost        running   3886   28 Sep 09:38:30
> manager      manager localhost        running   4063   28 Sep 09:38:32
> proxy-1      proxy   localhost        running   4384   28 Sep 09:38:34
> proxy-2      proxy   localhost        running   4386   28 Sep 09:38:34
> worker-1-1   worker  localhost        stopped
> worker-1-2   worker  localhost        stopped
> worker-1-3   worker  localhost        running   4751   28 Sep 09:38:36
> worker-1-4   worker  localhost        stopped
> worker-1-5   worker  localhost        stopped
> worker-1-6   worker  localhost        stopped
> worker-1-7   worker  localhost        stopped
> worker-1-8   worker  localhost        stopped
> worker-1-9   worker  localhost        stopped
> worker-1-10  worker  localhost        stopped
> worker-1-11  worker  localhost        stopped
> worker-1-12  worker  localhost        stopped
> worker-1-13  worker  localhost        stopped
> worker-1-14  worker  localhost        stopped
> worker-1-15  worker  localhost        stopped
> worker-1-16  worker  localhost        stopped
> worker-1-17  worker  localhost        stopped
> worker-1-18  worker  localhost        stopped
> worker-1-19  worker  localhost        stopped
> worker-1-20  worker  localhost        stopped
> 
> rest of them are failing with message:
> 
> ==== stderr.log
> 
> fatal error: problem with interface pf_ring::zc:enp5s0f0 (Bad address)
> 
> 
> 
> Best regards
> 
> Rado
> 
> 
> On 28 September 2017 at 15:14, Azoff, Justin S <jazoff at illinois.edu> wrote:
> 
> > On Sep 28, 2017, at 5:52 AM, radek <radoslawc at gmail.com> wrote:
> >
> > Hi!
> > Thank you for your reply.
> >
> > In 'full zerocopy' mode:
> >
> > zbalance_ipc cluster-27.conf:
> >
> > https://gist.github.com/radoslawc/afa7293fde9ba5bc9f51640d5fc63005
> >
> > node.cfg:
> >
> > https://gist.github.com/radoslawc/c7406452f01c14caa43c729c164d701b
> >
> > bro doctor output for above setup:
> >
> > https://gist.github.com/radoslawc/bb3e608dfa7ceca97378c26e98520fae
> 
> Ah.. so this is not good:
> 
> error: 99.17%, 7562 out of 7625 connections are half duplex
> 
> And this is not great either:
> 
> ok, only 0.00%, 0 out of 13 connections appear to be duplicate
> 
> It only looked at 13 connections because there were only 13 bidirectional connections in the log.
> 
> I think your problem is this:
> 
> interface=zc:27
> 
> That should not actually work with the pf_ring plugin.. in order to use the pf_ring plugin the interface needs to start with pf_ring:: I believe you need
> 
> interface=pf_ring::zc:27
> 
> So try that and see if that fixes everything.  If not, can you remove lb_procs and move to one worker for now to at least verify that that configuration works.
> 
> 
> > Bro doctor states that bro binary is not linked against pfring (which is correct, as configure doesn't give this option) instead I've used pf_ring plugin from aux:
> >
> > Bro-PF_RING.linux-x86_64.so
> > user at u1604:/opt/bro/lib/bro/plugins/Bro_PF_RING/lib$ ldd Bro-PF_RING.linux-x86_64.so
> >         linux-vdso.so.1 =>  (0x00007ffdd37f1000)
> >         libpfring.so => /usr/local/lib/libpfring.so (0x00007f85dbd5e000)
> >         libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f85db9dc000)
> >         libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f85db7c6000)
> >         libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f85db3fc000)
> >         libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f85db1df000)
> >         librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f85dafd7000)
> >         libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f85dadd3000)
> >         libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f85daaca000)
> >         /lib64/ld-linux-x86-64.so.2 (0x00007f85dc1dc000)
> 
> Ah, that is correct.  I need to have it separately check to see if bro -N lists the pf_ring plugin.
> 
> If the pf_ring::zc thing fixes things, I'll fix bro-doctor to check for that.
> 
> I think the check needs to be that if bro -N lists the pf_ring plugin, the interface MUST start with pf_ring::
> 
> The bro pf_ring plugin should probably do the same check.. I think there are a few issues with the pf_ring plugin.  I'm working on fixing one issue that causes the plugin to be broken if you are not using ZC.
> 
> 
> 
> > I'll rebuild bro with gperftools only, thank you for pointing that out.
> >
> > Best regard
> > Rado
> 
> —
> Justin Azoff
> 
>