[Bro] zbalance_ipc with multiple applications and Bro

[Adam Pumphrey]

Your Bro config looks like it should work.  From what I’ve seen that usually indicates an issue with pf_ring; possibly that zbalance_ipc is failing to run? 

A couple other things to check on the pf_ring side, all of which applies to your worker nodes. Sorry if any of this obvious, just throwing out ideas:

- pf_ring kernel module installed
- pf_ring-aware ZC NIC driver installed and in use by the physical interface (ethtool -i)
- ZC license installed
- huge memory pages configured

If successful zbalance_ipc should output (when not in daemon mode or stdout/stderr redirected) something like this, followed by traffic collection stats:

Starting balancer with 8 consumer queues..
You can now attach to the balancer your application instances as follows:
Application 0
        pfcount -i zc:99 at 0
        pfcount -i zc:99 at 1
        pfcount -i zc:99 at 2
        pfcount -i zc:99 at 3
Application 1
        pfcount -i zc:99 at 4
        pfcount -i zc:99 at 5
        pfcount -i zc:99 at 6
        pfcount -i zc:99 at 7 

Once zbalance_ipc is running you can use zcount_ipc as another way to validate what zbalance is doing.  If you can run zcount_ipc and get packets from each of the app instances, your Bro config should work too.  

Adam

> On Feb 11, 2015, at 12:18 PM, Michał Purzyński <michalpurzynski1 at gmail.com> wrote:
> 
> I'm clearly doing something wrong.
> 
> pfring-svn-latest/userland/examples_zc » ./zbalance_ipc -i zc:eth5 -c
> 99 -n 4,4 -m 1
> 
> 
> [nsm1-sfo-eth5]
> type=worker
> host=10.251.75.9
> interface=zc:99
> lb_method=pf_ring
> lb_procs=4
> 
> grep PFRINGFirstAppInstance broctl.cfg
> 
> PFRINGFirstAppInstance = 4
> 
> 
> fatal error: /opt/bro/bin/bro: problem with interface zc:99 -
> pcap_open_live: zc:99: No such device exists (SIOCGIFHWADDR: No such
> device)
> 
> And yeah, Bro is compiled against the pf_ring libpcap.
> 
> On Wed, Feb 11, 2015 at 5:31 PM, Adam Pumphrey <apumphrey at ivsec.com> wrote:
>> You probably need to take a look at the PFRINGFirstAppInstance in broctl.cfg, it defaults to 0.  If you’re looking to use the second app instance created by zbalance_ipc you’ll need to set that option to 4.
>> 
>> Also make sure the lb_method and lb_procs are set appropriately in node.cfg file, for example:
>> 
>> interface=zc:99
>> lb_method=pf_ring
>> lb_procs=4    # should be equivalent to the number of instances per ‘ring'
>> 
>> If you really want to use zero-copy you need to add the prefix “zc:” to the physical interface name; e.g.  zbalance_ipc -i zc:eth5.  There are other pre-req’s for that to work, like configuring huge memory pages and installing the pf_ring-aware ZC driver.
>> 
>> I’ve been testing with ZC also but having issues with Bro reporting increased packet loss rates as soon I enable a configuration like this.  Not sure if this is a hashing mode conflict, NIC/driver configuration issue or what..  I’d be interested to hear about your (or anyone else’s) results with such a setup.
>> 
>> Adam
>> 
>>> On Feb 11, 2015, at 10:14 AM, Michał Purzyński <michalpurzynski1 at gmail.com> wrote:
>>> 
>>> Hi.
>>> 
>>> I'm trying to start Bro and Suricata on one sensor, using the pf_ring
>>> ZC, like this
>>> 
>>> zbalance_ipc -i eth5 -c 99 -n 4,4 -m 1
>>> 
>>> where 99 is the cluster ID and -n <num>,<num> creates separate rings
>>> for each application. So far so good.
>>> 
>>> I should tell Bro to somehow bind to the zc:99 at 4, zc:99 at 5, zc:99 at 6,
>>> zc:99 at 7 interfaces. How can I do it?
>>> 
>>> Using zc:99 at 4 (AKA base, and let it increment automatically) does not work
>>> 
>>> fatal error: /opt/bro/bin/bro: problem with interface zc:99 at 4 -
>>> pcap_open_live: zc:99 at 4: No such device exists (SIOCGIFHWADDR: No such
>>> device)
>>> 
>>> Same for just zc:99 and not a surprise, Bro somehow needs to open
>>> sub-interfaces 4-7.
>>> 
>>> Is it even supported?
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150211/5a2a58fa/attachment.html