[Bro] zbalance_ipc with multiple applications and Bro
apumphrey at ivsec.com
Wed Feb 11 12:03:23 PST 2015
Your Bro config looks like it should work. From what I’ve seen that usually indicates an issue with pf_ring; possibly that zbalance_ipc is failing to run?
A couple other things to check on the pf_ring side, all of which applies to your worker nodes. Sorry if any of this obvious, just throwing out ideas:
- pf_ring kernel module installed
- pf_ring-aware ZC NIC driver installed and in use by the physical interface (ethtool -i)
- ZC license installed
- huge memory pages configured
If successful zbalance_ipc should output (when not in daemon mode or stdout/stderr redirected) something like this, followed by traffic collection stats:
Starting balancer with 8 consumer queues..
You can now attach to the balancer your application instances as follows:
pfcount -i zc:99 at 0
pfcount -i zc:99 at 1
pfcount -i zc:99 at 2
pfcount -i zc:99 at 3
pfcount -i zc:99 at 4
pfcount -i zc:99 at 5
pfcount -i zc:99 at 6
pfcount -i zc:99 at 7
Once zbalance_ipc is running you can use zcount_ipc as another way to validate what zbalance is doing. If you can run zcount_ipc and get packets from each of the app instances, your Bro config should work too.
> On Feb 11, 2015, at 12:18 PM, Michał Purzyński <michalpurzynski1 at gmail.com> wrote:
> I'm clearly doing something wrong.
> pfring-svn-latest/userland/examples_zc » ./zbalance_ipc -i zc:eth5 -c
> 99 -n 4,4 -m 1
> grep PFRINGFirstAppInstance broctl.cfg
> PFRINGFirstAppInstance = 4
> fatal error: /opt/bro/bin/bro: problem with interface zc:99 -
> pcap_open_live: zc:99: No such device exists (SIOCGIFHWADDR: No such
> And yeah, Bro is compiled against the pf_ring libpcap.
> On Wed, Feb 11, 2015 at 5:31 PM, Adam Pumphrey <apumphrey at ivsec.com> wrote:
>> You probably need to take a look at the PFRINGFirstAppInstance in broctl.cfg, it defaults to 0. If you’re looking to use the second app instance created by zbalance_ipc you’ll need to set that option to 4.
>> Also make sure the lb_method and lb_procs are set appropriately in node.cfg file, for example:
>> lb_procs=4 # should be equivalent to the number of instances per ‘ring'
>> If you really want to use zero-copy you need to add the prefix “zc:” to the physical interface name; e.g. zbalance_ipc -i zc:eth5. There are other pre-req’s for that to work, like configuring huge memory pages and installing the pf_ring-aware ZC driver.
>> I’ve been testing with ZC also but having issues with Bro reporting increased packet loss rates as soon I enable a configuration like this. Not sure if this is a hashing mode conflict, NIC/driver configuration issue or what.. I’d be interested to hear about your (or anyone else’s) results with such a setup.
>>> On Feb 11, 2015, at 10:14 AM, Michał Purzyński <michalpurzynski1 at gmail.com> wrote:
>>> I'm trying to start Bro and Suricata on one sensor, using the pf_ring
>>> ZC, like this
>>> zbalance_ipc -i eth5 -c 99 -n 4,4 -m 1
>>> where 99 is the cluster ID and -n <num>,<num> creates separate rings
>>> for each application. So far so good.
>>> I should tell Bro to somehow bind to the zc:99 at 4, zc:99 at 5, zc:99 at 6,
>>> zc:99 at 7 interfaces. How can I do it?
>>> Using zc:99 at 4 (AKA base, and let it increment automatically) does not work
>>> fatal error: /opt/bro/bin/bro: problem with interface zc:99 at 4 -
>>> pcap_open_live: zc:99 at 4: No such device exists (SIOCGIFHWADDR: No such
>>> Same for just zc:99 and not a surprise, Bro somehow needs to open
>>> sub-interfaces 4-7.
>>> Is it even supported?
>>> Bro mailing list
>>> bro at bro-ids.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Bro