[Bro] bro cluster with pf ring dna+libzero

[Li, Yee-Ting]

Hi Gary,

ah, that makes sense! -n 10,1 works great :) thanks very much.

is anyone using ZC pf_ring for bro?

cheers,

Yee.  


On Wednesday, 18 June 2014 at 17:27, Gary Faulkner wrote:

> Hello,
>  
> Capstats is a separate application as far as pfdnacluster_master is  
> concerned. You can tell pfdnacluster_master that you want to send the  
> same traffic to another application using the -n flag by using a "," and  
> then specifying how many instances of the second app you intend to run.  
> When you call pfdnacluster_master try "-n 10,1" instead of "-n 10". You  
> actually want to run two applications against the same traffic, but the  
> second app, capstats, will only run one process that needs to consume  
> all of the traffic instead of having slices of traffic load balanced  
> between multiple processes.
>  
> Regards,
> Gary
>  
>  
>  
> On 6/18/2014 6:02 PM, Li, Yee-Ting wrote:
> > we're deploying a new bro cluster and am a huge newbie on all of this; so please excuse my ignorance. i have yet to actually start capturing on the cluster (awaiting delivery of a front-end device)
> >  
> > on each worker i have the dna+libzero ixgbe driver installed and insmodded. so i run:
> >  
> > $ sudo insmod pf_ring.ko enable_tx_capture=0 min_num_slots=32768
> > $ sudo insmod ixgbe.ko RSS=1,1,1,1 num_rx_slots=32768 mtu=9000
> >  
> >  
> >  
> > $ sudo /usr/sbin/setcap cap_net_raw,cap_net_admin=eip /usr/bin/pfdnacluster_master
> > $ /usr/bin/pfdnacluster_master -d -P /var/run/pfdnacluster-dna0.pid -D bromaint -c 0 -i dna0 -n 10
> >  
> > i do the setcap as i am running bro as non-root user. looks good…
> >  
> > $ cat /proc/net/pf_ring/13979-dna0.1
> > Bound Device(s) :
> > Active : 1
> > Breed : DNA
> > Sampling Rate : 1
> > Capture Direction : RX+TX
> > Socket Mode : RX only
> > Appl. Name : dna-cluster-0-socket-0
> > IP Defragment : No
> > BPF Filtering : Disabled
> > # Sw Filt. Rules : 0
> > # Hw Filt. Rules : 0
> > Poll Pkt Watermark : 128
> > Num Poll Calls : 0
> > Channel Id : 0
> > Num RX Slots : 32768
> > Num TX Slots : 8192
> > Tot Memory : 672399360 bytes
> > Cluster: Tot Recvd : 11
> > Cluster: Tot Sent : 0
> >  
> >  
> >  
> > then on my manager i have the following nodes.cfg:
> >  
> > [manager]
> > type=manager
> > host=sec-broman
> >  
> > [proxy-0]
> > type=proxy
> > host=sec-broman
> >  
> > [proxy-1]
> > type=proxy
> > host=sec-broman
> >  
> > [sec-bro01-0]
> > type=worker
> > host=sec-bro01
> > interface=dnacluster:0
> > lb_method=pf_ring
> > lb_procs=10
> >  
> >  
> >  
> > using bro 2.3; so i believe the lb_pf_ring.py script understands the dnacluster interface spec.
> >  
> > so i do an 'broctl install' (as user bromaint) from the manager, then log onto my worker and run
> >  
> > $ sudo /usr/sbin/setcap cap_net_raw,cap_net_admin=eip /opt/bro/bin/capstats
> > $ sudo /usr/sbin/setcap cap_net_raw,cap_net_admin=eip /opt/bro/bin/bro
> >  
> >  
> >  
> > then a 'broctl start' on the manager. everything looks fine so far… then i run 'broctl capstats' and i get:
> >  
> > Interface kpps mbps (10s average)
> > ----------------------------------------
> > sec-bro01-0-9: capstats failed (error: dnacluster:0: No such device exists (SIOCGIFHWADDR: No such device))
> >  
> >  
> >  
> > looking at proc for the pid of that bro instance, i get:
> >  
> > $ ps aux | grep sec-bro01-0-9
> > bromaint 14696 0.0 0.0 108128 1496 ? S 15:28 0:00 bash /opt/bro/share/broctl/scripts/run-bro -1 -i dnacluster:0 at 9 -U .status -p broctl -p broctl-live -p local -p sec-bro01-0-9 local.bro broctl base/frameworks/cluster local-worker.bro broctl/auto
> >  
> > bromaint 14778 25.1 0.0 157736 56320 ? S 15:28 0:02 /opt/bro/bin/bro -i dnacluster:0 at 9 -U .status -p broctl -p broctl-live -p local -p sec-bro01-0-9 local.bro broctl base/frameworks/cluster local-worker.bro broctl/auto
> > bromaint 14846 14.3 0.0 161832 52996 ? SN 15:28 0:01 /opt/bro/bin/bro -i dnacluster:0 at 9 -U .status -p broctl -p broctl-live -p local -p sec-bro01-0-9 local.bro broctl base/frameworks/cluster local-worker.bro broctl/auto
> >  
> > $ cat /proc/net/pf_ring/14778-none.7
> > Bound Device(s) :
> > Active : 1
> > Breed : Non-DNA
> > Sampling Rate : 1
> > Capture Direction : RX+TX
> > Socket Mode : RX+TX
> > Appl. Name : bro-dnacluster:0 at 9
> > IP Defragment : No
> > BPF Filtering : Enabled
> > # Sw Filt. Rules : 0
> > # Hw Filt. Rules : 0
> > Poll Pkt Watermark : 1
> > Num Poll Calls : 2562490
> >  
> >  
> >  
> > what gives???
> >  
> > if i manually kill the bro process on the worker and rerun capstats, i get:
> >  
> > Interface kpps mbps (10s average)
> > ----------------------------------------
> > sec-bro01/dnacluster:0 0.0 0.0
> > Total 0.0 0.0
> >  
> >  
> > also, if i were to change the lb_procs to less than that of the pfdnacluster number of workers (-n), everything (seems to) work fine (bear in mind i'm not capturing any traffic at the moment). but would i loose any data? i'm using pf_ring 6.0.1.
> >  
> >  
> > thanks,
> >  
> > Yee.
> >  
> >  
> >  
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org (mailto:bro at bro-ids.org)
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>  
>  
>  
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org (mailto:bro at bro-ids.org)
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro