[Bro] 5 node cluster

Darrain Waters dwaters at bioteam.net
Sat Oct 8 10:52:26 PDT 2016


Great, would you point me in the right direction. Thank you.

On Sat, Oct 8, 2016 at 12:50 PM, Michał Purzyński <
michalpurzynski1 at gmail.com> wrote:

> You don't seem to use native Myricom support, there's a plugin for that.
>
> On 8 Oct 2016, at 19:44, Darrain Waters <dwaters at bioteam.net> wrote:
>
> Turns out it was a simple config issue (like most times & RTFM), and
> traffic is flowing to snf0. My workers were not using the snf0 interface as
> you must if you compiled using the myricom sniffer. Also changed the cpu
> pinning so thanks for that info. I also turned off the time source in the
> snf driver. Now I need to add Arista time source.  Thanks for your time.
>
> [worker-3]
> type=worker
> host=10.0.40.16
> interface=snf0     use to be eth2
> lb_method=myricom
> lb_procs=10
> pin_cpus=2,3,4,5,6,7,8,9,10,11     the cpu pinning was not right either
> #env_vars=LD_LIBRARY_PATH=/opt/snf/lib:$PATH, SNF_FLAGS=0x1,
> SNF_DATARING_SIZE=0x100000000, SNF_NUM_RINGS=10
>
> [worker-3]
>
> type=worker
>
> host=10.0.40.16
>
> interface=eth2
>
> lb_method=myricom
>
> lb_procs=10
>
> pin_cpus=7,8,9,10,11,18,19,20,21,22
>
> env_vars=LD_LIBRARY_PATH=/opt/snf/lib:$PATH, SNF_FLAGS=0x1,
> SNF_DATARING_SIZE=0x100000000, SNF_NUM_RINGS=10
> To:
>
> [worker-3]
> type=worker
> host=10.0.40.16
> interface=snf0
> lb_method=myricom
> lb_procs=10
> pin_cpus=2,3,4,5,6,7,8,9,10,11
>
>
>
>
> On Fri, Oct 7, 2016 at 10:34 PM, Azoff, Justin S <jazoff at illinois.edu>
> wrote:
>
>>
>> > On Oct 7, 2016, at 6:27 PM, Darrain Waters <dwaters at bioteam.net> wrote:
>> >
>> > Sorry, yeah I am getting comm logs and stderr on the manager. I do have
>> two NICS enabled on each system, one for management with IP and the other
>> is the myricom with no IP and in sniffer mode.
>> >
>> > Each of the workers do have the spool wirker directories but they are
>> empty.
>> >
>> > I use to be able to run this on the manager
>> >
>> > [bromgr at bromgr etc]$ sudo tcpdump -i eth2
>> >
>> >
>> > tcpdump: snf_ring_open_id(ring=-1) failed: Device or resource busy
>> >
>> >
>> >
>> > [BroControl] > netstats
>> >
>> >  worker-1-1: 1475878452.092051 recvd=1 dropped=17260812 link=17260813
>> >
>> > worker-1-10: 1475878452.292009 recvd=1 dropped=17260812 link=17260813
>> >
>> >  worker-1-2: 1475878452.493003 recvd=1 dropped=17260812 link=17260813
>>
>> Ah, ok.. so this isn't the firewall issue...  That's when "everything is
>> working but there are no logs" but in your case nothing is working :-)
>>
>> I'd stop bro and then make sure everything is stopped.  You can use
>> 'broctl ps.bro' to ensure that there are no stray procs lying around.  Then
>> at that point with nothing else running you should be able to run things
>> like 'tcpdump' or 'broctl capstats' and verify that you can capture packets.
>>
>> You should also be able to run tools like
>>
>> /opt/snf/bin/myri_nic_info
>> /opt/snf/bin/myri_counters
>> /opt/snf/bin/myri_bandwidth
>> /opt/snf/sbin/myri_license
>>
>> to ensure that the card+drivers are working properly as well as check
>> dmesg output and check to see if it is complaining about anything
>>
>> I don't recall every seeing that particular netstats output, but I bet
>> you'll be able to reproduce the problem with regular tcpdump.  Generally
>> speaking if tcpdump -w foo.pcap writes out packets that look ok, and you
>> can use bro -r against foo.pcap, bro it should work in realtime.
>>
>> The snf issues on the manager may be due to trying to use snf libs
>> against a regular NIC, I've had to use things like
>>
>> LD_PRELOAD=/usr/lib64/libpcap.so.1 tcpdump ...
>>
>> to force it to use standard libpcap.
>>
>> --
>> - Justin Azoff
>>
>>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161008/50464b43/attachment-0001.html 


More information about the Bro mailing list