[Bro] 5 node cluster

[Darrain Waters]

I did compile using below. Perhaps that is not enough, as I was following
the myricom instructions. See attached

On Sat, Oct 8, 2016 at 3:29 PM, Michał Purzyński <michalpurzynski1 at gmail.com
> wrote:

> http://lmgtfy.com/?q=Bro+Myricom
>
> Finds me
>
> https://www.bro.org/sphinx-git/components/bro-plugins/myricom/README.html
>
> You're welcome ;)
>
> Tested on 2.5 master and beta. Haven't tried on 2.4 although by the time
> you will build your cluster 2.5 will have been released.
>
> On 8 Oct 2016, at 19:52, Darrain Waters <dwaters at bioteam.net> wrote:
>
> Great, would you point me in the right direction. Thank you.
>
> On Sat, Oct 8, 2016 at 12:50 PM, Michał Purzyński <
> michalpurzynski1 at gmail.com> wrote:
>
>> You don't seem to use native Myricom support, there's a plugin for that.
>>
>> On 8 Oct 2016, at 19:44, Darrain Waters <dwaters at bioteam.net> wrote:
>>
>> Turns out it was a simple config issue (like most times & RTFM), and
>> traffic is flowing to snf0. My workers were not using the snf0 interface as
>> you must if you compiled using the myricom sniffer. Also changed the cpu
>> pinning so thanks for that info. I also turned off the time source in the
>> snf driver. Now I need to add Arista time source.  Thanks for your time.
>>
>> [worker-3]
>> type=worker
>> host=10.0.40.16
>> interface=snf0     use to be eth2
>> lb_method=myricom
>> lb_procs=10
>> pin_cpus=2,3,4,5,6,7,8,9,10,11     the cpu pinning was not right either
>> #env_vars=LD_LIBRARY_PATH=/opt/snf/lib:$PATH, SNF_FLAGS=0x1,
>> SNF_DATARING_SIZE=0x100000000, SNF_NUM_RINGS=10
>>
>> [worker-3]
>>
>> type=worker
>>
>> host=10.0.40.16
>>
>> interface=eth2
>>
>> lb_method=myricom
>>
>> lb_procs=10
>>
>> pin_cpus=7,8,9,10,11,18,19,20,21,22
>>
>> env_vars=LD_LIBRARY_PATH=/opt/snf/lib:$PATH, SNF_FLAGS=0x1,
>> SNF_DATARING_SIZE=0x100000000, SNF_NUM_RINGS=10
>> To:
>>
>> [worker-3]
>> type=worker
>> host=10.0.40.16
>> interface=snf0
>> lb_method=myricom
>> lb_procs=10
>> pin_cpus=2,3,4,5,6,7,8,9,10,11
>>
>>
>>
>>
>> On Fri, Oct 7, 2016 at 10:34 PM, Azoff, Justin S <jazoff at illinois.edu>
>> wrote:
>>
>>>
>>> > On Oct 7, 2016, at 6:27 PM, Darrain Waters <dwaters at bioteam.net>
>>> wrote:
>>> >
>>> > Sorry, yeah I am getting comm logs and stderr on the manager. I do
>>> have two NICS enabled on each system, one for management with IP and the
>>> other is the myricom with no IP and in sniffer mode.
>>> >
>>> > Each of the workers do have the spool wirker directories but they are
>>> empty.
>>> >
>>> > I use to be able to run this on the manager
>>> >
>>> > [bromgr at bromgr etc]$ sudo tcpdump -i eth2
>>> >
>>> >
>>> > tcpdump: snf_ring_open_id(ring=-1) failed: Device or resource busy
>>> >
>>> >
>>> >
>>> > [BroControl] > netstats
>>> >
>>> >  worker-1-1: 1475878452.092051 recvd=1 dropped=17260812 link=17260813
>>> >
>>> > worker-1-10: 1475878452.292009 recvd=1 dropped=17260812 link=17260813
>>> >
>>> >  worker-1-2: 1475878452.493003 recvd=1 dropped=17260812 link=17260813
>>>
>>> Ah, ok.. so this isn't the firewall issue...  That's when "everything is
>>> working but there are no logs" but in your case nothing is working :-)
>>>
>>> I'd stop bro and then make sure everything is stopped.  You can use
>>> 'broctl ps.bro' to ensure that there are no stray procs lying around.  Then
>>> at that point with nothing else running you should be able to run things
>>> like 'tcpdump' or 'broctl capstats' and verify that you can capture packets.
>>>
>>> You should also be able to run tools like
>>>
>>> /opt/snf/bin/myri_nic_info
>>> /opt/snf/bin/myri_counters
>>> /opt/snf/bin/myri_bandwidth
>>> /opt/snf/sbin/myri_license
>>>
>>> to ensure that the card+drivers are working properly as well as check
>>> dmesg output and check to see if it is complaining about anything
>>>
>>> I don't recall every seeing that particular netstats output, but I bet
>>> you'll be able to reproduce the problem with regular tcpdump.  Generally
>>> speaking if tcpdump -w foo.pcap writes out packets that look ok, and you
>>> can use bro -r against foo.pcap, bro it should work in realtime.
>>>
>>> The snf issues on the manager may be due to trying to use snf libs
>>> against a regular NIC, I've had to use things like
>>>
>>> LD_PRELOAD=/usr/lib64/libpcap.so.1 tcpdump ...
>>>
>>> to force it to use standard libpcap.
>>>
>>> --
>>> - Justin Azoff
>>>
>>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161009/68bd9686/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: BRO App Note.pdf
Type: application/pdf
Size: 280306 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161009/68bd9686/attachment-0001.pdf