[Zeek] "bro-cluster-in-a-box-setup" to "zeek-cluster-in-a-box-setup"?

[Andrew Klaus]

Hi Michal,

We're working with Zeek on the same project as Paul and have opted to use
Zeek within a Docker container since it works better for our workflow. It's
my first time using AF_PACKET, so it would be nice to have a second set of
eyes from you on it if you don't mind :)

I've bridged all interfaces that need to be analyzed:
https://github.com/cybera/jsp-zeek/blob/master/host/60-zeek-bridge.yaml

Then I've disabled hardware features on the NIC, which is done on each
boot: https://github.com/cybera/jsp-zeek/blob/master/host/ethtool.sh

Using "interface=af_packet::br0", and pinning CPUs for the workers,
manager, and proxy:
https://github.com/cybera/jsp-zeek/blob/master/docker/files/etc/node.cfg

We don't have a ton of traffic being analyzed yet, but want to make sure we
have a decent setup for when we start ingesting more data. I've pieced this
together from various Zeek articles I've read, so hopefully it's not too
much of a Frankenstein's Monster ;)

Any help would be appreciated!

Cheers,
Andrew


> Date: Wed, 5 Feb 2020 14:12:53 -0800
> From: Micha? Purzy?ski <michalpurzynski1 at gmail.com>
> Subject: Re: [Zeek] "bro-cluster-in-a-box-setup" to
>         "zeek-cluster-in-a-box-setup"?
> To: Scott Wang <scwang+bro at sfu.ca>
> Cc: Paul Sibley <Paul.Sibley at canarie.ca>, "zeek at zeek.org"
>         <zeek at zeek.org>
> Message-ID:
>         <
> CAJ6bFK3V3FBZr8W84zj2C9y+4HxuSQ3kweEyO+fYxMfcOZ0t5Q at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> There's a law that if you say pf_ring and af_packet 3 times, Michal shows
> up.
>
> I don't see many (any?) reasons for using pf_ring, TBH, if you have a
> modern kernel or a decent network card (Mellanox, Intel, etc). And I still
> owe the community the article to show how to use the af_packet correctly :/
>
> The case where one has inputs from multiple taps, to multiple network ports
> will be handled the same way by af_packet, if interfaces are bonded or
> bridged and by pf_ring. None of them buffers data and processes them at L4
> and deals with out of order, etc.
>
> On Wed, Feb 5, 2020 at 2:04 PM Scott Wang <scwang+bro at sfu.ca> wrote:
>
> > At the Canarie workshop, Steve Smoot from Corelight suggested using
> > pf_ring still. Any thoughts/comments on switching to af_packet?
> Advantages
> > vs Disadvantages?
> >
> > Regards,
> > Scott
> >
> > On Feb 05, 2020, at 12:48, Justin Azoff <justin at corelight.com> wrote:
> >
> > Hi!
> >
> > It shouldn't be that hard to update to 3.x..
> >
> > - bro-pkg should be swapped out with the renamed zkg
> > - the python2 references can likely be changed to 3
> > - caf no longer needs to be installed separately
> > - geoip and databases needs to be swapped out with maxminddb versions,
> > might need a license
> > - probably worth it to switch to af_packet from pf_ring.. pf_ring was
> only
> > used initially to easily support capturing directly from both halves of a
> > tap, which might not be a requirement anymore.
> >
> > My schedule is a bit crazy for the next week, but once I have some time
> to
> > work on it I should be able to get things updated pretty quickly..
> There's
> > really not much to it.
> >
> >
> >
> > On Wed, Feb 5, 2020 at 12:38 PM Paul Sibley <Paul.Sibley at canarie.ca>
> > wrote:
> >
> >> Hello Zeek Community,
> >>
> >>
> >>
> >> I am working on a project where Zeek has been deployed in two phases.
> >> During the first phase, some participants used ?
> >> https://github.com/ncsa/bro-cluster-in-a-box-setup? script to assist
> in,
> >> and automate a lot of the installation process.
> >>
> >> Since then we have entered the phase in our project where more
> >> participants have been added, CentOS 8 is preferred, and we are using
> Zeek
> >> 3.0.1.
> >>
> >> I wonder if any consideration, or work has been done, in updating the
> >> bro-cluster-in-a-box script to work with the updated OS and Zeek
> version.
> >> Any information would be appreciated.
> >>
> >>
> >>
> >> Thanks in advance,
> >>
> >> Paul Sibley
> >> _______________________________________________
> >> Zeek mailing list
> >> zeek at zeek.org
> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> >> <http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek>
> >
> >
> >
> > --
> > Justin
> > _______________________________________________
> > Zeek mailing list
> > zeek at zeek.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> >
> >
> > _______________________________________________
> > Zeek mailing list
> > zeek at zeek.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200205/18e567a5/attachment.html
>
> ------------------------------
>
> _______________________________________________
> Zeek mailing list
> Zeek at zeek.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
>
> End of Zeek Digest, Vol 166, Issue 8
> ************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20200205/aa84fc09/attachment-0001.html