[Bro] Bro Digest, Vol 123, Issue 24

Fri Jul 15 10:03:56 PDT 2016

Well regarding Splunk add-on for BRO-IDS, I asked following question on
Splunkbase and still waiting for an answer so thought might be worth
sharing it here as well:
Starting with the environment, I have an indexer cluster of 3 indexers, two
independent search heads, and one Universal forwarder.
My question is where the BRO IDS app goes and how it works?
What I have done is - I have installed the app on both of my search heads
(as per general convention while dealing with apps), and my Universal
Forwarder is monitoring the Bro log directory (yes I have installed UF on
my Bro sensor machine).
I am getting the monitored Bro logs in my indexers and am able to search
them via search heads, but the app is just sitting there doing nothing it
seems.

The documentation I have read so far says that you need to install app on
the heavy forwarder that is monitoring your log dir and have to set the
inputs path in the app instead of heavy forwarder's input. (So I think it's
stupid for the people who just want to have a forwarder installed on their
bro sensor for just forwarding bro logs and for that we need to install
heavy forwarder with the app, and that too app will be doing all the
forwarding and parsing and heavy forwarder will be just sitting there
providing Python support to the app to do its stuff).

So my question is: is my above configuration even workable with Bro IDS
add-on or do I have to just chuck the idea of using the add-on because I
don't want to run a heavy forwarder on my Bro machines?

Thanks,
Fatema.

On Fri, Jul 15, 2016 at 5:55 AM, <bro-request at bro.org> wrote:

> Send Bro mailing list submissions to
>         bro at bro.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> or, via email, send a message with subject or body 'help' to
>         bro-request at bro.org
>
> You can reach the person managing the list at
>         bro-owner at bro.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Bro digest..."
>
>
> Today's Topics:
>
>    1. Re: problem ingesting bro json logs into splunk (Steve Brant)
>    2. Re: PF_RING ZC Config (Alfredo Cardigliano)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 14 Jul 2016 17:07:41 -0600
> From: Steve Brant <steve at brant.nu>
> Subject: Re: [Bro] problem ingesting bro json logs into splunk
> To: Brandon Lattin <lattin at umn.edu>
> Cc: philosnef <philosnef at yahoo.com>, "bro at bro.org" <bro at bro.org>
> Message-ID:
>         <CAA=
> spH96fjZWyMduzDmmT+HCgUufdY2aNaOkf00zD07ZmnmONg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> I've attached a modified version of the Splunk TA for bro, that
> accommodates bro logs in json format. Let me know if you have any problem
> with it.
>
> Thanks,
> Steve
>
> ~/SB
>
> On Thu, Jul 14, 2016 at 8:14 AM, Brandon Lattin <lattin at umn.edu> wrote:
>
> > Do you have the Splunk installed? (
> https://splunkbase.splunk.com/app/1617/
> > )
> >
> > The TA will dynamically create sourcetypes based on the log name.
> >
> > # Dynamic source typing based on log filename
> > # Match: conn.log, bro.conn.log,
> > # md5.bro.conn.log, whatever.conn.log
> > [BroAutoType]
> > DEST_KEY = MetaData:Sourcetype
> > SOURCE_KEY = MetaData:Source
> > REGEX = ([a-zA-Z0-9-]+)(?:\.[0-9-]*)?(?:\.[0-9\:-]*)?\.log
> > FORMAT = sourcetype::bro_$1
> > WRITE_META = true
> >
> >
> > On Thu, Jul 14, 2016 at 8:33 AM, philosnef <philosnef at yahoo.com> wrote:
> >
> >> We are getting a spurious sourcetype when ingesting bro json logs into
> >> splunk.
> >>
> >> Specifically, we are getting a sourcetype of bro_00. There is no log
> file
> >> named this, and the splunkforwarder is just pushing the raw logs for
> >> indexing into splunk. There is no massaging of the log data. Anyone know
> >> why this sourcetype is popping up?
> >>
> >> _______________________________________________
> >> Bro mailing list
> >> bro at bro-ids.org
> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >>
> >
> >
> >
> > --
> > Brandon Lattin
> > Security Analyst
> > University of Minnesota - University Information Security
> > Office: 612-626-6672
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160714/ea658fc8/attachment-0001.html
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: TA-bro_json.tar.gz
> Type: application/x-gzip
> Size: 4241 bytes
> Desc: not available
> Url :
> http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160714/ea658fc8/attachment-0001.gz
>
> ------------------------------
>
> Message: 2
> Date: Fri, 15 Jul 2016 11:55:50 +0200
> From: Alfredo Cardigliano <cardigliano at ntop.org>
> Subject: Re: [Bro] PF_RING ZC Config
> To: "Slagell, Adam J" <slagell at illinois.edu>
> Cc: "bro at bro.org" <bro at bro.org>
> Message-ID: <9B8FC355-9897-4FCC-9B77-6799DA201088 at ntop.org>
> Content-Type: text/plain; charset="windows-1252"
>
> Hi guys
> I drafted a REAME based on PF_RING ZC at
> https://github.com/ntop/PF_RING/blob/dev/doc/README.bro
> feel free to edit it (sending pull requests).
>
> Thank you
> Alfredo
>
> > On 08 Jul 2016, at 21:23, Slagell, Adam J <slagell at illinois.edu> wrote:
> >
> > Thanks. I don?t want to forget to come back to this.
> >
> >> On Jul 8, 2016, at 12:57 PM, Gary Faulkner <gfaulkner.nsm at gmail.com>
> wrote:
> >>
> >> https://bro-tracker.atlassian.net/browse/BIT-1642
> >>
> >>
> >> On 7/8/16 12:35 PM, Slagell, Adam J wrote:
> >>> Could you create a ticket for this in the tracker?
> >>>
> >>> On Jul 8, 2016, at 12:26 PM, Gary Faulkner <gfaulkner.nsm at gmail.com
> <mailto:gfaulkner.nsm at gmail.com>> wrote:
> >>>
> >>>
> >>> Related to Dave's query, but not really an answer, sorry Dave...
> >>>
> >>> It might be worth revisiting this doc and updating for ZC:
> >>>
> >>> https://www.bro.org/documentation/load-balancing.html<
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.bro.org_documentation_load-2Dbalancing.html&d=CwMD-g&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=jKm5KEaN1h0UfG6EGbAMSpW5NbXR4pzULAvbXFWNnvg&s=x-uS-7-lC174enHZ7TiRa3RLswl6nRsyBvGQ2_T2W-E&e=
> >
> >>>
> >>> A few things have changed on the PF_RING DNA side in broctl in regards
> to naming support "dnacl" instead of "dnacluster" due to problems with name
> length for dnaclusters with greater than 10 queues, and with the most
> recent releases of PF_RING (6.4+), DNA appears to have been removed finally
> in favor of the newer ZC according to the change notes. From what I recall
> reading I don't believe it is terribly different outside of substituting ZC
> drivers (and tweaking huge-pages in the driver load script) in favor of
> DNA, and using zbalance_ipc instead of pfdnacluster_master. I want to say
> the naming in node.cfg becomes zc:<clusterid> instead of dnacl:<clusterid>.
> >>>
> >>> Also, speaking of ZC, NTOP has a blog post that might be worth taking
> a look at concerning alternate ways of implementing ZC / zbalance_ipc with
> bro to work around a problem that can occur when bro workers crash and get
> automatically restarted.
> >>>
> >>>
> http://www.ntop.org/pf_ring/best-practices-for-using-bro_ids-with-pf_ring-zc-reliably/
> <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ntop.org_pf-5Fring_best-2Dpractices-2Dfor-2Dusing-2Dbro-5Fids-2Dwith-2Dpf-5Fring-2Dzc-2Dreliably_&d=CwMD-g&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=jKm5KEaN1h0UfG6EGbAMSpW5NbXR4pzULAvbXFWNnvg&s=oOBYvlJMigTXYIzqgtcGz3iNzZpTQrMlSPBWRYkOFA4&e=
> >
> >>>
> >>> I haven't quite made the transition to ZC from DNA yet, otherwise I'd
> take a stab at submitting updated docs and trying to assist more here. I
> have plans to make the switch later this summer though.
> >>>
> >>> ~Gary
> >>>
> >>> On 7/7/16 5:25 PM, Dave Crawford wrote:
> >>>
> >>> Just wanted to update the list that I quit spending cycles on this and
> for the time being reverted back to running our clusters with the
> non-commercial version of pf_ring.
> >>>
> >>> I can only comment on my experience, but I discovered there is an
> extreme lack of quality documentation and the "commercial support" that
> came with the 10 licenses was nearly non-existent.
> >>>
> >>> Lessons have been learned and when the need to expand comes we'll be
> looking at other commercial solutions to replace our X520's with.
> >>>
> >>> -Dave
> >>>
> >>>
> >>>
> >>> On Jun 24, 2016, at 8:28 AM, Dave Crawford <bro at pingtrip.com><mailto:
> bro at pingtrip.com> wrote:
> >>>
> >>> Would anyone happen to have documentation for configuring ZC and Bro?
> I have NTop's PF_RING and ixgbe driver packages installed, the proper
> license in /etc/pf_ring, and have compiled Bro with the NTop libraries but
> I'm seeing the kernel error below along with a ton of ?split routing?
> messages in weird.conf, so I suspect the flows aren?t being load balanced
> correctly.
> >>>
> >>> Jun 22 15:10:03 win-csignsm-01 kernel: [11060.244524] [PF_RING] Unable
> to activate two or more ZC sockets on the same interface eth6/link direction
> >>>
> >>> The monitored NIC is an Intel X520-LR1.
> >>>
> >>> Contents of /etc/pf_ring/zc/ixgbe/ixgbe.conf:
> >>> RSS=10 allow_unsupported_sfp=0
> >>>
> >>> Contents of /etc/pf_ring/hugepages.conf
> >>> node=1 hugepages=1024
> >>>
> >>>
> >>> And Bro is configured as:
> >>> [MID_INT]
> >>> type=worker
> >>> host=10.20.30.123
> >>> interface=zc:eth6
> >>> lb_method=pf_ring
> >>> lb_procs=10
> >>> pin_cpus=10,11,12,13,14,15,16,17,18,19
> >>>
> >>> Thanks!
> >>> -Dave
> >>> _______________________________________________
> >>> Bro mailing list
> >>> bro at bro-ids.org<mailto:bro at bro-ids.org>
> >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro<
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.ICSI.Berkeley.EDU_mailman_listinfo_bro&d=CwMD-g&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=jKm5KEaN1h0UfG6EGbAMSpW5NbXR4pzULAvbXFWNnvg&s=V2Ec5cOcnYqN7P3EpauWtYZUCrRNMUzy_pvRrgdH_C8&e=
> >
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Bro mailing list
> >>> bro at bro-ids.org<mailto:bro at bro-ids.org>
> >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro<
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.ICSI.Berkeley.EDU_mailman_listinfo_bro&d=CwMD-g&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=jKm5KEaN1h0UfG6EGbAMSpW5NbXR4pzULAvbXFWNnvg&s=V2Ec5cOcnYqN7P3EpauWtYZUCrRNMUzy_pvRrgdH_C8&e=
> >
> >>>
> >>> _______________________________________________
> >>> Bro mailing list
> >>> bro at bro-ids.org<mailto:bro at bro-ids.org>
> >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >>>
> >>> ------
> >>>
> >>> Adam J. Slagell
> >>> Chief Information Security Officer
> >>> Director, Cybersecurity Division
> >>> National Center for Supercomputing Applications
> >>> University of Illinois at Urbana-Champaign
> >>> www.slagell.info<http://www.slagell.info>
> >>>
> >>> "Under the Illinois Freedom of Information Act (FOIA), any written
> communication to or from University employees regarding University business
> is a public record and may be subject to public disclosure."
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >
> > ------
> >
> > Adam J. Slagell
> > Chief Information Security Officer
> > Director, Cybersecurity Division
> > National Center for Supercomputing Applications
> > University of Illinois at Urbana-Champaign
> > www.slagell.info <http://www.slagell.info/>
> >
> > "Under the Illinois Freedom of Information Act (FOIA), any written
> communication to or from University employees regarding University business
> is a public record and may be subject to public disclosure."
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org <mailto:bro at bro-ids.org>
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro <
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160715/955bdb0d/attachment.html
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 842 bytes
> Desc: Message signed with OpenPGP using GPGMail
> Url :
> http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160715/955bdb0d/attachment.bin
>
> ------------------------------
>
> _______________________________________________
> Bro mailing list
> Bro at bro.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> End of Bro Digest, Vol 123, Issue 24
> ************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160715/499d93cd/attachment-0001.html 