[Bro] Bro Digest, Vol 123, Issue 24

Sat Jul 16 22:53:34 PDT 2016

Hi Fatema-

I've answered your posting at
https://answers.splunk.com/answers/427161/where-to-install-and-configure-the-splunk-add-on-f.html#answer-430982

Please respond there if you need some additional detail.

Thanks,
Steve

~/SB

On Fri, Jul 15, 2016 at 11:03 AM, fatema bannatwala <
fatema.bannatwala at gmail.com> wrote:

> Well regarding Splunk add-on for BRO-IDS, I asked following question on
> Splunkbase and still waiting for an answer so thought might be worth
> sharing it here as well:
> Starting with the environment, I have an indexer cluster of 3 indexers,
> two independent search heads, and one Universal forwarder.
> My question is where the BRO IDS app goes and how it works?
> What I have done is - I have installed the app on both of my search heads
> (as per general convention while dealing with apps), and my Universal
> Forwarder is monitoring the Bro log directory (yes I have installed UF on
> my Bro sensor machine).
> I am getting the monitored Bro logs in my indexers and am able to search
> them via search heads, but the app is just sitting there doing nothing it
> seems.
>
> The documentation I have read so far says that you need to install app on
> the heavy forwarder that is monitoring your log dir and have to set the
> inputs path in the app instead of heavy forwarder's input. (So I think it's
> stupid for the people who just want to have a forwarder installed on their
> bro sensor for just forwarding bro logs and for that we need to install
> heavy forwarder with the app, and that too app will be doing all the
> forwarding and parsing and heavy forwarder will be just sitting there
> providing Python support to the app to do its stuff).
>
> So my question is: is my above configuration even workable with Bro IDS
> add-on or do I have to just chuck the idea of using the add-on because I
> don't want to run a heavy forwarder on my Bro machines?
>
> Thanks,
> Fatema.
>
> On Fri, Jul 15, 2016 at 5:55 AM, <bro-request at bro.org> wrote:
>
>> Send Bro mailing list submissions to
>>         bro at bro.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>         http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> or, via email, send a message with subject or body 'help' to
>>         bro-request at bro.org
>>
>> You can reach the person managing the list at
>>         bro-owner at bro.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of Bro digest..."
>>
>>
>> Today's Topics:
>>
>>    1. Re: problem ingesting bro json logs into splunk (Steve Brant)
>>    2. Re: PF_RING ZC Config (Alfredo Cardigliano)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Thu, 14 Jul 2016 17:07:41 -0600
>> From: Steve Brant <steve at brant.nu>
>> Subject: Re: [Bro] problem ingesting bro json logs into splunk
>> To: Brandon Lattin <lattin at umn.edu>
>> Cc: philosnef <philosnef at yahoo.com>, "bro at bro.org" <bro at bro.org>
>> Message-ID:
>>         <CAA=
>> spH96fjZWyMduzDmmT+HCgUufdY2aNaOkf00zD07ZmnmONg at mail.gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> I've attached a modified version of the Splunk TA for bro, that
>> accommodates bro logs in json format. Let me know if you have any problem
>> with it.
>>
>> Thanks,
>> Steve
>>
>> ~/SB
>>
>> On Thu, Jul 14, 2016 at 8:14 AM, Brandon Lattin <lattin at umn.edu> wrote:
>>
>> > Do you have the Splunk installed? (
>> https://splunkbase.splunk.com/app/1617/
>> > )
>> >
>> > The TA will dynamically create sourcetypes based on the log name.
>> >
>> > # Dynamic source typing based on log filename
>> > # Match: conn.log, bro.conn.log,
>> > # md5.bro.conn.log, whatever.conn.log
>> > [BroAutoType]
>> > DEST_KEY = MetaData:Sourcetype
>> > SOURCE_KEY = MetaData:Source
>> > REGEX = ([a-zA-Z0-9-]+)(?:\.[0-9-]*)?(?:\.[0-9\:-]*)?\.log
>> > FORMAT = sourcetype::bro_$1
>> > WRITE_META = true
>> >
>> >
>> > On Thu, Jul 14, 2016 at 8:33 AM, philosnef <philosnef at yahoo.com> wrote:
>> >
>> >> We are getting a spurious sourcetype when ingesting bro json logs into
>> >> splunk.
>> >>
>> >> Specifically, we are getting a sourcetype of bro_00. There is no log
>> file
>> >> named this, and the splunkforwarder is just pushing the raw logs for
>> >> indexing into splunk. There is no massaging of the log data. Anyone
>> know
>> >> why this sourcetype is popping up?
>> >>
>> >> _______________________________________________
>> >> Bro mailing list
>> >> bro at bro-ids.org
>> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> >>
>> >
>> >
>> >
>> > --
>> > Brandon Lattin
>> > Security Analyst
>> > University of Minnesota - University Information Security
>> > Office: 612-626-6672
>> >
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> >
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160714/ea658fc8/attachment-0001.html
>> -------------- next part --------------
>> A non-text attachment was scrubbed...
>> Name: TA-bro_json.tar.gz
>> Type: application/x-gzip
>> Size: 4241 bytes
>> Desc: not available
>> Url :
>> http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160714/ea658fc8/attachment-0001.gz
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Fri, 15 Jul 2016 11:55:50 +0200
>> From: Alfredo Cardigliano <cardigliano at ntop.org>
>> Subject: Re: [Bro] PF_RING ZC Config
>> To: "Slagell, Adam J" <slagell at illinois.edu>
>> Cc: "bro at bro.org" <bro at bro.org>
>> Message-ID: <9B8FC355-9897-4FCC-9B77-6799DA201088 at ntop.org>
>> Content-Type: text/plain; charset="windows-1252"
>>
>> Hi guys
>> I drafted a REAME based on PF_RING ZC at
>> https://github.com/ntop/PF_RING/blob/dev/doc/README.bro
>> feel free to edit it (sending pull requests).
>>
>> Thank you
>> Alfredo
>>
>> > On 08 Jul 2016, at 21:23, Slagell, Adam J <slagell at illinois.edu> wrote:
>> >
>> > Thanks. I don?t want to forget to come back to this.
>> >
>> >> On Jul 8, 2016, at 12:57 PM, Gary Faulkner <gfaulkner.nsm at gmail.com>
>> wrote:
>> >>
>> >> https://bro-tracker.atlassian.net/browse/BIT-1642
>> >>
>> >>
>> >> On 7/8/16 12:35 PM, Slagell, Adam J wrote:
>> >>> Could you create a ticket for this in the tracker?
>> >>>
>> >>> On Jul 8, 2016, at 12:26 PM, Gary Faulkner <gfaulkner.nsm at gmail.com
>> <mailto:gfaulkner.nsm at gmail.com>> wrote:
>> >>>
>> >>>
>> >>> Related to Dave's query, but not really an answer, sorry Dave...
>> >>>
>> >>> It might be worth revisiting this doc and updating for ZC:
>> >>>
>> >>> https://www.bro.org/documentation/load-balancing.html<
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.bro.org_documentation_load-2Dbalancing.html&d=CwMD-g&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=jKm5KEaN1h0UfG6EGbAMSpW5NbXR4pzULAvbXFWNnvg&s=x-uS-7-lC174enHZ7TiRa3RLswl6nRsyBvGQ2_T2W-E&e=
>> >
>> >>>
>> >>> A few things have changed on the PF_RING DNA side in broctl in
>> regards to naming support "dnacl" instead of "dnacluster" due to problems
>> with name length for dnaclusters with greater than 10 queues, and with the
>> most recent releases of PF_RING (6.4+), DNA appears to have been removed
>> finally in favor of the newer ZC according to the change notes. From what I
>> recall reading I don't believe it is terribly different outside of
>> substituting ZC drivers (and tweaking huge-pages in the driver load script)
>> in favor of DNA, and using zbalance_ipc instead of pfdnacluster_master. I
>> want to say the naming in node.cfg becomes zc:<clusterid> instead of
>> dnacl:<clusterid>.
>> >>>
>> >>> Also, speaking of ZC, NTOP has a blog post that might be worth taking
>> a look at concerning alternate ways of implementing ZC / zbalance_ipc with
>> bro to work around a problem that can occur when bro workers crash and get
>> automatically restarted.
>> >>>
>> >>>
>> http://www.ntop.org/pf_ring/best-practices-for-using-bro_ids-with-pf_ring-zc-reliably/
>> <
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ntop.org_pf-5Fring_best-2Dpractices-2Dfor-2Dusing-2Dbro-5Fids-2Dwith-2Dpf-5Fring-2Dzc-2Dreliably_&d=CwMD-g&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=jKm5KEaN1h0UfG6EGbAMSpW5NbXR4pzULAvbXFWNnvg&s=oOBYvlJMigTXYIzqgtcGz3iNzZpTQrMlSPBWRYkOFA4&e=
>> >
>> >>>
>> >>> I haven't quite made the transition to ZC from DNA yet, otherwise I'd
>> take a stab at submitting updated docs and trying to assist more here. I
>> have plans to make the switch later this summer though.
>> >>>
>> >>> ~Gary
>> >>>
>> >>> On 7/7/16 5:25 PM, Dave Crawford wrote:
>> >>>
>> >>> Just wanted to update the list that I quit spending cycles on this
>> and for the time being reverted back to running our clusters with the
>> non-commercial version of pf_ring.
>> >>>
>> >>> I can only comment on my experience, but I discovered there is an
>> extreme lack of quality documentation and the "commercial support" that
>> came with the 10 licenses was nearly non-existent.
>> >>>
>> >>> Lessons have been learned and when the need to expand comes we'll be
>> looking at other commercial solutions to replace our X520's with.
>> >>>
>> >>> -Dave
>> >>>
>> >>>
>> >>>
>> >>> On Jun 24, 2016, at 8:28 AM, Dave Crawford <bro at pingtrip.com><mailto:
>> bro at pingtrip.com> wrote:
>> >>>
>> >>> Would anyone happen to have documentation for configuring ZC and Bro?
>> I have NTop's PF_RING and ixgbe driver packages installed, the proper
>> license in /etc/pf_ring, and have compiled Bro with the NTop libraries but
>> I'm seeing the kernel error below along with a ton of ?split routing?
>> messages in weird.conf, so I suspect the flows aren?t being load balanced
>> correctly.
>> >>>
>> >>> Jun 22 15:10:03 win-csignsm-01 kernel: [11060.244524] [PF_RING]
>> Unable to activate two or more ZC sockets on the same interface eth6/link
>> direction
>> >>>
>> >>> The monitored NIC is an Intel X520-LR1.
>> >>>
>> >>> Contents of /etc/pf_ring/zc/ixgbe/ixgbe.conf:
>> >>> RSS=10 allow_unsupported_sfp=0
>> >>>
>> >>> Contents of /etc/pf_ring/hugepages.conf
>> >>> node=1 hugepages=1024
>> >>>
>> >>>
>> >>> And Bro is configured as:
>> >>> [MID_INT]
>> >>> type=worker
>> >>> host=10.20.30.123
>> >>> interface=zc:eth6
>> >>> lb_method=pf_ring
>> >>> lb_procs=10
>> >>> pin_cpus=10,11,12,13,14,15,16,17,18,19
>> >>>
>> >>> Thanks!
>> >>> -Dave
>> >>> _______________________________________________
>> >>> Bro mailing list
>> >>> bro at bro-ids.org<mailto:bro at bro-ids.org>
>> >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.ICSI.Berkeley.EDU_mailman_listinfo_bro&d=CwMD-g&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=jKm5KEaN1h0UfG6EGbAMSpW5NbXR4pzULAvbXFWNnvg&s=V2Ec5cOcnYqN7P3EpauWtYZUCrRNMUzy_pvRrgdH_C8&e=
>> >
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> Bro mailing list
>> >>> bro at bro-ids.org<mailto:bro at bro-ids.org>
>> >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro<
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.ICSI.Berkeley.EDU_mailman_listinfo_bro&d=CwMD-g&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=jKm5KEaN1h0UfG6EGbAMSpW5NbXR4pzULAvbXFWNnvg&s=V2Ec5cOcnYqN7P3EpauWtYZUCrRNMUzy_pvRrgdH_C8&e=
>> >
>> >>>
>> >>> _______________________________________________
>> >>> Bro mailing list
>> >>> bro at bro-ids.org<mailto:bro at bro-ids.org>
>> >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> >>>
>> >>> ------
>> >>>
>> >>> Adam J. Slagell
>> >>> Chief Information Security Officer
>> >>> Director, Cybersecurity Division
>> >>> National Center for Supercomputing Applications
>> >>> University of Illinois at Urbana-Champaign
>> >>> www.slagell.info<http://www.slagell.info>
>> >>>
>> >>> "Under the Illinois Freedom of Information Act (FOIA), any written
>> communication to or from University employees regarding University business
>> is a public record and may be subject to public disclosure."
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>
>> >
>> > ------
>> >
>> > Adam J. Slagell
>> > Chief Information Security Officer
>> > Director, Cybersecurity Division
>> > National Center for Supercomputing Applications
>> > University of Illinois at Urbana-Champaign
>> > www.slagell.info <http://www.slagell.info/>
>> >
>> > "Under the Illinois Freedom of Information Act (FOIA), any written
>> communication to or from University employees regarding University business
>> is a public record and may be subject to public disclosure."
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org <mailto:bro at bro-ids.org>
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro <
>> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160715/955bdb0d/attachment.html
>> -------------- next part --------------
>> A non-text attachment was scrubbed...
>> Name: signature.asc
>> Type: application/pgp-signature
>> Size: 842 bytes
>> Desc: Message signed with OpenPGP using GPGMail
>> Url :
>> http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160715/955bdb0d/attachment.bin
>>
>> ------------------------------
>>
>> _______________________________________________
>> Bro mailing list
>> Bro at bro.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>
>> End of Bro Digest, Vol 123, Issue 24
>> ************************************
>>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160716/ed44dc11/attachment-0001.html 