[Bro] problem ingesting bro json logs into splunk

Gross, Brett gross.b at ghc.org
Thu Jul 14 08:33:57 PDT 2016

We’ve used bro and splunk at our organization for a couple years now. We utilize the Splunk props and transforms configs to ingest the bro log in the format we want or with the additional attributes and aliases.

REGEX = ^#.*
DEST_KEY = queue
FORMAT = nullQueue

DELIMS = "\t"
FIELDS = ts,uid,id.orig_h,id.orig_p,id.resp_h,id.resp_p,proto,service,duration,orig_bytes,resp_bytes,conn_state,local_orig,local_resp,missed_bytes,history,orig_pkts,orig_ip_bytes,resp_pkts,resp_ip_bytes,tunnel_parents,orig_cc,resp_cc,sensorname

REPORT-bro_conn_extract = bro_conn_extractions
TRANSFORMS-sourcetype = remove_hash_comments

KV_MODE = none


index = bro_conn
sourcetype = bro_conn
_TCP_ROUTING = primary_indexers


From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of philosnef
Sent: Thursday, July 14, 2016 7:24 AM
Cc: bro at bro.org
Subject: Re: [Bro] problem ingesting bro json logs into splunk

There are no 00.log files in Bro, so the automatic generation of the sourcetype bro_00 makes no sense. It does not follow the standard sourcetype pinning that all the other log files generate. find . -name "00*" in the parent logs directory reports zero logs of this type. This only occured when we moved off of Bro standard log format to JSON format.

On Thursday, July 14, 2016 10:14 AM, Brandon Lattin <lattin at umn.edu<mailto:lattin at umn.edu>> wrote:

Do you have the Splunk installed? (https://splunkbase.splunk.com/app/1617/)

The TA will dynamically create sourcetypes based on the log name.

# Dynamic source typing based on log filename
# Match: conn.log, bro.conn.log,
# md5.bro.conn.log, whatever.conn.log
DEST_KEY = MetaData:Sourcetype
SOURCE_KEY = MetaData:Source
REGEX = ([a-zA-Z0-9-]+)(?:\.[0-9-]*)?(?:\.[0-9\:-]*)?\.log
FORMAT = sourcetype::bro_$1

On Thu, Jul 14, 2016 at 8:33 AM, philosnef <philosnef at yahoo.com<mailto:philosnef at yahoo.com>> wrote:
We are getting a spurious sourcetype when ingesting bro json logs into splunk.

Specifically, we are getting a sourcetype of bro_00. There is no log file named this, and the splunkforwarder is just pushing the raw logs for indexing into splunk. There is no massaging of the log data. Anyone know why this sourcetype is popping up?

Bro mailing list
bro at bro-ids.org<mailto:bro at bro-ids.org>

Brandon Lattin
Security Analyst
University of Minnesota - University Information Security
Office: 612-626-6672


GHC Confidentiality Statement

This message and any attached files might contain confidential information protected by federal and state law. The information is intended only for the use of the individual(s) or entities originally named as addressees. The improper disclosure of such information may be subject to civil or criminal penalties. If this message reached you in error, please contact the sender and destroy this message. Disclosing, copying, forwarding, or distributing the information by unauthorized individuals or entities is strictly prohibited by law.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160714/05d86df4/attachment-0001.html 

More information about the Bro mailing list