[Bro] problem ingesting bro json logs into splunk

[Brandon Lattin]

The Bro TA is assuming TSV extractions. The move to JSON probably is
causing the Splunk auto-sourcetyper to do some funky things.

[source::...bro.*.log]
SHOULD_LINEMERGE = false
TRUNCATE = 0
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_FORMAT = %s.%6N
TRANSFORMS-BroAutoType = BroAutoType, TrashComments
INDEXED_EXTRACTIONS = TSV
FIELD_HEADER_REGEX = ^#fields\t(.*)
FIELD_DELIMITER = \t
FIELD_QUOTE = \t

On Thu, Jul 14, 2016 at 9:23 AM, philosnef <philosnef at yahoo.com> wrote:

> There are no 00.log files in Bro, so the automatic generation of the
> sourcetype bro_00 makes no sense. It does not follow the standard
> sourcetype pinning that all the other log files generate. find . -name
> "00*" in the parent logs directory reports zero logs of this type. This
> only occured when we moved off of Bro standard log format to JSON format.
>
>
> On Thursday, July 14, 2016 10:14 AM, Brandon Lattin <lattin at umn.edu>
> wrote:
>
>
> Do you have the Splunk installed? (https://splunkbase.splunk.com/app/1617/
> )
>
> The TA will dynamically create sourcetypes based on the log name.
>
> # Dynamic source typing based on log filename
> # Match: conn.log, bro.conn.log,
> # md5.bro.conn.log, whatever.conn.log
> [BroAutoType]
> DEST_KEY = MetaData:Sourcetype
> SOURCE_KEY = MetaData:Source
> REGEX = ([a-zA-Z0-9-]+)(?:\.[0-9-]*)?(?:\.[0-9\:-]*)?\.log
> FORMAT = sourcetype::bro_$1
> WRITE_META = true
>
>
> On Thu, Jul 14, 2016 at 8:33 AM, philosnef <philosnef at yahoo.com> wrote:
>
> We are getting a spurious sourcetype when ingesting bro json logs into
> splunk.
>
> Specifically, we are getting a sourcetype of bro_00. There is no log file
> named this, and the splunkforwarder is just pushing the raw logs for
> indexing into splunk. There is no massaging of the log data. Anyone know
> why this sourcetype is popping up?
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> <http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
>
>
>
>
> --
> Brandon Lattin
> Security Analyst
> University of Minnesota - University Information Security
> Office: 612-626-6672
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Brandon Lattin
Security Analyst
University of Minnesota - University Information Security
Office: 612-626-6672
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160714/051c6aa3/attachment.html