[Bro] Get the license usage down in Splunk when indexing Bro logs

[fatema bannatwala]

We only do filtering on conn logs, as they are the heaviest (in our
environment at least), before indexing it in Splunk.
Also, if you are ingesting files.log as well, then you can build some
similar filters in props and transforms for the
mime-type you can ignore (like plain/text etc), that will also reduce some
of the volume indexed by your Splunk cluster.
I do not know much about the cloud deployment, hence can't comment on that.

Regards,
Fatema.

On Wed, Jul 12, 2017 at 3:51 PM, Mike Eriksson <mike at swedishmike.org> wrote:

> Hi Fatema,
>
> Thats looks ace - I'll definitely have to have a try at implementing that.
> Hopefully we'll be able to get that done even though we're on Cloud
> instances.
>
> Many thanks for this - it's really apprecaited.
>
> Cheers, Mike
>
> On Wed, Jul 12, 2017 at 8:43 PM fatema bannatwala <
> fatema.bannatwala at gmail.com> wrote:
>
>> Hi Mike,
>>
>> We also have something similar for brologs indexing in Splunk.
>> What we do currently is to drop all the connections whose history had
>> just a "Syn" and nothing else,
>> i.e dropping all the tcp connections that were just connection attempts.
>>
>> And the way we implemented it in Splunk, is with following filter on the
>> indexers:
>>
>> In props.conf:
>> [bro_conn_sourcetype]
>> TRANSFORMS-null= bro_conn_setnull
>>
>> In transforms.conf
>> [bro_conn_setnull]
>> REGEX = \b[S]{1}\b
>> DEST_KEY = queue
>> FORMAT = nullQueue
>>
>> Hope this helps.
>>
>> Thanks,
>> Fatema.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170712/8c3f8cad/attachment-0001.html