[Bro] Get the license usage down in Splunk when indexing Bro logs

[Mike Eriksson]

Fatema,

Trying to filter out on types in the the files.log as well sounds like a
great idea.

We're a bit more limited as to what we can do ourselves when it comes to
cloud Splunk but I'm sure they're more than happy to sell some PS time if
need be. ;)

Once again - many thanks for a very helpful suggestion.

Cheers, Mike

On Wed, Jul 12, 2017 at 9:14 PM fatema bannatwala <
fatema.bannatwala at gmail.com> wrote:

> We only do filtering on conn logs, as they are the heaviest (in our
> environment at least), before indexing it in Splunk.
> Also, if you are ingesting files.log as well, then you can build some
> similar filters in props and transforms for the
> mime-type you can ignore (like plain/text etc), that will also reduce some
> of the volume indexed by your Splunk cluster.
> I do not know much about the cloud deployment, hence can't comment on that.
>
> Regards,
> Fatema.
>
>
> On Wed, Jul 12, 2017 at 3:51 PM, Mike Eriksson <mike at swedishmike.org>
> wrote:
>
>> Hi Fatema,
>>
>> Thats looks ace - I'll definitely have to have a try at implementing
>> that. Hopefully we'll be able to get that done even though we're on Cloud
>> instances.
>>
>> Many thanks for this - it's really apprecaited.
>>
>> Cheers, Mike
>>
>> On Wed, Jul 12, 2017 at 8:43 PM fatema bannatwala <
>> fatema.bannatwala at gmail.com> wrote:
>>
>>> Hi Mike,
>>>
>>> We also have something similar for brologs indexing in Splunk.
>>> What we do currently is to drop all the connections whose history had
>>> just a "Syn" and nothing else,
>>> i.e dropping all the tcp connections that were just connection attempts.
>>>
>>> And the way we implemented it in Splunk, is with following filter on the
>>> indexers:
>>>
>>> In props.conf:
>>> [bro_conn_sourcetype]
>>> TRANSFORMS-null= bro_conn_setnull
>>>
>>> In transforms.conf
>>> [bro_conn_setnull]
>>> REGEX = \b[S]{1}\b
>>> DEST_KEY = queue
>>> FORMAT = nullQueue
>>>
>>> Hope this helps.
>>>
>>> Thanks,
>>> Fatema.
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170712/40fdf668/attachment.html