[Bro] Get the license usage down in Splunk when indexing Bro logs

[Mike Eriksson]

Jon,

Many thanks for your reply - that looks really useful. Many thanks for your
examples, I'll have a look at them.

Cheers, Mike

On Tue, Jul 11, 2017 at 7:07 PM Zeolla at GMail.com <zeolla at gmail.com> wrote:

> In order to reduce logging load for some of our logging plugins we've
> applied filters that do things like drop S0 connections from conn, only
> send traffic to/from the Internet, or only send traffic to/from sensitive
> zones.  I have an example of one of our configs here
> <https://github.com/JonZeolla/Development/blob/master/bro/logs-to-kafka.bro>
> (the example also filters out IPv6 in a way that works for pre-2.5 - now
> there are is_v{4,6}_subnet() functions to handle this).
>
> Jon
>
> On Tue, Jul 11, 2017 at 1:43 PM Hosom, Stephen M <hosom at battelle.org>
> wrote:
>
>> Some people choose to implement Bro log filters that can result in a
>> significant reduction in log volume. For example, if you filter all S0
>> connections originating from outside of your organization (and you also
>> happen to listen outside of a firewall) this could reduce a substantial
>> amount of log volume.
>>
>> -----Original Message-----
>> From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of
>> Slagell, Adam J
>> Sent: Tuesday, July 11, 2017 11:03 AM
>> To: Mike Eriksson <mike at swedishmike.org>
>> Cc: bro at bro.org
>> Subject: Re: [Bro] Get the license usage down in Splunk when indexing Bro
>> logs
>>
>> Message received from outside the Battelle network. Carefully examine it
>> before you open any links or attachments.
>>
>> Well if you are collecting net flows and conn.logs, you could get rid of
>> one. If you are recapturing syslogs with Bro and sending them to Splunk,
>> you could trim duplication there. If you are finding a ton of certificate
>> information in your bro logs, you might realize some cost savings there.
>> But I don’t have much advice beyond don’t send the same info twice and look
>> for large amounts of data that you don’t really use in Splunk, like maybe
>> certificates.
>>
>>
>> > On Jul 11, 2017, at 5:48 AM, Mike Eriksson <mike at swedishmike.org>
>> wrote:
>> >
>> > Hi all,
>> >
>> > We're currently working on deploying Bro sensors to various offices and
>> I've come to realise that the Bro logs are quite 'expensive' when it comes
>> to Splunk licenses. To say the least.
>> >
>> > We have discussed various solutions but most of them fall down on us
>> losing the ability to correlate events unless we shift all the logs in to
>> Splunk.
>> >
>> > At the moment we're running it pretty much 'out of the box' so we can
>> save some GB's per day to turn of certain scripts, but it will probably not
>> be enough.
>> >
>> > Someone mentioned that turning on JSON logging instead of the standard
>> logging on Bro could save considerable amounts of space on your SIEM. Have
>> any of you guys tested this and can you back that  statement up?
>> >
>> > I was hoping that someone else had encountered this before and had come
>> up with some solution(s) to this issue?
>> >
>> > Thanks in advance, Mike
>> >
>> >
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> --
>
> Jon
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170712/c5e0fb2d/attachment-0001.html