[Bro] Get the license usage down in Splunk when indexing Bro logs

[Mike Eriksson]

Osama,

I'm starting to lean towards this as being the next step to look into once
I've looked at the suggestions from the list.

More suggestions are of course very welcome ;-)

Cheers, Mike

On Tue, Jul 11, 2017 at 11:08 PM Osama Elnaggar <oelnaggar04 at gmail.com>
wrote:

> To build on some of the suggestions, you may also want to tier your
> logging.  You can send high value logs which can kick off an investigation
> to Splunk while sending less valuable logs which will help in
> investigations but not start them to something like ELK or ELSA / ODE.
> This will add a little overhead to your operations and involves more moving
> parts but can significantly reduce your licensing costs.
>
> --
> Osama Elnaggar
>
> On July 12, 2017 at 4:57:21 AM, Joshua Buysse (buysse at umn.edu) wrote:
>
> I think I know where Mike's misunderstanding comes from - the JSON logs
> are larger original size (for license volume), but will use less space on
> indexer disk than the default TSV because the extractions are search-time
> instead of index-time.
>
> On Tue, Jul 11, 2017 at 9:57 AM, Dave Crawford <bro at pingtrip.com> wrote:
>
>>
>>
>> The JSON logs will always be larger than the default tab delimited. With
>> JSON every log event includes the "column" names versus a set of headers in
>> he delimited format.
>>
>> > On Jul 11, 2017, at 6:48 AM, Mike Eriksson <mike at swedishmike.org>
>> wrote:
>> >
>> > Hi all,
>> >
>> > We're currently working on deploying Bro sensors to various offices and
>> I've come to realise that the Bro logs are quite 'expensive' when it comes
>> to Splunk licenses. To say the least.
>> >
>> > We have discussed various solutions but most of them fall down on us
>> losing the ability to correlate events unless we shift all the logs in to
>> Splunk.
>> >
>> > At the moment we're running it pretty much 'out of the box' so we can
>> save some GB's per day to turn of certain scripts, but it will probably not
>> be enough.
>> >
>> > Someone mentioned that turning on JSON logging instead of the standard
>> logging on Bro could save considerable amounts of space on your SIEM. Have
>> any of you guys tested this and can you back that  statement up?
>> >
>> > I was hoping that someone else had encountered this before and had come
>> up with some solution(s) to this issue?
>> >
>> > Thanks in advance, Mike
>> >
>> >
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
>
>
> --
> Joshua Buysse
> University of Minnesota - University Information Security
>
> "On two occasions I have been asked, 'Pray, Mr. Babbage, if you
> put into the machine wrong figures, will the right answers come
> out?' I am not able rightly to apprehend the kind of confusion of
> ideas that could provoke such a question."
>   - Charles Babbage
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170712/a5aa4c7d/attachment.html