From neslog at gmail.com  Tue Dec  4 08:41:44 2018
From: neslog at gmail.com (Neslog)
Date: Tue, 4 Dec 2018 11:41:44 -0500
Subject: [Bro] Mapping TLS scanners JA3 => User-Agent
Message-ID: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>

Morning everyone!

I've been working with a colleague mapping scanning activity.  We are able
to capture JA3 fingerprint and match it up with the cleartext User-Agent
strings.

I'm considering throwing together a database with this information and
wanted to get insight from others to see if it's worth it.  User-Agent
strings can obviously change so the mapping may be a bit weak.

Please let me know what the list thinks.  Worth it or not?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181204/3411d33b/attachment.html 

From anthony.kasza at gmail.com  Tue Dec  4 11:50:19 2018
From: anthony.kasza at gmail.com (anthony kasza)
Date: Tue, 4 Dec 2018 12:50:19 -0700
Subject: [Bro] Mapping TLS scanners JA3 => User-Agent
In-Reply-To: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
References: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
Message-ID: <CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>

This would be hugely valuable for analysis. If you could include host
information such as OS version that would be useful too.

-AK

On Tue, Dec 4, 2018, 09:58 Neslog <neslog at gmail.com wrote:

> Morning everyone!
>
> I've been working with a colleague mapping scanning activity.  We are able
> to capture JA3 fingerprint and match it up with the cleartext User-Agent
> strings.
>
> I'm considering throwing together a database with this information and
> wanted to get insight from others to see if it's worth it.  User-Agent
> strings can obviously change so the mapping may be a bit weak.
>
> Please let me know what the list thinks.  Worth it or not?
>
> Thanks!
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181204/32687a3b/attachment.html 

From vlad at es.net  Tue Dec  4 12:58:18 2018
From: vlad at es.net (Vlad Grigorescu)
Date: Tue, 4 Dec 2018 20:58:18 +0000
Subject: [Bro] Mapping TLS scanners JA3 => User-Agent
In-Reply-To: <CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>
References: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
	<CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>
Message-ID: <CAPqbkwvNRWsyK5a3yj1C1GL3JOWSrFfk_PEg4=86+bX_K4GY7Q@mail.gmail.com>

Hi,

Check out Trisul NSM's data:
https://github.com/trisulnsm/trisul-scripts/blob/master/lua/frontend_scripts/reassembly/ja3/prints/ja3fingerprint.json

  --Vlad

On Tue, Dec 4, 2018 at 8:05 PM anthony kasza <anthony.kasza at gmail.com>
wrote:

> This would be hugely valuable for analysis. If you could include host
> information such as OS version that would be useful too.
>
> -AK
>
> On Tue, Dec 4, 2018, 09:58 Neslog <neslog at gmail.com wrote:
>
>> Morning everyone!
>>
>> I've been working with a colleague mapping scanning activity.  We are
>> able to capture JA3 fingerprint and match it up with the cleartext
>> User-Agent strings.
>>
>> I'm considering throwing together a database with this information and
>> wanted to get insight from others to see if it's worth it.  User-Agent
>> strings can obviously change so the mapping may be a bit weak.
>>
>> Please let me know what the list thinks.  Worth it or not?
>>
>> Thanks!
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181204/aebfe19d/attachment.html 

From johanna at icir.org  Tue Dec  4 13:14:56 2018
From: johanna at icir.org (Johanna Amann)
Date: Tue, 4 Dec 2018 13:14:56 -0800
Subject: [Bro] Mapping TLS scanners JA3 => User-Agent
In-Reply-To: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
References: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
Message-ID: <20181204211456.d3yj7mzxjtyw55ev@wifi89.sys.ICSI.Berkeley.EDU>

Hi,

to chime in here a bit - I think this can be useful - but please give the
data in an as detailed format as possible. So - if that is possible,
please do not just include the JA3 hash and the user-agent, but also
include the parts that make up the JA3 hash (and consider including more
information).

That makes it possible to, e.g. see how close several fingerprints are to
each other, which can be useful.

Also - as a more generit remark - one has to be quite careful on how to
interpret such fingerprints; in our experience, collisions (several pieces
of software that use the same underlying library, or have the same
fingerprint for different reasons) are quite common; in our measurements
for a recent paper (http://icir.org/johanna/papers/imc18tlsdeployment.pdf)
it was so common that we did not use it for a whole bunch of data analysis
that we planned.

On a side-note - we also published a list of TLS fingerprints that were
generated for that paper; it is accessible at
https://github.com/platonK/tls_fingerprints and might potentially be of
interest to some people of the list.

However, the same caeveat applies - one has to be a bit careful on how to
interpret the data.

Johanna

On Tue, Dec 04, 2018 at 11:41:44AM -0500, Neslog wrote:
> Morning everyone!
> 
> I've been working with a colleague mapping scanning activity.  We are able
> to capture JA3 fingerprint and match it up with the cleartext User-Agent
> strings.
> 
> I'm considering throwing together a database with this information and
> wanted to get insight from others to see if it's worth it.  User-Agent
> strings can obviously change so the mapping may be a bit weak.
> 
> Please let me know what the list thinks.  Worth it or not?
> 
> Thanks!

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From michalpurzynski1 at gmail.com  Tue Dec  4 13:18:51 2018
From: michalpurzynski1 at gmail.com (=?utf-8?Q?Micha=C5=82_Purzy=C5=84ski?=)
Date: Tue, 4 Dec 2018 16:18:51 -0500
Subject: [Bro] Mapping TLS scanners JA3 => User-Agent
In-Reply-To: <CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>
References: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
	<CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>
Message-ID: <C8746EFF-8A2E-4F4B-805C-DAF7F19EDE9E@gmail.com>

That can address one of the biggest weaknesses of JA3 - the lack of a good database that is current.

There were some databases floating around, but none of them have been updated for a while.

> On Dec 4, 2018, at 2:50 PM, anthony kasza <anthony.kasza at gmail.com> wrote:
> 
> This would be hugely valuable for analysis. If you could include host information such as OS version that would be useful too.
> 
> -AK
> 
>> On Tue, Dec 4, 2018, 09:58 Neslog <neslog at gmail.com wrote:
>> Morning everyone!
>> 
>> I've been working with a colleague mapping scanning activity.  We are able to capture JA3 fingerprint and match it up with the cleartext User-Agent strings.  
>> 
>> I'm considering throwing together a database with this information and wanted to get insight from others to see if it's worth it.  User-Agent strings can obviously change so the mapping may be a bit weak. 
>> 
>> Please let me know what the list thinks.  Worth it or not?
>> 
>> Thanks!
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181204/8d62a3c2/attachment.html 

From blackhole.em at gmail.com  Tue Dec  4 14:01:43 2018
From: blackhole.em at gmail.com (Joe Blow)
Date: Tue, 4 Dec 2018 17:01:43 -0500
Subject: [Bro] Mapping TLS scanners JA3 => User-Agent
In-Reply-To: <CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>
References: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
	<CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>
Message-ID: <CAMCSCoXSO5w9d74ytQ18yGmTL_Lv+LORxo65meXM_SuP1799KQ@mail.gmail.com>

Can you please share pcaps of the JA3s you've seen?  Feel free to DM me.
If you've already collected these handshakes, I'd love to look closer at
them.

Thanks in advance.

Cheers,

JB

On Tue, Dec 4, 2018 at 3:06 PM anthony kasza <anthony.kasza at gmail.com>
wrote:

> This would be hugely valuable for analysis. If you could include host
> information such as OS version that would be useful too.
>
> -AK
>
> On Tue, Dec 4, 2018, 09:58 Neslog <neslog at gmail.com wrote:
>
>> Morning everyone!
>>
>> I've been working with a colleague mapping scanning activity.  We are
>> able to capture JA3 fingerprint and match it up with the cleartext
>> User-Agent strings.
>>
>> I'm considering throwing together a database with this information and
>> wanted to get insight from others to see if it's worth it.  User-Agent
>> strings can obviously change so the mapping may be a bit weak.
>>
>> Please let me know what the list thinks.  Worth it or not?
>>
>> Thanks!
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181204/345f7b39/attachment.html 

From michalpurzynski1 at gmail.com  Tue Dec  4 14:45:46 2018
From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=)
Date: Tue, 4 Dec 2018 17:45:46 -0500
Subject: [Bro] Mapping TLS scanners JA3 => User-Agent
In-Reply-To: <CAMCSCoXSO5w9d74ytQ18yGmTL_Lv+LORxo65meXM_SuP1799KQ@mail.gmail.com>
References: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
	<CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>
	<CAMCSCoXSO5w9d74ytQ18yGmTL_Lv+LORxo65meXM_SuP1799KQ@mail.gmail.com>
Message-ID: <CAJ6bFK0wgMFNqB7SgrMf2sQpZ6b56CrBAFX0s+3o4EJtRSzDYA@mail.gmail.com>

And you will of course add them to a public database of signatures, Joe, right?
On Tue, Dec 4, 2018 at 5:38 PM Joe Blow <blackhole.em at gmail.com> wrote:
>
> Can you please share pcaps of the JA3s you've seen?  Feel free to DM me.  If you've already collected these handshakes, I'd love to look closer at them.
>
> Thanks in advance.
>
> Cheers,
>
> JB
>
> On Tue, Dec 4, 2018 at 3:06 PM anthony kasza <anthony.kasza at gmail.com> wrote:
>>
>> This would be hugely valuable for analysis. If you could include host information such as OS version that would be useful too.
>>
>> -AK
>>
>> On Tue, Dec 4, 2018, 09:58 Neslog <neslog at gmail.com wrote:
>>>
>>> Morning everyone!
>>>
>>> I've been working with a colleague mapping scanning activity.  We are able to capture JA3 fingerprint and match it up with the cleartext User-Agent strings.
>>>
>>> I'm considering throwing together a database with this information and wanted to get insight from others to see if it's worth it.  User-Agent strings can obviously change so the mapping may be a bit weak.
>>>
>>> Please let me know what the list thinks.  Worth it or not?
>>>
>>> Thanks!
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

From neslog at gmail.com  Tue Dec  4 18:45:18 2018
From: neslog at gmail.com (Neslog)
Date: Tue, 4 Dec 2018 21:45:18 -0500
Subject: [Bro] Mapping TLS scanners JA3 => User-Agent
In-Reply-To: <CAJ6bFK0wgMFNqB7SgrMf2sQpZ6b56CrBAFX0s+3o4EJtRSzDYA@mail.gmail.com>
References: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
	<CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>
	<CAMCSCoXSO5w9d74ytQ18yGmTL_Lv+LORxo65meXM_SuP1799KQ@mail.gmail.com>
	<CAJ6bFK0wgMFNqB7SgrMf2sQpZ6b56CrBAFX0s+3o4EJtRSzDYA@mail.gmail.com>
Message-ID: <CALgfRGJTW41CKziqJHm1TUh6aKMpLLdTkGzmq25imY5JWBPC_A@mail.gmail.com>

Thank you all for the feedback.  The goal of this work is to provide a more
realtime aggregation of JA3 information and mappings.

I've spoken with Johanna previously and completely agree aggregating the
client hello details.  This data set could be really great for research as
she said above.

My thought would be to try and host something like the SSL Notary.  This
would be continually growing while most JA3 databases are stagnant, or at
most periodically updated.  There are several different mappings that I'd
be interested in tracking.  I'd like to build a couple of scripts that
could be distributed to feed back into a database.  Some of the mappings
I'd like to see are the following:

JA3, JA3 string+(additional details), User-Agent strings.
JA3, JA3 string+(additional details), Host Application Info

Then make this data available via some type of API.  We could provide a
REST API or maybe a DNS type lookup.  It'd be quite an undertaking but if
others find it of interest and can contribute I'd be able to get more
cycles for it.

Thoughts on this approach?


On Tue, Dec 4, 2018 at 5:54 PM Micha? Purzy?ski <michalpurzynski1 at gmail.com>
wrote:

> And you will of course add them to a public database of signatures, Joe,
> right?
> On Tue, Dec 4, 2018 at 5:38 PM Joe Blow <blackhole.em at gmail.com> wrote:
> >
> > Can you please share pcaps of the JA3s you've seen?  Feel free to DM
> me.  If you've already collected these handshakes, I'd love to look closer
> at them.
> >
> > Thanks in advance.
> >
> > Cheers,
> >
> > JB
> >
> > On Tue, Dec 4, 2018 at 3:06 PM anthony kasza <anthony.kasza at gmail.com>
> wrote:
> >>
> >> This would be hugely valuable for analysis. If you could include host
> information such as OS version that would be useful too.
> >>
> >> -AK
> >>
> >> On Tue, Dec 4, 2018, 09:58 Neslog <neslog at gmail.com wrote:
> >>>
> >>> Morning everyone!
> >>>
> >>> I've been working with a colleague mapping scanning activity.  We are
> able to capture JA3 fingerprint and match it up with the cleartext
> User-Agent strings.
> >>>
> >>> I'm considering throwing together a database with this information and
> wanted to get insight from others to see if it's worth it.  User-Agent
> strings can obviously change so the mapping may be a bit weak.
> >>>
> >>> Please let me know what the list thinks.  Worth it or not?
> >>>
> >>> Thanks!
> >>> _______________________________________________
> >>> Bro mailing list
> >>> bro at bro-ids.org
> >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >>
> >> _______________________________________________
> >> Bro mailing list
> >> bro at bro-ids.org
> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181204/417df95d/attachment.html 

From cstayyab at gmail.com  Wed Dec  5 00:59:39 2018
From: cstayyab at gmail.com (Muhammad Tayyab Sheikh)
Date: Wed, 5 Dec 2018 13:59:39 +0500
Subject: [Bro] Stripping SSL on network level
Message-ID: <CAPO_G0eEaWTW1t0+c1ibhNLMTcESU=9-NkUCi=X4c5TdJoOZ+g@mail.gmail.com>

I was wondering if it is possible for bro to do monitoring at network level
and also strip SSL from all the machines in network and log unencrypted
data?

Has something be done to achieve this or are there any plans?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181205/84e6b65a/attachment.html 

From michalpurzynski1 at gmail.com  Wed Dec  5 12:18:03 2018
From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=)
Date: Wed, 5 Dec 2018 15:18:03 -0500
Subject: [Bro] When is the file hash value available for the X509
	certificate?
Message-ID: <CAJ6bFK3DQMLK=x_i2=CjhfGReq2oV9bz-L-SmG5S83i5DLiLxQ@mail.gmail.com>

Hey!

I think this is a question mostly for Johanna, but feel free to to pitch in :)

I discovered recently, that over 70% (!!) of my files.log are for X509
certificates. I decided to stop logging events to files.log where the
MIME type is anything that smells like a X509 and that cut down my
SIEM intake by not less than 20%

The only downside I see is now I do not have the file hash of the X509
certificate logged.

I tried several approaches but I cannot find a way to consistently
access the X509 file hash value before the X509 record is written to
the log.

Ideally I would just add that hash to the x509 as an extra field and
have the best of both worlds (and possibly the fuid as well).

Is that something that can be even done?

--
M.

From neslog at gmail.com  Wed Dec  5 12:56:08 2018
From: neslog at gmail.com (Neslog)
Date: Wed, 5 Dec 2018 15:56:08 -0500
Subject: [Bro] Mapping TLS scanners JA3 => User-Agent
In-Reply-To: <CALgfRGJTW41CKziqJHm1TUh6aKMpLLdTkGzmq25imY5JWBPC_A@mail.gmail.com>
References: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
	<CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>
	<CAMCSCoXSO5w9d74ytQ18yGmTL_Lv+LORxo65meXM_SuP1799KQ@mail.gmail.com>
	<CAJ6bFK0wgMFNqB7SgrMf2sQpZ6b56CrBAFX0s+3o4EJtRSzDYA@mail.gmail.com>
	<CALgfRGJTW41CKziqJHm1TUh6aKMpLLdTkGzmq25imY5JWBPC_A@mail.gmail.com>
Message-ID: <CALgfRG+e085TS3qs-JmWqui8TUMS77wXUM7auHJe-hqMo-KTbQ@mail.gmail.com>

Interesting SANS web
https://www.google.com/url?q=https://www.sans.org/webcasts/109365&sa=D&source=hangouts&ust=1544128109847000&usg=AFQjCNHoIK10Lt9gIz4q2RgACnjvnf_SwA


On Tue, Dec 4, 2018 at 9:45 PM Neslog <neslog at gmail.com> wrote:

> Thank you all for the feedback.  The goal of this work is to provide a
> more realtime aggregation of JA3 information and mappings.
>
> I've spoken with Johanna previously and completely agree aggregating the
> client hello details.  This data set could be really great for research as
> she said above.
>
> My thought would be to try and host something like the SSL Notary.  This
> would be continually growing while most JA3 databases are stagnant, or at
> most periodically updated.  There are several different mappings that I'd
> be interested in tracking.  I'd like to build a couple of scripts that
> could be distributed to feed back into a database.  Some of the mappings
> I'd like to see are the following:
>
> JA3, JA3 string+(additional details), User-Agent strings.
> JA3, JA3 string+(additional details), Host Application Info
>
> Then make this data available via some type of API.  We could provide a
> REST API or maybe a DNS type lookup.  It'd be quite an undertaking but if
> others find it of interest and can contribute I'd be able to get more
> cycles for it.
>
> Thoughts on this approach?
>
>
> On Tue, Dec 4, 2018 at 5:54 PM Micha? Purzy?ski <
> michalpurzynski1 at gmail.com> wrote:
>
>> And you will of course add them to a public database of signatures, Joe,
>> right?
>> On Tue, Dec 4, 2018 at 5:38 PM Joe Blow <blackhole.em at gmail.com> wrote:
>> >
>> > Can you please share pcaps of the JA3s you've seen?  Feel free to DM
>> me.  If you've already collected these handshakes, I'd love to look closer
>> at them.
>> >
>> > Thanks in advance.
>> >
>> > Cheers,
>> >
>> > JB
>> >
>> > On Tue, Dec 4, 2018 at 3:06 PM anthony kasza <anthony.kasza at gmail.com>
>> wrote:
>> >>
>> >> This would be hugely valuable for analysis. If you could include host
>> information such as OS version that would be useful too.
>> >>
>> >> -AK
>> >>
>> >> On Tue, Dec 4, 2018, 09:58 Neslog <neslog at gmail.com wrote:
>> >>>
>> >>> Morning everyone!
>> >>>
>> >>> I've been working with a colleague mapping scanning activity.  We are
>> able to capture JA3 fingerprint and match it up with the cleartext
>> User-Agent strings.
>> >>>
>> >>> I'm considering throwing together a database with this information
>> and wanted to get insight from others to see if it's worth it.  User-Agent
>> strings can obviously change so the mapping may be a bit weak.
>> >>>
>> >>> Please let me know what the list thinks.  Worth it or not?
>> >>>
>> >>> Thanks!
>> >>> _______________________________________________
>> >>> Bro mailing list
>> >>> bro at bro-ids.org
>> >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> >>
>> >> _______________________________________________
>> >> Bro mailing list
>> >> bro at bro-ids.org
>> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> >
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181205/08f6ebd1/attachment.html 

From michalpurzynski1 at gmail.com  Wed Dec  5 13:20:50 2018
From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=)
Date: Wed, 5 Dec 2018 16:20:50 -0500
Subject: [Bro] When is the file hash value available for the X509
	certificate?
In-Reply-To: <CAJ6bFK3DQMLK=x_i2=CjhfGReq2oV9bz-L-SmG5S83i5DLiLxQ@mail.gmail.com>
References: <CAJ6bFK3DQMLK=x_i2=CjhfGReq2oV9bz-L-SmG5S83i5DLiLxQ@mail.gmail.com>
Message-ID: <CAJ6bFK3VP8EtHnmAh=5H-N9yDHHEJ4sgrWmJuNkt+Xm7J0BF6g@mail.gmail.com>

One more thing

I created this script and it seems to work -
http://try.bro.org/#/trybro/saved/283934

Can I get some feedback, how reliable it will be? It does seem to work on a
single production sensor.



On Wed, Dec 5, 2018 at 3:18 PM Micha? Purzy?ski <michalpurzynski1 at gmail.com>
wrote:

> Hey!
>
> I think this is a question mostly for Johanna, but feel free to to pitch
> in :)
>
> I discovered recently, that over 70% (!!) of my files.log are for X509
> certificates. I decided to stop logging events to files.log where the
> MIME type is anything that smells like a X509 and that cut down my
> SIEM intake by not less than 20%
>
> The only downside I see is now I do not have the file hash of the X509
> certificate logged.
>
> I tried several approaches but I cannot find a way to consistently
> access the X509 file hash value before the X509 record is written to
> the log.
>
> Ideally I would just add that hash to the x509 as an extra field and
> have the best of both worlds (and possibly the fuid as well).
>
> Is that something that can be even done?
>
> --
> M.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181205/9dc8329a/attachment.html 

From neslog at gmail.com  Wed Dec  5 13:45:44 2018
From: neslog at gmail.com (Neslog)
Date: Wed, 5 Dec 2018 16:45:44 -0500
Subject: [Bro] Mapping TLS scanners JA3 => User-Agent
In-Reply-To: <CALgfRG+e085TS3qs-JmWqui8TUMS77wXUM7auHJe-hqMo-KTbQ@mail.gmail.com>
References: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
	<CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>
	<CAMCSCoXSO5w9d74ytQ18yGmTL_Lv+LORxo65meXM_SuP1799KQ@mail.gmail.com>
	<CAJ6bFK0wgMFNqB7SgrMf2sQpZ6b56CrBAFX0s+3o4EJtRSzDYA@mail.gmail.com>
	<CALgfRGJTW41CKziqJHm1TUh6aKMpLLdTkGzmq25imY5JWBPC_A@mail.gmail.com>
	<CALgfRG+e085TS3qs-JmWqui8TUMS77wXUM7auHJe-hqMo-KTbQ@mail.gmail.com>
Message-ID: <CALgfRGLBmmQ7RjNL-QCp6JpAbNqYCKcNrjx+2MyN=WHNVB2vxw@mail.gmail.com>

This reiterates some of the concerns we had and why I'm looking at building
this project.  He had some good ideas about clues which are available
within the extension values.  It gave me some ideas on how to apply it and
extend JA3.

Love to hear what others think.  Feel free to ping me directly.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181205/05067ec3/attachment.html 

From johanna at icir.org  Wed Dec  5 13:55:01 2018
From: johanna at icir.org (Johanna Amann)
Date: Wed, 05 Dec 2018 13:55:01 -0800
Subject: [Bro] When is the file hash value available for the X509
 certificate?
In-Reply-To: <A2FCC09C-B5E2-4AAE-A10C-ED000120E9EC@icsi.berkeley.edu>
References: <CAJ6bFK3DQMLK=x_i2=CjhfGReq2oV9bz-L-SmG5S83i5DLiLxQ@mail.gmail.com>
	<CAJ6bFK3VP8EtHnmAh=5H-N9yDHHEJ4sgrWmJuNkt+Xm7J0BF6g@mail.gmail.com>
	<A2FCC09C-B5E2-4AAE-A10C-ED000120E9EC@icsi.berkeley.edu>
Message-ID: <84472F5D-7FC7-43D3-9319-C7E996532199@icir.org>

That should be completely reliable.

Johanna


On 5 Dec 2018, at 13:53, Johanna Amann wrote:

> That should be completely reliable.
>
> Johanna
>
> On 5 Dec 2018, at 13:20, Micha? Purzy?ski wrote:
>
>> One more thing
>>
>> I created this script and it seems to work -
>> http://try.bro.org/#/trybro/saved/283934
>>
>> Can I get some feedback, how reliable it will be? It does seem to 
>> work on a
>> single production sensor.
>>
>>
>>
>> On Wed, Dec 5, 2018 at 3:18 PM Micha? Purzy?ski 
>> <michalpurzynski1 at gmail.com>
>> wrote:
>>
>>> Hey!
>>>
>>> I think this is a question mostly for Johanna, but feel free to to 
>>> pitch
>>> in :)
>>>
>>> I discovered recently, that over 70% (!!) of my files.log are for 
>>> X509
>>> certificates. I decided to stop logging events to files.log where 
>>> the
>>> MIME type is anything that smells like a X509 and that cut down my
>>> SIEM intake by not less than 20%
>>>
>>> The only downside I see is now I do not have the file hash of the 
>>> X509
>>> certificate logged.
>>>
>>> I tried several approaches but I cannot find a way to consistently
>>> access the X509 file hash value before the X509 record is written to
>>> the log.
>>>
>>> Ideally I would just add that hash to the x509 as an extra field and
>>> have the best of both worlds (and possibly the fuid as well).
>>>
>>> Is that something that can be even done?
>>>
>>> --
>>> M.
>>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

From jlay at slave-tothe-box.net  Wed Dec  5 14:54:44 2018
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 05 Dec 2018 15:54:44 -0700
Subject: [Bro] Gotchas for 2.5.5 to 2.6 (notes from the field)
Message-ID: <080a9009996294ef42a66a10eac0157a@slave-tothe-box.net>

Not many.  From my notes...might help someone out there.  Going from 
non-bro-pkg to bro-pkg was the bulk of the excitement.  Also if you've 
ran bro-pkg with sudo instead of just as root you'll have to tweak out 
the config file.  Try as I might to bro-pkg upgrade ja3 it did not fly, 
but an uninstall and reinstall worked.  The list of packages are ones I 
use, betting folks use things other than my tiny list.  Thank you.

James

remove current /opt/bro/lib/bro/plugins/Bro_AF_Packet

update /root/.bro-pkg/config
bro_dist = /home/home/build/bro-2.6 <- remained on old build dir even 
after config and install

remove all from local.bro
@load packages <- not this
#@load packages/intel-seen-more/seen <- these

as root:
pip install bro-pkg
bro-pkg refresh
bro-pkg install bro-af_packet-plugin
bro-pkg remove ja3
bro-pkg install ja3
bro-pkg upgrade domain-tld
bro-pkg upgrade intel-seen-more
bro-pkg load ja3
bro-pkg load domain-tld
bro-pkg load intel-seen-more


From robin at corelight.com  Wed Dec  5 17:19:20 2018
From: robin at corelight.com (Robin Sommer)
Date: Wed, 5 Dec 2018 17:19:20 -0800
Subject: [Bro] Job posting: Open Source Zeek Developer
Message-ID: <20181206011920.GM81843@corelight.com>

Corelight is looking for an experienced software developer interested
in working with us on open source Zeek.

See https://www.corelight.com/company/careers/1457622 for more
information.

Robin

-- 
Robin Sommer * Corelight, Inc. * robin at corelight.com * www.corelight.com

From seth at corelight.com  Thu Dec  6 05:51:51 2018
From: seth at corelight.com (Seth Hall)
Date: Thu, 06 Dec 2018 08:51:51 -0500
Subject: [Bro] Gotchas for 2.5.5 to 2.6 (notes from the field)
In-Reply-To: <080a9009996294ef42a66a10eac0157a@slave-tothe-box.net>
References: <080a9009996294ef42a66a10eac0157a@slave-tothe-box.net>
Message-ID: <FBD1D408-A3DC-41E5-9146-4F284A463944@corelight.com>

Thanks for the notes!  Always helpful.

  .Seth

On 5 Dec 2018, at 17:54, James Lay wrote:

> Not many.  From my notes...might help someone out there.  Going from
> non-bro-pkg to bro-pkg was the bulk of the excitement.  Also if you've
> ran bro-pkg with sudo instead of just as root you'll have to tweak out
> the config file.  Try as I might to bro-pkg upgrade ja3 it did not fly,
> but an uninstall and reinstall worked.  The list of packages are ones I
> use, betting folks use things other than my tiny list.  Thank you.
>
> James
>
> remove current /opt/bro/lib/bro/plugins/Bro_AF_Packet
>
> update /root/.bro-pkg/config
> bro_dist = /home/home/build/bro-2.6 <- remained on old build dir even
> after config and install
>
> remove all from local.bro
> @load packages <- not this
> #@load packages/intel-seen-more/seen <- these
>
> as root:
> pip install bro-pkg
> bro-pkg refresh
> bro-pkg install bro-af_packet-plugin
> bro-pkg remove ja3
> bro-pkg install ja3
> bro-pkg upgrade domain-tld
> bro-pkg upgrade intel-seen-more
> bro-pkg load ja3
> bro-pkg load domain-tld
> bro-pkg load intel-seen-more
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

--
Seth Hall * Corelight, Inc * www.corelight.com

From johanna at icir.org  Thu Dec  6 07:05:15 2018
From: johanna at icir.org (Johanna Amann)
Date: Thu, 6 Dec 2018 07:05:15 -0800
Subject: [Bro] When is the file hash value available for the X509
 certificate?
In-Reply-To: <CAJ6bFK3DQMLK=x_i2=CjhfGReq2oV9bz-L-SmG5S83i5DLiLxQ@mail.gmail.com>
References: <CAJ6bFK3DQMLK=x_i2=CjhfGReq2oV9bz-L-SmG5S83i5DLiLxQ@mail.gmail.com>
Message-ID: <20181206150515.h2mbutil5rqizw3s@Tranquility.local>

Hi Michal,

> Ideally I would just add that hash to the x509 as an extra field and
> have the best of both worlds (and possibly the fuid as well).

One small additional question here - does the solution that you have now
satisfy this, or did you want the information in some other log-file (e.g.
ssl.log)?

Johanna

From neslog at gmail.com  Thu Dec  6 07:49:20 2018
From: neslog at gmail.com (Neslog)
Date: Thu, 6 Dec 2018 10:49:20 -0500
Subject: [Bro] Tracking TLS extensions
Message-ID: <CALgfRG+KJkVCEjdBuXJqMVYQng1MmJ-z7Tkkp9xVizfJouAL=w@mail.gmail.com>

Is there a table of TLS extensions out there already?  I didn't see
anything right away and starting to work on it.  Didn't want to duplicate
someone's work if it's already available.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181206/26f56dbe/attachment.html 

From neslog at gmail.com  Thu Dec  6 08:45:58 2018
From: neslog at gmail.com (Neslog)
Date: Thu, 6 Dec 2018 11:45:58 -0500
Subject: [Bro] Tracking TLS extensions
In-Reply-To: <CALgfRG+KJkVCEjdBuXJqMVYQng1MmJ-z7Tkkp9xVizfJouAL=w@mail.gmail.com>
References: <CALgfRG+KJkVCEjdBuXJqMVYQng1MmJ-z7Tkkp9xVizfJouAL=w@mail.gmail.com>
Message-ID: <CALgfRG+uH+N7biERZ6PbQD-EXc-pxHLsYGS19XO+jsOMLd0Fuw@mail.gmail.com>

Found it... table located in scripts/base/protocols/ssl/consts.bro.
Reference the table SSL::extensions.

On Thu, Dec 6, 2018 at 10:49 AM Neslog <neslog at gmail.com> wrote:

> Is there a table of TLS extensions out there already?  I didn't see
> anything right away and starting to work on it.  Didn't want to duplicate
> someone's work if it's already available.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181206/8900a530/attachment.html 

From michalpurzynski1 at gmail.com  Thu Dec  6 09:58:32 2018
From: michalpurzynski1 at gmail.com (=?utf-8?Q?Micha=C5=82_Purzy=C5=84ski?=)
Date: Thu, 6 Dec 2018 12:58:32 -0500
Subject: [Bro] When is the file hash value available for the X509
	certificate?
In-Reply-To: <20181206150515.h2mbutil5rqizw3s@Tranquility.local>
References: <CAJ6bFK3DQMLK=x_i2=CjhfGReq2oV9bz-L-SmG5S83i5DLiLxQ@mail.gmail.com>
	<20181206150515.h2mbutil5rqizw3s@Tranquility.local>
Message-ID: <FC5C4E84-F28E-4E95-A794-1DE647273DB5@gmail.com>

Yeah, it works for me. How complicated would it be to add everything to the ssl log, out of curiosity?

> On Dec 6, 2018, at 10:05 AM, Johanna Amann <johanna at icir.org> wrote:
> 
> Hi Michal,
> 
>> Ideally I would just add that hash to the x509 as an extra field and
>> have the best of both worlds (and possibly the fuid as well).
> 
> One small additional question here - does the solution that you have now
> satisfy this, or did you want the information in some other log-file (e.g.
> ssl.log)?
> 
> Johanna


From johanna at icir.org  Thu Dec  6 10:38:57 2018
From: johanna at icir.org (Johanna Amann)
Date: Thu, 6 Dec 2018 10:38:57 -0800
Subject: [Bro] When is the file hash value available for the X509
 certificate?
In-Reply-To: <FC5C4E84-F28E-4E95-A794-1DE647273DB5@gmail.com>
References: <CAJ6bFK3DQMLK=x_i2=CjhfGReq2oV9bz-L-SmG5S83i5DLiLxQ@mail.gmail.com>
	<20181206150515.h2mbutil5rqizw3s@Tranquility.local>
	<FC5C4E84-F28E-4E95-A794-1DE647273DB5@gmail.com>
Message-ID: <20181206183857.cvnichc65tjcbkj5@Trafalgar.local>

Unless I am forgetting something big - not all that complicated... for
some measure of complicated. It might need extending a few records to have
the data in the right place... I would try doing it similarly to how the
certificate subject is currently put into ssl.log.

Johanna

On Thu, Dec 06, 2018 at 12:58:32PM -0500, Micha? Purzy?ski wrote:
> Yeah, it works for me. How complicated would it be to add everything to the ssl log, out of curiosity?
> 
> > On Dec 6, 2018, at 10:05 AM, Johanna Amann <johanna at icir.org> wrote:
> > 
> > Hi Michal,
> > 
> >> Ideally I would just add that hash to the x509 as an extra field and
> >> have the best of both worlds (and possibly the fuid as well).
> > 
> > One small additional question here - does the solution that you have now
> > satisfy this, or did you want the information in some other log-file (e.g.
> > ssl.log)?
> > 
> > Johanna
> 

From jsiwek at corelight.com  Thu Dec  6 11:56:42 2018
From: jsiwek at corelight.com (Jon Siwek)
Date: Thu, 6 Dec 2018 13:56:42 -0600
Subject: [Bro] GitHub Zeek/Bro Account Transition
Message-ID: <CAMzgZ0KFCE3Uvy1_g-45q5rcc_H-cdVOLOi4VETh-PCJmH-J1Q@mail.gmail.com>

The official location for Zeek (Bro) Git repositories is now here:

    https://github.com/zeek

The "bro" GitHub account (https://github.com/bro) is now only used to
mirror "zeek" repositories.  Please use the later for new issues and
pull requests.

Many repositories with "bro" in their name have replaced that with
"zeek" at their new GitHub location.  For example, the main "bro"
repository is now here:

    https://github.com/zeek/zeek

Also, the repositories hosted at git.bro.org are now just mirrors of
the GitHub account.

Existing git clones may continue to work, but here's an example of how
to update a clone of bro/zeek to the new URL (recommended):

    git remote set-url origin https://github.com/zeek/zeek
    git pull
    git submodule sync --recursive
    git submodule update --recursive --init

Note there is still quite a bit of renaming to be done within the
various codebases and documentation, but if you notice anything that's
outright broken as the renaming effort advances, feel free to report
it as it may have been overlooked.

- Jon

From daniel.guerra69 at gmail.com  Fri Dec  7 02:43:36 2018
From: daniel.guerra69 at gmail.com (Daniel Guerra)
Date: Fri, 7 Dec 2018 11:43:36 +0100
Subject: [Bro] Mapping TLS scanners JA3 => User-Agent
In-Reply-To: <CALgfRGLBmmQ7RjNL-QCp6JpAbNqYCKcNrjx+2MyN=WHNVB2vxw@mail.gmail.com>
References: <CALgfRGK2dXMsFQiFHbAkx_cWNkpyejgDex0+5QDOicMG1c889g@mail.gmail.com>
	<CAEZw2bwpN9ndW0YxRwToQARkWUGsSqKXOQtk9C0j1OZo2-JPXQ@mail.gmail.com>
	<CAMCSCoXSO5w9d74ytQ18yGmTL_Lv+LORxo65meXM_SuP1799KQ@mail.gmail.com>
	<CAJ6bFK0wgMFNqB7SgrMf2sQpZ6b56CrBAFX0s+3o4EJtRSzDYA@mail.gmail.com>
	<CALgfRGJTW41CKziqJHm1TUh6aKMpLLdTkGzmq25imY5JWBPC_A@mail.gmail.com>
	<CALgfRG+e085TS3qs-JmWqui8TUMS77wXUM7auHJe-hqMo-KTbQ@mail.gmail.com>
	<CALgfRGLBmmQ7RjNL-QCp6JpAbNqYCKcNrjx+2MyN=WHNVB2vxw@mail.gmail.com>
Message-ID: <19ae37d2-18ca-047b-a42e-736fb7469811@gmail.com>

I use JA3's and they can be very useful. The problem is indeed the lack of

a good database. Creating one is lots of work. I once tried to gather
all known

databases. Still its not enough. To use the ja3 in the right way is :
study a known

app you are interested and add the ja3 to the db.

An other way is to do is automated by using public pcap achives and
search for

user-agents.


My salesforce fork with database:
https://github.com/danielguerra69/ja3

Regards,

Daniel
Op 05-12-18 om 22:45 schreef Neslog:
> This reiterates some of the concerns we had and why I'm looking at
> building this project.? He had some good ideas about clues which are
> available within the extension values.? It gave me some ideas on how
> to apply it and extend JA3. ?
>
> Love to hear what others think.? Feel free to ping me directly.
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181207/484e7145/attachment.html 

From konrad.weglowski at gmail.com  Fri Dec  7 12:07:18 2018
From: konrad.weglowski at gmail.com (Konrad Weglowski)
Date: Fri, 7 Dec 2018 15:07:18 -0500
Subject: [Bro] Bro 2.5.5 documentation link?
Message-ID: <CA+fGMJRrSK5CjqRxPzE-c6is4zHHXuZZ91noF_mcZK_rnRAieQ@mail.gmail.com>

Hey,

I noticed all the documentation has been updated to 2.6 on bro.org...is
there a way to access docs for version 2.5.5?

Thank You

Konrad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181207/aefa0840/attachment.html 

From jsiwek at corelight.com  Fri Dec  7 13:36:53 2018
From: jsiwek at corelight.com (Jon Siwek)
Date: Fri, 7 Dec 2018 15:36:53 -0600
Subject: [Bro] Bro 2.5.5 documentation link?
In-Reply-To: <CA+fGMJRrSK5CjqRxPzE-c6is4zHHXuZZ91noF_mcZK_rnRAieQ@mail.gmail.com>
References: <CA+fGMJRrSK5CjqRxPzE-c6is4zHHXuZZ91noF_mcZK_rnRAieQ@mail.gmail.com>
Message-ID: <CAMzgZ0Lxer7ZfdZQ9S52nVFCHthvPf2EqhB+6h_kkRo_jTwtPA@mail.gmail.com>

On Fri, Dec 7, 2018 at 2:21 PM Konrad Weglowski
<konrad.weglowski at gmail.com> wrote:

> I noticed all the documentation has been updated to 2.6 on bro.org...is there a way to access docs for version 2.5.5?

I added a link to an archive for offline viewing [1].  The raw sources
for this documentation are also in the doc/ subdir of the Bro/Zeek
source tree.

- Jon

[1] https://www.bro.org/downloads/bro-2.5.5-docs.tar.gz

From konrad.weglowski at gmail.com  Fri Dec  7 13:38:02 2018
From: konrad.weglowski at gmail.com (Konrad Weglowski)
Date: Fri, 7 Dec 2018 16:38:02 -0500
Subject: [Bro] Bro 2.5.5 documentation link?
In-Reply-To: <CAMzgZ0Lxer7ZfdZQ9S52nVFCHthvPf2EqhB+6h_kkRo_jTwtPA@mail.gmail.com>
References: <CA+fGMJRrSK5CjqRxPzE-c6is4zHHXuZZ91noF_mcZK_rnRAieQ@mail.gmail.com>
	<CAMzgZ0Lxer7ZfdZQ9S52nVFCHthvPf2EqhB+6h_kkRo_jTwtPA@mail.gmail.com>
Message-ID: <CA+fGMJS3g2jz4KbH-OG+tP4R--bmPKrY=btD6WHBxHDA2m-q1w@mail.gmail.com>

Thank you

Konrad

On Fri, Dec 7, 2018 at 4:37 PM Jon Siwek <jsiwek at corelight.com> wrote:

> On Fri, Dec 7, 2018 at 2:21 PM Konrad Weglowski
> <konrad.weglowski at gmail.com> wrote:
>
> > I noticed all the documentation has been updated to 2.6 on bro.org...is
> there a way to access docs for version 2.5.5?
>
> I added a link to an archive for offline viewing [1].  The raw sources
> for this documentation are also in the doc/ subdir of the Bro/Zeek
> source tree.
>
> - Jon
>
> [1] https://www.bro.org/downloads/bro-2.5.5-docs.tar.gz
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181207/f548e021/attachment.html 

From zeolla at gmail.com  Sat Dec  8 15:31:42 2018
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Sat, 8 Dec 2018 18:31:42 -0500
Subject: [Bro] Bro 2.5.5 documentation link?
In-Reply-To: <CA+fGMJS3g2jz4KbH-OG+tP4R--bmPKrY=btD6WHBxHDA2m-q1w@mail.gmail.com>
References: <CA+fGMJRrSK5CjqRxPzE-c6is4zHHXuZZ91noF_mcZK_rnRAieQ@mail.gmail.com>
	<CAMzgZ0Lxer7ZfdZQ9S52nVFCHthvPf2EqhB+6h_kkRo_jTwtPA@mail.gmail.com>
	<CA+fGMJS3g2jz4KbH-OG+tP4R--bmPKrY=btD6WHBxHDA2m-q1w@mail.gmail.com>
Message-ID: <CAM1mTCrq=3P1eya8WPn-uw-JtRgZE19owdeYVAYewqT4AYTKpQ@mail.gmail.com>

Is there a reason why there isn't versioned documentation on the website?

Jon

On Fri, Dec 7, 2018, 5:05 PM Konrad Weglowski <konrad.weglowski at gmail.com>
wrote:

> Thank you
>
> Konrad
>
> On Fri, Dec 7, 2018 at 4:37 PM Jon Siwek <jsiwek at corelight.com> wrote:
>
>> On Fri, Dec 7, 2018 at 2:21 PM Konrad Weglowski
>> <konrad.weglowski at gmail.com> wrote:
>>
>> > I noticed all the documentation has been updated to 2.6 on bro.org...is
>> there a way to access docs for version 2.5.5?
>>
>> I added a link to an archive for offline viewing [1].  The raw sources
>> for this documentation are also in the doc/ subdir of the Bro/Zeek
>> source tree.
>>
>> - Jon
>>
>> [1] https://www.bro.org/downloads/bro-2.5.5-docs.tar.gz
>>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-- 

Jon Zeolla
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181208/12c220c3/attachment.html 

From bill.de.ping at gmail.com  Sun Dec  9 07:12:42 2018
From: bill.de.ping at gmail.com (william de ping)
Date: Sun, 9 Dec 2018 17:12:42 +0200
Subject: [Bro]  - recommended DB for Bro logs
Message-ID: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>

Hi all,

I would appreciate recommendations for a DB server that is most suited for
ingesting and digesting Bro logs.

I know of some use cases involving splunk and the Splunk Bro app, but price
and performance wise (10GBps incoming traffic) it does not seem to be the
best solution out there.

Does anyone have any experience with Bro and  ElasticSearch | Redis | MySQL
?

I am looking into different solutions and would appreciate your thoughts.

Thanks in advance
B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181209/4348ed4f/attachment.html 

From ipninichuck at gmail.com  Sun Dec  9 07:38:03 2018
From: ipninichuck at gmail.com (ivan ninichuck)
Date: Sun, 9 Dec 2018 07:38:03 -0800
Subject: [Bro] - recommended DB for Bro logs
In-Reply-To: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
References: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
Message-ID: <CAOJiQr3d-0mqNwHujtPRSLKGX=GU+0K-dNDFkFyaV+5Gw63X8Q@mail.gmail.com>

Elastic Search is fantastic. Very good displaying of information, and the
newest version has alerts, some graph analysis and basic machine learning.
Let me know if you need help getting started.

On Sun, Dec 9, 2018, 7:20 AM william de ping <bill.de.ping at gmail.com wrote:

> Hi all,
>
> I would appreciate recommendations for a DB server that is most suited for
> ingesting and digesting Bro logs.
>
> I know of some use cases involving splunk and the Splunk Bro app, but
> price and performance wise (10GBps incoming traffic) it does not seem to be
> the best solution out there.
>
> Does anyone have any experience with Bro and  ElasticSearch | Redis |
> MySQL ?
>
> I am looking into different solutions and would appreciate your thoughts.
>
> Thanks in advance
> B
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181209/cdb90c57/attachment.html 

From cgaylord at vt.edu  Sun Dec  9 07:47:42 2018
From: cgaylord at vt.edu (Clark Gaylord)
Date: Sun, 9 Dec 2018 10:47:42 -0500
Subject: [Bro] - recommended DB for Bro logs
In-Reply-To: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
References: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
Message-ID: <CADzU5g7ZJ7FrQorh8buZKByKgWJupN8T5Jdy0HOmoSxckiLK-w@mail.gmail.com>

I have done some proof of concept work with PostgreSQL (mostly in AWS RDS)
and have been very happy with the results so far. Of course the rub is you
need to set up the schema, but it is pretty straightforward to ingest after
that from the JSON.

What I've done is load JSON into a text field of a temp table, then cast
that as JSON on insert (there was a little trick to getting this right that
I don't recall off the top of my head). My load process is currently out of
service but I can try to look up my code for this if you need it.

Anyway, works like a champ since PG has not only JSON but inet and cidr
data types!
https://www.postgresql.org/docs/11.1/datatype-net-types.html

You could do a document database that would handle the JSON gracefully, but
then you're constantly paying the parse tax. Works great if you don't
actually want to use your data, though. :-)

If you use standard bro text files you've got more parsing to do but it's
certainly doable. I like having JSON bro output to avoid that heavy lifting.

Cheers
Clark

-- 

--
Clark Gaylord
cgaylord at vt.edu
... Autocorrect may have improved this message ...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181209/0a5eb7b6/attachment.html 

From zeolla at gmail.com  Sun Dec  9 08:53:35 2018
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Sun, 9 Dec 2018 11:53:35 -0500
Subject: [Bro] - recommended DB for Bro logs
In-Reply-To: <CADzU5g7ZJ7FrQorh8buZKByKgWJupN8T5Jdy0HOmoSxckiLK-w@mail.gmail.com>
References: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
	<CADzU5g7ZJ7FrQorh8buZKByKgWJupN8T5Jdy0HOmoSxckiLK-w@mail.gmail.com>
Message-ID: <CAM1mTCoOhYb+zq2Hrch5FhtEcu2Qd7F29MrXXvRLSFXhJN62xQ@mail.gmail.com>

I've put bro data in Solr,.ElasticSearch, HDFS, Splunk, and Mongodb with
success but for different use cases.  What are you looking to do with the
data?

The Apache Metron project supports bro logs natively and can index in hdfs,
solr, or elasticsearch.  If you don't want to buy into the entire project
(a bit of a heavy lift if you don't already run Ambari and Hadoop or aren't
interested in security data analytics) there may be reusable components
that are helpful.  Let me know if you're interested in digging in and I can
help.  A part of this project is the kafka writer plugin, used as a buffer
between bro and an indexed store.
https://packages.bro.org/packages/view/7388aa77-4fb7-11e8-88be-0a645a3f3086

This isn't meant to be a commercial, I've heard great things about bro data
going into Postgres and redis as well.

See also:
https://packages.bro.org/tags/view/737d1f7c-4fb7-11e8-88be-0a645a3f3086
https://packages.bro.org/tags/view/738aaeb0-4fb7-11e8-88be-0a645a3f3086

Jon

On Sun, Dec 9, 2018, 10:56 AM Clark Gaylord <cgaylord at vt.edu> wrote:

> I have done some proof of concept work with PostgreSQL (mostly in AWS RDS)
> and have been very happy with the results so far. Of course the rub is you
> need to set up the schema, but it is pretty straightforward to ingest after
> that from the JSON.
>
> What I've done is load JSON into a text field of a temp table, then cast
> that as JSON on insert (there was a little trick to getting this right that
> I don't recall off the top of my head). My load process is currently out of
> service but I can try to look up my code for this if you need it.
>
> Anyway, works like a champ since PG has not only JSON but inet and cidr
> data types!
> https://www.postgresql.org/docs/11.1/datatype-net-types.html
>
> You could do a document database that would handle the JSON gracefully,
> but then you're constantly paying the parse tax. Works great if you don't
> actually want to use your data, though. :-)
>
> If you use standard bro text files you've got more parsing to do but it's
> certainly doable. I like having JSON bro output to avoid that heavy lifting.
>
> Cheers
> Clark
>
> --
>
> --
> Clark Gaylord
> cgaylord at vt.edu
> ... Autocorrect may have improved this message ...
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-- 

Jon Zeolla
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181209/5386f17f/attachment-0001.html 

From cgaylord at vt.edu  Sun Dec  9 09:47:04 2018
From: cgaylord at vt.edu (Clark Gaylord)
Date: Sun, 9 Dec 2018 12:47:04 -0500
Subject: [Bro] - recommended DB for Bro logs
In-Reply-To: <CAM1mTCoOhYb+zq2Hrch5FhtEcu2Qd7F29MrXXvRLSFXhJN62xQ@mail.gmail.com>
References: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
	<CADzU5g7ZJ7FrQorh8buZKByKgWJupN8T5Jdy0HOmoSxckiLK-w@mail.gmail.com>
	<CAM1mTCoOhYb+zq2Hrch5FhtEcu2Qd7F29MrXXvRLSFXhJN62xQ@mail.gmail.com>
Message-ID: <CADzU5g5q7fdQZA+UZX=ROzmBV75e+HAg4vQE+GUuYtsLKuWU-A@mail.gmail.com>

Looks like Metron doesn't support IPv6. Hence useless in 2018. Too bad.

--
Clark Gaylord
cgaylord at vt.edu
... autocorrect may have improved this message ...

On Sun, Dec 9, 2018, 11:53 Zeolla at GMail.com <zeolla at gmail.com wrote:

> I've put bro data in Solr,.ElasticSearch, HDFS, Splunk, and Mongodb with
> success but for different use cases.  What are you looking to do with the
> data?
>
> The Apache Metron project supports bro logs natively and can index in
> hdfs, solr, or elasticsearch.  If you don't want to buy into the entire
> project (a bit of a heavy lift if you don't already run Ambari and Hadoop
> or aren't interested in security data analytics) there may be reusable
> components that are helpful.  Let me know if you're interested in digging
> in and I can help.  A part of this project is the kafka writer plugin, used
> as a buffer between bro and an indexed store.
> https://packages.bro.org/packages/view/7388aa77-4fb7-11e8-88be-0a645a3f3086
>
> This isn't meant to be a commercial, I've heard great things about bro
> data going into Postgres and redis as well.
>
> See also:
> https://packages.bro.org/tags/view/737d1f7c-4fb7-11e8-88be-0a645a3f3086
> https://packages.bro.org/tags/view/738aaeb0-4fb7-11e8-88be-0a645a3f3086
>
> Jon
>
> On Sun, Dec 9, 2018, 10:56 AM Clark Gaylord <cgaylord at vt.edu> wrote:
>
>> I have done some proof of concept work with PostgreSQL (mostly in AWS
>> RDS) and have been very happy with the results so far. Of course the rub is
>> you need to set up the schema, but it is pretty straightforward to ingest
>> after that from the JSON.
>>
>> What I've done is load JSON into a text field of a temp table, then cast
>> that as JSON on insert (there was a little trick to getting this right that
>> I don't recall off the top of my head). My load process is currently out of
>> service but I can try to look up my code for this if you need it.
>>
>> Anyway, works like a champ since PG has not only JSON but inet and cidr
>> data types!
>> https://www.postgresql.org/docs/11.1/datatype-net-types.html
>>
>> You could do a document database that would handle the JSON gracefully,
>> but then you're constantly paying the parse tax. Works great if you don't
>> actually want to use your data, though. :-)
>>
>> If you use standard bro text files you've got more parsing to do but it's
>> certainly doable. I like having JSON bro output to avoid that heavy lifting.
>>
>> Cheers
>> Clark
>>
>> --
>>
>> --
>> Clark Gaylord
>> cgaylord at vt.edu
>> ... Autocorrect may have improved this message ...
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> --
>
> Jon Zeolla
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181209/19543bdd/attachment.html 

From daniel.guerra69 at gmail.com  Sun Dec  9 10:45:06 2018
From: daniel.guerra69 at gmail.com (Daniel Guerra)
Date: Sun, 9 Dec 2018 19:45:06 +0100
Subject: [Bro] - recommended DB for Bro logs
In-Reply-To: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
References: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
Message-ID: <CAKd-9_kWqQC05GsNkz4vfYS3xdH4+JTja9f-ZikYA-CE3YEW6w@mail.gmail.com>

I like kibana as frontend. So the choice would be elastic. I switched to
elassandra. Elastic is way to slow for bro. With a file buffer or a broker
like Kafka all goes well.If you use elastic, split the bro types e.g. Conn
ssl etc. This is to avoid mapping collisions.
MySQL is a great database, consider timebased databases, because after
100mil records the performance goes down.


On Sun, Dec 9, 2018, 4:20 PM william de ping <bill.de.ping at gmail.com wrote:

> Hi all,
>
> I would appreciate recommendations for a DB server that is most suited for
> ingesting and digesting Bro logs.
>
> I know of some use cases involving splunk and the Splunk Bro app, but
> price and performance wise (10GBps incoming traffic) it does not seem to be
> the best solution out there.
>
> Does anyone have any experience with Bro and  ElasticSearch | Redis |
> MySQL ?
>
> I am looking into different solutions and would appreciate your thoughts.
>
> Thanks in advance
> B
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181209/fa384494/attachment.html 

From zeolla at gmail.com  Sun Dec  9 12:24:56 2018
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Sun, 9 Dec 2018 15:24:56 -0500
Subject: [Bro] - recommended DB for Bro logs
In-Reply-To: <CAKd-9_kWqQC05GsNkz4vfYS3xdH4+JTja9f-ZikYA-CE3YEW6w@mail.gmail.com>
References: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
	<CAKd-9_kWqQC05GsNkz4vfYS3xdH4+JTja9f-ZikYA-CE3YEW6w@mail.gmail.com>
Message-ID: <CAM1mTCrJCypkNdbrHOtrTNq6OCJVz_gk8Vfz0Gi5ivWSEu2crA@mail.gmail.com>

RE: Clark, not to hijack the thread but that isn't true.  Assuming you're
referring to the note in the plugin that says "Metron currently doesn't
support IPv6 source or destination IPs in the default enrichments" this
just means there isn't a built-in *example enrichment* that supports IPv6.
The platform itself has full IPv6 support end to end without issue, I have
been doing it for years.  If you want to chat more on this we should talk
elsewhere.

Jon

On Sun, Dec 9, 2018 at 1:53 PM Daniel Guerra <daniel.guerra69 at gmail.com>
wrote:

> I like kibana as frontend. So the choice would be elastic. I switched to
> elassandra. Elastic is way to slow for bro. With a file buffer or a broker
> like Kafka all goes well.If you use elastic, split the bro types e.g.
> Conn ssl etc. This is to avoid mapping collisions.
> MySQL is a great database, consider timebased databases, because after
> 100mil records the performance goes down.
>
>
> On Sun, Dec 9, 2018, 4:20 PM william de ping <bill.de.ping at gmail.com
> wrote:
>
>> Hi all,
>>
>> I would appreciate recommendations for a DB server that is most suited
>> for ingesting and digesting Bro logs.
>>
>> I know of some use cases involving splunk and the Splunk Bro app, but
>> price and performance wise (10GBps incoming traffic) it does not seem to be
>> the best solution out there.
>>
>> Does anyone have any experience with Bro and  ElasticSearch | Redis |
>> MySQL ?
>>
>> I am looking into different solutions and would appreciate your thoughts.
>>
>> Thanks in advance
>> B
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-- 

Jon Zeolla
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181209/1764c3dd/attachment.html 

From cgaylord at vt.edu  Sun Dec  9 12:39:27 2018
From: cgaylord at vt.edu (Clark Gaylord)
Date: Sun, 9 Dec 2018 15:39:27 -0500
Subject: [Bro] - recommended DB for Bro logs
In-Reply-To: <CAM1mTCrJCypkNdbrHOtrTNq6OCJVz_gk8Vfz0Gi5ivWSEu2crA@mail.gmail.com>
References: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
	<CAKd-9_kWqQC05GsNkz4vfYS3xdH4+JTja9f-ZikYA-CE3YEW6w@mail.gmail.com>
	<CAM1mTCrJCypkNdbrHOtrTNq6OCJVz_gk8Vfz0Gi5ivWSEu2crA@mail.gmail.com>
Message-ID: <CADzU5g5MV7-aLPqVwkUqu+XDkG18UTGcNZUCEgP1wDYYTXNpYg@mail.gmail.com>

Thank you for the clarification! That's a great reason to "hijack" the
convo.  :-)

I admit I was rather shocked to read that and I'm glad to hear I was
mistaken. I have been intrigued by Metron and would like to take a closer
look. Alas the Hadoop requirement is a pretty heavy lift - it would be my
only actual Hadoop requirement (in a peta-scale data analytics environment,
btw...) Though in fairness I'd like to have a legitimate reason to do more
than play with Hadoop and conclude it doesn't fit our other use cases.

--ckg

On Sun, Dec 9, 2018, 15:32 Zeolla at GMail.com <zeolla at gmail.com> wrote:

> RE: Clark, not to hijack the thread but that isn't true.  Assuming you're
> referring to the note in the plugin that says "Metron currently doesn't
> support IPv6 source or destination IPs in the default enrichments" this
> just means there isn't a built-in *example enrichment* that supports
> IPv6.  The platform itself has full IPv6 support end to end without issue,
> I have been doing it for years.  If you want to chat more on this we should
> talk elsewhere.
>
> Jon
>
> On Sun, Dec 9, 2018 at 1:53 PM Daniel Guerra <daniel.guerra69 at gmail.com>
> wrote:
>
>> I like kibana as frontend. So the choice would be elastic. I switched to
>> elassandra. Elastic is way to slow for bro. With a file buffer or a broker
>> like Kafka all goes well.If you use elastic, split the bro types e.g.
>> Conn ssl etc. This is to avoid mapping collisions.
>> MySQL is a great database, consider timebased databases, because after
>> 100mil records the performance goes down.
>>
>>
>> On Sun, Dec 9, 2018, 4:20 PM william de ping <bill.de.ping at gmail.com
>> wrote:
>>
>>> Hi all,
>>>
>>> I would appreciate recommendations for a DB server that is most suited
>>> for ingesting and digesting Bro logs.
>>>
>>> I know of some use cases involving splunk and the Splunk Bro app, but
>>> price and performance wise (10GBps incoming traffic) it does not seem to be
>>> the best solution out there.
>>>
>>> Does anyone have any experience with Bro and  ElasticSearch | Redis |
>>> MySQL ?
>>>
>>> I am looking into different solutions and would appreciate your thoughts.
>>>
>>> Thanks in advance
>>> B
>>>
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> --
>
> Jon Zeolla
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-- 

--
Clark Gaylord
cgaylord at vt.edu
... Autocorrect may have improved this message ...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181209/d2e1d81f/attachment.html 

From bkeep at alias454studios.com  Sun Dec  9 13:51:11 2018
From: bkeep at alias454studios.com (bkeep)
Date: Sun, 9 Dec 2018 15:51:11 -0600
Subject: [Bro] - recommended DB for Bro logs
In-Reply-To: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
References: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
Message-ID: <81868cbc-e251-82df-bf35-655dc63a44b2@alias454studios.com>

I've had some success using Graylog. I send BRO logs via rsyslog to a 
Graylog collector and utilize pipeline processing rules in Graylog for 
message enrichment. https://github.com/alias454/graylog-bro-content-pack.

On 12/9/18 9:12 AM, william de ping wrote:
> Hi all,
>
> I would appreciate recommendations for a DB server that is most suited 
> for ingesting and digesting Bro logs.
>
> I know of some use cases involving splunk and the Splunk Bro app, but 
> price and performance wise (10GBps incoming traffic) it does not seem 
> to be the best solution out there.
>
> Does anyone have any experience with Bro and? ElasticSearch | Redis | 
> MySQL ?
>
> I am looking into different solutions and would appreciate your thoughts.
>
> Thanks in advance
> B
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181209/11034db5/attachment.html 

From jsiwek at corelight.com  Mon Dec 10 07:41:21 2018
From: jsiwek at corelight.com (Jon Siwek)
Date: Mon, 10 Dec 2018 09:41:21 -0600
Subject: [Bro] Bro 2.5.5 documentation link?
In-Reply-To: <CAM1mTCrq=3P1eya8WPn-uw-JtRgZE19owdeYVAYewqT4AYTKpQ@mail.gmail.com>
References: <CA+fGMJRrSK5CjqRxPzE-c6is4zHHXuZZ91noF_mcZK_rnRAieQ@mail.gmail.com>
	<CAMzgZ0Lxer7ZfdZQ9S52nVFCHthvPf2EqhB+6h_kkRo_jTwtPA@mail.gmail.com>
	<CA+fGMJS3g2jz4KbH-OG+tP4R--bmPKrY=btD6WHBxHDA2m-q1w@mail.gmail.com>
	<CAM1mTCrq=3P1eya8WPn-uw-JtRgZE19owdeYVAYewqT4AYTKpQ@mail.gmail.com>
Message-ID: <CAMzgZ0JJt5_WOE92+gX7N+04duj4zAMkFmsx0gF-_2aHcfCKSA@mail.gmail.com>

On Sat, Dec 8, 2018 at 5:31 PM Zeolla at GMail.com <zeolla at gmail.com> wrote:
>
> Is there a reason why there isn't versioned documentation on the website?

Historically, not much reason other than little demand combined with
lack of cycles to spend on changing the website autobuild/integration
scripts (there's a bit of unexpected/hidden complexity there).

This week, I'm looking into moving those docs to Read the Docs and it
should be simpler to host multiple versions there.

- Jon

From zeolla at gmail.com  Mon Dec 10 07:43:52 2018
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Mon, 10 Dec 2018 10:43:52 -0500
Subject: [Bro] Bro 2.5.5 documentation link?
In-Reply-To: <CAMzgZ0JJt5_WOE92+gX7N+04duj4zAMkFmsx0gF-_2aHcfCKSA@mail.gmail.com>
References: <CA+fGMJRrSK5CjqRxPzE-c6is4zHHXuZZ91noF_mcZK_rnRAieQ@mail.gmail.com>
	<CAMzgZ0Lxer7ZfdZQ9S52nVFCHthvPf2EqhB+6h_kkRo_jTwtPA@mail.gmail.com>
	<CA+fGMJS3g2jz4KbH-OG+tP4R--bmPKrY=btD6WHBxHDA2m-q1w@mail.gmail.com>
	<CAM1mTCrq=3P1eya8WPn-uw-JtRgZE19owdeYVAYewqT4AYTKpQ@mail.gmail.com>
	<CAMzgZ0JJt5_WOE92+gX7N+04duj4zAMkFmsx0gF-_2aHcfCKSA@mail.gmail.com>
Message-ID: <CAM1mTCopttDrcQ1PQxmrh2JWBGk9x6a3O71HvGyyAwgJs7CgNg@mail.gmail.com>

That sounds reasonable to me.  Thanks,

Jon

On Mon, Dec 10, 2018 at 10:41 AM Jon Siwek <jsiwek at corelight.com> wrote:

> On Sat, Dec 8, 2018 at 5:31 PM Zeolla at GMail.com <zeolla at gmail.com> wrote:
> >
> > Is there a reason why there isn't versioned documentation on the website?
>
> Historically, not much reason other than little demand combined with
> lack of cycles to spend on changing the website autobuild/integration
> scripts (there's a bit of unexpected/hidden complexity there).
>
> This week, I'm looking into moving those docs to Read the Docs and it
> should be simpler to host multiple versions there.
>
> - Jon
>
-- 

Jon Zeolla
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181210/bf81c81f/attachment.html 

From hovsep.sanjay.levi at gmail.com  Thu Dec 13 07:25:04 2018
From: hovsep.sanjay.levi at gmail.com (Hovsep Levi)
Date: Thu, 13 Dec 2018 15:25:04 +0000
Subject: [Bro] Bro logs - enable_local_logging and remove_default_filter
Message-ID: <CAJ19UZB_SSvRzTtMEFJvf4tX+d1fdp9=LRK_=CtXaiBUmz8k6w@mail.gmail.com>

Hi all,


Can you please help to explain how to disable local logging ?  I am using
the KafkaWriter Bro plugin for many years now without a problem but after
an upgrade to Bro 2.6 there is a problem.

The logs that are excluded from sending to Kafka are the logs that are
being written to disk.  In Bro config language that means the logs that are
not explicitly defined in KafkaLogger::logs_to_send.

Example from local.bro for KafkaLogger:

redef KafkaLogger::logs_to_send( CaptureLoss::LOG, etc... )


Historically I modify the KafkaLogger plugin slightly to support disabling
the writing of logs to disk by adding a function call to
"Log::remove_default_filter" for each log.  With Bro 2.6 this no longer
seems to work the way it once did.

So I check the documentation at
https://www.bro.org/sphinx-git/scripts/base/frameworks/logging/main.bro.html
and see remove_default_filter still exists and also notice two variables
that might be relevant to my issue.

Log::enable_local_logging
<https://www.bro.org/sphinx-git/scripts/base/frameworks/logging/main.bro.html#id-Log::enable_local_logging>:
bool <https://www.bro.org/sphinx-git/script-reference/types.html#type-bool>
&redef
<https://www.bro.org/sphinx-git/script-reference/attributes.html#attr-&redef>
If
true, local logging is by default enabled for all filters.
Log::enable_remote_logging
<https://www.bro.org/sphinx-git/scripts/base/frameworks/logging/main.bro.html#id-Log::enable_remote_logging>:
bool <https://www.bro.org/sphinx-git/script-reference/types.html#type-bool>
&redef
<https://www.bro.org/sphinx-git/script-reference/attributes.html#attr-&redef>
If
true, remote logging is by default enabled for all filters.

But when I try to set Log::enable_local_logging=0 within the KafkaLogger
plugin loop for each log I get an error.


Thanks in advance.

-Hovsep
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181213/9bfbf6e9/attachment.html 

From johanna at icir.org  Thu Dec 13 13:51:08 2018
From: johanna at icir.org (Johanna Amann)
Date: Thu, 13 Dec 2018 13:51:08 -0800
Subject: [Bro] Bro logs - enable_local_logging and remove_default_filter
In-Reply-To: <CAJ19UZB_SSvRzTtMEFJvf4tX+d1fdp9=LRK_=CtXaiBUmz8k6w@mail.gmail.com>
References: <CAJ19UZB_SSvRzTtMEFJvf4tX+d1fdp9=LRK_=CtXaiBUmz8k6w@mail.gmail.com>
Message-ID: <20181213215108.x27ucegishqprnlx@user159.sys.ICSI.Berkeley.EDU>

Hi Hovsep,

[...]

> Historically I modify the KafkaLogger plugin slightly to support disabling
> the writing of logs to disk by adding a function call to
> "Log::remove_default_filter" for each log.  With Bro 2.6 this no longer
> seems to work the way it once did.

I just looked and I did not really see any big way in which this changed.
Could you perhaps provide a code-snippet that does not work anymore?

I also just tried a minimal example script and Log::remove_default_filter
seems to work as expected.

[...]

> But when I try to set Log::enable_local_logging=0 within the KafkaLogger
> plugin loop for each log I get an error.

This is probably a misunderstanding. Log::enable_local_logging is not a
per-log setting - so there is nothing to loop over.

If you do a

redef Log::enable_local_logging = F;

The setting will persist. That being said, you will very probably not want
to enable this, it means something slightly different than what you
expect. Remote logging means that a log is sent to a remote Bro
instance; local logging means that logging is performed by the current
node. If you set enable_local_logging to false on a node, it will not
output any kind of logs directly itself - this includes sending logs to
Kafka - from a Bro point of view, these are local logs (the logging is
performed by the local node).

By default this is set to "T" in standalone mode; in clusters the setting
is "T" on Logger nodes and "F" on all other nodes. Which is very probably
like you want it.

I hope this helps,
 Johanna

From johanna at icir.org  Thu Dec 13 14:44:55 2018
From: johanna at icir.org (Johanna Amann)
Date: Thu, 13 Dec 2018 14:44:55 -0800
Subject: [Bro] Stripping SSL on network level
In-Reply-To: <CAPO_G0eEaWTW1t0+c1ibhNLMTcESU=9-NkUCi=X4c5TdJoOZ+g@mail.gmail.com>
References: <CAPO_G0eEaWTW1t0+c1ibhNLMTcESU=9-NkUCi=X4c5TdJoOZ+g@mail.gmail.com>
Message-ID: <20181213224455.gnjvmim2y54wg6cy@user159.sys.ICSI.Berkeley.EDU>

Hi,

> I was wondering if it is possible for bro to do monitoring at network level
> and also strip SSL from all the machines in network and log unencrypted
> data?

Bro itself does not support any kind of SSL/TLS decryption. If it is fed
unencrypted data (e.g. sitting behing a SSL terminator) it will happily
log it.

> Has something be done to achieve this or are there any plans?

There are no plans current plans that I know of to implement this.

Johanna

From michalpurzynski1 at gmail.com  Thu Dec 13 15:10:44 2018
From: michalpurzynski1 at gmail.com (=?utf-8?Q?Micha=C5=82_Purzy=C5=84ski?=)
Date: Thu, 13 Dec 2018 15:10:44 -0800
Subject: [Bro] Stripping SSL on network level
In-Reply-To: <20181213224455.gnjvmim2y54wg6cy@user159.sys.ICSI.Berkeley.EDU>
References: <CAPO_G0eEaWTW1t0+c1ibhNLMTcESU=9-NkUCi=X4c5TdJoOZ+g@mail.gmail.com>
	<20181213224455.gnjvmim2y54wg6cy@user159.sys.ICSI.Berkeley.EDU>
Message-ID: <878D546A-8DE8-4D18-BAF4-CC6FE5533102@gmail.com>

Well, the design of SSL makes it impossible to strip it at the network level.

How?s that usually done is, there?s a proxy that terminates each connection and initiates a new one, generating certificates on the fly for every destination site, signed by a CA sitting on that proxy, that?s trusted by clients.

What that means - you need to configure your clients to trust that CA anyway.

Before someone mentions SSLstrip - it looks for HTTP connections before they are 302 to the SSL endpoint. If connection is SSL end to end, it won?t do anything.

> On Dec 13, 2018, at 2:44 PM, Johanna Amann <johanna at icir.org> wrote:
> 
> Hi,
> 
>> I was wondering if it is possible for bro to do monitoring at network level
>> and also strip SSL from all the machines in network and log unencrypted
>> data?
> 
> Bro itself does not support any kind of SSL/TLS decryption. If it is fed
> unencrypted data (e.g. sitting behing a SSL terminator) it will happily
> log it.
> 
>> Has something be done to achieve this or are there any plans?
> 
> There are no plans current plans that I know of to implement this.
> 
> Johanna
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From michalpurzynski1 at gmail.com  Thu Dec 13 17:59:22 2018
From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=)
Date: Thu, 13 Dec 2018 17:59:22 -0800
Subject: [Bro] Zeek monitoring
Message-ID: <CAJ6bFK3DHCbXWZGBkTU3HE-ATFRQfggTv2088f5Odc6bkpNYPQ@mail.gmail.com>

Hey!

How do you monitor your Zeek instance?

What are the things you would be looking for?

I'd like to compile a list of 'standard' checks (and have that added
to the official documentation later) that are considered a good
practice.

This is what we are currently doing, or planning to do:

Hardware checks - nothing exciting here
- interface status on the packet broker side
- is the traffic on each interface within expected range? alert if too
low (someone unplugged a tap?)
- is the signal level what it should be? alert is signal level is too
low (important for fibre)

OS level checks - is each sensor up, memory usage, CPU usage, disk space, etc

App level checks
- how many processes with the name 'bro' are running?
- have we seen 'I am alive' events in the last N minutes?

What I am also thinking about
- monitoring the log lag (Justin has a script for that)
- have we seen at least N events of type M from sensor K in a time
window O minutes? Repeat for each log file, for each sensor
- packet loss (expect me to publish some code soon, because you are
doing it wrong ;)
- errors (hardware counters for each card, for example)
- crashing processes - I have seen process Y crashed N times in the
last 15 minutes
- weird.log size or just how fast is it growing
- reporter.log size or just the growth velocity (useful to catch
errors in scripts and undefined variable access)

I'm also thinking about monitoring how frequently the broctl cron has
to respawn processes. Is there any useful way to tell that?

Feel free to comment, critique or add to the list.

--
M.

From sbarker at nettitude.com  Fri Dec 14 02:11:06 2018
From: sbarker at nettitude.com (Samual Barker)
Date: Fri, 14 Dec 2018 10:11:06 +0000
Subject: [Bro] Adding interface to bro logs
Message-ID: <7b5c0455f98c426c8f1c01e5f6d6fedb@nettitude.com>

Hi


Does anyone know how to add the name of the interface Bro is listening on to the logs? I currently have a server listening on multiple interfaces and would be useful to have the interface logged so that I can retrieve the pcap for any event more easily


Many thanks

Sam
___________________________________________________________________________________
Lloyd?s Register and variants of it are trading names of Lloyd?s Register Group Limited, its subsidiaries and affiliates. 
Nettitude Limited, registered in England, registered number 4705154
Registered office: 1 Jephson Court, Tancred Close, Leamington Spa, Warwickshire, CV31 3RZ. A member of the Lloyd?s Register group.
 
Lloyd?s Register Group Limited, its affiliates and subsidiaries and their respective officers, employees or agents are individually and collectively, referred to in this clause as ?Lloyd?s Register?. Lloyd?s Register assumes no responsibility and shall not be liable to any person for any loss, damage or expense caused by reliance on the information or advice in this document or howsoever provided, unless that person has signed a contract with the relevant Lloyd?s Register entity for the provision of this information or advice and in that case any responsibility or liability is exclusively on the terms and conditions set out in that contract. 
___________________________________________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181214/5588b059/attachment.html 

From ericooi at gmail.com  Fri Dec 14 09:03:34 2018
From: ericooi at gmail.com (Eric Ooi)
Date: Fri, 14 Dec 2018 11:03:34 -0600
Subject: [Bro] Adding interface to bro logs
In-Reply-To: <7b5c0455f98c426c8f1c01e5f6d6fedb@nettitude.com>
References: <7b5c0455f98c426c8f1c01e5f6d6fedb@nettitude.com>
Message-ID: <CAEuvLSbD5RvURYQCQfkR-5k=E_BQ5hcJ7y2LBG7dH2=Yfgtm+A@mail.gmail.com>

Check out Example 2:
https://blog.zeek.org/2012/02/filtering-logs-with-bro.html

On Fri, Dec 14, 2018 at 4:15 AM Samual Barker <sbarker at nettitude.com> wrote:

> Hi
>
>
> Does anyone know how to add the name of the interface Bro is listening on
> to the logs? I currently have a server listening on multiple interfaces and
> would be useful to have the interface logged so that I can retrieve the
> pcap for any event more easily
>
>
> Many thanks
>
> Sam
>
>
> ___________________________________________________________________________________
>
> Lloyd?s Register and variants of it are trading names of Lloyd?s Register
> Group Limited, its subsidiaries and affiliates.
> Nettitude Limited, registered in England, registered number 4705154
> Registered office: 1 Jephson Court, Tancred Close, Leamington Spa,
> Warwickshire, CV31 3RZ. A member of the Lloyd?s Register group.
>
>
>
> Lloyd?s Register Group Limited, its affiliates and subsidiaries and their
> respective officers, employees or agents are individually and collectively,
> referred to in this clause as ?Lloyd?s Register?. Lloyd?s Register assumes
> no responsibility and shall not be liable to any person for any loss,
> damage or expense caused by reliance on the information or advice in this
> document or howsoever provided, unless that person has signed a contract
> with the relevant Lloyd?s Register entity for the provision of this
> information or advice and in that case any responsibility or liability is
> exclusively on the terms and conditions set out in that contract.
>
>
> ___________________________________________________________________________________
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181214/6cb315ab/attachment-0001.html 

From al.kefallonitis at gmail.com  Mon Dec 17 05:04:55 2018
From: al.kefallonitis at gmail.com (Alex Kefallonitis)
Date: Mon, 17 Dec 2018 15:04:55 +0200
Subject: [Bro] General Whitelisting IP's or Domains
In-Reply-To: <CAHv=MuoK6mjkCKb5NZ3-n5koKNRE_vOZDhyXn3upW+_vic0nGQ@mail.gmail.com>
References: <CAHv=Muqh4i=xYqL9QvO-4qFzHyQxqdcG2A7yMTsz+i135ZLaFQ@mail.gmail.com>
	<SN6PR11MB3216AACEA7381A435F3C4AADA5D20@SN6PR11MB3216.namprd11.prod.outlook.com>
	<CAHv=MuoK6mjkCKb5NZ3-n5koKNRE_vOZDhyXn3upW+_vic0nGQ@mail.gmail.com>
Message-ID: <CAHv=Muqd-QKDfaurGgsOt8Nuaiv4xq7oOX5WRq=mn1A5fqGUEQ@mail.gmail.com>

So i cannot find any other way for generic whitelisting i am not so sure
how dns could work. Any suggestions ?

???? ???, 29 ??? 2018 ???? 7:34 ?.?., ?/? Alex Kefallonitis <
al.kefallonitis at gmail.com> ??????:

> Hi and thanks for the response
>
>  I want to be able to apply the whitelist in all of the above as generic
> solution when something is spamming or hits as false positive. So is there
> any generic solution ?
>
> Thanks in advanced,
> Alex Kefallonitis
>
> ???? ???, 29 ??? 2018 ???? 7:30 ?.?., ?/? Azoff, Justin S <
> jazoff at illinois.edu> ??????:
>
>> > Is there a generic way to whitelist certain IP's/Subets or Domains in
>> local.bro for the whole Bro configuration as not to produce logs and or
>> notices.
>> >
>> > For e.g whitelist 8.8.8.8 or  google.com ?
>>
>> It depends.. if you wanted to ignore ALL traffic to 8.8.8.8 you could add
>> this:
>>
>>     redef restrict_filters += [ ["not-google-dns"] = "not (host 8.8.8.8)"
>> ];
>>
>> Ignoring a 'google.com' is possible as well, but a little more involved
>> since it
>> could appear in dns, ssl, or http logs.  Is there a particular kind of
>> log that
>> you are seeing domains in that you want to ignore, or all of the above?
>>
>> --
>> - Justin
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181217/b0881d6b/attachment.html 

From andy at unimatrixzero.co.uk  Mon Dec 17 06:27:40 2018
From: andy at unimatrixzero.co.uk (Andy Millett)
Date: Mon, 17 Dec 2018 14:27:40 +0000
Subject: [Bro] BPF Syntax/Runtime Problem
Message-ID: <791899D7-E022-40E0-8FA9-00C1FDF353F0@unimatrixzero.co.uk>

An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181217/261c60d9/attachment.html 

From promero at cenic.org  Mon Dec 17 10:11:23 2018
From: promero at cenic.org (Philip Romero)
Date: Mon, 17 Dec 2018 10:11:23 -0800
Subject: [Bro] Bro Digest, Vol 152, Issue 15
In-Reply-To: <mailman.82467.1544807002.837.bro@bro.org>
References: <mailman.82467.1544807002.837.bro@bro.org>
Message-ID: <52df3c5f-9e47-891f-ffe9-0efff83cf040@cenic.org>

Sam,

I'm not sure if you got what you were looking for or if this input of
mine will help, but I use the "worker" tag to help me identify which
interface the logged event was seen on. The events in the conn log show
the worker name for the event seen when logging. There is also a unique
number for each process so in the below node.cfg example the logs would
include a field that states "worker-1-1", "worker-1-2", "worker-2-1", or
"worker-2-2". When I see worker-1 in the log I know it was seen on eth1
and when I see worker-2 in the log I know it was seen on eth2.

Hope this helped.

Example node.cfg:
[manager]
type=manager
host=localhost
#
[proxy-1]
type=proxy
host=localhost
#
[worker-1]
lb_method=pf_ring
lb_procs=2
pin_cpus=0,1
type=worker
host=localhost
interface=eth1
#
[worker-2]
lb_method=pf_ring
lb_procs=2
pin_cpus=2,3
type=worker
host=localhost
interface=eth2

On 12/14/18 9:03 AM, bro-request at bro.org wrote:
> Send Bro mailing list submissions to
> 	bro at bro.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> or, via email, send a message with subject or body 'help' to
> 	bro-request at bro.org
>
> You can reach the person managing the list at
> 	bro-owner at bro.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Bro digest..."
>
>
> Today's Topics:
>
>    1. Re: Bro logs - enable_local_logging and remove_default_filter
>       (Johanna Amann)
>    2. Re: Stripping SSL on network level (Johanna Amann)
>    3. Re: Stripping SSL on network level (Micha? Purzy?ski)
>    4. Zeek monitoring (Micha? Purzy?ski)
>    5. Adding interface to bro logs (Samual Barker)
>    6. Re: Adding interface to bro logs (Eric Ooi)
>
>
> ------------------------------
>
> Message: 5
> Date: Fri, 14 Dec 2018 10:11:06 +0000
> From: Samual Barker <sbarker at nettitude.com>
> Subject: [Bro] Adding interface to bro logs
> To: "bro at bro.org" <bro at bro.org>
> Message-ID: <7b5c0455f98c426c8f1c01e5f6d6fedb at nettitude.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi
>
>
> Does anyone know how to add the name of the interface Bro is listening on to the logs? I currently have a server listening on multiple interfaces and would be useful to have the interface logged so that I can retrieve the pcap for any event more easily
>
>
> Many thanks
>
> Sam
>
-- 
Philip Romero, CISSP, CISA
Sr. Information Security Analyst
CENIC
promero at cenic.org
Phone: (714) 220-3430
Mobile: (562) 237-9290

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181217/45d42655/attachment.html 

From jmellander at lbl.gov  Mon Dec 17 12:50:04 2018
From: jmellander at lbl.gov (Jim Mellander)
Date: Mon, 17 Dec 2018 12:50:04 -0800
Subject: [Bro] BPF Syntax/Runtime Problem
In-Reply-To: <791899D7-E022-40E0-8FA9-00C1FDF353F0@unimatrixzero.co.uk>
References: <791899D7-E022-40E0-8FA9-00C1FDF353F0@unimatrixzero.co.uk>
Message-ID: <CADju=b5L1xO=V+AU0hXhPV517otabFicDbrAE+BBfdLkswkJ6Q@mail.gmail.com>

Hi:

I ran your filter on a local bro instance with no problems, although based
on your description, shouldn't you have parentheses around the subnets in
the restrict_filters["unmonitored nets"] expression? , i.e.

restrict_filters["unmonitored nets"] = "not (net 10.230.128.0/23 or net
10.230.64.0/23 or net 10.230.48.0/23 or net 10.230.130.0/23 or net
10.230.40.0/24 or net 10.230.237.0/24 or net 10.237.128.0/24 or net
10.230.108.0/24 or net 10.230.37.0/24 or net 10.230.38.0/24 or net
10.230.168.0/24 or net 10.230.72.0/24 or net 10.230.199.0/24 or net
10.230.177.0/24 or net 10.230.178.0/24 or net 10.230.179.0/24 or net
10.230.189.0/24 or net 10.230.183.0/24 or net 10.230.151.0/24 or net
10.230.165.0/24 or net 10.230.197.0/24 or net 10.230.167.0/24 or net
10.230.181.0/24 or net 10.230.31.0/24 or net 10.230.26.0/24 or net
10.230.180.0/24 or net 10.230.157.0/24 or net 10.230.159.0/24 or net
10.230.60.0/24 or net 10.230.150.0/24 or net 10.230.184.0/24 or net
10.230.202.0/24 or net 10.230.16.0/24 or net 10.230.156.0/24 or net
10.237.171.0/24 or net 10.230.76.0/24 or net 10.230.222.0/24 or net
10.230.186.0/24 or net 10.230.24.0/24 or net 10.237.162.0/24 or net
10.230.22.0/24 or net 10.230.112.0/23 or net 10.230.120.0/24 or net
10.230.163.0/24 or net 10.230.17.0/24 or net 10.230.152.0/24 or host
224.0.0.252)";

You might also take the filter in packet_filter.log and use that as the
filter for a tcpdump and see if you are, in fact, capturing the traffic you
expect.

Hope this helps,

Jim


On Mon, Dec 17, 2018 at 6:37 AM Andy Millett <andy at unimatrixzero.co.uk>
wrote:

> Hi guys,
>
> We have a number of distributed Bro IDS sensors running on Raspberry Pi
> hardware at over 50 MPLS sites which are small or medium size links
> (anything up to 50Mbps). We have another 100 sites which don?t have sensors
> deployed (yet), so we?re trying to capture as much additional information
> for our ELK stack at the corporate HQ where most traffic goes. With this, I
> want to bypass logging of subnets which already have a remote sensor
> deployed to reduce duplication in ELK. I?ve been trying to use the BPF
> syntax, but don?t appear to be very successful.
>
> For starters, I?ve tried this -
>
> event bro_init() &priority=-12
>        {
>        restrict_filters["ignore proxy node"] = "not (host 10.230.91.2)";
> restrict_filters["unmonitored nets"] = "not net 10.230.128.0/23 or net
> 10.230.64.0/23 or net 10.230.48.0/23 or net 10.230.130.0/23 or net
> 10.230.40.0/24 or net 10.230.237.0/24 or net 10.237.128.0/24 or net
> 10.230.108.0/24 or net 10.230.37.0/24 or net 10.230.38.0/24 or net
> 10.230.168.0/24 or net 10.230.72.0/24 or net 10.230.199.0/24 or net
> 10.230.177.0/24 or net 10.230.178.0/24 or net 10.230.179.0/24 or net
> 10.230.189.0/24 or net 10.230.183.0/24 or net 10.230.151.0/24 or net
> 10.230.165.0/24 or net 10.230.197.0/24 or net 10.230.167.0/24 or net
> 10.230.181.0/24 or net 10.230.31.0/24 or net 10.230.26.0/24 or net
> 10.230.180.0/24 or net 10.230.157.0/24 or net 10.230.159.0/24 or net
> 10.230.60.0/24 or net 10.230.150.0/24 or net 10.230.184.0/24 or net
> 10.230.202.0/24 or net 10.230.16.0/24 or net 10.230.156.0/24 or net
> 10.237.171.0/24 or net 10.230.76.0/24 or net 10.230.222.0/24 or net
> 10.230.186.0/24 or net 10.230.24.0/24 or net 10.237.162.0/24 or net
> 10.230.22.0/24 or net 10.230.112.0/23 or net 10.230.120.0/24 or net
> 10.230.163.0/24 or net 10.230.17.0/24 or net 10.230.152.0/24 or host
> 224.0.0.252";
>        PacketFilter::install();
>        }
>
> With such a sizeable filter, bro does checkout OK (broctl check), and it
> starts, but the spool directory never receives any traffic files. All we
> get is -
>
> root at bro00:/var/spool/bro/bro# ls
> communication.log  stderr.log  stdout.log
>
> The stderr.log ends with -
>
> Warning: Kernel filter failed: Cannot allocate memory
> received termination signal
> 0 packets received on interface not open, 0 dropped
>
> If I reduce the filters to just a couple of subnets (no more than 6), it
> works just fine.
>
> Any ideas greatly appreciated.
>
> Andy
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181217/92e39dd1/attachment.html 

From andy at unimatrixzero.co.uk  Mon Dec 17 14:20:16 2018
From: andy at unimatrixzero.co.uk (Andy Millett)
Date: Mon, 17 Dec 2018 22:20:16 +0000
Subject: [Bro] BPF Syntax/Runtime Problem
In-Reply-To: <CADju=b5L1xO=V+AU0hXhPV517otabFicDbrAE+BBfdLkswkJ6Q@mail.gmail.com>
References: <791899D7-E022-40E0-8FA9-00C1FDF353F0@unimatrixzero.co.uk>
	<CADju=b5L1xO=V+AU0hXhPV517otabFicDbrAE+BBfdLkswkJ6Q@mail.gmail.com>
Message-ID: <D5B044F9-9848-4285-85FD-2CFADC545475@unimatrixzero.co.uk>

Hi Jim, 

Thanks a lot for the response. 

I removed the parentheses as suggested, and restarted the host itself. I do get a couple of files after the boot, one is the notice file - 

0.000000	-	-	-	-	-	-	-	-	-	PacketFilter::Install_Failure	Installing packet filter failed	(ip or not ip) and ((not (host 10.230.91.2) or (host 10.230.100.131)) and (not net 10.230.128.0/23 or net 10.230.64.0/23 or net 10.230.48.0/23 or net 10.230.130.0/23 or net 10.230.40.0/24 or net 10.230.237.0/24 or net 10.237.128.0/24 or net 10.230.108.0/24 or net 10.230.37.0/24 or net 10.230.38.0/24 or net 10.230.168.0/24 or net 10.230.72.0/24 or net 10.230.199.0/24 or net 10.230.177.0/24 or net 10.230.178.0/24 or net 10.230.179.0/24 or net 10.230.189.0/24 or net 10.230.183.0/24 or net 10.230.151.0/24 or net 10.230.165.0/24 or net 10.230.197.0/24 or net 10.230.167.0/24 or net 10.230.181.0/24 or net 10.230.31.0/24 or net 10.230.26.0/24 or net 10.230.180.0/24 or net 10.230.157.0/24 or net 10.230.159.0/24 or net 10.230.60.0/24 or net 10.230.150.0/24 or net 10.230.184.0/24 or net 10.230.202.0/24 or net 10.230.16.0/24 or net 10.230.156.0/24 or net 10.237.171.0/24 or net 10.230.76.0/24 or net 10.230.222.0/24 or net 10.230.186.0/24 or net 10.230.24.0/24 or net 10.237.162.0/24 or net 10.230.22.0/24 or net 10.230.112.0/23 or net 10.230.120.0/24 or net 10.230.163.0/24 or net 10.230.17.0/24 or net 10.230.152.0/24 or host 224.0.0.252))	-	-	-	-	bro	Notice::ACTION_LO3600.000000	F	-	-	-	-	-

If I try to run the filter in tcpdump, I get 

Warning: Kernel filter failed: Cannot allocate memory
tcpdump: can't remove kernel filter: No such file or directory

The stderr.log file logs the same -

Warning: Kernel filter failed: Cannot allocate memory

The server is a VM with 16GB memory. Nothing else running on it but Bro (based OS is Kali 2018). 

Best regards
Andy



> On 17 Dec 2018, at 20:50, Jim Mellander <jmellander at lbl.gov> wrote:
> 
> Hi:
> 
> I ran your filter on a local bro instance with no problems, although based on your description, shouldn't you have parentheses around the subnets in the restrict_filters["unmonitored nets"] expression? , i.e.
> 
> restrict_filters["unmonitored nets"] = "not (net 10.230.128.0/23 <http://10.230.128.0/23> or net 10.230.64.0/23 <http://10.230.64.0/23> or net 10.230.48.0/23 <http://10.230.48.0/23> or net 10.230.130.0/23 <http://10.230.130.0/23> or net 10.230.40.0/24 <http://10.230.40.0/24> or net 10.230.237.0/24 <http://10.230.237.0/24> or net 10.237.128.0/24 <http://10.237.128.0/24> or net 10.230.108.0/24 <http://10.230.108.0/24> or net 10.230.37.0/24 <http://10.230.37.0/24> or net 10.230.38.0/24 <http://10.230.38.0/24> or net 10.230.168.0/24 <http://10.230.168.0/24> or net 10.230.72.0/24 <http://10.230.72.0/24> or net 10.230.199.0/24 <http://10.230.199.0/24> or net 10.230.177.0/24 <http://10.230.177.0/24> or net 10.230.178.0/24 <http://10.230.178.0/24> or net 10.230.179.0/24 <http://10.230.179.0/24> or net 10.230.189.0/24 <http://10.230.189.0/24> or net 10.230.183.0/24 <http://10.230.183.0/24> or net 10.230.151.0/24 <http://10.230.151.0/24> or net 10.230.165.0/24 <http://10.230.165.0/24> or net 10.230.197.0/24 <http://10.230.197.0/24> or net 10.230.167.0/24 <http://10.230.167.0/24> or net 10.230.181.0/24 <http://10.230.181.0/24> or net 10.230.31.0/24 <http://10.230.31.0/24> or net 10.230.26.0/24 <http://10.230.26.0/24> or net 10.230.180.0/24 <http://10.230.180.0/24> or net 10.230.157.0/24 <http://10.230.157.0/24> or net 10.230.159.0/24 <http://10.230.159.0/24> or net 10.230.60.0/24 <http://10.230.60.0/24> or net 10.230.150.0/24 <http://10.230.150.0/24> or net 10.230.184.0/24 <http://10.230.184.0/24> or net 10.230.202.0/24 <http://10.230.202.0/24> or net 10.230.16.0/24 <http://10.230.16.0/24> or net 10.230.156.0/24 <http://10.230.156.0/24> or net 10.237.171.0/24 <http://10.237.171.0/24> or net 10.230.76.0/24 <http://10.230.76.0/24> or net 10.230.222.0/24 <http://10.230.222.0/24> or net 10.230.186.0/24 <http://10.230.186.0/24> or net 10.230.24.0/24 <http://10.230.24.0/24> or net 10.237.162.0/24 <http://10.237.162.0/24> or net 10.230.22.0/24 <http://10.230.22.0/24> or net 10.230.112.0/23 <http://10.230.112.0/23> or net 10.230.120.0/24 <http://10.230.120.0/24> or net 10.230.163.0/24 <http://10.230.163.0/24> or net 10.230.17.0/24 <http://10.230.17.0/24> or net 10.230.152.0/24 <http://10.230.152.0/24> or host 224.0.0.252)";
> 
> You might also take the filter in packet_filter.log and use that as the filter for a tcpdump and see if you are, in fact, capturing the traffic you expect.
> 
> Hope this helps,
> 
> Jim
> 
> 
> On Mon, Dec 17, 2018 at 6:37 AM Andy Millett <andy at unimatrixzero.co.uk <mailto:andy at unimatrixzero.co.uk>> wrote:
> Hi guys, 
> 
> We have a number of distributed Bro IDS sensors running on Raspberry Pi hardware at over 50 MPLS sites which are small or medium size links (anything up to 50Mbps). We have another 100 sites which don?t have sensors deployed (yet), so we?re trying to capture as much additional information for our ELK stack at the corporate HQ where most traffic goes. With this, I want to bypass logging of subnets which already have a remote sensor deployed to reduce duplication in ELK. I?ve been trying to use the BPF syntax, but don?t appear to be very successful. 
> 
> For starters, I?ve tried this - 
> 
> event bro_init() &priority=-12
>        {
>        restrict_filters["ignore proxy node"] = "not (host 10.230.91.2)";
> 	restrict_filters["unmonitored nets"] = "not net 10.230.128.0/23 <http://10.230.128.0/23> or net 10.230.64.0/23 <http://10.230.64.0/23> or net 10.230.48.0/23 <http://10.230.48.0/23> or net 10.230.130.0/23 <http://10.230.130.0/23> or net 10.230.40.0/24 <http://10.230.40.0/24> or net 10.230.237.0/24 <http://10.230.237.0/24> or net 10.237.128.0/24 <http://10.237.128.0/24> or net 10.230.108.0/24 <http://10.230.108.0/24> or net 10.230.37.0/24 <http://10.230.37.0/24> or net 10.230.38.0/24 <http://10.230.38.0/24> or net 10.230.168.0/24 <http://10.230.168.0/24> or net 10.230.72.0/24 <http://10.230.72.0/24> or net 10.230.199.0/24 <http://10.230.199.0/24> or net 10.230.177.0/24 <http://10.230.177.0/24> or net 10.230.178.0/24 <http://10.230.178.0/24> or net 10.230.179.0/24 <http://10.230.179.0/24> or net 10.230.189.0/24 <http://10.230.189.0/24> or net 10.230.183.0/24 <http://10.230.183.0/24> or net 10.230.151.0/24 <http://10.230.151.0/24> or net 10.230.165.0/24 <http://10.230.165.0/24> or net 10.230.197.0/24 <http://10.230.197.0/24> or net 10.230.167.0/24 <http://10.230.167.0/24> or net 10.230.181.0/24 <http://10.230.181.0/24> or net 10.230.31.0/24 <http://10.230.31.0/24> or net 10.230.26.0/24 <http://10.230.26.0/24> or net 10.230.180.0/24 <http://10.230.180.0/24> or net 10.230.157.0/24 <http://10.230.157.0/24> or net 10.230.159.0/24 <http://10.230.159.0/24> or net 10.230.60.0/24 <http://10.230.60.0/24> or net 10.230.150.0/24 <http://10.230.150.0/24> or net 10.230.184.0/24 <http://10.230.184.0/24> or net 10.230.202.0/24 <http://10.230.202.0/24> or net 10.230.16.0/24 <http://10.230.16.0/24> or net 10.230.156.0/24 <http://10.230.156.0/24> or net 10.237.171.0/24 <http://10.237.171.0/24> or net 10.230.76.0/24 <http://10.230.76.0/24> or net 10.230.222.0/24 <http://10.230.222.0/24> or net 10.230.186.0/24 <http://10.230.186.0/24> or net 10.230.24.0/24 <http://10.230.24.0/24> or net 10.237.162.0/24 <http://10.237.162.0/24> or net 10.230.22.0/24 <http://10.230.22.0/24> or net 10.230.112.0/23 <http://10.230.112.0/23> or net 10.230.120.0/24 <http://10.230.120.0/24> or net 10.230.163.0/24 <http://10.230.163.0/24> or net 10.230.17.0/24 <http://10.230.17.0/24> or net 10.230.152.0/24 <http://10.230.152.0/24> or host 224.0.0.252";
>        PacketFilter::install();
>        }
> 
> With such a sizeable filter, bro does checkout OK (broctl check), and it starts, but the spool directory never receives any traffic files. All we get is - 
> 
> root at bro00:/var/spool/bro/bro# ls
> communication.log  stderr.log  stdout.log
> 
> The stderr.log ends with - 
> 
> Warning: Kernel filter failed: Cannot allocate memory
> received termination signal
> 0 packets received on interface not open, 0 dropped
> 
> If I reduce the filters to just a couple of subnets (no more than 6), it works just fine. 
> 
> Any ideas greatly appreciated. 
> 
> Andy
> 
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org <mailto:bro at bro-ids.org>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro <http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181217/d9951045/attachment-0001.html 

From bill.de.ping at gmail.com  Mon Dec 17 23:00:50 2018
From: bill.de.ping at gmail.com (william de ping)
Date: Tue, 18 Dec 2018 09:00:50 +0200
Subject: [Bro] - recommended DB for Bro logs
In-Reply-To: <81868cbc-e251-82df-bf35-655dc63a44b2@alias454studios.com>
References: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
	<81868cbc-e251-82df-bf35-655dc63a44b2@alias454studios.com>
Message-ID: <CAKZA8xiG_b3fTs4y71eVqhS4DoKsxSorLfAR+srk_-rRNq3YBw@mail.gmail.com>

Thank you all for your suggestions !

I've decided to simultaneously deploy several solutions with the same
traffic and benchmark them in retrospect.
Candidates are oracle db, elk and splunk.
Since no writer exists for all of the above DB's, I will use the kafka
writer and use kafka queue as a middle man for each of the database
consumers.

I will update when results are in.
Feel free to respond with any further insights

B

On Mon, Dec 10, 2018 at 12:06 AM bkeep <bkeep at alias454studios.com> wrote:

> I've had some success using Graylog. I send BRO logs via rsyslog to a
> Graylog collector and utilize pipeline processing rules in Graylog for
> message enrichment. https://github.com/alias454/graylog-bro-content-pack.
> On 12/9/18 9:12 AM, william de ping wrote:
>
> Hi all,
>
> I would appreciate recommendations for a DB server that is most suited for
> ingesting and digesting Bro logs.
>
> I know of some use cases involving splunk and the Splunk Bro app, but
> price and performance wise (10GBps incoming traffic) it does not seem to be
> the best solution out there.
>
> Does anyone have any experience with Bro and  ElasticSearch | Redis |
> MySQL ?
>
> I am looking into different solutions and would appreciate your thoughts.
>
> Thanks in advance
> B
>
>
> _______________________________________________
> Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181218/a9773bfb/attachment.html 

From zeolla at gmail.com  Tue Dec 18 03:25:09 2018
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Tue, 18 Dec 2018 06:25:09 -0500
Subject: [Bro] - recommended DB for Bro logs
In-Reply-To: <CAKZA8xiG_b3fTs4y71eVqhS4DoKsxSorLfAR+srk_-rRNq3YBw@mail.gmail.com>
References: <CAKZA8xgeT2BpPUWsXDksGALRmkZFvWBDkVEP1_b0SeLsFqs0Xg@mail.gmail.com>
	<81868cbc-e251-82df-bf35-655dc63a44b2@alias454studios.com>
	<CAKZA8xiG_b3fTs4y71eVqhS4DoKsxSorLfAR+srk_-rRNq3YBw@mail.gmail.com>
Message-ID: <CAM1mTCrUD2j+cwPiw43qGKQKnkf_PztS2fv4XpG9uyLxKfU57w@mail.gmail.com>

Regarding the kafka plugin, please be aware of
https://github.com/apache/metron-bro-plugin-kafka/pull/20

The issue only happens on shutdown, and that PR currently fixes it but more
root cause analysis is needed.

That issue is in the latest release, 0.2.  master has some improved tests
and features, and once the above PR is merged we will be releasing 0.3.

Jon

On Tue, Dec 18, 2018, 2:08 AM william de ping <bill.de.ping at gmail.com>
wrote:

> Thank you all for your suggestions !
>
> I've decided to simultaneously deploy several solutions with the same
> traffic and benchmark them in retrospect.
> Candidates are oracle db, elk and splunk.
> Since no writer exists for all of the above DB's, I will use the kafka
> writer and use kafka queue as a middle man for each of the database
> consumers.
>
> I will update when results are in.
> Feel free to respond with any further insights
>
> B
>
> On Mon, Dec 10, 2018 at 12:06 AM bkeep <bkeep at alias454studios.com> wrote:
>
>> I've had some success using Graylog. I send BRO logs via rsyslog to a
>> Graylog collector and utilize pipeline processing rules in Graylog for
>> message enrichment. https://github.com/alias454/graylog-bro-content-pack.
>> On 12/9/18 9:12 AM, william de ping wrote:
>>
>> Hi all,
>>
>> I would appreciate recommendations for a DB server that is most suited
>> for ingesting and digesting Bro logs.
>>
>> I know of some use cases involving splunk and the Splunk Bro app, but
>> price and performance wise (10GBps incoming traffic) it does not seem to be
>> the best solution out there.
>>
>> Does anyone have any experience with Bro and  ElasticSearch | Redis |
>> MySQL ?
>>
>> I am looking into different solutions and would appreciate your thoughts.
>>
>> Thanks in advance
>> B
>>
>>
>> _______________________________________________
>> Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-- 

Jon Zeolla
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181218/923f20f0/attachment.html 

From jmellander at lbl.gov  Tue Dec 18 11:20:10 2018
From: jmellander at lbl.gov (Jim Mellander)
Date: Tue, 18 Dec 2018 11:20:10 -0800
Subject: [Bro] BPF Syntax/Runtime Problem
In-Reply-To: <D5B044F9-9848-4285-85FD-2CFADC545475@unimatrixzero.co.uk>
References: <791899D7-E022-40E0-8FA9-00C1FDF353F0@unimatrixzero.co.uk>
	<CADju=b5L1xO=V+AU0hXhPV517otabFicDbrAE+BBfdLkswkJ6Q@mail.gmail.com>
	<D5B044F9-9848-4285-85FD-2CFADC545475@unimatrixzero.co.uk>
Message-ID: <CADju=b5aRftpE4LFETgTVHGvrjEHh_kbBdUx81mEoe9+4nSDgg@mail.gmail.com>

Seems likely that the common denominator between bro & tcpdump is your
libpcap library.  Has that been updated?  Alternatively, you could try
compiling and linking against the latest libpcap from tcpdump.org.  Could
also be some sort of kernel issue, although that seems unlikely.

See https://seclists.org/tcpdump/2008/q4/180 for further info on the error
message

Hope this helps

On Mon, Dec 17, 2018 at 2:20 PM Andy Millett <andy at unimatrixzero.co.uk>
wrote:

> Hi Jim,
>
> Thanks a lot for the response.
>
> I removed the parentheses as suggested, and restarted the host itself. I
> do get a couple of files after the boot, one is the notice file -
>
> 0.000000 - - - - - - - - - PacketFilter::Install_Failure Installing
> packet filter failed (ip or not ip) and ((not (host 10.230.91.2) or (host
> 10.230.100.131)) and (not net 10.230.128.0/23 or net 10.230.64.0/23 or
> net 10.230.48.0/23 or net 10.230.130.0/23 or net 10.230.40.0/24 or net
> 10.230.237.0/24 or net 10.237.128.0/24 or net 10.230.108.0/24 or net
> 10.230.37.0/24 or net 10.230.38.0/24 or net 10.230.168.0/24 or net
> 10.230.72.0/24 or net 10.230.199.0/24 or net 10.230.177.0/24 or net
> 10.230.178.0/24 or net 10.230.179.0/24 or net 10.230.189.0/24 or net
> 10.230.183.0/24 or net 10.230.151.0/24 or net 10.230.165.0/24 or net
> 10.230.197.0/24 or net 10.230.167.0/24 or net 10.230.181.0/24 or net
> 10.230.31.0/24 or net 10.230.26.0/24 or net 10.230.180.0/24 or net
> 10.230.157.0/24 or net 10.230.159.0/24 or net 10.230.60.0/24 or net
> 10.230.150.0/24 or net 10.230.184.0/24 or net 10.230.202.0/24 or net
> 10.230.16.0/24 or net 10.230.156.0/24 or net 10.237.171.0/24 or net
> 10.230.76.0/24 or net 10.230.222.0/24 or net 10.230.186.0/24 or net
> 10.230.24.0/24 or net 10.237.162.0/24 or net 10.230.22.0/24 or net
> 10.230.112.0/23 or net 10.230.120.0/24 or net 10.230.163.0/24 or net
> 10.230.17.0/24 or net 10.230.152.0/24 or host 224.0.0.252)) - - - - bro
> Notice::ACTION_LO3600.000000 F - - - - -
>
> If I try to run the filter in tcpdump, I get
>
> Warning: Kernel filter failed: Cannot allocate memory
> tcpdump: can't remove kernel filter: No such file or directory
>
> The stderr.log file logs the same -
>
> Warning: Kernel filter failed: Cannot allocate memory
>
> The server is a VM with 16GB memory. Nothing else running on it but Bro
> (based OS is Kali 2018).
>
> Best regards
> Andy
>
>
>
> On 17 Dec 2018, at 20:50, Jim Mellander <jmellander at lbl.gov> wrote:
>
> Hi:
>
> I ran your filter on a local bro instance with no problems, although based
> on your description, shouldn't you have parentheses around the subnets in
> the restrict_filters["unmonitored nets"] expression? , i.e.
>
> restrict_filters["unmonitored nets"] = "not (net 10.230.128.0/23 or net
> 10.230.64.0/23 or net 10.230.48.0/23 or net 10.230.130.0/23 or net
> 10.230.40.0/24 or net 10.230.237.0/24 or net 10.237.128.0/24 or net
> 10.230.108.0/24 or net 10.230.37.0/24 or net 10.230.38.0/24 or net
> 10.230.168.0/24 or net 10.230.72.0/24 or net 10.230.199.0/24 or net
> 10.230.177.0/24 or net 10.230.178.0/24 or net 10.230.179.0/24 or net
> 10.230.189.0/24 or net 10.230.183.0/24 or net 10.230.151.0/24 or net
> 10.230.165.0/24 or net 10.230.197.0/24 or net 10.230.167.0/24 or net
> 10.230.181.0/24 or net 10.230.31.0/24 or net 10.230.26.0/24 or net
> 10.230.180.0/24 or net 10.230.157.0/24 or net 10.230.159.0/24 or net
> 10.230.60.0/24 or net 10.230.150.0/24 or net 10.230.184.0/24 or net
> 10.230.202.0/24 or net 10.230.16.0/24 or net 10.230.156.0/24 or net
> 10.237.171.0/24 or net 10.230.76.0/24 or net 10.230.222.0/24 or net
> 10.230.186.0/24 or net 10.230.24.0/24 or net 10.237.162.0/24 or net
> 10.230.22.0/24 or net 10.230.112.0/23 or net 10.230.120.0/24 or net
> 10.230.163.0/24 or net 10.230.17.0/24 or net 10.230.152.0/24 or host
> 224.0.0.252)";
>
> You might also take the filter in packet_filter.log and use that as the
> filter for a tcpdump and see if you are, in fact, capturing the traffic you
> expect.
>
> Hope this helps,
>
> Jim
>
>
> On Mon, Dec 17, 2018 at 6:37 AM Andy Millett <andy at unimatrixzero.co.uk>
> wrote:
>
>> Hi guys,
>>
>> We have a number of distributed Bro IDS sensors running on Raspberry Pi
>> hardware at over 50 MPLS sites which are small or medium size links
>> (anything up to 50Mbps). We have another 100 sites which don?t have sensors
>> deployed (yet), so we?re trying to capture as much additional information
>> for our ELK stack at the corporate HQ where most traffic goes. With this, I
>> want to bypass logging of subnets which already have a remote sensor
>> deployed to reduce duplication in ELK. I?ve been trying to use the BPF
>> syntax, but don?t appear to be very successful.
>>
>> For starters, I?ve tried this -
>>
>> event bro_init() &priority=-12
>>        {
>>        restrict_filters["ignore proxy node"] = "not (host 10.230.91.2)";
>> restrict_filters["unmonitored nets"] = "not net 10.230.128.0/23 or net
>> 10.230.64.0/23 or net 10.230.48.0/23 or net 10.230.130.0/23 or net
>> 10.230.40.0/24 or net 10.230.237.0/24 or net 10.237.128.0/24 or net
>> 10.230.108.0/24 or net 10.230.37.0/24 or net 10.230.38.0/24 or net
>> 10.230.168.0/24 or net 10.230.72.0/24 or net 10.230.199.0/24 or net
>> 10.230.177.0/24 or net 10.230.178.0/24 or net 10.230.179.0/24 or net
>> 10.230.189.0/24 or net 10.230.183.0/24 or net 10.230.151.0/24 or net
>> 10.230.165.0/24 or net 10.230.197.0/24 or net 10.230.167.0/24 or net
>> 10.230.181.0/24 or net 10.230.31.0/24 or net 10.230.26.0/24 or net
>> 10.230.180.0/24 or net 10.230.157.0/24 or net 10.230.159.0/24 or net
>> 10.230.60.0/24 or net 10.230.150.0/24 or net 10.230.184.0/24 or net
>> 10.230.202.0/24 or net 10.230.16.0/24 or net 10.230.156.0/24 or net
>> 10.237.171.0/24 or net 10.230.76.0/24 or net 10.230.222.0/24 or net
>> 10.230.186.0/24 or net 10.230.24.0/24 or net 10.237.162.0/24 or net
>> 10.230.22.0/24 or net 10.230.112.0/23 or net 10.230.120.0/24 or net
>> 10.230.163.0/24 or net 10.230.17.0/24 or net 10.230.152.0/24 or host
>> 224.0.0.252";
>>        PacketFilter::install();
>>        }
>>
>> With such a sizeable filter, bro does checkout OK (broctl check), and it
>> starts, but the spool directory never receives any traffic files. All we
>> get is -
>>
>> root at bro00:/var/spool/bro/bro# ls
>> communication.log  stderr.log  stdout.log
>>
>> The stderr.log ends with -
>>
>> Warning: Kernel filter failed: Cannot allocate memory
>> received termination signal
>> 0 packets received on interface not open, 0 dropped
>>
>> If I reduce the filters to just a couple of subnets (no more than 6), it
>> works just fine.
>>
>> Any ideas greatly appreciated.
>>
>> Andy
>>
>>
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> <http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181218/a3f9e23e/attachment-0001.html 

From hovsep.sanjay.levi at gmail.com  Tue Dec 18 13:18:36 2018
From: hovsep.sanjay.levi at gmail.com (Hovsep Levi)
Date: Tue, 18 Dec 2018 21:18:36 +0000
Subject: [Bro] Bro logs - enable_local_logging and remove_default_filter
In-Reply-To: <20181213215108.x27ucegishqprnlx@user159.sys.ICSI.Berkeley.EDU>
References: <CAJ19UZB_SSvRzTtMEFJvf4tX+d1fdp9=LRK_=CtXaiBUmz8k6w@mail.gmail.com>
	<20181213215108.x27ucegishqprnlx@user159.sys.ICSI.Berkeley.EDU>
Message-ID: <CAJ19UZB4WP2qYnBppQDoj7p3fxBrECvHgarX728M1qNQRTFiYQ@mail.gmail.com>

On Thu, Dec 13, 2018 at 9:51 PM Johanna Amann <johanna at icir.org> wrote:

>
> I just looked and I did not really see any big way in which this changed.
> Could you perhaps provide a code-snippet that does not work anymore?
>
>
I modify the KafkaLogger script (logs-to-kafka.bro) and add
Log::remove_default_filter before the call to Log::add_filter.



> I also just tried a minimal example script and Log::remove_default_filter
> seems to work as expected.
>
>
It works for some of the logs except:

ls -l bro/logs/current/
total 511992
-rw-r--r--  1 bro  bro    1032325 Dec 18 07:15 broker.log
-rw-r--r--  1 bro  bro  666385163 Dec 18 07:15 conn.log
-rw-r--r--  1 bro  bro      12994 Dec 18 07:15 dce_rpc.log
-rw-r--r--  1 bro  bro  223181005 Dec 18 07:15 files.log
-rw-r--r--  1 bro  bro       5780 Dec 18 07:15 smb_files.log
-rw-r--r--  1 bro  bro       3283 Dec 18 07:15 smb_mapping.log
-rw-r--r--  1 bro  bro    5077483 Dec 18 07:15 stderr.log
-rw-r--r--  1 bro  bro        187 Dec 13 14:45 stdout.log



[...]
>
> > But when I try to set Log::enable_local_logging=0 within the KafkaLogger
> > plugin loop for each log I get an error.
>
> This is probably a misunderstanding. Log::enable_local_logging is not a
> per-log setting - so there is nothing to loop over.
>
>
>
Ok, thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181218/e193b2dd/attachment.html 

From johanna at icir.org  Wed Dec 19 04:38:42 2018
From: johanna at icir.org (Johanna Amann)
Date: Wed, 19 Dec 2018 13:38:42 +0100
Subject: [Bro] Zeek mailing list renaming and outage
Message-ID: <20181219123842.eizalnmhjjwdm5tp@Tranquility.fritz.box>

Hello everyone,

As part of renaming Bro to Zeek, we are going to change the addresses and
names of all our mailing lists.

The important thing first: this change is going to happen tomorrow
(Thursday), 12/20. The mailing list rename will happen during a system
maintenance window at ICSI (which is hosting the mailing list). Thus the
mailing lists (including the archives) will be unavailable for a few hours
during the day on Thursday. Mails sent to the lists during this time
should be queued and delivered when the system is back up.

The current Bro mailing lists will be renamed as follows:
bro at bro.org -> zeek at zeek.org
bro-dev at bro.org -> zeek-dev at zeek.org
bro-commits at bro.org -> zeek-commits at zeek.org
bro-announce at bro.org -> zeek-announce at zeek.org

Similarly we will change the mailing list subject tags from Bro to Zeek.

There will be redirects from the old mailing list names to the new ones,
so mails sent to the old addresses will not be lost.

Johanna

From jsiwek at corelight.com  Wed Dec 19 10:24:28 2018
From: jsiwek at corelight.com (Jon Siwek)
Date: Wed, 19 Dec 2018 12:24:28 -0600
Subject: [Bro] Bro 2.6.1 release
Message-ID: <CAMzgZ0Lkvd7=1QgHA-RVFiBYbLi2eRYPTiowOrQW_2iUnFgAzw@mail.gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bro v2.6.1 is available for download:

    https://www.zeek.org/download/index.html
    https://www.zeek.org/downloads/bro-2.6.1.tar.gz

This release updates the embedded SQLite to version 3.26.0 to
address the "Magellan" remote code execution vulnerability.  The
stock Bro configuration/scripts don't use SQLite by default, but
custom user scripts/packages may.

This release also updates Broker to v1.1.2, which includes a
minor bug fix in its Python bindings and improved support for
building it as a static library.
-----BEGIN PGP SIGNATURE-----

iQIcBAEBAgAGBQJcGowAAAoJEKfUHOR63zbz3EUP/iVCvDHFPwl/i8QQUQUTz7gQ
bdyE95USUd4J7+tZEnd3f0yoqTDmFGxWfioLlr0U+Qpz7K8fgEb7lnSIin7O4vb+
nuPM3fRrl3uh8P0PjALWvmxqS7LQMiJOV72XCoBF/2illWny8G/57inkeaoqYeYl
/bHMoS7xPb3BZVXjj/v2aLMXvyeDY9Xv3cqzmKEiKHAE09WxMUNFsWHQhyLqLunU
0bJqSRjQwy3nMDq9lUGUXdqXiVILufkVN7kXu8WgjTn2Cis1D8ZRDNhYSKW1PFQA
kCKqxXjuBe97MAYlls/nv7IadivJx22h7A/sogjxJh7oTmjyDC4UKWnjH09zWjdh
UYfS51D1bdDDv4yFLgCXGPkg02cUIRAO7w12XCfgTe1g0hoJOcl4faBWvNa3xBZV
z5Qge+1mrW/k5MEtCSFnRPGvxD7SeZ7Dj+9PQcA9wY2iO6YNKQz7DYZjb8i6Gnaf
L+4yO51B2qasUQICW0sZiYkg5LU847DyGrcfTE4z9ImlsSwpIxagbfG/3zxtIAow
vpF5Su2g1/C9jXxJJovcthWf/HM3/VaBWwDpd8K7zmIssbMbd/apQvwitnMdM6y2
GJY9i7LPgSJTCkRyLrE1jvKBbuF3225VUfq1n4dwT6EByyJ9TxLgHfFA/gHYiG8Z
skDvIlyix1XUZl5UpLLI
=TFfn
-----END PGP SIGNATURE-----

From jlay at slave-tothe-box.net  Wed Dec 19 11:37:23 2018
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 19 Dec 2018 12:37:23 -0700
Subject: [Bro] Bro 2.6.1 release
In-Reply-To: <CAMzgZ0Lkvd7=1QgHA-RVFiBYbLi2eRYPTiowOrQW_2iUnFgAzw@mail.gmail.com>
References: <CAMzgZ0Lkvd7=1QgHA-RVFiBYbLi2eRYPTiowOrQW_2iUnFgAzw@mail.gmail.com>
Message-ID: <ccea4a650c59b947d38b12d110424fea@slave-tothe-box.net>

On 2018-12-19 11:24, Jon Siwek wrote:
> Bro v2.6.1 is available for download:
> 
>     https://www.zeek.org/download/index.html
>     https://www.zeek.org/downloads/bro-2.6.1.tar.gz
> 
> This release updates the embedded SQLite to version 3.26.0 to
> address the "Magellan" remote code execution vulnerability.  The
> stock Bro configuration/scripts don't use SQLite by default, but
> custom user scripts/packages may.
> 
> This release also updates Broker to v1.1.2, which includes a
> minor bug fix in its Python bindings and improved support for
> building it as a static library.

And the first casualty:

bro-af_packet-plugin

fatal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1: 
cannot load plugin library 
/usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so: 
/usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so: 
undefined symbol: bro_version_2_6_plugin_6

bro-pkg upgrade bro-af_packet-plugin
All packages already up-to-date.

Now what :)

James

From shirkdog.bsd at gmail.com  Wed Dec 19 12:06:00 2018
From: shirkdog.bsd at gmail.com (Michael Shirk)
Date: Wed, 19 Dec 2018 15:06:00 -0500
Subject: [Bro] Bro 2.6.1 release
In-Reply-To: <ccea4a650c59b947d38b12d110424fea@slave-tothe-box.net>
References: <CAMzgZ0Lkvd7=1QgHA-RVFiBYbLi2eRYPTiowOrQW_2iUnFgAzw@mail.gmail.com>
	<ccea4a650c59b947d38b12d110424fea@slave-tothe-box.net>
Message-ID: <CAL8PkUWG_bwVHsH=uHg6YQCh08ApA5gSLMaY0d562kRWkQbhdA@mail.gmail.com>

You need to rebuild the package/plugin, I think this was just added to
the zeek package manager...or will be soon

On Wed, Dec 19, 2018 at 2:58 PM James Lay <jlay at slave-tothe-box.net> wrote:
>
> On 2018-12-19 11:24, Jon Siwek wrote:
> > Bro v2.6.1 is available for download:
> >
> >     https://www.zeek.org/download/index.html
> >     https://www.zeek.org/downloads/bro-2.6.1.tar.gz
> >
> > This release updates the embedded SQLite to version 3.26.0 to
> > address the "Magellan" remote code execution vulnerability.  The
> > stock Bro configuration/scripts don't use SQLite by default, but
> > custom user scripts/packages may.
> >
> > This release also updates Broker to v1.1.2, which includes a
> > minor bug fix in its Python bindings and improved support for
> > building it as a static library.
>
> And the first casualty:
>
> bro-af_packet-plugin
>
> fatal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1:
> cannot load plugin library
> /usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so:
> /usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so:
> undefined symbol: bro_version_2_6_plugin_6
>
> bro-pkg upgrade bro-af_packet-plugin
> All packages already up-to-date.
>
> Now what :)
>
> James
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



-- 
Michael Shirk
Daemon Security, Inc.
https://www.daemon-security.com

From jlay at slave-tothe-box.net  Wed Dec 19 12:11:18 2018
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 19 Dec 2018 13:11:18 -0700
Subject: [Bro] Bro 2.6.1 release
In-Reply-To: <CAL8PkUWG_bwVHsH=uHg6YQCh08ApA5gSLMaY0d562kRWkQbhdA@mail.gmail.com>
References: <CAMzgZ0Lkvd7=1QgHA-RVFiBYbLi2eRYPTiowOrQW_2iUnFgAzw@mail.gmail.com>
	<ccea4a650c59b947d38b12d110424fea@slave-tothe-box.net>
	<CAL8PkUWG_bwVHsH=uHg6YQCh08ApA5gSLMaY0d562kRWkQbhdA@mail.gmail.com>
Message-ID: <1406f99ccb3c15b94951e36bff964287@slave-tothe-box.net>

On 2018-12-19 13:06, Michael Shirk wrote:
> You need to rebuild the package/plugin, I think this was just added to
> the zeek package manager...or will be soon
> 
> On Wed, Dec 19, 2018 at 2:58 PM James Lay <jlay at slave-tothe-box.net> 
> wrote:
>> 
>> On 2018-12-19 11:24, Jon Siwek wrote:
>> > Bro v2.6.1 is available for download:
>> >
>> >     https://www.zeek.org/download/index.html
>> >     https://www.zeek.org/downloads/bro-2.6.1.tar.gz
>> >
>> > This release updates the embedded SQLite to version 3.26.0 to
>> > address the "Magellan" remote code execution vulnerability.  The
>> > stock Bro configuration/scripts don't use SQLite by default, but
>> > custom user scripts/packages may.
>> >
>> > This release also updates Broker to v1.1.2, which includes a
>> > minor bug fix in its Python bindings and improved support for
>> > building it as a static library.
>> 
>> And the first casualty:
>> 
>> bro-af_packet-plugin
>> 
>> fatal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1:
>> cannot load plugin library
>> /usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so:
>> /usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so:
>> undefined symbol: bro_version_2_6_plugin_6
>> 
>> bro-pkg upgrade bro-af_packet-plugin
>> All packages already up-to-date.
>> 
>> Now what :)
>> 
>> James


Ok...so...this was installed using the spiffy bro-pkg, so "rebuilding" 
isn't an option if I intend to stick with bro-pkg.  If plugins aren't 
going to keep in sync with the core app proper then that might be an 
issue (especially in my case it looks like).

James

From jsiwek at corelight.com  Wed Dec 19 12:17:58 2018
From: jsiwek at corelight.com (Jon Siwek)
Date: Wed, 19 Dec 2018 14:17:58 -0600
Subject: [Bro] Bro 2.6.1 release
In-Reply-To: <ccea4a650c59b947d38b12d110424fea@slave-tothe-box.net>
References: <CAMzgZ0Lkvd7=1QgHA-RVFiBYbLi2eRYPTiowOrQW_2iUnFgAzw@mail.gmail.com>
	<ccea4a650c59b947d38b12d110424fea@slave-tothe-box.net>
Message-ID: <CAMzgZ0KJHh6v=bj+wihr6vKYGWee81aaXa1F6qDxq4WtPTV6WA@mail.gmail.com>

On Wed, Dec 19, 2018 at 1:58 PM James Lay <jlay at slave-tothe-box.net> wrote:

> And the first casualty:
>
> bro-af_packet-plugin
>
> fatal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1:
> cannot load plugin library
> /usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so:
> /usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so:
> undefined symbol: bro_version_2_6_plugin_6
>
> bro-pkg upgrade bro-af_packet-plugin
> All packages already up-to-date.

Try re-compiling/installing the plugin/package.

Plugins currently get compiled such that they reference a specific Bro
(plugin API) version and only linking against that version of Bro
provides it.  i.e. once compiled, the plugin only works against a
specific Bro version

- Jon

From dopheide at gmail.com  Wed Dec 19 12:35:53 2018
From: dopheide at gmail.com (Mike Dopheide)
Date: Wed, 19 Dec 2018 14:35:53 -0600
Subject: [Bro] Bro 2.6.1 release
In-Reply-To: <1406f99ccb3c15b94951e36bff964287@slave-tothe-box.net>
References: <CAMzgZ0Lkvd7=1QgHA-RVFiBYbLi2eRYPTiowOrQW_2iUnFgAzw@mail.gmail.com>
	<ccea4a650c59b947d38b12d110424fea@slave-tothe-box.net>
	<CAL8PkUWG_bwVHsH=uHg6YQCh08ApA5gSLMaY0d562kRWkQbhdA@mail.gmail.com>
	<1406f99ccb3c15b94951e36bff964287@slave-tothe-box.net>
Message-ID: <CAPy2kFYmPexoq56XC28H7ghHsJCts=9JuUWuCcDuXqKADb+XWA@mail.gmail.com>

Installing a bro-pkg that is a plugin does rebuild it.  They aren't
distributed as binaries.

-Dop

On Wed, Dec 19, 2018 at 2:32 PM James Lay <jlay at slave-tothe-box.net> wrote:

> On 2018-12-19 13:06, Michael Shirk wrote:
> > You need to rebuild the package/plugin, I think this was just added to
> > the zeek package manager...or will be soon
> >
> > On Wed, Dec 19, 2018 at 2:58 PM James Lay <jlay at slave-tothe-box.net>
> > wrote:
> >>
> >> On 2018-12-19 11:24, Jon Siwek wrote:
> >> > Bro v2.6.1 is available for download:
> >> >
> >> >     https://www.zeek.org/download/index.html
> >> >     https://www.zeek.org/downloads/bro-2.6.1.tar.gz
> >> >
> >> > This release updates the embedded SQLite to version 3.26.0 to
> >> > address the "Magellan" remote code execution vulnerability.  The
> >> > stock Bro configuration/scripts don't use SQLite by default, but
> >> > custom user scripts/packages may.
> >> >
> >> > This release also updates Broker to v1.1.2, which includes a
> >> > minor bug fix in its Python bindings and improved support for
> >> > building it as a static library.
> >>
> >> And the first casualty:
> >>
> >> bro-af_packet-plugin
> >>
> >> fatal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1:
> >> cannot load plugin library
> >> /usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/
> Bro-AF_Packet.linux-x86_64.so:
> >> /usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/
> Bro-AF_Packet.linux-x86_64.so:
> >> undefined symbol: bro_version_2_6_plugin_6
> >>
> >> bro-pkg upgrade bro-af_packet-plugin
> >> All packages already up-to-date.
> >>
> >> Now what :)
> >>
> >> James
>
>
> Ok...so...this was installed using the spiffy bro-pkg, so "rebuilding"
> isn't an option if I intend to stick with bro-pkg.  If plugins aren't
> going to keep in sync with the core app proper then that might be an
> issue (especially in my case it looks like).
>
> James
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181219/c6c9a942/attachment.html 

From jlay at slave-tothe-box.net  Wed Dec 19 12:42:28 2018
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 19 Dec 2018 13:42:28 -0700
Subject: [Bro] Bro 2.6.1 release
In-Reply-To: <CAPy2kFYmPexoq56XC28H7ghHsJCts=9JuUWuCcDuXqKADb+XWA@mail.gmail.com>
References: <CAMzgZ0Lkvd7=1QgHA-RVFiBYbLi2eRYPTiowOrQW_2iUnFgAzw@mail.gmail.com>
	<ccea4a650c59b947d38b12d110424fea@slave-tothe-box.net>
	<CAL8PkUWG_bwVHsH=uHg6YQCh08ApA5gSLMaY0d562kRWkQbhdA@mail.gmail.com>
	<1406f99ccb3c15b94951e36bff964287@slave-tothe-box.net>
	<CAPy2kFYmPexoq56XC28H7ghHsJCts=9JuUWuCcDuXqKADb+XWA@mail.gmail.com>
Message-ID: <fdd5cd415bc7894a422090fcc74dcc41@slave-tothe-box.net>

On 2018-12-19 13:35, Mike Dopheide wrote:
> Installing a bro-pkg that is a plugin does rebuild it.  They aren't
> distributed as binaries.
> 
> -Dop

Ah....I did not know that.  So in the future for folks (I've already git 
cloned and compiled) a bro-pkg remove/bro-pkg install does the 
trick...good to know thank you.

James

> 
> On Wed, Dec 19, 2018 at 2:32 PM James Lay <jlay at slave-tothe-box.net>
> wrote:
> 
>> On 2018-12-19 13:06, Michael Shirk wrote:
>>> You need to rebuild the package/plugin, I think this was just
>> added to
>>> the zeek package manager...or will be soon
>>> 
>>> On Wed, Dec 19, 2018 at 2:58 PM James Lay
>> <jlay at slave-tothe-box.net>
>>> wrote:
>>>> 
>>>> On 2018-12-19 11:24, Jon Siwek wrote:
>>>>> Bro v2.6.1 is available for download:
>>>>> 
>>>>>     https://www.zeek.org/download/index.html
>>>>>     https://www.zeek.org/downloads/bro-2.6.1.tar.gz
>>>>> 
>>>>> This release updates the embedded SQLite to version 3.26.0 to
>>>>> address the "Magellan" remote code execution vulnerability.
>> The
>>>>> stock Bro configuration/scripts don't use SQLite by default,
>> but
>>>>> custom user scripts/packages may.
>>>>> 
>>>>> This release also updates Broker to v1.1.2, which includes a
>>>>> minor bug fix in its Python bindings and improved support for
>>>>> building it as a static library.
>>>> 
>>>> And the first casualty:
>>>> 
>>>> bro-af_packet-plugin
>>>> 
>>>> fatal error in /usr/local/bro/share/bro/base/init-bare.bro, line
>> 1:
>>>> cannot load plugin library
>>>> 
>> 
> /usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so
>> [1]:
>>>> 
>> 
> /usr/local/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so
>> [1]:
>>>> undefined symbol: bro_version_2_6_plugin_6
>>>> 
>>>> bro-pkg upgrade bro-af_packet-plugin
>>>> All packages already up-to-date.
>>>> 
>>>> Now what :)
>>>> 
>>>> James
>> 
>> Ok...so...this was installed using the spiffy bro-pkg, so
>> "rebuilding"
>> isn't an option if I intend to stick with bro-pkg.  If plugins
>> aren't
>> going to keep in sync with the core app proper then that might be an
>> 
>> issue (especially in my case it looks like).
>> 
>> James
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> 
> Links:
> ------
> [1] http://Bro-AF_Packet.linux-x86_64.so

From jsiwek at corelight.com  Wed Dec 19 12:47:31 2018
From: jsiwek at corelight.com (Jon Siwek)
Date: Wed, 19 Dec 2018 14:47:31 -0600
Subject: [Bro] Bro 2.6.1 release
In-Reply-To: <1406f99ccb3c15b94951e36bff964287@slave-tothe-box.net>
References: <CAMzgZ0Lkvd7=1QgHA-RVFiBYbLi2eRYPTiowOrQW_2iUnFgAzw@mail.gmail.com>
	<ccea4a650c59b947d38b12d110424fea@slave-tothe-box.net>
	<CAL8PkUWG_bwVHsH=uHg6YQCh08ApA5gSLMaY0d562kRWkQbhdA@mail.gmail.com>
	<1406f99ccb3c15b94951e36bff964287@slave-tothe-box.net>
Message-ID: <CAMzgZ0+Ns1VUn6pZBM_GRnk68P6D5Q2tuG1swSCGb7tMU=naWg@mail.gmail.com>

On Wed, Dec 19, 2018 at 2:32 PM James Lay <jlay at slave-tothe-box.net> wrote:

> Ok...so...this was installed using the spiffy bro-pkg, so "rebuilding"
> isn't an option if I intend to stick with bro-pkg.

Why is it not an option?  There isn't a "rebuild" command (yet), but
the alternative given at [1] should be equivalent AFAIK, just in two
separate commands:

    bro-pkg bundle my.bundle && bro-pkg unbundle my.bundle

- Jon

[1] https://github.com/zeek/package-manager/issues/38

From jlay at slave-tothe-box.net  Wed Dec 19 13:10:25 2018
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 19 Dec 2018 14:10:25 -0700
Subject: [Bro] Bro 2.6.1 release
In-Reply-To: <CAMzgZ0+Ns1VUn6pZBM_GRnk68P6D5Q2tuG1swSCGb7tMU=naWg@mail.gmail.com>
References: <CAMzgZ0Lkvd7=1QgHA-RVFiBYbLi2eRYPTiowOrQW_2iUnFgAzw@mail.gmail.com>
	<ccea4a650c59b947d38b12d110424fea@slave-tothe-box.net>
	<CAL8PkUWG_bwVHsH=uHg6YQCh08ApA5gSLMaY0d562kRWkQbhdA@mail.gmail.com>
	<1406f99ccb3c15b94951e36bff964287@slave-tothe-box.net>
	<CAMzgZ0+Ns1VUn6pZBM_GRnk68P6D5Q2tuG1swSCGb7tMU=naWg@mail.gmail.com>
Message-ID: <90b77251bb2875d91ff98c8626a036d5@slave-tothe-box.net>

On 2018-12-19 13:47, Jon Siwek wrote:
> On Wed, Dec 19, 2018 at 2:32 PM James Lay <jlay at slave-tothe-box.net> 
> wrote:
> 
>> Ok...so...this was installed using the spiffy bro-pkg, so "rebuilding"
>> isn't an option if I intend to stick with bro-pkg.
> 
> Why is it not an option?  There isn't a "rebuild" command (yet), but
> the alternative given at [1] should be equivalent AFAIK, just in two
> separate commands:
> 
>     bro-pkg bundle my.bundle && bro-pkg unbundle my.bundle
> 
> - Jon
> 
> [1] https://github.com/zeek/package-manager/issues/38

Bleh...lemme start a new thread for this thanks all.

James

From Robert.Cotter at endace.com  Thu Dec 20 13:35:23 2018
From: Robert.Cotter at endace.com (Robert Cotter)
Date: Thu, 20 Dec 2018 21:35:23 +0000
Subject: [Zeek] Timetable for RPM release for 2.6.x
Message-ID: <e70f0f3a16f1443099730c9a5d337e6c@endace.com>

Is there a timetable available yet for the release of RPM's zeek/bro's latest release via the repo ?


Regards

Robert Cotter
Sales Engineer -  APAC Region


http://www.endace.com/


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20181220/2e315d56/attachment.html 

From jlay at slave-tothe-box.net  Thu Dec 20 12:55:40 2018
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 20 Dec 2018 13:55:40 -0700
Subject: [Zeek] Bro upgrades, and plugins vs. packages, and bro-pkg
Message-ID: <05d972a78319a2939a1b01cc9f2ce8f3@slave-tothe-box.net>

So here we go.  I've attacked this with my lab and here are some 
thoughts/results.  Current state:

bro-2.6 installed from source (config option --prefix=/opt/bro)
bro-af_packet-plugin ja3 intel-seen-more domain-tld installed via 
bro-pkg

after upgrading to bro-2.6.1 errors like the below:

fatal error in /opt/bro/share/bro/base/init-bare.bro, line 1: cannot 
load plugin library 
/opt/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so: 
/opt/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so: 
undefined symbol: bro_version_2_6_plugin_6

next up, remove and reinstall bro-af_backup-plugin:

root@# bro-pkg remove bro-af_packet-plugin
The following packages will be REMOVED:
   bro/j-gras/bro-af_packet-plugin

Proceed? [Y/n] y
Removed "bro/j-gras/bro-af_packet-plugin"
root@# bro-pkg install bro-af_packet-plugin
The following packages will be INSTALLED:
   bro/j-gras/bro-af_packet-plugin (1.3.0)

Proceed? [Y/n] y
Running unit tests for "bro/j-gras/bro-af_packet-plugin"
[  0%] scripts.show-plugin ... failed
   % 'bro -NN Bro::AF_Packet > output' failed unexpectedly (exit code 1)
   % cat .stderr
   fatal error in /opt/bro/share/bro/base/init-bare.bro, line 1: cannot 
load plugin library 
/root/.bro-pkg/testing/bro-af_packet-plugin/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so: 
/root/.bro-pkg/testing/bro-af_packet-plugin/plugins/packages/bro-af_packet-plugin//lib/Bro-AF_Packet.linux-x86_64.so: 
undefined symbol: bro_version_2_6_plugin_6

1 of 1 test failed
error: bro/j-gras/bro-af_packet-plugin tests failed, inspect contents of 
/root/.bro-pkg/testing/bro-af_packet-plugin for details
Proceed to install anyway? [N/y] n
Abort.

a thought occurs....modify /root/.bro-pkg/config -> bro_dist = 
/build/bro-2.6.1

all works.  So long story short, the upgrade process going forward 
should be:

./configure, make, make install
bro-pkg autoconfig
bro-pkg refresh

on from there.  It might be worthwhile to annotate somewhere in the 
README or create an UPGRADE in the tarball to reflect that bro-pkg will 
need some attention as well during the upgrade process.  Thank you!

James

From johanna at icir.org  Thu Dec 20 23:07:26 2018
From: johanna at icir.org (Johanna Amann)
Date: Fri, 21 Dec 2018 08:07:26 +0100
Subject: [Zeek] Timetable for RPM release for 2.6.x
In-Reply-To: <e70f0f3a16f1443099730c9a5d337e6c@endace.com>
References: <e70f0f3a16f1443099730c9a5d337e6c@endace.com>
Message-ID: <8A5FE2ED-881C-43D8-AA24-879380B49998@icir.org>

Hi,

> Is there a timetable available yet for the release of RPM's zeek/bro's 
> latest release via the repo ?

I will do this within the week, they will be available some time on the 
week-end.

Did you by any chance look at the nightly packages? I asked for feedback 
on them on zeek-dev a few days ago :)

Johanna

From ericooi at gmail.com  Fri Dec 21 10:04:24 2018
From: ericooi at gmail.com (Eric Ooi)
Date: Fri, 21 Dec 2018 12:04:24 -0600
Subject: [Zeek] Bro upgrades, and plugins vs. packages, and bro-pkg
In-Reply-To: <05d972a78319a2939a1b01cc9f2ce8f3@slave-tothe-box.net>
References: <05d972a78319a2939a1b01cc9f2ce8f3@slave-tothe-box.net>
Message-ID: <CAEuvLSZZmFsfr2O5G0N1_XjvAXw+2o81N2xpGSRoBTb6wgmYiQ@mail.gmail.com>

Thanks James.  The is helpful.

On Fri, Dec 21, 2018 at 1:08 AM James Lay <jlay at slave-tothe-box.net> wrote:

> So here we go.  I've attacked this with my lab and here are some
> thoughts/results.  Current state:
>
> bro-2.6 installed from source (config option --prefix=/opt/bro)
> bro-af_packet-plugin ja3 intel-seen-more domain-tld installed via
> bro-pkg
>
> after upgrading to bro-2.6.1 errors like the below:
>
> fatal error in /opt/bro/share/bro/base/init-bare.bro, line 1: cannot
> load plugin library
> /opt/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/
> Bro-AF_Packet.linux-x86_64.so:
> /opt/bro/lib/bro/plugins/packages/bro-af_packet-plugin//lib/
> Bro-AF_Packet.linux-x86_64.so:
> undefined symbol: bro_version_2_6_plugin_6
>
> next up, remove and reinstall bro-af_backup-plugin:
>
> root@# bro-pkg remove bro-af_packet-plugin
> The following packages will be REMOVED:
>    bro/j-gras/bro-af_packet-plugin
>
> Proceed? [Y/n] y
> Removed "bro/j-gras/bro-af_packet-plugin"
> root@# bro-pkg install bro-af_packet-plugin
> The following packages will be INSTALLED:
>    bro/j-gras/bro-af_packet-plugin (1.3.0)
>
> Proceed? [Y/n] y
> Running unit tests for "bro/j-gras/bro-af_packet-plugin"
> [  0%] scripts.show-plugin ... failed
>    % 'bro -NN Bro::AF_Packet > output' failed unexpectedly (exit code 1)
>    % cat .stderr
>    fatal error in /opt/bro/share/bro/base/init-bare.bro, line 1: cannot
> load plugin library
>
> /root/.bro-pkg/testing/bro-af_packet-plugin/plugins/packages/bro-af_packet-plugin//lib/
> Bro-AF_Packet.linux-x86_64.so:
>
> /root/.bro-pkg/testing/bro-af_packet-plugin/plugins/packages/bro-af_packet-plugin//lib/
> Bro-AF_Packet.linux-x86_64.so:
> undefined symbol: bro_version_2_6_plugin_6
>
> 1 of 1 test failed
> error: bro/j-gras/bro-af_packet-plugin tests failed, inspect contents of
> /root/.bro-pkg/testing/bro-af_packet-plugin for details
> Proceed to install anyway? [N/y] n
> Abort.
>
> a thought occurs....modify /root/.bro-pkg/config -> bro_dist =
> /build/bro-2.6.1
>
> all works.  So long story short, the upgrade process going forward
> should be:
>
> ./configure, make, make install
> bro-pkg autoconfig
> bro-pkg refresh
>
> on from there.  It might be worthwhile to annotate somewhere in the
> README or create an UPGRADE in the tarball to reflect that bro-pkg will
> need some attention as well during the upgrade process.  Thank you!
>
> James
> _______________________________________________
> Zeek mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20181221/ed2ede69/attachment.html 

From al.kefallonitis at gmail.com  Fri Dec 21 04:04:00 2018
From: al.kefallonitis at gmail.com (Alex Kefallonitis)
Date: Fri, 21 Dec 2018 14:04:00 +0200
Subject: [Zeek] files.log
Message-ID: <CAHv=Mup178e2DojXY-5WVLON4GERHgu0TeiHm3pLcLuT6zCGgw@mail.gmail.com>

Hi is there a way to exclude only application/pkix-cert from files.log ?

Thanks in advanced
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20181221/f6788c5c/attachment.html 

From anthony.kasza at gmail.com  Sat Dec 22 10:10:53 2018
From: anthony.kasza at gmail.com (anthony kasza)
Date: Sat, 22 Dec 2018 13:10:53 -0500
Subject: [Zeek] files.log
In-Reply-To: <CAHv=Mup178e2DojXY-5WVLON4GERHgu0TeiHm3pLcLuT6zCGgw@mail.gmail.com>
References: <CAHv=Mup178e2DojXY-5WVLON4GERHgu0TeiHm3pLcLuT6zCGgw@mail.gmail.com>
Message-ID: <CAEZw2byan8u9GWP-uJEOZ6qStuBvWDMBOLLfmzukXE2W1P78Ew@mail.gmail.com>

You may find these walkthroughs helpful.

https://blog.zeek.org/2012/02/filtering-logs-with-bro.html
https://www.zeek.org/manual/release/frameworks/file-analysis.html#file-lifecycle-events

-AK

On Sat, Dec 22, 2018, 11:45 Alex Kefallonitis <al.kefallonitis at gmail.com
wrote:

> Hi is there a way to exclude only application/pkix-cert from files.log ?
>
> Thanks in advanced
> _______________________________________________
> Zeek mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20181222/1f7f05d3/attachment.html 

From michalpurzynski1 at gmail.com  Sat Dec 22 16:03:41 2018
From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=)
Date: Sat, 22 Dec 2018 16:03:41 -0800
Subject: [Zeek] files.log
In-Reply-To: <CAEZw2byan8u9GWP-uJEOZ6qStuBvWDMBOLLfmzukXE2W1P78Ew@mail.gmail.com>
References: <CAHv=Mup178e2DojXY-5WVLON4GERHgu0TeiHm3pLcLuT6zCGgw@mail.gmail.com>
	<CAEZw2byan8u9GWP-uJEOZ6qStuBvWDMBOLLfmzukXE2W1P78Ew@mail.gmail.com>
Message-ID: <CAJ6bFK3hC4vE0rnQC7cYp41ckR5p04SnNL=9HO4EGRt1Yi+M5w@mail.gmail.com>

This is what I do. It allowed me to cut the number of files events by
like 70% and the total SIEM intake by a whooping 30%

You most definitely want to filter those out. Be aware there are some
drawbacks, though, like you just lost the trivial ability to correlate
x509-files-ssl but you combat that by correlating the "id" field from
the x509 log with the cert_chain_fuids field from the ssl.log



module LogFilter;


event bro_init()

{

        Log::remove_default_filter(Files::LOG);

        Log::add_filter(Files::LOG, [$name = "files-noise",

                                      $pred(rec: Files::Info) = {

                                        for (tx_host in rec$tx_hosts) {

                                                if ((rec?$mime_type)
&& ((rec$mime_type == "application/pkix-cert") || (rec$mime_type ==
"application/x-x509-ca-cert") || (rec$mime_type ==
"application/x-x509-user-cert") ))

                                                    return F;

                                            return T;

                                            }

                                        return T;

                                      }

                                    ]);

}

On Sat, Dec 22, 2018 at 10:19 AM anthony kasza <anthony.kasza at gmail.com> wrote:
>
> You may find these walkthroughs helpful.
>
> https://blog.zeek.org/2012/02/filtering-logs-with-bro.html
> https://www.zeek.org/manual/release/frameworks/file-analysis.html#file-lifecycle-events
>
> -AK
>
> On Sat, Dec 22, 2018, 11:45 Alex Kefallonitis <al.kefallonitis at gmail.com wrote:
>>
>> Hi is there a way to exclude only application/pkix-cert from files.log ?
>>
>> Thanks in advanced
>> _______________________________________________
>> Zeek mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
> _______________________________________________
> Zeek mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

From michael.alaly at gmail.com  Wed Dec 26 11:41:41 2018
From: michael.alaly at gmail.com (Michael Alaly)
Date: Wed, 26 Dec 2018 13:41:41 -0600
Subject: [Zeek] DNS forwarding + weird.log
Message-ID: <CALXa+CubYjBmgvy79zWHQaW2F52P=1NRdcf3=1cZ=6sT7d9pxQ@mail.gmail.com>

Does anyone have a recommended way to handle a sensor that also runs a DNS
resolver/forwarder?

Since the requests "originate" at the sensor there is no other side of the
traffic for Zeek to see. This generates a weird.log possible_split_routing
entry for every forwarded DNS request.

Is this generally avoided by moving DNS off the firewall/sensor, or are
there other ways of handling this?

Thanks,
Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/zeek/attachments/20181226/cc134358/attachment.html