From jedwards2728 at gmail.com  Tue Nov  1 03:49:53 2016
From: jedwards2728 at gmail.com (John Edwards)
Date: Tue, 1 Nov 2016 21:49:53 +1100
Subject: [Bro] Bro Digest, Vol 126, Issue 56
In-Reply-To: <mailman.1.1477940401.8311.bro@bro.org>
References: <mailman.1.1477940401.8311.bro@bro.org>
Message-ID: <CAAcg0eLTznEZ=ogQGT0mjxLNLN20+jsB9vxanzqGyAOY6Y0Q+g@mail.gmail.com>

All resolved now, i noticed the cronjob was in place for broctl tasks and
also even though i configured the node.cfg back from a cluster to a
standalone instance and re-ran deploy it had PID's for both standalone and
clustered processes. So i rebooted the system and it was logging and
gzipping in the json output i want and now consuming a lot less resources
and disk on our SIEM. ASCII had a 3:1 compression ratio of inflation! so
json is much more efficient use of space

On Tue, Nov 1, 2016 at 6:00 AM, <bro-request at bro.org> wrote:

> Send Bro mailing list submissions to
>         bro at bro.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> or, via email, send a message with subject or body 'help' to
>         bro-request at bro.org
>
> You can reach the person managing the list at
>         bro-owner at bro.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Bro digest..."
>
>
> Today's Topics:
>
>    1.  bro logging gzip (erik clark)
>    2. Re: bro logging gzip (John Edwards)
>    3. af_packet/pf_ring equivalency (erik clark)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 31 Oct 2016 08:02:11 -0400
> From: erik clark <philosnef at gmail.com>
> Subject: [Bro]  bro logging gzip
> To: jedwards2728 at gmail.com, bro at bro.org
> Message-ID:
>         <CAK6atxrrDS8hF9QjWdB4f-V6W9msjwY0a+PXzVYdFMmafM+5JA@
> mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> broctl cron cronjob? Pretty sure this is what controls rollover and
> compression.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161031/b59e71cb/attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Mon, 31 Oct 2016 23:05:03 +1100
> From: John Edwards <jedwards2728 at gmail.com>
> Subject: Re: [Bro] bro logging gzip
> To: erik clark <philosnef at gmail.com>
> Cc: bro at bro.org
> Message-ID:
>         <CAAcg0e+8LDPYEu8xSFAe6bXfUcsMdduJdQ1wo
> JC9O4eLwG3c1Q at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Oh I just found that 10 minutes ago. I overlooked it as I have built two
> standalone systems and just forgot about cron. Then you emailed :) thanks
> for reminding me
>
> On Monday, 31 October 2016, erik clark <philosnef at gmail.com> wrote:
>
> > broctl cron cronjob? Pretty sure this is what controls rollover and
> > compression.
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161031/209deb54/attachment-0001.html
>
> ------------------------------
>
> Message: 3
> Date: Mon, 31 Oct 2016 12:38:22 -0400
> From: erik clark <philosnef at gmail.com>
> Subject: [Bro] af_packet/pf_ring equivalency
> To: bro at bro.org
> Message-ID:
>         <CAK6atxp_rmpg+aLQDN_dhJCOCg-7JebhZZn0u2EJFYgCZSuA0Q at mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> I am using pf_ring with pfcount to do traffic analysis (pps/throughput)
> since it is very reliable.
>
> Does af_packet have an equivalent for this? I dont want to use broctl
> capstats unless there is absolutely no other option.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161031/ee8dd1ed/attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> Bro mailing list
> Bro at bro.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> End of Bro Digest, Vol 126, Issue 56
> ************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161101/2c9129b4/attachment.html 

From philosnef at gmail.com  Tue Nov  1 06:09:59 2016
From: philosnef at gmail.com (erik clark)
Date: Tue, 1 Nov 2016 09:09:59 -0400
Subject: [Bro] af_packet/pf_ring equivalency
In-Reply-To: <CAJ6bFK2rtasKMADB3G+hX1ewowXd=J-Q371isGhj7GJmGveuiQ@mail.gmail.com>
References: <CAK6atxp_rmpg+aLQDN_dhJCOCg-7JebhZZn0u2EJFYgCZSuA0Q@mail.gmail.com>
	<CAJ6bFK2rtasKMADB3G+hX1ewowXd=J-Q371isGhj7GJmGveuiQ@mail.gmail.com>
Message-ID: <CAK6atxpOEqr05WL7BUkRyqzQhDBfgvxAgxXxe00+xZA8qcK-Hg@mail.gmail.com>

Interestingly, bwm-ng does not give me traffic numbers for my sniff
interface.... I am trying to get ifpps, but I dont want to have to compile
it and would like to find a rhel6 package of it. Sadly, it isnt in EPEL's
netsniff-ng package group.

On Mon, Oct 31, 2016 at 7:21 PM, Micha? Purzy?ski <
michalpurzynski1 at gmail.com> wrote:

> ifpps for generic bandwidth and pps monitoring. Never, ever, use iptraf.
> ifpps has been written by the netsniff-ng author and it speaks for itself.
>
> bwm-ng seems to be good, haven't compared the accuracy and the perf data
> acquisition.
>
>
> For monitoring drops
>
> ethtool -S <int> to detect drops in card's FIFO and sometimes, reasons for
> them.
>
> https://github.com/netoptimizer/network-testing/
> blob/master/bin/softnet_stat.pl
>
> to detect drops at the softirq layer
>
> Bro's stats.log to detect drops at the af_packet layer
>
> Bro capture_loss to detect drops in all above + drops before packets reach
> your sensor.
>
> Monitoring drops is complex and there is no single metric that tells you
> all. Some of this is true for pfring as well, people just don't know. I've
> seen sensors with 2-3% drops (in Suricata) but 40% drops in FIFO and they
> were like "we're doing fine". Well, so here's a bad news... ;-)
>
>
>
> On Mon, Oct 31, 2016 at 5:38 PM, erik clark <philosnef at gmail.com> wrote:
>
>> I am using pf_ring with pfcount to do traffic analysis (pps/throughput)
>> since it is very reliable.
>>
>> Does af_packet have an equivalent for this? I dont want to use broctl
>> capstats unless there is absolutely no other option.
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161101/24661246/attachment-0001.html 

From jazoff at illinois.edu  Tue Nov  1 06:25:02 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 1 Nov 2016 13:25:02 +0000
Subject: [Bro] Bro Digest, Vol 126, Issue 56
In-Reply-To: <CAAcg0eLTznEZ=ogQGT0mjxLNLN20+jsB9vxanzqGyAOY6Y0Q+g@mail.gmail.com>
References: <mailman.1.1477940401.8311.bro@bro.org>
	<CAAcg0eLTznEZ=ogQGT0mjxLNLN20+jsB9vxanzqGyAOY6Y0Q+g@mail.gmail.com>
Message-ID: <867626FB-6BC7-4339-A883-333A0DAA9D89@illinois.edu>


> On Nov 1, 2016, at 6:49 AM, John Edwards <jedwards2728 at gmail.com> wrote:
> 
> ASCII had a 3:1 compression ratio of inflation! so json is much more efficient use of space

The json log entries need to include the field names in every record.  There is no possible way that the json logs are more space efficient.

-- 
- Justin Azoff



From seth at icir.org  Tue Nov  1 07:31:35 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 1 Nov 2016 10:31:35 -0400
Subject: [Bro] extract smtp objects
In-Reply-To: <CAK6atxr4A81vj1mfX5PgtMVmW7G1E7E6bftijCkpfsGN79CvkQ@mail.gmail.com>
References: <CAK6atxp+rAmxaXt5Jg6yvPj62DhrA1EiR==KJ3mmpSE9jL+f+g@mail.gmail.com>
	<CAK6atxrPG561xp5YhbbTUV5pA3oUbhZbLPPUjo3+qPG6_bEgLA@mail.gmail.com>
	<CAK6atxrtX_-x7t1U9ctmbC60jV-gs3aq+HQFiowgmk5Oy6jGYQ@mail.gmail.com>
	<CAK6atxr4A81vj1mfX5PgtMVmW7G1E7E6bftijCkpfsGN79CvkQ@mail.gmail.com>
Message-ID: <95CD8A71-1E53-4C1B-9A08-ED7901426194@icir.org>


> On Oct 28, 2016, at 11:25 AM, erik clark <philosnef at gmail.com> wrote:
> 
> Sorry for the clutter. I did this a different way with extract from file analyzer. I will just script some glue with conn.log, smtp.log, and fuid. I had originally wanted to scrap the data out of the raw smtp message (and would still prefer to do that) with other tools entirely, so if someone has a way to do that, that would be fantastic. :)

You are hinting towards a design change that I've wanted to see for quite a while where the MIME analyzer would turn into a file analyzer and the MIME content carried over SMTP would be fed into the MIME file analyzer.  This would have the nice side effect of making it simple to extract the full MIME message through the normal file extraction channels.

Unfortunately this design change hasn't happened yet and isn't slated for the near term.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From philosnef at gmail.com  Tue Nov  1 07:33:48 2016
From: philosnef at gmail.com (erik clark)
Date: Tue, 1 Nov 2016 10:33:48 -0400
Subject: [Bro] extract smtp objects
In-Reply-To: <95CD8A71-1E53-4C1B-9A08-ED7901426194@icir.org>
References: <CAK6atxp+rAmxaXt5Jg6yvPj62DhrA1EiR==KJ3mmpSE9jL+f+g@mail.gmail.com>
	<CAK6atxrPG561xp5YhbbTUV5pA3oUbhZbLPPUjo3+qPG6_bEgLA@mail.gmail.com>
	<CAK6atxrtX_-x7t1U9ctmbC60jV-gs3aq+HQFiowgmk5Oy6jGYQ@mail.gmail.com>
	<CAK6atxr4A81vj1mfX5PgtMVmW7G1E7E6bftijCkpfsGN79CvkQ@mail.gmail.com>
	<95CD8A71-1E53-4C1B-9A08-ED7901426194@icir.org>
Message-ID: <CAK6atxqBBY4xb57jeM6=sz2w5v7qnJVcXPfGd5yJv_u+9TSVbA@mail.gmail.com>

How difficult would this be to do?

On Tue, Nov 1, 2016 at 10:31 AM, Seth Hall <seth at icir.org> wrote:

>
> > On Oct 28, 2016, at 11:25 AM, erik clark <philosnef at gmail.com> wrote:
> >
> > Sorry for the clutter. I did this a different way with extract from file
> analyzer. I will just script some glue with conn.log, smtp.log, and fuid. I
> had originally wanted to scrap the data out of the raw smtp message (and
> would still prefer to do that) with other tools entirely, so if someone has
> a way to do that, that would be fantastic. :)
>
> You are hinting towards a design change that I've wanted to see for quite
> a while where the MIME analyzer would turn into a file analyzer and the
> MIME content carried over SMTP would be fed into the MIME file analyzer.
> This would have the nice side effect of making it simple to extract the
> full MIME message through the normal file extraction channels.
>
> Unfortunately this design change hasn't happened yet and isn't slated for
> the near term.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161101/370453f2/attachment.html 

From seth at icir.org  Tue Nov  1 07:35:32 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 1 Nov 2016 10:35:32 -0400
Subject: [Bro] Have a cluster infrastructure read pcaps
In-Reply-To: <CAKZA8xh3h=uuy46-ze4coizOEwZGgaAWJznmH2_8A1pHHQ9+HA@mail.gmail.com>
References: <CAK6atxqKjgD97dDBUd9Pr4peWHZLX0uDWa-BbQVsGZkgnFO7ZQ@mail.gmail.com>
	<CAKZA8xi6=dqyg1J_X0RyirdT3PFwG8pQv9ojgVEjuzYE7hTnRQ@mail.gmail.com>
	<CAK6atxp=Z-Xukp4HHcXdSuixxgVwfkRWiBwSfDt=QK-J_ZjF6A@mail.gmail.com>
	<CAKZA8xh3h=uuy46-ze4coizOEwZGgaAWJznmH2_8A1pHHQ9+HA@mail.gmail.com>
Message-ID: <F70B1A9D-4106-42C8-99E8-AA21B4453C11@icir.org>


> On Oct 31, 2016, at 7:34 AM, william de ping <bill.de.ping at gmail.com> wrote:
> 
> I was hoping for some solution that will keep bro process loaded and running and feeding it with pcaps.
> This way I can at least skip the reoccurring loading process.

You are going to have trouble keeping the logs with the original pcap in this case.  You could have sessions that cross the pcaps like this....

PCAP 1 -> TCP session establishment
PCAP 2 -> lots of session data
PCAP 3 -> TCP session teardown - The conn log entry will be written here!

Your logs won't match up as closely as you'd like and could become very confusing.  I would argue that this offline packet loading situation is a situation that you want to avoid at all costs, but if you have to live within that situation, I would argue that you want to keep the Bro processes up and treat the sequential files as a stream and don't try to tie logs to a particular file.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Tue Nov  1 07:38:58 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 1 Nov 2016 10:38:58 -0400
Subject: [Bro] extract smtp objects
In-Reply-To: <CAK6atxqBBY4xb57jeM6=sz2w5v7qnJVcXPfGd5yJv_u+9TSVbA@mail.gmail.com>
References: <CAK6atxp+rAmxaXt5Jg6yvPj62DhrA1EiR==KJ3mmpSE9jL+f+g@mail.gmail.com>
	<CAK6atxrPG561xp5YhbbTUV5pA3oUbhZbLPPUjo3+qPG6_bEgLA@mail.gmail.com>
	<CAK6atxrtX_-x7t1U9ctmbC60jV-gs3aq+HQFiowgmk5Oy6jGYQ@mail.gmail.com>
	<CAK6atxr4A81vj1mfX5PgtMVmW7G1E7E6bftijCkpfsGN79CvkQ@mail.gmail.com>
	<95CD8A71-1E53-4C1B-9A08-ED7901426194@icir.org>
	<CAK6atxqBBY4xb57jeM6=sz2w5v7qnJVcXPfGd5yJv_u+9TSVbA@mail.gmail.com>
Message-ID: <ED5B6C66-9613-4150-8915-6B794ABCE07A@icir.org>


> On Nov 1, 2016, at 10:33 AM, erik clark <philosnef at gmail.com> wrote:
> 
> How difficult would this be to do? 

Probably quite a bit of work and maybe 80-90% of it would be in the analyzer which is hand written in C++.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From xuchen890530 at gmail.com  Tue Nov  1 08:03:02 2016
From: xuchen890530 at gmail.com (Chen Xu)
Date: Tue, 1 Nov 2016 11:03:02 -0400
Subject: [Bro] Convert integer to string
Message-ID: <CAJ5Y-2M4rLLH3R3Aq8Yd3C8GfweSwX7Ed=kex+rvOOL6O_=0Yg@mail.gmail.com>

Hello all,

I am new to bro. I have a simple question. Is there any function which can
convert integer to string?

Thanks,

Chen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161101/2c41e582/attachment.html 

From matt.clemons at gmail.com  Tue Nov  1 08:06:09 2016
From: matt.clemons at gmail.com (Matt Clemons)
Date: Tue, 1 Nov 2016 10:06:09 -0500
Subject: [Bro] accept failed, Too many open files 24
Message-ID: <CANiyPJazU9SQP2HED5aSFAP5M-wH8F2QEX25tOdAvoOyCFV38g@mail.gmail.com>

Lo All,

Started receiving this error after adding a worker yesterday.  If I remove
the worker and deploy, no issues.

Communications.log: "accept failed, Too many open files 24"


Running CentOS6.  Bro 2.4.1.  17 Physical worker systems.  150 total worker
processes.

When adding the 18th worker (6 additional worker processes) logs slow to a
crawl and the communications log is filled with the failure message.

I've experimented with limits.conf and set a high soft and hard limits of
open files.  Also tried doubling the defaults, and many different
combinations to no avail.  Most of these caused bro to hang and stop
logging.  Others had no affect on the problem.

Has anyone had to deal with this issue or have some ideas?  Is there some
hidden setting in bro where I can set open file limits?

-- 
Regards,

Matt Clemons
(816) 200-0789
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161101/25788555/attachment.html 

From troyj at maine.edu  Tue Nov  1 08:08:22 2016
From: troyj at maine.edu (Troy Jordan)
Date: Tue, 1 Nov 2016 11:08:22 -0400
Subject: [Bro] Convert integer to string
In-Reply-To: <CAJ5Y-2M4rLLH3R3Aq8Yd3C8GfweSwX7Ed=kex+rvOOL6O_=0Yg@mail.gmail.com>
References: <CAJ5Y-2M4rLLH3R3Aq8Yd3C8GfweSwX7Ed=kex+rvOOL6O_=0Yg@mail.gmail.com>
Message-ID: <1e92e26d-0985-6054-aa2e-e8749df92dd9@maine.edu>

Chen,

There is a fmt function for formatting strings that will handle signed
and unsigned integers:

https://www.bro.org/sphinx/scripts/base/bif/bro.bif.bro.html#id-fmt

- Troy



On 11/1/2016 11:03 AM, Chen Xu wrote:
> Hello all,
>
> I am new to bro. I have a simple question. Is there any function which
> can convert integer to string?
>
> Thanks,
>
> Chen
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-- 


                     	  Troy Jordan 
                   t r o y j @ m a i n e . e d u
			   GIAC GCIH,GCIA
------------------------------------------------------------
                Network Systems Security Analyst
             Information Technology Security Office
                    University of Maine System
------------------------------------------------------------
233 Science Building           |     voice: 207.561.3590
Portland, ME 04103             |     fax:   509.351.3650



"As you all know, Security Is Mortals chiefest Enemy"
 William Shakespeare, Macbeth


From vladg at illinois.edu  Tue Nov  1 11:31:50 2016
From: vladg at illinois.edu (Vlad Grigorescu)
Date: Tue, 01 Nov 2016 13:31:50 -0500
Subject: [Bro] Protocol Analyzer Compilation Issue types.bif
In-Reply-To: <58217af9-2580-8bc9-f29f-422b9178970b@gmx.com>
References: <58217af9-2580-8bc9-f29f-422b9178970b@gmx.com>
Message-ID: <m0bmxzf3vt.fsf@vladg.net>

Hi Valerio,

Personally, I usually use binpac_quickstart, which should take care of
this for you:

https://github.com/grigorescu/binpac_quickstart

  --Vlad

Valerio <valerio.click at gmx.com> writes:

> Hi all,
>
> in writing a custom protocol analyzer for BRO, I came across a strange
> behaviour at compilation time.
>
> It seems that the order in which you specify events.bif and types.bif
> files in CMakeLists.txt matters. In fact, if I have:
>
> bro_plugin_begin(BroCustomProt)
> 	bro_plugin_cc(CustomProt.cc Plugin.cc)
> 	bro_plugin_bif(types.bif)
> 	bro_plugin_bif(events.bif)
> 	[...]
> bro_plugin_end()
>
> and I try to compile bro I get the following error:
>
> [...]
>  /build/src/analyzer/protocol/customprot/customprot_pac.h fatal error:
> types.bif.h: File or directory do not exist
>
>
> If instead I modify CMakeLists.txt by swapping events.bif and types.bif
> as in:
>
> bro_plugin_begin(BroCustomProt)
> 	bro_plugin_cc(CustomProt.cc Plugin.cc)
> 	bro_plugin_bif(events.bif)
> 	bro_plugin_bif(types.bif)
> 	[...]
> bro_plugin_end()
>
> the compilation succeeds.
>
> Is there any ordering issue in writing the CMakeLists.txt for a BRO
> Protocol Analyzer that needs to be taken into account?
>
> best regards,
> Valerio
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 800 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161101/56ec1e86/attachment.bin 

From jedwards2728 at gmail.com  Tue Nov  1 12:03:41 2016
From: jedwards2728 at gmail.com (John Edwards)
Date: Wed, 2 Nov 2016 06:03:41 +1100
Subject: [Bro] Bro Digest, Vol 127, Issue 2
In-Reply-To: <mailman.60413.1478012909.819.bro@bro.org>
References: <mailman.60413.1478012909.819.bro@bro.org>
Message-ID: <CAAcg0eJs6JTe33g3E=kmtPhbLQ7vipjFfh9TayN2YvAzkreC2w@mail.gmail.com>

Ive just configured it so I will see how the logging performs now. I was
basing my information of saving space from here

https://github.com/jahshuah/splunk-ta-bro-json/blob/master/README.md


Cheers

John

On Wednesday, 2 November 2016, <bro-request at bro.org> wrote:

> Send Bro mailing list submissions to
>         bro at bro.org <javascript:;>
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> or, via email, send a message with subject or body 'help' to
>         bro-request at bro.org <javascript:;>
>
> You can reach the person managing the list at
>         bro-owner at bro.org <javascript:;>
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Bro digest..."
>
>
> Today's Topics:
>
>    1. Re: Bro Digest, Vol 126, Issue 56 (Azoff, Justin S)
>    2. Re: extract smtp objects (Seth Hall)
>    3. Re: extract smtp objects (erik clark)
>    4. Re: Have a cluster infrastructure read pcaps (Seth Hall)
>    5. Re: extract smtp objects (Seth Hall)
>    6. Convert integer to string (Chen Xu)
>    7. accept failed, Too many open files 24 (Matt Clemons)
>    8. Re: Convert integer to string (Troy Jordan)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 1 Nov 2016 13:25:02 +0000
> From: "Azoff, Justin S" <jazoff at illinois.edu <javascript:;>>
> Subject: Re: [Bro] Bro Digest, Vol 126, Issue 56
> To: John Edwards <jedwards2728 at gmail.com <javascript:;>>
> Cc: "bro at bro.org <javascript:;>" <bro at bro.org <javascript:;>>
> Message-ID: <867626FB-6BC7-4339-A883-333A0DAA9D89 at illinois.edu
> <javascript:;>>
> Content-Type: text/plain; charset="us-ascii"
>
>
> > On Nov 1, 2016, at 6:49 AM, John Edwards <jedwards2728 at gmail.com
> <javascript:;>> wrote:
> >
> > ASCII had a 3:1 compression ratio of inflation! so json is much more
> efficient use of space
>
> The json log entries need to include the field names in every record.
> There is no possible way that the json logs are more space efficient.
>
> --
> - Justin Azoff
>
>
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 1 Nov 2016 10:31:35 -0400
> From: Seth Hall <seth at icir.org <javascript:;>>
> Subject: Re: [Bro] extract smtp objects
> To: erik clark <philosnef at gmail.com <javascript:;>>
> Cc: bro at bro.org <javascript:;>
> Message-ID: <95CD8A71-1E53-4C1B-9A08-ED7901426194 at icir.org <javascript:;>>
> Content-Type: text/plain; charset=us-ascii
>
>
> > On Oct 28, 2016, at 11:25 AM, erik clark <philosnef at gmail.com
> <javascript:;>> wrote:
> >
> > Sorry for the clutter. I did this a different way with extract from file
> analyzer. I will just script some glue with conn.log, smtp.log, and fuid. I
> had originally wanted to scrap the data out of the raw smtp message (and
> would still prefer to do that) with other tools entirely, so if someone has
> a way to do that, that would be fantastic. :)
>
> You are hinting towards a design change that I've wanted to see for quite
> a while where the MIME analyzer would turn into a file analyzer and the
> MIME content carried over SMTP would be fed into the MIME file analyzer.
> This would have the nice side effect of making it simple to extract the
> full MIME message through the normal file extraction channels.
>
> Unfortunately this design change hasn't happened yet and isn't slated for
> the near term.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
>
>
> ------------------------------
>
> Message: 3
> Date: Tue, 1 Nov 2016 10:33:48 -0400
> From: erik clark <philosnef at gmail.com <javascript:;>>
> Subject: Re: [Bro] extract smtp objects
> To: Seth Hall <seth at icir.org <javascript:;>>
> Cc: bro at bro.org <javascript:;>
> Message-ID:
>         <CAK6atxqBBY4xb57jeM6=
> sz2w5v7qnJVcXPfGd5yJv_u+9TSVbA at mail.gmail.com <javascript:;>>
> Content-Type: text/plain; charset="utf-8"
>
> How difficult would this be to do?
>
> On Tue, Nov 1, 2016 at 10:31 AM, Seth Hall <seth at icir.org <javascript:;>>
> wrote:
>
> >
> > > On Oct 28, 2016, at 11:25 AM, erik clark <philosnef at gmail.com
> <javascript:;>> wrote:
> > >
> > > Sorry for the clutter. I did this a different way with extract from
> file
> > analyzer. I will just script some glue with conn.log, smtp.log, and
> fuid. I
> > had originally wanted to scrap the data out of the raw smtp message (and
> > would still prefer to do that) with other tools entirely, so if someone
> has
> > a way to do that, that would be fantastic. :)
> >
> > You are hinting towards a design change that I've wanted to see for quite
> > a while where the MIME analyzer would turn into a file analyzer and the
> > MIME content carried over SMTP would be fed into the MIME file analyzer.
> > This would have the nice side effect of making it simple to extract the
> > full MIME message through the normal file extraction channels.
> >
> > Unfortunately this design change hasn't happened yet and isn't slated for
> > the near term.
> >
> >   .Seth
> >
> > --
> > Seth Hall
> > International Computer Science Institute
> > (Bro) because everyone has a network
> > http://www.bro.org/
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161101/370453f2/attachment-0001.html
>
> ------------------------------
>
> Message: 4
> Date: Tue, 1 Nov 2016 10:35:32 -0400
> From: Seth Hall <seth at icir.org <javascript:;>>
> Subject: Re: [Bro] Have a cluster infrastructure read pcaps
> To: william de ping <bill.de.ping at gmail.com <javascript:;>>
> Cc: erik clark <philosnef at gmail.com <javascript:;>>, bro at bro.org
> <javascript:;>
> Message-ID: <F70B1A9D-4106-42C8-99E8-AA21B4453C11 at icir.org <javascript:;>>
> Content-Type: text/plain; charset=us-ascii
>
>
> > On Oct 31, 2016, at 7:34 AM, william de ping <bill.de.ping at gmail.com
> <javascript:;>> wrote:
> >
> > I was hoping for some solution that will keep bro process loaded and
> running and feeding it with pcaps.
> > This way I can at least skip the reoccurring loading process.
>
> You are going to have trouble keeping the logs with the original pcap in
> this case.  You could have sessions that cross the pcaps like this....
>
> PCAP 1 -> TCP session establishment
> PCAP 2 -> lots of session data
> PCAP 3 -> TCP session teardown - The conn log entry will be written here!
>
> Your logs won't match up as closely as you'd like and could become very
> confusing.  I would argue that this offline packet loading situation is a
> situation that you want to avoid at all costs, but if you have to live
> within that situation, I would argue that you want to keep the Bro
> processes up and treat the sequential files as a stream and don't try to
> tie logs to a particular file.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
>
>
> ------------------------------
>
> Message: 5
> Date: Tue, 1 Nov 2016 10:38:58 -0400
> From: Seth Hall <seth at icir.org <javascript:;>>
> Subject: Re: [Bro] extract smtp objects
> To: erik clark <philosnef at gmail.com <javascript:;>>
> Cc: bro at bro.org <javascript:;>
> Message-ID: <ED5B6C66-9613-4150-8915-6B794ABCE07A at icir.org <javascript:;>>
> Content-Type: text/plain; charset=us-ascii
>
>
> > On Nov 1, 2016, at 10:33 AM, erik clark <philosnef at gmail.com
> <javascript:;>> wrote:
> >
> > How difficult would this be to do?
>
> Probably quite a bit of work and maybe 80-90% of it would be in the
> analyzer which is hand written in C++.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
>
>
> ------------------------------
>
> Message: 6
> Date: Tue, 1 Nov 2016 11:03:02 -0400
> From: Chen Xu <xuchen890530 at gmail.com <javascript:;>>
> Subject: [Bro] Convert integer to string
> To: bro at bro.org <javascript:;>
> Message-ID:
>         <CAJ5Y-2M4rLLH3R3Aq8Yd3C8GfweSwX7Ed=kex+rvOOL6O_=
> 0Yg at mail.gmail.com <javascript:;>>
> Content-Type: text/plain; charset="utf-8"
>
> Hello all,
>
> I am new to bro. I have a simple question. Is there any function which can
> convert integer to string?
>
> Thanks,
>
> Chen
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161101/2c41e582/attachment-0001.html
>
> ------------------------------
>
> Message: 7
> Date: Tue, 1 Nov 2016 10:06:09 -0500
> From: Matt Clemons <matt.clemons at gmail.com <javascript:;>>
> Subject: [Bro] accept failed, Too many open files 24
> To: "bro at bro.org <javascript:;>" <bro at bro.org <javascript:;>>
> Message-ID:
>         <
> CANiyPJazU9SQP2HED5aSFAP5M-wH8F2QEX25tOdAvoOyCFV38g at mail.gmail.com
> <javascript:;>>
> Content-Type: text/plain; charset="utf-8"
>
> Lo All,
>
> Started receiving this error after adding a worker yesterday.  If I remove
> the worker and deploy, no issues.
>
> Communications.log: "accept failed, Too many open files 24"
>
>
> Running CentOS6.  Bro 2.4.1.  17 Physical worker systems.  150 total worker
> processes.
>
> When adding the 18th worker (6 additional worker processes) logs slow to a
> crawl and the communications log is filled with the failure message.
>
> I've experimented with limits.conf and set a high soft and hard limits of
> open files.  Also tried doubling the defaults, and many different
> combinations to no avail.  Most of these caused bro to hang and stop
> logging.  Others had no affect on the problem.
>
> Has anyone had to deal with this issue or have some ideas?  Is there some
> hidden setting in bro where I can set open file limits?
>
> --
> Regards,
>
> Matt Clemons
> (816) 200-0789
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161101/25788555/attachment-0001.html
>
> ------------------------------
>
> Message: 8
> Date: Tue, 1 Nov 2016 11:08:22 -0400
> From: Troy Jordan <troyj at maine.edu <javascript:;>>
> Subject: Re: [Bro] Convert integer to string
> To: bro at bro.org <javascript:;>
> Message-ID: <1e92e26d-0985-6054-aa2e-e8749df92dd9 at maine.edu <javascript:;>
> >
> Content-Type: text/plain; charset=windows-1252
>
> Chen,
>
> There is a fmt function for formatting strings that will handle signed
> and unsigned integers:
>
> https://www.bro.org/sphinx/scripts/base/bif/bro.bif.bro.html#id-fmt
>
> - Troy
>
>
>
> On 11/1/2016 11:03 AM, Chen Xu wrote:
> > Hello all,
> >
> > I am new to bro. I have a simple question. Is there any function which
> > can convert integer to string?
> >
> > Thanks,
> >
> > Chen
> >
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org <javascript:;>
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> --
>
>
>                           Troy Jordan
>                    t r o y j @ m a i n e . e d u
>                            GIAC GCIH,GCIA
> ------------------------------------------------------------
>                 Network Systems Security Analyst
>              Information Technology Security Office
>                     University of Maine System
> ------------------------------------------------------------
> 233 Science Building           |     voice: 207.561.3590
> Portland, ME 04103             |     fax:   509.351.3650
>
>
>
> "As you all know, Security Is Mortals chiefest Enemy"
>  William Shakespeare, Macbeth
>
>
>
> ------------------------------
>
> _______________________________________________
> Bro mailing list
> Bro at bro.org <javascript:;>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> End of Bro Digest, Vol 127, Issue 2
> ***********************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161102/6e364c91/attachment-0001.html 

From seth at icir.org  Tue Nov  1 13:25:04 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 1 Nov 2016 16:25:04 -0400
Subject: [Bro] accept failed, Too many open files 24
In-Reply-To: <CANiyPJazU9SQP2HED5aSFAP5M-wH8F2QEX25tOdAvoOyCFV38g@mail.gmail.com>
References: <CANiyPJazU9SQP2HED5aSFAP5M-wH8F2QEX25tOdAvoOyCFV38g@mail.gmail.com>
Message-ID: <F8E67CA3-685C-46FE-8BD5-4EFD652C3846@icir.org>


> On Nov 1, 2016, at 11:06 AM, Matt Clemons <matt.clemons at gmail.com> wrote:
> 
> Has anyone had to deal with this issue or have some ideas?  Is there some hidden setting in bro where I can set open file limits

There was an issue with file handles being left open when they weren't needed which will be fixed in 2.5 (and is fixed in the 2.5 beta).  Also, in 2.5 you can run a cluster with a dedicated "logger" node which should improve some of the cluster behavior with such a large cluster.

I would recommend jumping in and trying out the beta.  

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From jazoff at illinois.edu  Tue Nov  1 14:16:21 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 1 Nov 2016 21:16:21 +0000
Subject: [Bro] accept failed, Too many open files 24
In-Reply-To: <F8E67CA3-685C-46FE-8BD5-4EFD652C3846@icir.org>
References: <CANiyPJazU9SQP2HED5aSFAP5M-wH8F2QEX25tOdAvoOyCFV38g@mail.gmail.com>
	<F8E67CA3-685C-46FE-8BD5-4EFD652C3846@icir.org>
Message-ID: <D6DC42B1-6DC2-424F-8A2F-FC1DF54B8028@illinois.edu>


> On Nov 1, 2016, at 4:25 PM, Seth Hall <seth at icir.org> wrote:
> 
> 
>> On Nov 1, 2016, at 11:06 AM, Matt Clemons <matt.clemons at gmail.com> wrote:
>> 
>> Has anyone had to deal with this issue or have some ideas?  Is there some hidden setting in bro where I can set open file limits
> 
> There was an issue with file handles being left open when they weren't needed which will be fixed in 2.5 (and is fixed in the 2.5 beta).  Also, in 2.5 you can run a cluster with a dedicated "logger" node which should improve some of the cluster behavior with such a large cluster.
> 
> I would recommend jumping in and trying out the beta.  

The 2 file descriptor leakage fixes were committed just after the 2.5 beta was released

commit 520ed43eae4ce7bcd8bb22cfd9cb6d138c4a4fd7
Author: Daniel Thayer <dnthayer at illinois.edu>
Date:   Wed Aug 31 16:30:10 2016 -0500

    Added another missing fclose in scan.l

commit b3a7d07e66b027a56e57ba010998639ff0d6da86
Author: Daniel Thayer <dnthayer at illinois.edu>
Date:   Wed Aug 31 14:07:44 2016 -0500



-- 
- Justin Azoff



From jlay at slave-tothe-box.net  Wed Nov  2 07:13:32 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 02 Nov 2016 08:13:32 -0600
Subject: [Bro] accept failed, Too many open files 24
In-Reply-To: <D6DC42B1-6DC2-424F-8A2F-FC1DF54B8028@illinois.edu>
References: <CANiyPJazU9SQP2HED5aSFAP5M-wH8F2QEX25tOdAvoOyCFV38g@mail.gmail.com>
	<F8E67CA3-685C-46FE-8BD5-4EFD652C3846@icir.org>
	<D6DC42B1-6DC2-424F-8A2F-FC1DF54B8028@illinois.edu>
Message-ID: <5c832b84179169c8ad8c83a921a173d8@localhost>

On 2016-11-01 15:16, Azoff, Justin S wrote:
>> On Nov 1, 2016, at 4:25 PM, Seth Hall <seth at icir.org> wrote:
>> 
>> 
>>> On Nov 1, 2016, at 11:06 AM, Matt Clemons <matt.clemons at gmail.com> 
>>> wrote:
>>> 
>>> Has anyone had to deal with this issue or have some ideas?  Is there 
>>> some hidden setting in bro where I can set open file limits
>> 
>> There was an issue with file handles being left open when they weren't 
>> needed which will be fixed in 2.5 (and is fixed in the 2.5 beta).  
>> Also, in 2.5 you can run a cluster with a dedicated "logger" node 
>> which should improve some of the cluster behavior with such a large 
>> cluster.
>> 
>> I would recommend jumping in and trying out the beta.
> 
> The 2 file descriptor leakage fixes were committed just after the 2.5
> beta was released
> 
> commit 520ed43eae4ce7bcd8bb22cfd9cb6d138c4a4fd7
> Author: Daniel Thayer <dnthayer at illinois.edu>
> Date:   Wed Aug 31 16:30:10 2016 -0500
> 
>     Added another missing fclose in scan.l
> 
> commit b3a7d07e66b027a56e57ba010998639ff0d6da86
> Author: Daniel Thayer <dnthayer at illinois.edu>
> Date:   Wed Aug 31 14:07:44 2016 -0500

Give this a look:

http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/

James

From matt.clemons at gmail.com  Wed Nov  2 08:53:47 2016
From: matt.clemons at gmail.com (Matt Clemons)
Date: Wed, 2 Nov 2016 10:53:47 -0500
Subject: [Bro] accept failed, Too many open files 24
In-Reply-To: <5c832b84179169c8ad8c83a921a173d8@localhost>
References: <CANiyPJazU9SQP2HED5aSFAP5M-wH8F2QEX25tOdAvoOyCFV38g@mail.gmail.com>
	<F8E67CA3-685C-46FE-8BD5-4EFD652C3846@icir.org>
	<D6DC42B1-6DC2-424F-8A2F-FC1DF54B8028@illinois.edu>
	<5c832b84179169c8ad8c83a921a173d8@localhost>
Message-ID: <CANiyPJb6PzVcMgqpNrJRi6_ZeX0c-SXVFe5vTs4qiO749v6Azw@mail.gmail.com>

James,
I tried all those options.  Bro would hang when I changed the soft limit,
and no matter what hard limit I set, it still produced errors.  I'm going
to do a parallel install of 2.5 and see if that fixes it.  I have already
tested, but was waiting for the full release.

Thanks for you responses.

On Wed, Nov 2, 2016 at 9:13 AM, James Lay <jlay at slave-tothe-box.net> wrote:

> On 2016-11-01 15:16, Azoff, Justin S wrote:
> >> On Nov 1, 2016, at 4:25 PM, Seth Hall <seth at icir.org> wrote:
> >>
> >>
> >>> On Nov 1, 2016, at 11:06 AM, Matt Clemons <matt.clemons at gmail.com>
> >>> wrote:
> >>>
> >>> Has anyone had to deal with this issue or have some ideas?  Is there
> >>> some hidden setting in bro where I can set open file limits
> >>
> >> There was an issue with file handles being left open when they weren't
> >> needed which will be fixed in 2.5 (and is fixed in the 2.5 beta).
> >> Also, in 2.5 you can run a cluster with a dedicated "logger" node
> >> which should improve some of the cluster behavior with such a large
> >> cluster.
> >>
> >> I would recommend jumping in and trying out the beta.
> >
> > The 2 file descriptor leakage fixes were committed just after the 2.5
> > beta was released
> >
> > commit 520ed43eae4ce7bcd8bb22cfd9cb6d138c4a4fd7
> > Author: Daniel Thayer <dnthayer at illinois.edu>
> > Date:   Wed Aug 31 16:30:10 2016 -0500
> >
> >     Added another missing fclose in scan.l
> >
> > commit b3a7d07e66b027a56e57ba010998639ff0d6da86
> > Author: Daniel Thayer <dnthayer at illinois.edu>
> > Date:   Wed Aug 31 14:07:44 2016 -0500
>
> Give this a look:
>
> http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/
>
> James
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Regards,

Matt Clemons
(816) 200-0789
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161102/fb9a8104/attachment.html 

From jlay at slave-tothe-box.net  Wed Nov  2 10:07:04 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 02 Nov 2016 11:07:04 -0600
Subject: [Bro] accept failed, Too many open files 24
In-Reply-To: <CANiyPJb6PzVcMgqpNrJRi6_ZeX0c-SXVFe5vTs4qiO749v6Azw@mail.gmail.com>
References: <CANiyPJazU9SQP2HED5aSFAP5M-wH8F2QEX25tOdAvoOyCFV38g@mail.gmail.com>
	<F8E67CA3-685C-46FE-8BD5-4EFD652C3846@icir.org>
	<D6DC42B1-6DC2-424F-8A2F-FC1DF54B8028@illinois.edu>
	<5c832b84179169c8ad8c83a921a173d8@localhost>
	<CANiyPJb6PzVcMgqpNrJRi6_ZeX0c-SXVFe5vTs4qiO749v6Azw@mail.gmail.com>
Message-ID: <c7a3ab1d7363a78c692817e8cf3a6adb@localhost>

You bet...sorry it didn't help.  Would like to know what the resolution 
was once you find it.

James

On 2016-11-02 09:53, Matt Clemons wrote:
> James,
> I tried all those options.  Bro would hang when I changed the soft
> limit, and no matter what hard limit I set, it still produced errors.
> I'm going to do a parallel install of 2.5 and see if that fixes it.  I
> have already tested, but was waiting for the full release.
> 
> Thanks for you responses.
> 
> On Wed, Nov 2, 2016 at 9:13 AM, James Lay <jlay at slave-tothe-box.net>
> wrote:
> 
>> On 2016-11-01 15:16, Azoff, Justin S wrote:
>>>> On Nov 1, 2016, at 4:25 PM, Seth Hall <seth at icir.org> wrote:
>>>> 
>>>> 
>>>>> On Nov 1, 2016, at 11:06 AM, Matt Clemons
>> <matt.clemons at gmail.com>
>>>>> wrote:
>>>>> 
>>>>> Has anyone had to deal with this issue or have some ideas?  Is
>> there
>>>>> some hidden setting in bro where I can set open file limits
>>>> 
>>>> There was an issue with file handles being left open when they
>> weren't
>>>> needed which will be fixed in 2.5 (and is fixed in the 2.5 beta).
>>>> Also, in 2.5 you can run a cluster with a dedicated "logger" node
>>>> which should improve some of the cluster behavior with such a
>> large
>>>> cluster.
>>>> 
>>>> I would recommend jumping in and trying out the beta.
>>> 
>>> The 2 file descriptor leakage fixes were committed just after the
>> 2.5
>>> beta was released
>>> 
>>> commit 520ed43eae4ce7bcd8bb22cfd9cb6d138c4a4fd7
>>> Author: Daniel Thayer <dnthayer at illinois.edu>
>>> Date:   Wed Aug 31 16:30:10 2016 -0500
>>> 
>>> Added another missing fclose in scan.l
>>> 
>>> commit b3a7d07e66b027a56e57ba010998639ff0d6da86
>>> Author: Daniel Thayer <dnthayer at illinois.edu>
>>> Date:   Wed Aug 31 14:07:44 2016 -0500
>> 
>> Give this a look:
>> 
>> http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/
>> [1]
>> 
>> James
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro [2]
> 
> --
> 
> Regards,
> 
> Matt Clemons
> (816) 200-0789
> 
> 
> Links:
> ------
> [1] http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/
> [2] http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

From francois.pennaneach at free.fr  Wed Nov  2 17:19:45 2016
From: francois.pennaneach at free.fr (=?UTF-8?Q?Fran=c3=a7ois_Pennaneach?=)
Date: Thu, 3 Nov 2016 01:19:45 +0100
Subject: [Bro] binPAC : more than one &require attribute on a field
Message-ID: <86842898-2d45-b079-e029-97b243cfdaad@free.fr>

Hi all,


I'm a Bro beginner. I got a small problem when writing binPAC. See below.

I'm using master branch of binPAC.


In the binPAC grammar, nothing prevents from applying many &requires 
attributes to the same field.
However,  in such a case the generated C++ code is incorrect.

type MyArray = record {
     a: uint16 &requires(c) &requires(d);
     b: uint16;
} &let {
     c : uint16 = b * 2;
     d : uint16 = b * 3;
};

The generated code is :
     // Parse "a"
     // Parse "b"
     b_ = FixByteOrder(t_byteorder, *((uint16 const *) ((t_begin_of_data 
+ 2))));
     // Evaluate 'let' and 'withinput' fields
     d_ = b() * 3;
     a_ = FixByteOrder(t_byteorder, *((uint16 const *) (t_begin_of_data)));
     // Evaluate 'let' and 'withinput' fields

     // Evaluate 'let' and 'withinput' fields
     c_ = b() * 2;


In pac_types.cc, only the last &requires attribute is kept, the previous 
ones are forgotten.
Replacing attr_requires_ of type Expr with a ListExpr solves the problem 
and produces the (expected) C++ code below :

     // Parse "a"
     // Parse "b"
     b_ = FixByteOrder(t_byteorder, *((uint16 const *) ((t_begin_of_data 
+ 2))));
     // Evaluate 'let' and 'withinput' fields
     c_ = b() * 2;
     d_ = b() * 3;
     a_ = FixByteOrder(t_byteorder, *((uint16 const *) (t_begin_of_data)));


I have written a small patch for this problem. I can submit it if you 
agree with my changes.


Thank you.


From lagoon7 at gmail.com  Wed Nov  2 19:46:07 2016
From: lagoon7 at gmail.com (Ludwig Goon)
Date: Wed, 2 Nov 2016 22:46:07 -0400
Subject: [Bro] Weird Log rotation in Bro IDS 2.5 Beta
Message-ID: <CAOHA7OkfbYGFU9XaG3jvE1EJqdDqC8_iDfs5QbUrtzWwveyfUg@mail.gmail.com>

Bro IDS was  storing all of my files in the logs/current directory for the
last couple of days. I restarted bro to see if the files would rotate into
the proper directories instead I got this.


drwxr-xr-x  2 root root 36864 Nov  2 22:21 2000-00-
drwxr-xr-x  2 root root  4096 Nov  2 22:21 2000-25-
drwxr-xr-x  2 root root  4096 Nov  2 22:21 2000-33-
drwxr-xr-x  2 root root  4096 Nov  2 22:20 2000-35-
drwxr-xr-x  2 root root  4096 Nov  2 22:21 2000-36-
drwxr-xr-x  2 root root  4096 Nov  2 22:21 2010-22-
drwxr-xr-x  2 root root  4096 Nov  2 22:21 2011-27-
drwxr-xr-x  2 root root  4096 Nov  2 22:20 2011-37-
drwxr-xr-x  2 root root  4096 Oct 29 00:00 2016-10-28
drwxr-xr-x  2 root root 20480 Oct 30 00:00 2016-10-29
drwxr-xr-x  2 root root 20480 Oct 30 23:00 2016-10-30
drwxr-xr-x  2 root root  4096 Oct 31 22:54 2016-10-31
drwxr-xr-x  2 root root  4096 Nov  2 22:20 2031-32-
drwxr-xr-x  2 root root  4096 Nov  2 22:21 2039-49-
drwxr-xr-x  2 root root  4096 Nov  2 22:21 2057-18-
drwxr-xr-x  2 root root  4096 Nov  2 22:21 2057-19-
lrwxrwxrwx  1 root root    19 Nov  2 22:22 current -> /data/bro/spool/bro

Anyone got any ideas why this happened?

Also I used "broctl deploy" first then that ran for a couple of days then I
ran "broctl stop" that when the weirdness happened after I ran broctl
deploy.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161102/e00afe3c/attachment.html 

From seth at icir.org  Thu Nov  3 04:57:07 2016
From: seth at icir.org (Seth Hall)
Date: Thu, 3 Nov 2016 07:57:07 -0400
Subject: [Bro] Weird Log rotation in Bro IDS 2.5 Beta
In-Reply-To: <CAOHA7OkfbYGFU9XaG3jvE1EJqdDqC8_iDfs5QbUrtzWwveyfUg@mail.gmail.com>
References: <CAOHA7OkfbYGFU9XaG3jvE1EJqdDqC8_iDfs5QbUrtzWwveyfUg@mail.gmail.com>
Message-ID: <D4F0E81D-D6A5-48E5-A86A-F9D6D0457948@icir.org>


> On Nov 2, 2016, at 10:46 PM, Ludwig Goon <lagoon7 at gmail.com> wrote:
> 
> Anyone got any ideas why this happened?

Are you writing your logs as JSON?  If you are, are you doing something like this too?

	redef LogAscii::json_timestamps = JSON::TS_ISO8601;

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Thu Nov  3 04:59:18 2016
From: seth at icir.org (Seth Hall)
Date: Thu, 3 Nov 2016 07:59:18 -0400
Subject: [Bro] Weird Log rotation in Bro IDS 2.5 Beta
In-Reply-To: <D4F0E81D-D6A5-48E5-A86A-F9D6D0457948@icir.org>
References: <CAOHA7OkfbYGFU9XaG3jvE1EJqdDqC8_iDfs5QbUrtzWwveyfUg@mail.gmail.com>
	<D4F0E81D-D6A5-48E5-A86A-F9D6D0457948@icir.org>
Message-ID: <AA6F9FB8-A741-41DF-8933-0DDE4D3B21F9@icir.org>


> On Nov 3, 2016, at 7:57 AM, Seth Hall <seth at icir.org> wrote:
> 
> 
>> On Nov 2, 2016, at 10:46 PM, Ludwig Goon <lagoon7 at gmail.com> wrote:
>> 
>> Anyone got any ideas why this happened?
> 
> Are you writing your logs as JSON?  If you are, are you doing something like this too?
> 
> 	redef LogAscii::json_timestamps = JSON::TS_ISO8601;

Wait, sorry. I emailed too quickly.  I was asking those questions because they are related to a bug, but I see that you are running the 2.5 beta and the bug is fixed there.  Are you making any changes to how you write out logs though?  If you are, that could point to another instance of the same bug that we missed.

Thanks,
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From philosnef at gmail.com  Thu Nov  3 05:14:55 2016
From: philosnef at gmail.com (erik clark)
Date: Thu, 3 Nov 2016 08:14:55 -0400
Subject: [Bro] conn history
Message-ID: <CAK6atxqYEiw2F++cTv_9bkEiZWrpoM2qvTnCWGtd78TT5jFRfg@mail.gmail.com>

What does a history of - imply about a connection in conn.log? I have a
significant number of conn events with that for a history, and I am
wondering if this is possibly because of duplicate packets. Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161103/0ec0ae6f/attachment.html 

From seth at icir.org  Thu Nov  3 05:34:05 2016
From: seth at icir.org (Seth Hall)
Date: Thu, 3 Nov 2016 08:34:05 -0400
Subject: [Bro] conn history
In-Reply-To: <CAK6atxqYEiw2F++cTv_9bkEiZWrpoM2qvTnCWGtd78TT5jFRfg@mail.gmail.com>
References: <CAK6atxqYEiw2F++cTv_9bkEiZWrpoM2qvTnCWGtd78TT5jFRfg@mail.gmail.com>
Message-ID: <C17276D9-46F5-4F7A-9B20-62E795E4C042@icir.org>


> On Nov 3, 2016, at 8:14 AM, erik clark <philosnef at gmail.com> wrote:
> 
> What does a history of - imply about a connection in conn.log? I have a significant number of conn events with that for a history, and I am wondering if this is possibly because of duplicate packets. Thanks!

I'm not sure off hand.  I checked some code and can't quite explain it.  Can you send me some of your conn log entries off list where you are seeing this?  I wonder if there are any other clues in the log.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From lagoon7 at gmail.com  Thu Nov  3 06:35:54 2016
From: lagoon7 at gmail.com (Ludwig Goon)
Date: Thu, 3 Nov 2016 09:35:54 -0400
Subject: [Bro] Weird Log rotation in Bro IDS 2.5 Beta
In-Reply-To: <AA6F9FB8-A741-41DF-8933-0DDE4D3B21F9@icir.org>
References: <CAOHA7OkfbYGFU9XaG3jvE1EJqdDqC8_iDfs5QbUrtzWwveyfUg@mail.gmail.com>
	<D4F0E81D-D6A5-48E5-A86A-F9D6D0457948@icir.org>
	<AA6F9FB8-A741-41DF-8933-0DDE4D3B21F9@icir.org>
Message-ID: <CAOHA7Ok8+s9t9hLsEFHGKtAoBHfQ5d=FCL=uH6K=FJqCBSgUSQ@mail.gmail.com>

Yes I am writing logs as JSON. But I don't have the ISO part on the end.
What is the correct way that line should read and is that the fix?

On Thursday, November 3, 2016, Seth Hall <seth at icir.org> wrote:

>
> > On Nov 3, 2016, at 7:57 AM, Seth Hall <seth at icir.org <javascript:;>>
> wrote:
> >
> >
> >> On Nov 2, 2016, at 10:46 PM, Ludwig Goon <lagoon7 at gmail.com
> <javascript:;>> wrote:
> >>
> >> Anyone got any ideas why this happened?
> >
> > Are you writing your logs as JSON?  If you are, are you doing something
> like this too?
> >
> >       redef LogAscii::json_timestamps = JSON::TS_ISO8601;
>
> Wait, sorry. I emailed too quickly.  I was asking those questions because
> they are related to a bug, but I see that you are running the 2.5 beta and
> the bug is fixed there.  Are you making any changes to how you write out
> logs though?  If you are, that could point to another instance of the same
> bug that we missed.
>
> Thanks,
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161103/775a61e4/attachment.html 

From jlay at slave-tothe-box.net  Thu Nov  3 06:59:19 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 03 Nov 2016 07:59:19 -0600
Subject: [Bro] New install connection summary
Message-ID: <b03c7601e074dd4c4e2b21fc862604ce@localhost>

New install config'd with --prefix=/opt/bro...first summary I received 
this:

Traceback (most recent call last):
   File "/opt/bro/bin/trace-summary", line 19, in <module>
     import SubnetTree
   File "/opt/bro/lib/broctl/SubnetTree.py", line 28, in <module>
     _SubnetTree = swig_import_helper()
   File "/opt/bro/lib/broctl/SubnetTree.py", line 24, in 
swig_import_helper
     _mod = imp.load_module('_SubnetTree', fp, pathname, description)
ImportError: dynamic module does not define init function 
(init_SubnetTree) Command exited with non-zero status 1
0:00.17 real, 0.03 user, 0.01 sys, 0K total memory

Clearly I'm missing something, but not sure what at this stage...any 
hints on this?  Thank you.

James

From seth at icir.org  Thu Nov  3 07:55:40 2016
From: seth at icir.org (Seth Hall)
Date: Thu, 3 Nov 2016 10:55:40 -0400
Subject: [Bro] Weird Log rotation in Bro IDS 2.5 Beta
In-Reply-To: <CAOHA7Ok8+s9t9hLsEFHGKtAoBHfQ5d=FCL=uH6K=FJqCBSgUSQ@mail.gmail.com>
References: <CAOHA7OkfbYGFU9XaG3jvE1EJqdDqC8_iDfs5QbUrtzWwveyfUg@mail.gmail.com>
	<D4F0E81D-D6A5-48E5-A86A-F9D6D0457948@icir.org>
	<AA6F9FB8-A741-41DF-8933-0DDE4D3B21F9@icir.org>
	<CAOHA7Ok8+s9t9hLsEFHGKtAoBHfQ5d=FCL=uH6K=FJqCBSgUSQ@mail.gmail.com>
Message-ID: <E1C345E8-E618-4BEA-A4DD-728384F035D7@icir.org>


> On Nov 3, 2016, at 9:35 AM, Ludwig Goon <lagoon7 at gmail.com> wrote:
> 
> Yes I am writing logs as JSON. But I don't have the ISO part on the end. What is the correct way that line should read and is that the fix?

Nope, that line I provided is not a fix.  We fixed an issue related to timestamp rendering in the 2.5 release.  Whatever problem you are encountering is unknown.  Is anyone else on the list seeing issues like this?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From lagoon7 at gmail.com  Thu Nov  3 08:59:38 2016
From: lagoon7 at gmail.com (Ludwig Goon)
Date: Thu, 3 Nov 2016 11:59:38 -0400
Subject: [Bro] Weird Log rotation in Bro IDS 2.5 Beta
In-Reply-To: <E1C345E8-E618-4BEA-A4DD-728384F035D7@icir.org>
References: <CAOHA7OkfbYGFU9XaG3jvE1EJqdDqC8_iDfs5QbUrtzWwveyfUg@mail.gmail.com>
	<D4F0E81D-D6A5-48E5-A86A-F9D6D0457948@icir.org>
	<AA6F9FB8-A741-41DF-8933-0DDE4D3B21F9@icir.org>
	<CAOHA7Ok8+s9t9hLsEFHGKtAoBHfQ5d=FCL=uH6K=FJqCBSgUSQ@mail.gmail.com>
	<E1C345E8-E618-4BEA-A4DD-728384F035D7@icir.org>
Message-ID: <CAOHA7On4ypLBfURTPk2WyF3LXvVaaWtn8EV6ezzu8EE7ZqDeGg@mail.gmail.com>

OK please let me know if you need further information and if this qualifies
as a bug.

Here is my redef I am using to write log files in JSON.
redef LogAscii::use_json = T;

>From your line if I want to write the timestamp other than EPOCH then I
should use the line you mentioned in addition to the redef I have.



On Thu, Nov 3, 2016 at 10:55 AM, Seth Hall <seth at icir.org> wrote:

>
> > On Nov 3, 2016, at 9:35 AM, Ludwig Goon <lagoon7 at gmail.com> wrote:
> >
> > Yes I am writing logs as JSON. But I don't have the ISO part on the end.
> What is the correct way that line should read and is that the fix?
>
> Nope, that line I provided is not a fix.  We fixed an issue related to
> timestamp rendering in the 2.5 release.  Whatever problem you are
> encountering is unknown.  Is anyone else on the list seeing issues like
> this?
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161103/99659819/attachment.html 

From johanna at icir.org  Thu Nov  3 11:19:29 2016
From: johanna at icir.org (Johanna Amann)
Date: Thu, 3 Nov 2016 11:19:29 -0700
Subject: [Bro] Bro 2.5 Beta2 available
Message-ID: <20161103181929.lzd4izsdtpzvl74p@Beezling.local>

The second beta for Bro 2.5 is now available for testing and can be
downloaded at:

    https://bro.org/download/index.html

Binary packages are also available at:

     https://bro.org/download/beta-packages.html

If you already installed the binary packages of the earlier beta, the new
beta version can be installed by an invoking the update functionality of
the package manager of your distribution.

The notable changes since the first 2.5 beta are:

- Lots of fixes for the SMB analuyzer. Note that the SMB analyzer still is
  disabled by default.

- Support for (draft) TLS 1.3

- Lots of various small fixes in Bro and Broctl.

For more information see the NEWS and CHANGES files:
https://www.bro.org/documentation/beta/NEWS.bro.html
https://www.bro.org/documentation/beta/CHANGES.bro.txt

Feel free to use this mailing list or the bug tracker (tracker.bro.org) to
provide feedback or report problems. We are happy to receive feedback and
expect only minor changes before the release.

Johanna

From luis.e.jimenez01 at gmail.com  Thu Nov  3 12:27:57 2016
From: luis.e.jimenez01 at gmail.com (Luis Jimenez)
Date: Thu, 3 Nov 2016 15:27:57 -0400
Subject: [Bro] Layer 2 Info
Message-ID: <CAOrRJ9QoSUpcJWUWd2qF_W7o2tReRGNG8W7avOO-SfwF=A31FQ@mail.gmail.com>

Is there any way to glean layer 2 information from bro? Or maybe a reliable
means of correlating IPs to hostnames?

We are running into issues where dynamic IP addressing is severely
hindering the ability to track behavior identified by analysis of bro logs.

Thanks for the help!

Luis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161103/f4ecd9ad/attachment.html 

From jlay at slave-tothe-box.net  Thu Nov  3 13:33:59 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 03 Nov 2016 14:33:59 -0600
Subject: [Bro] New install connection summary
In-Reply-To: <b03c7601e074dd4c4e2b21fc862604ce@localhost>
References: <b03c7601e074dd4c4e2b21fc862604ce@localhost>
Message-ID: <35a000c6daa0d2060548b5f525521c14@localhost>

On 2016-11-03 07:59, James Lay wrote:
> New install config'd with --prefix=/opt/bro...first summary I received
> this:
> 
> Traceback (most recent call last):
>    File "/opt/bro/bin/trace-summary", line 19, in <module>
>      import SubnetTree
>    File "/opt/bro/lib/broctl/SubnetTree.py", line 28, in <module>
>      _SubnetTree = swig_import_helper()
>    File "/opt/bro/lib/broctl/SubnetTree.py", line 24, in
> swig_import_helper
>      _mod = imp.load_module('_SubnetTree', fp, pathname, description)
> ImportError: dynamic module does not define init function
> (init_SubnetTree) Command exited with non-zero status 1
> 0:00.17 real, 0.03 user, 0.01 sys, 0K total memory
> 
> Clearly I'm missing something, but not sure what at this stage...any
> hints on this?  Thank you.
> 
> James
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


Any takers on this?  I got about 3.5 hours before the box in UTC time 
gets to run the summary again.  Thank you.

James

From jazoff at illinois.edu  Thu Nov  3 13:43:25 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Thu, 3 Nov 2016 20:43:25 +0000
Subject: [Bro] New install connection summary
In-Reply-To: <35a000c6daa0d2060548b5f525521c14@localhost>
References: <b03c7601e074dd4c4e2b21fc862604ce@localhost>
	<35a000c6daa0d2060548b5f525521c14@localhost>
Message-ID: <D18E53A5-6290-4236-8058-183C8CE935F0@illinois.edu>

What distribution is this and what version(s) of python are installed?


-- 
- Justin Azoff

> On Nov 3, 2016, at 4:33 PM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> On 2016-11-03 07:59, James Lay wrote:
>> New install config'd with --prefix=/opt/bro...first summary I received
>> this:
>> 
>> Traceback (most recent call last):
>>   File "/opt/bro/bin/trace-summary", line 19, in <module>
>>     import SubnetTree
>>   File "/opt/bro/lib/broctl/SubnetTree.py", line 28, in <module>
>>     _SubnetTree = swig_import_helper()
>>   File "/opt/bro/lib/broctl/SubnetTree.py", line 24, in
>> swig_import_helper
>>     _mod = imp.load_module('_SubnetTree', fp, pathname, description)
>> ImportError: dynamic module does not define init function
>> (init_SubnetTree) Command exited with non-zero status 1
>> 0:00.17 real, 0.03 user, 0.01 sys, 0K total memory
>> 
>> Clearly I'm missing something, but not sure what at this stage...any
>> hints on this?  Thank you.
>> 
>> James
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> 
> Any takers on this?  I got about 3.5 hours before the box in UTC time 
> gets to run the summary again.  Thank you.
> 
> James
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From jlay at slave-tothe-box.net  Thu Nov  3 13:47:12 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 03 Nov 2016 14:47:12 -0600
Subject: [Bro] New install connection summary
In-Reply-To: <D18E53A5-6290-4236-8058-183C8CE935F0@illinois.edu>
References: <b03c7601e074dd4c4e2b21fc862604ce@localhost>
	<35a000c6daa0d2060548b5f525521c14@localhost>
	<D18E53A5-6290-4236-8058-183C8CE935F0@illinois.edu>
Message-ID: <a332dc97c2d108e74d74b277c3e31ac8@localhost>

Thanks Justin.  Here's the info:

Ubuntu 16.04.1 LTS 64 bit

[20:45:36 ~/current$] python2 --version
Python 2.7.12

[20:45:39 ~/current$] python3 --version
Python 3.5.2

Everything else is working great...I have a test machine at home, 
granted with beta's and git's, with the same OS and python versions with 
no issues.  It's just this one thing..crazytown!

James

On 2016-11-03 14:43, Azoff, Justin S wrote:
> What distribution is this and what version(s) of python are installed?
> 
> 
> --
> - Justin Azoff
> 
>> On Nov 3, 2016, at 4:33 PM, James Lay <jlay at slave-tothe-box.net> 
>> wrote:
>> 
>> On 2016-11-03 07:59, James Lay wrote:
>>> New install config'd with --prefix=/opt/bro...first summary I 
>>> received
>>> this:
>>> 
>>> Traceback (most recent call last):
>>>   File "/opt/bro/bin/trace-summary", line 19, in <module>
>>>     import SubnetTree
>>>   File "/opt/bro/lib/broctl/SubnetTree.py", line 28, in <module>
>>>     _SubnetTree = swig_import_helper()
>>>   File "/opt/bro/lib/broctl/SubnetTree.py", line 24, in
>>> swig_import_helper
>>>     _mod = imp.load_module('_SubnetTree', fp, pathname, description)
>>> ImportError: dynamic module does not define init function
>>> (init_SubnetTree) Command exited with non-zero status 1
>>> 0:00.17 real, 0.03 user, 0.01 sys, 0K total memory
>>> 
>>> Clearly I'm missing something, but not sure what at this stage...any
>>> hints on this?  Thank you.
>>> 
>>> James
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> 
>> 
>> Any takers on this?  I got about 3.5 hours before the box in UTC time
>> gets to run the summary again.  Thank you.
>> 
>> James
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

From jazoff at illinois.edu  Thu Nov  3 13:51:13 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Thu, 3 Nov 2016 20:51:13 +0000
Subject: [Bro] New install connection summary
In-Reply-To: <a332dc97c2d108e74d74b277c3e31ac8@localhost>
References: <b03c7601e074dd4c4e2b21fc862604ce@localhost>
	<35a000c6daa0d2060548b5f525521c14@localhost>
	<D18E53A5-6290-4236-8058-183C8CE935F0@illinois.edu>
	<a332dc97c2d108e74d74b277c3e31ac8@localhost>
Message-ID: <80176A37-118E-47DC-9E2D-A8EB41BCD26C@illinois.edu>


> On Nov 3, 2016, at 4:47 PM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> Thanks Justin.  Here's the info:
> 
> Ubuntu 16.04.1 LTS 64 bit
> 
> [20:45:36 ~/current$] python2 --version
> Python 2.7.12
> 
> [20:45:39 ~/current$] python3 --version
> Python 3.5.2
> 
> Everything else is working great...I have a test machine at home, granted with beta's and git's, with the same OS and python versions with no issues.  It's just this one thing..crazytown!

Ah.. that's great (that you have a similar working machine)

Does 'python' default to the same version on both machines?

Can you compare your installed packages?  I wonder if you have python-dev on one machine and python3-dev on the other, or a different version of swig.

-- 
- Justin Azoff



From jlay at slave-tothe-box.net  Thu Nov  3 14:03:57 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 03 Nov 2016 15:03:57 -0600
Subject: [Bro] New install connection summary
In-Reply-To: <80176A37-118E-47DC-9E2D-A8EB41BCD26C@illinois.edu>
References: <b03c7601e074dd4c4e2b21fc862604ce@localhost>
	<35a000c6daa0d2060548b5f525521c14@localhost>
	<D18E53A5-6290-4236-8058-183C8CE935F0@illinois.edu>
	<a332dc97c2d108e74d74b277c3e31ac8@localhost>
	<80176A37-118E-47DC-9E2D-A8EB41BCD26C@illinois.edu>
Message-ID: <f993db3e1ff86566c89552b5b0189467@localhost>

On 2016-11-03 14:51, Azoff, Justin S wrote:
>> On Nov 3, 2016, at 4:47 PM, James Lay <jlay at slave-tothe-box.net> 
>> wrote:
>> 
>> Thanks Justin.  Here's the info:
>> 
>> Ubuntu 16.04.1 LTS 64 bit
>> 
>> [20:45:36 ~/current$] python2 --version
>> Python 2.7.12
>> 
>> [20:45:39 ~/current$] python3 --version
>> Python 3.5.2
>> 
>> Everything else is working great...I have a test machine at home, 
>> granted with beta's and git's, with the same OS and python versions 
>> with no issues.  It's just this one thing..crazytown!
> 
> Ah.. that's great (that you have a similar working machine)
> 
> Does 'python' default to the same version on both machines?
> 
> Can you compare your installed packages?  I wonder if you have
> python-dev on one machine and python3-dev on the other, or a different
> version of swig.

Yep...they both have pretty much the same thing...safe one is 
standalone, the other using pfring with workers and all that jazz.  It's 
kind of weird.  Like I said...everything else works great.  Is there a 
debug mode I can run?  Thank you.

From jazoff at illinois.edu  Thu Nov  3 14:08:30 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Thu, 3 Nov 2016 21:08:30 +0000
Subject: [Bro] New install connection summary
In-Reply-To: <f993db3e1ff86566c89552b5b0189467@localhost>
References: <b03c7601e074dd4c4e2b21fc862604ce@localhost>
	<35a000c6daa0d2060548b5f525521c14@localhost>
	<D18E53A5-6290-4236-8058-183C8CE935F0@illinois.edu>
	<a332dc97c2d108e74d74b277c3e31ac8@localhost>
	<80176A37-118E-47DC-9E2D-A8EB41BCD26C@illinois.edu>
	<f993db3e1ff86566c89552b5b0189467@localhost>
Message-ID: <6BA65CD5-FB50-46EA-9929-05057D5B3DDE@illinois.edu>


> On Nov 3, 2016, at 5:03 PM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> Yep...they both have pretty much the same thing...safe one is standalone, the other using pfring with workers and all that jazz.  It's kind of weird.  Like I said...everything else works great.  Is there a debug mode I can run?  Thank you.

Not really.. one thing that might help would be to run

ldd /opt/bro/lib/broctl/_SubnetTree.so

on both machines and see if they are different in any way.


-- 
- Justin Azoff



From jlay at slave-tothe-box.net  Thu Nov  3 14:13:29 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 03 Nov 2016 15:13:29 -0600
Subject: [Bro] New install connection summary
In-Reply-To: <6BA65CD5-FB50-46EA-9929-05057D5B3DDE@illinois.edu>
References: <b03c7601e074dd4c4e2b21fc862604ce@localhost>
	<35a000c6daa0d2060548b5f525521c14@localhost>
	<D18E53A5-6290-4236-8058-183C8CE935F0@illinois.edu>
	<a332dc97c2d108e74d74b277c3e31ac8@localhost>
	<80176A37-118E-47DC-9E2D-A8EB41BCD26C@illinois.edu>
	<f993db3e1ff86566c89552b5b0189467@localhost>
	<6BA65CD5-FB50-46EA-9929-05057D5B3DDE@illinois.edu>
Message-ID: <c0bb421c450519a30af91ef22c4e3140@localhost>

On 2016-11-03 15:08, Azoff, Justin S wrote:
>> On Nov 3, 2016, at 5:03 PM, James Lay <jlay at slave-tothe-box.net> 
>> wrote:
>> 
>> Yep...they both have pretty much the same thing...safe one is 
>> standalone, the other using pfring with workers and all that jazz.  
>> It's kind of weird.  Like I said...everything else works great.  Is 
>> there a debug mode I can run?  Thank you.
> 
> Not really.. one thing that might help would be to run
> 
> ldd /opt/bro/lib/broctl/_SubnetTree.so
> 
> on both machines and see if they are different in any way.


Interesting...working box:
         linux-vdso.so.1 =>  (0x00007ffe783fc000)
         libpython2.7.so.1.0 => 
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007f49e0ab6000)
         libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 
(0x00007f49e0899000)
         libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 
(0x00007f49e0516000)
         libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 
(0x00007f49e0300000)
         libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 
(0x00007f49dff37000)
         libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 
(0x00007f49dfd1c000)
         libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 
(0x00007f49dfb18000)
         libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 
(0x00007f49df915000)
         libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 
(0x00007f49df60b000)
         /lib64/ld-linux-x86-64.so.2 (0x000055f3a7b25000)

Non-working:
         linux-vdso.so.1 =>  (0x00007ffe6289b000)
         libpython3.5m.so.1.0 => 
/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0 (0x00007f6a55e28000)
         libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 
(0x00007f6a55c0b000)
         libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 
(0x00007f6a55888000)
         libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 
(0x00007f6a55672000)
         libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 
(0x00007f6a552a9000)
         libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 
(0x00007f6a5507f000)
         libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 
(0x00007f6a54e65000)
         libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 
(0x00007f6a54c61000)
         libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 
(0x00007f6a54a5d000)
         libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 
(0x00007f6a54754000)
         /lib64/ld-linux-x86-64.so.2 (0x000055d9ea7fa000)

no workie with python3 perhaps?

James

From jives at security.berkeley.edu  Thu Nov  3 15:36:48 2016
From: jives at security.berkeley.edu (John Ives)
Date: Thu, 3 Nov 2016 15:36:48 -0700
Subject: [Bro] changing log output
Message-ID: <44b643e4-4251-3379-fa38-b838a6206306@security.berkeley.edu>

I'm trying to configure bro to work within some proposed privacy
policies. For example, one of the things we may not be allowed to store
is the http traffic logs. I want bro to still know (internally) what is
happening in these streams so that it can use it for other functions
(like sqli detection which loads http), just not output the normal logs.
 I had thought to do this through Notice::ignored_types
in local.bro, however the following is still outputting the http.log file.

redef Notice::ignored_types += {
  SSL::Invalid_Server_Cert,
  HTTP::LOG,
 };

Additionally, I suspect that while this method (if I get it to work) may
result in sqli notices, I am not sure it will result in me getting the
attack data. For example, if a sqli attack is detected, I would like the
http.log style string to be output to a file.

Any suggestions on how first to prevent the http.log file creation and
then make sure the offending traffic is recorded for detect-sqli.bro?

Yours,

John

-- 
------------------------------------------------------------------------
John Ives
Information Security & Policy			    Phone (510) 229-8676
University of California, Berkeley
------------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 481 bytes
Desc: OpenPGP digital signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161103/4f1f3681/attachment.bin 

From jlamps at sandia.gov  Thu Nov  3 15:56:49 2016
From: jlamps at sandia.gov (Lamps, Jereme)
Date: Thu, 3 Nov 2016 22:56:49 +0000
Subject: [Bro] [EXTERNAL]  changing log output
In-Reply-To: <44b643e4-4251-3379-fa38-b838a6206306@security.berkeley.edu>
References: <44b643e4-4251-3379-fa38-b838a6206306@security.berkeley.edu>
Message-ID: <D4411C98.9824%jlamps@sandia.gov>

This will do it for you I think:

event bro_init() {
 Log::disable_stream(HTTP::LOG);
}



Jereme

On 11/3/16, 4:36 PM, "bro-bounces at bro.org on behalf of John Ives"
<bro-bounces at bro.org on behalf of jives at security.berkeley.edu> wrote:

>I'm trying to configure bro to work within some proposed privacy
>policies. For example, one of the things we may not be allowed to store
>is the http traffic logs. I want bro to still know (internally) what is
>happening in these streams so that it can use it for other functions
>(like sqli detection which loads http), just not output the normal logs.
> I had thought to do this through Notice::ignored_types
>in local.bro, however the following is still outputting the http.log file.
>
>redef Notice::ignored_types += {
>  SSL::Invalid_Server_Cert,
>  HTTP::LOG,
> };
>
>Additionally, I suspect that while this method (if I get it to work) may
>result in sqli notices, I am not sure it will result in me getting the
>attack data. For example, if a sqli attack is detected, I would like the
>http.log style string to be output to a file.
>
>Any suggestions on how first to prevent the http.log file creation and
>then make sure the offending traffic is recorded for detect-sqli.bro?
>
>Yours,
>
>John
>
>-- 
>------------------------------------------------------------------------
>John Ives
>Information Security & Policy			    Phone (510) 229-8676
>University of California, Berkeley
>------------------------------------------------------------------------
>



From jives at security.berkeley.edu  Thu Nov  3 16:30:30 2016
From: jives at security.berkeley.edu (John Ives)
Date: Thu, 3 Nov 2016 16:30:30 -0700
Subject: [Bro] [EXTERNAL] changing log output
In-Reply-To: <D4411C98.9824%jlamps@sandia.gov>
References: <44b643e4-4251-3379-fa38-b838a6206306@security.berkeley.edu>
	<D4411C98.9824%jlamps@sandia.gov>
Message-ID: <9ca17cfb-3d2e-535e-1bb6-4894b124d9b6@security.berkeley.edu>

Jereme,

Thank you very much that seems to have done the trick for disabling the
http.log.

John


On 11/3/16 3:56 PM, Lamps, Jereme wrote:
> This will do it for you I think:
> 
> event bro_init() {
>  Log::disable_stream(HTTP::LOG);
> }
> 
> 
> 
> Jereme
> 
> On 11/3/16, 4:36 PM, "bro-bounces at bro.org on behalf of John Ives"
> <bro-bounces at bro.org on behalf of jives at security.berkeley.edu> wrote:
> 
>> I'm trying to configure bro to work within some proposed privacy
>> policies. For example, one of the things we may not be allowed to store
>> is the http traffic logs. I want bro to still know (internally) what is
>> happening in these streams so that it can use it for other functions
>> (like sqli detection which loads http), just not output the normal logs.
>> I had thought to do this through Notice::ignored_types
>> in local.bro, however the following is still outputting the http.log file.
>>
>> redef Notice::ignored_types += {
>>  SSL::Invalid_Server_Cert,
>>  HTTP::LOG,
>> };
>>
>> Additionally, I suspect that while this method (if I get it to work) may
>> result in sqli notices, I am not sure it will result in me getting the
>> attack data. For example, if a sqli attack is detected, I would like the
>> http.log style string to be output to a file.
>>
>> Any suggestions on how first to prevent the http.log file creation and
>> then make sure the offending traffic is recorded for detect-sqli.bro?
>>
>> Yours,
>>
>> John
>>
>> -- 
>> ------------------------------------------------------------------------
>> John Ives
>> Information Security & Policy			    Phone (510) 229-8676
>> University of California, Berkeley
>> ------------------------------------------------------------------------
>>
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 

-- 
------------------------------------------------------------------------
John Ives
Information Security & Policy			    Phone (510) 229-8676
University of California, Berkeley
------------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 481 bytes
Desc: OpenPGP digital signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161103/a46d9594/attachment.bin 

From jan.grashoefer at gmail.com  Thu Nov  3 18:02:43 2016
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Fri, 4 Nov 2016 02:02:43 +0100
Subject: [Bro] Layer 2 Info
In-Reply-To: <CAOrRJ9QoSUpcJWUWd2qF_W7o2tReRGNG8W7avOO-SfwF=A31FQ@mail.gmail.com>
References: <CAOrRJ9QoSUpcJWUWd2qF_W7o2tReRGNG8W7avOO-SfwF=A31FQ@mail.gmail.com>
Message-ID: <8f54d155-3f0f-3062-a345-062435a54272@gmail.com>

Hi Luis,

> Is there any way to glean layer 2 information from bro? Or maybe a reliable
> means of correlating IPs to hostnames?

Bro 2.5 (beta2 available) will support logging of MAC addresses:
https://github.com/bro/bro/blob/master/scripts/site/local.bro#L98

Best regards,
Jan

From dnthayer at illinois.edu  Thu Nov  3 18:23:26 2016
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Thu, 3 Nov 2016 20:23:26 -0500
Subject: [Bro] New install connection summary
In-Reply-To: <c0bb421c450519a30af91ef22c4e3140@localhost>
References: <b03c7601e074dd4c4e2b21fc862604ce@localhost>
	<35a000c6daa0d2060548b5f525521c14@localhost>
	<D18E53A5-6290-4236-8058-183C8CE935F0@illinois.edu>
	<a332dc97c2d108e74d74b277c3e31ac8@localhost>
	<80176A37-118E-47DC-9E2D-A8EB41BCD26C@illinois.edu>
	<f993db3e1ff86566c89552b5b0189467@localhost>
	<6BA65CD5-FB50-46EA-9929-05057D5B3DDE@illinois.edu>
	<c0bb421c450519a30af91ef22c4e3140@localhost>
Message-ID: <8ccdb1b8-479b-301e-cc87-4bc556cbc576@illinois.edu>

On 11/3/16 4:13 PM, James Lay wrote:
> On 2016-11-03 15:08, Azoff, Justin S wrote:
>>> On Nov 3, 2016, at 5:03 PM, James Lay <jlay at slave-tothe-box.net>
>>> wrote:
>>>
>>> Yep...they both have pretty much the same thing...safe one is
>>> standalone, the other using pfring with workers and all that jazz.
>>> It's kind of weird.  Like I said...everything else works great.  Is
>>> there a debug mode I can run?  Thank you.
>>
>> Not really.. one thing that might help would be to run
>>
>> ldd /opt/bro/lib/broctl/_SubnetTree.so
>>
>> on both machines and see if they are different in any way.
>
>
> Interesting...working box:
>          linux-vdso.so.1 =>  (0x00007ffe783fc000)
>          libpython2.7.so.1.0 =>
> /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007f49e0ab6000)
>          libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
> (0x00007f49e0899000)
>          libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> (0x00007f49e0516000)
>          libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
> (0x00007f49e0300000)
>          libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
> (0x00007f49dff37000)
>          libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
> (0x00007f49dfd1c000)
>          libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
> (0x00007f49dfb18000)
>          libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1
> (0x00007f49df915000)
>          libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
> (0x00007f49df60b000)
>          /lib64/ld-linux-x86-64.so.2 (0x000055f3a7b25000)
>
> Non-working:
>          linux-vdso.so.1 =>  (0x00007ffe6289b000)
>          libpython3.5m.so.1.0 =>
> /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0 (0x00007f6a55e28000)
>          libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
> (0x00007f6a55c0b000)
>          libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> (0x00007f6a55888000)
>          libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
> (0x00007f6a55672000)
>          libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
> (0x00007f6a552a9000)
>          libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1
> (0x00007f6a5507f000)
>          libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
> (0x00007f6a54e65000)
>          libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
> (0x00007f6a54c61000)
>          libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1
> (0x00007f6a54a5d000)
>          libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
> (0x00007f6a54754000)
>          /lib64/ld-linux-x86-64.so.2 (0x000055d9ea7fa000)
>
> no workie with python3 perhaps?
>
> James


It won't work if you build bro with Python 3 and then run
trace-summary with Python 2.  Have you tried the beta2 that
was released today?

From jlay at slave-tothe-box.net  Fri Nov  4 04:36:23 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 04 Nov 2016 05:36:23 -0600
Subject: [Bro] New install connection summary
In-Reply-To: <8ccdb1b8-479b-301e-cc87-4bc556cbc576@illinois.edu>
References: <b03c7601e074dd4c4e2b21fc862604ce@localhost>
	<35a000c6daa0d2060548b5f525521c14@localhost>
	<D18E53A5-6290-4236-8058-183C8CE935F0@illinois.edu>
	<a332dc97c2d108e74d74b277c3e31ac8@localhost>
	<80176A37-118E-47DC-9E2D-A8EB41BCD26C@illinois.edu>
	<f993db3e1ff86566c89552b5b0189467@localhost>
	<6BA65CD5-FB50-46EA-9929-05057D5B3DDE@illinois.edu>
	<c0bb421c450519a30af91ef22c4e3140@localhost>
	<8ccdb1b8-479b-301e-cc87-4bc556cbc576@illinois.edu>
Message-ID: <1478259383.3611.2.camel@slave-tothe-box.net>

On Thu, 2016-11-03 at 20:23 -0500, Daniel Thayer wrote:
> On 11/3/16 4:13 PM, James Lay wrote:
> > 
> > On 2016-11-03 15:08, Azoff, Justin S wrote:
> > > 
> > > > 
> > > > On Nov 3, 2016, at 5:03 PM, James Lay <jlay at slave-tothe-box.net
> > > > >
> > > > wrote:
> > > > 
> > > > Yep...they both have pretty much the same thing...safe one is
> > > > standalone, the other using pfring with workers and all that
> > > > jazz.
> > > > It's kind of weird.??Like I said...everything else works
> > > > great.??Is
> > > > there a debug mode I can run???Thank you.
> > > Not really.. one thing that might help would be to run
> > > 
> > > ldd /opt/bro/lib/broctl/_SubnetTree.so
> > > 
> > > on both machines and see if they are different in any way.
> > 
> > Interesting...working box:
> > ?????????linux-vdso.so.1 =>??(0x00007ffe783fc000)
> > ?????????libpython2.7.so.1.0 =>
> > /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007f49e0ab6000)
> > ?????????libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
> > (0x00007f49e0899000)
> > ?????????libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> > (0x00007f49e0516000)
> > ?????????libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
> > (0x00007f49e0300000)
> > ?????????libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
> > (0x00007f49dff37000)
> > ?????????libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
> > (0x00007f49dfd1c000)
> > ?????????libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
> > (0x00007f49dfb18000)
> > ?????????libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1
> > (0x00007f49df915000)
> > ?????????libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
> > (0x00007f49df60b000)
> > ?????????/lib64/ld-linux-x86-64.so.2 (0x000055f3a7b25000)
> > 
> > Non-working:
> > ?????????linux-vdso.so.1 =>??(0x00007ffe6289b000)
> > ?????????libpython3.5m.so.1.0 =>
> > /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0 (0x00007f6a55e28000)
> > ?????????libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
> > (0x00007f6a55c0b000)
> > ?????????libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> > (0x00007f6a55888000)
> > ?????????libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
> > (0x00007f6a55672000)
> > ?????????libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
> > (0x00007f6a552a9000)
> > ?????????libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1
> > (0x00007f6a5507f000)
> > ?????????libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
> > (0x00007f6a54e65000)
> > ?????????libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
> > (0x00007f6a54c61000)
> > ?????????libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1
> > (0x00007f6a54a5d000)
> > ?????????libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
> > (0x00007f6a54754000)
> > ?????????/lib64/ld-linux-x86-64.so.2 (0x000055d9ea7fa000)
> > 
> > no workie with python3 perhaps?
> > 
> > James
> 
> It won't work if you build bro with Python 3 and then run
> trace-summary with Python 2.??Have you tried the beta2 that
> was released today?
Thanks Daniel,
The issue was I had python3 dev install, but not python2. ?So bro
compiled with 3, but ran with 2...makes sense why it didn't work. ?I'm
currently testing the beta and git elsewhere..looking forward to the
new release...thank you.
James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/0569e7b1/attachment-0001.html 

From simone.rotondo at aizoongroup.com  Fri Nov  4 08:12:33 2016
From: simone.rotondo at aizoongroup.com (Rotondo Simone)
Date: Fri, 4 Nov 2016 15:12:33 +0000
Subject: [Bro] Bro stable release date
Message-ID: <1a7c8869817f45468a7bd21e8cdd9de0@SRVEX03.aizoon.local>

Hi,

I'd like to switch from Bro release 2.4 to 2.5 because of its new features, but currently v2.5 is a beta version yet.

Do you know if Bro 2.5 will be released as "STABLE" in near term ?
Do you have a rough idea for release date ?

Thanks for your support and your work.

Best Regards
Simone

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/49eb7cfb/attachment.html 

From jlay at slave-tothe-box.net  Fri Nov  4 08:30:24 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 04 Nov 2016 09:30:24 -0600
Subject: [Bro] Bro stable release date
In-Reply-To: <1a7c8869817f45468a7bd21e8cdd9de0@SRVEX03.aizoon.local>
References: <1a7c8869817f45468a7bd21e8cdd9de0@SRVEX03.aizoon.local>
Message-ID: <af64a8fdac804e6a74a1156a33dfef7b@localhost>

On 2016-11-04 09:12, Rotondo Simone wrote:
> Hi,
> 
> I'd like to switch from Bro release 2.4 to 2.5 because of its new
> features, but currently v2.5 is a beta version yet.
> 
> Do you know if Bro 2.5 will be released as "STABLE" in near term ?
> 
> Do you have a rough idea for release date ?
> 
> Thanks for your support and your work.
> 
> Best Regards
> 
> Simone
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

Ya +1 to this...people itch when I talk about putting a beta into 
production.

James

From slagell at illinois.edu  Fri Nov  4 08:37:25 2016
From: slagell at illinois.edu (Slagell, Adam J)
Date: Fri, 4 Nov 2016 15:37:25 +0000
Subject: [Bro] Bro stable release date
In-Reply-To: <1a7c8869817f45468a7bd21e8cdd9de0@SRVEX03.aizoon.local>
References: <1a7c8869817f45468a7bd21e8cdd9de0@SRVEX03.aizoon.local>
Message-ID: <95562F8B-4596-4742-8C1B-54116B2FCD09@illinois.edu>


> On Nov 4, 2016, at 10:12 AM, Rotondo Simone <simone.rotondo at aizoongroup.com> wrote:
> 
> Do you know if Bro 2.5 will be released as ?STABLE? in near term ?
> Do you have a rough idea for release date ?


Weeks, not months, assuming nothing goes wrong. But we want the second beta to at least get a couple of weeks of testing in production at multiple sites.
------

Adam J. Slagell
Chief Information Security Officer
Director, Cybersecurity Division
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info

"Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." 








-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/788e47f9/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3579 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/788e47f9/attachment.bin 

From jlay at slave-tothe-box.net  Fri Nov  4 08:57:27 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 04 Nov 2016 09:57:27 -0600
Subject: [Bro] Bro stable release date
In-Reply-To: <95562F8B-4596-4742-8C1B-54116B2FCD09@illinois.edu>
References: <1a7c8869817f45468a7bd21e8cdd9de0@SRVEX03.aizoon.local>
	<95562F8B-4596-4742-8C1B-54116B2FCD09@illinois.edu>
Message-ID: <907990e095a9793c8ebad032b47018b7@localhost>

Adam which would be more helpful for Bro...to test the official packaged 
beta, or git?

James

On 2016-11-04 09:37, Slagell, Adam J wrote:
>> On Nov 4, 2016, at 10:12 AM, Rotondo Simone
>> <simone.rotondo at aizoongroup.com> wrote:
>> 
>> Do you know if Bro 2.5 will be released as ?STABLE? in near term
>> ?
>> Do you have a rough idea for release date ?
> 
> Weeks, not months, assuming nothing goes wrong. But we want the second
> beta to at least get a couple of weeks of testing in production at
> multiple sites.
> 
> ------
> 
> Adam J. Slagell
> Chief Information Security Officer
> Director, Cybersecurity Division
> National Center for Supercomputing Applications
> University of Illinois at Urbana-Champaign
> www.slagell.info [1]
> 
> "Under the Illinois Freedom of Information Act (FOIA), any written
> communication to or from University employees regarding University
> business is a public record and may be subject to public disclosure."
> 
> 
> 
> Links:
> ------
> [1] http://www.slagell.info
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

From slagell at illinois.edu  Fri Nov  4 09:05:52 2016
From: slagell at illinois.edu (Slagell, Adam J)
Date: Fri, 4 Nov 2016 16:05:52 +0000
Subject: [Bro] Bro stable release date
In-Reply-To: <907990e095a9793c8ebad032b47018b7@localhost>
References: <1a7c8869817f45468a7bd21e8cdd9de0@SRVEX03.aizoon.local>
	<95562F8B-4596-4742-8C1B-54116B2FCD09@illinois.edu>
	<907990e095a9793c8ebad032b47018b7@localhost>
Message-ID: <4D5D776A-D2CC-412A-AE30-A0D5BF34FE4B@illinois.edu>

If you are keeping up with master, that?s fine. The reason we released the second beta was because a lot of fixes had already been pushed into the master since the first beta, and we wanted to get people testing those.

> On Nov 4, 2016, at 10:57 AM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> Adam which would be more helpful for Bro...to test the official packaged 
> beta, or git?
> 
> James
> 
> On 2016-11-04 09:37, Slagell, Adam J wrote:
>>> On Nov 4, 2016, at 10:12 AM, Rotondo Simone
>>> <simone.rotondo at aizoongroup.com> wrote:
>>> 
>>> Do you know if Bro 2.5 will be released as ?STABLE? in near term
>>> ?
>>> Do you have a rough idea for release date ?
>> 
>> Weeks, not months, assuming nothing goes wrong. But we want the second
>> beta to at least get a couple of weeks of testing in production at
>> multiple sites.
>> 
>> ------
>> 
>> Adam J. Slagell
>> Chief Information Security Officer
>> Director, Cybersecurity Division
>> National Center for Supercomputing Applications
>> University of Illinois at Urbana-Champaign
>> www.slagell.info <http://www.slagell.info/> [1]
>> 
>> "Under the Illinois Freedom of Information Act (FOIA), any written
>> communication to or from University employees regarding University
>> business is a public record and may be subject to public disclosure."
>> 
>> 
>> 
>> Links:
>> ------
>> [1] http://www.slagell.info <http://www.slagell.info/>
>> 
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org <mailto:bro at bro-ids.org>
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro <http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org <mailto:bro at bro-ids.org>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro <http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
------

Adam J. Slagell
Chief Information Security Officer
Director, Cybersecurity Division
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info

"Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." 








-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/77ac1964/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3579 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/77ac1964/attachment-0001.bin 

From jlay at slave-tothe-box.net  Fri Nov  4 09:11:01 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 04 Nov 2016 10:11:01 -0600
Subject: [Bro] Bro stable release date
In-Reply-To: <4D5D776A-D2CC-412A-AE30-A0D5BF34FE4B@illinois.edu>
References: <1a7c8869817f45468a7bd21e8cdd9de0@SRVEX03.aizoon.local>
	<95562F8B-4596-4742-8C1B-54116B2FCD09@illinois.edu>
	<907990e095a9793c8ebad032b47018b7@localhost>
	<4D5D776A-D2CC-412A-AE30-A0D5BF34FE4B@illinois.edu>
Message-ID: <bfe27c4a8aefd914ee4c0184ccf7caa4@localhost>

Good deal...I'll stick with master then thanks.

James

On 2016-11-04 10:05, Slagell, Adam J wrote:
> If you are keeping up with master, that?s fine. The reason we
> released the second beta was because a lot of fixes had already been
> pushed into the master since the first beta, and we wanted to get
> people testing those.
> 
>> On Nov 4, 2016, at 10:57 AM, James Lay <jlay at slave-tothe-box.net>
>> wrote:
>> Adam which would be more helpful for Bro...to test the official
>> packaged
>> beta, or git?
>> 
>> James
>> 
>> On 2016-11-04 09:37, Slagell, Adam J wrote:
>> On Nov 4, 2016, at 10:12 AM, Rotondo Simone
>> <simone.rotondo at aizoongroup.com> wrote:
>> 
>> Do you know if Bro 2.5 will be released as ?STABLE? in near term
>> ?
>> Do you have a rough idea for release date ?
>> 
>> Weeks, not months, assuming nothing goes wrong. But we want the
>> second
>> beta to at least get a couple of weeks of testing in production at
>> multiple sites.
>> 
>> ------
>> 
>> Adam J. Slagell
>> Chief Information Security Officer
>> Director, Cybersecurity Division
>> National Center for Supercomputing Applications
>> University of Illinois at Urbana-Champaign
>> www.slagell.info [1] [1]
>> 
>> "Under the Illinois Freedom of Information Act (FOIA), any written
>> communication to or from University employees regarding University
>> business is a public record and may be subject to public
>> disclosure."
>> 
>> Links:
>> ------
>> [1] http://www.slagell.info [1]
>> 
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro [2]
>  _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro [2]
> 
> ------
> 
> Adam J. Slagell
> Chief Information Security Officer
> Director, Cybersecurity Division
> National Center for Supercomputing Applications
> University of Illinois at Urbana-Champaign
> www.slagell.info [3]
> 
> "Under the Illinois Freedom of Information Act (FOIA), any written
> communication to or from University employees regarding University
> business is a public record and may be subject to public disclosure."
> 
> 
> 
> Links:
> ------
> [1] http://www.slagell.info/
> [2] http://mailman.icsi.berkeley.edu/mailman/listinfo/bro
> [3] http://www.slagell.info

From dwdixon at umich.edu  Fri Nov  4 09:11:25 2016
From: dwdixon at umich.edu (Drew Dixon)
Date: Fri, 4 Nov 2016 12:11:25 -0400
Subject: [Bro] Layer 2 Info
In-Reply-To: <8f54d155-3f0f-3062-a345-062435a54272@gmail.com>
References: <CAOrRJ9QoSUpcJWUWd2qF_W7o2tReRGNG8W7avOO-SfwF=A31FQ@mail.gmail.com>
	<8f54d155-3f0f-3062-a345-062435a54272@gmail.com>
Message-ID: <CA+pNqTMcjoQqwz1nCwnxHtBJyCg7tQ=-tn29TsHiGufPM9NSeg@mail.gmail.com>

This may not help solve the problem you're having but just FYI Bro 2.5 also
logs VLAN ID's now, from the new functionality section at the link below:

"Bro now tracks VLAN IDs. To record them inside the connection log, load
protocols/conn/vlan-logging.bro."

https://www.bro.org/documentation/beta/NEWS.bro.html

-Drew


On Thu, Nov 3, 2016 at 9:02 PM, Jan Grash?fer <jan.grashoefer at gmail.com>
wrote:

> Hi Luis,
>
> > Is there any way to glean layer 2 information from bro? Or maybe a
> reliable
> > means of correlating IPs to hostnames?
>
> Bro 2.5 (beta2 available) will support logging of MAC addresses:
> https://github.com/bro/bro/blob/master/scripts/site/local.bro#L98
>
> Best regards,
> Jan
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/c95cc356/attachment.html 

From dwdixon at umich.edu  Fri Nov  4 09:28:06 2016
From: dwdixon at umich.edu (Drew Dixon)
Date: Fri, 4 Nov 2016 12:28:06 -0400
Subject: [Bro] af_packet/pf_ring equivalency
In-Reply-To: <CAK6atxpOEqr05WL7BUkRyqzQhDBfgvxAgxXxe00+xZA8qcK-Hg@mail.gmail.com>
References: <CAK6atxp_rmpg+aLQDN_dhJCOCg-7JebhZZn0u2EJFYgCZSuA0Q@mail.gmail.com>
	<CAJ6bFK2rtasKMADB3G+hX1ewowXd=J-Q371isGhj7GJmGveuiQ@mail.gmail.com>
	<CAK6atxpOEqr05WL7BUkRyqzQhDBfgvxAgxXxe00+xZA8qcK-Hg@mail.gmail.com>
Message-ID: <CA+pNqTMPnjKSAAQ1+YCdAf9dUSBBqHp+0oFu67n7aLOHzmezMQ@mail.gmail.com>

The documentation for installing netsniff-ng is not great so I don't blame
you, however, it's not all that bad if you just run the one liner to
install all the dependancies for your respective distro and then
compile/build just ifpps using the following:

One-liner installation for *all* dependencies on Debian:

  $ sudo apt-get install ccache flex bison libnl-3-dev \
  libnl-genl-3-dev libnl-route-3-dev libgeoip-dev \
  libnetfilter-conntrack-dev libncurses5-dev liburcu-dev \
  libnacl-dev libpcap-dev zlib1g-dev libcli-dev libnet1-dev

One-liner installation for *all* dependencies on Fedora:

  $ sudo yum install ccache flex bison ccache libnl3-devel \
  GeoIP-devel libnetfilter_conntrack-devel ncurses-devel \
  userspace-rcu-devel nacl-devel libpcap-devel zlib-devel \
  libcli-devel libnet-devel

Compile/build options:

./configure
sudo make ifpps
sudo make ifpps_install

OR (I'd recommend this if you want more tools but the tunneling stuff, FYI
it also creates dependency issues [at least on Fedora based distros] so
exclude it using this)

./configure
sudo make allbutcurvetun
sudo make install_allbutcurvetun


https://github.com/netsniff-ng/netsniff-ng
https://github.com/netsniff-ng/netsniff-ng/blob/master/INSTALL

On Tue, Nov 1, 2016 at 9:09 AM, erik clark <philosnef at gmail.com> wrote:

> Interestingly, bwm-ng does not give me traffic numbers for my sniff
> interface.... I am trying to get ifpps, but I dont want to have to compile
> it and would like to find a rhel6 package of it. Sadly, it isnt in EPEL's
> netsniff-ng package group.
>
> On Mon, Oct 31, 2016 at 7:21 PM, Micha? Purzy?ski <
> michalpurzynski1 at gmail.com> wrote:
>
>> ifpps for generic bandwidth and pps monitoring. Never, ever, use iptraf.
>> ifpps has been written by the netsniff-ng author and it speaks for itself.
>>
>> bwm-ng seems to be good, haven't compared the accuracy and the perf data
>> acquisition.
>>
>>
>> For monitoring drops
>>
>> ethtool -S <int> to detect drops in card's FIFO and sometimes, reasons
>> for them.
>>
>> https://github.com/netoptimizer/network-testing/blob/master/
>> bin/softnet_stat.pl
>>
>> to detect drops at the softirq layer
>>
>> Bro's stats.log to detect drops at the af_packet layer
>>
>> Bro capture_loss to detect drops in all above + drops before packets
>> reach your sensor.
>>
>> Monitoring drops is complex and there is no single metric that tells you
>> all. Some of this is true for pfring as well, people just don't know. I've
>> seen sensors with 2-3% drops (in Suricata) but 40% drops in FIFO and they
>> were like "we're doing fine". Well, so here's a bad news... ;-)
>>
>>
>>
>> On Mon, Oct 31, 2016 at 5:38 PM, erik clark <philosnef at gmail.com> wrote:
>>
>>> I am using pf_ring with pfcount to do traffic analysis (pps/throughput)
>>> since it is very reliable.
>>>
>>> Does af_packet have an equivalent for this? I dont want to use broctl
>>> capstats unless there is absolutely no other option.
>>>
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>
>>
>>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/34ec564a/attachment.html 

From tgdesrochers at gmail.com  Fri Nov  4 09:34:29 2016
From: tgdesrochers at gmail.com (Tim Desrochers)
Date: Fri, 4 Nov 2016 12:34:29 -0400
Subject: [Bro] [bro] Finding specific protocols
Message-ID: <CANrcqc2tamK+SQr3f2-gunbKuhS6FKhRUyFDk9ffozEO_GeU-w@mail.gmail.com>

Is there something in Bro that will identify protocol 47, GRE tunnels.  I
see a tunnel log and conn_service_name fields that show teredo tunnels, but
is there anything I can search to find specific GRE tunnels?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/34a42221/attachment.html 

From jlay at slave-tothe-box.net  Fri Nov  4 10:58:11 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 04 Nov 2016 11:58:11 -0600
Subject: [Bro] [bro] Finding specific protocols
In-Reply-To: <CANrcqc2tamK+SQr3f2-gunbKuhS6FKhRUyFDk9ffozEO_GeU-w@mail.gmail.com>
References: <CANrcqc2tamK+SQr3f2-gunbKuhS6FKhRUyFDk9ffozEO_GeU-w@mail.gmail.com>
Message-ID: <474a9647e8333fe9abad0997a57dd29c@localhost>

On 2016-11-04 10:34, Tim Desrochers wrote:
> Is there something in Bro that will identify protocol 47, GRE tunnels.
>  I see a tunnel log and conn_service_name fields that show teredo
> tunnels, but is there anything I can search to find specific GRE
> tunnels?
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

I'm trying to get this to work with protosigs, but not having much luck 
matching:

signature protosig_gre {
   header ip[10] == 47
   event "match"
}

signature protosig_gre {
   header ip[10:1] == 47
   event "match"
}

so far no luck ...byte 10 in the IP header should work...still digging:

https://www.cloudshark.org/captures/000721f1edfb

James

From jazoff at illinois.edu  Fri Nov  4 12:03:34 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Fri, 4 Nov 2016 19:03:34 +0000
Subject: [Bro] Worker OOM crashes
In-Reply-To: <EB96F595-8BB8-412B-8164-E7D801D302DF@pingtrip.com>
References: <EB96F595-8BB8-412B-8164-E7D801D302DF@pingtrip.com>
Message-ID: <68253A99-8547-401B-BA78-4380756F0F81@illinois.edu>

I worked with Dave on this a bit.. had no luck getting bro to dump a core, but then I realized that the tcmalloc log line WAS the stack trace.  Still working on getting a real core dump, but this is a start.

18446744072956297216 is FFFFFFFFD31A4000 in hex




stderr.log:tcmalloc: large alloc 18446744072956297216 bytes == (nil) @  0x7fc9e487bdcb 0x7fc9e487bf1b 0x7fc9e487c965 0x7fc9e48ae9c5 0x7f1fef 0x867006 0x86780e 0x7f1a86 0x7f1da6 0x7f120a 0x7efbae 0x7ed324 0x866ea5 0x56440c 0x600317 0x60123e 0x5cfa04 0x837da3 0x5cfebf 0x52ecb0 0x7fc9e3865b45 0x5373ad (nil)

(gdb) info symbol 0x7fc9e487bdcb
No symbol matches 0x7fc9e487bdcb.

(gdb) info symbol 0x7fc9e487bf1b
No symbol matches 0x7fc9e487bf1b.

(gdb) info symbol 0x7fc9e487c965
No symbol matches 0x7fc9e487c965.

(gdb) info symbol 0x7fc9e48ae9c5
No symbol matches 0x7fc9e48ae9c5.

(gdb) info symbol 0x7f1fef
analyzer::tcp::ContentLine_Analyzer::DeliverStream(int, unsigned char const*, bool) + 207 in section .text

(gdb) info symbol 0x867006
analyzer::Analyzer::NextStream(int, unsigned char const*, bool) + 102 in section .text

(gdb) info symbol 0x86780e
analyzer::Analyzer::ForwardStream(int, unsigned char const*, bool) + 158 in section .text

(gdb) info symbol 0x7f1a86
analyzer::tcp::TCP_Reassembler::DeliverBlock(unsigned long, int, unsigned char const*) + 118 in section .text

(gdb) info symbol 0x7f1da6
analyzer::tcp::TCP_Reassembler::BlockInserted(DataBlock*) + 102 in section .text

(gdb) info symbol 0x7f120a
analyzer::tcp::TCP_Reassembler::DataSent(double, unsigned long, int, unsigned char const*, bool) + 106 in section .text

(gdb) info symbol 0x7efbae
analyzer::tcp::TCP_Endpoint::DataSent(double, unsigned long, int, int, unsigned char const*, IP_Hdr const*, tcphdr const*) + 110 in section .text

(gdb) info symbol 0x7ed324
analyzer::tcp::TCP_Analyzer::DeliverPacket(int, unsigned char const*, bool, unsigned long, IP_Hdr const*, int) + 4932 in section .text

(gdb) info symbol 0x866ea5
analyzer::Analyzer::NextPacket(int, unsigned char const*, bool, unsigned long, IP_Hdr const*, int) + 149 in section .text

(gdb) info symbol 0x56440c
Connection::NextPacket(double, int, IP_Hdr const*, int, int, unsigned char const*&, int&, int&, pcap_pkthdr const*, unsigned char const*, int) + 188 in section .text

(gdb) info symbol 0x600317
NetSessions::DoNextPacket(double, pcap_pkthdr const*, IP_Hdr const*, unsigned char const*, int, EncapsulationStack const*) + 2279 in section .text

(gdb) info symbol 0x60123e
NetSessions::NextPacket(double, pcap_pkthdr const*, unsigned char const*, int) + 798 in section .text

(gdb) info symbol 0x5cfa04
net_packet_dispatch(double, pcap_pkthdr const*, unsigned char const*, int, iosource::PktSrc*) + 244 in section .text

(gdb) info symbol 0x837da3
iosource::PktSrc::Process() + 323 in section .text

(gdb) info symbol 0x5cfebf 
net_run() + 111 in section .text

(gdb) info symbol 0x52ecb0
main + 7232 in section .text

(gdb) info symbol 0x7fc9e3865b45
No symbol matches 0x7fc9e3865b45.

(gdb) info symbol 0x5373ad
_start + 41 in section .text

-- 
- Justin Azoff

> On Oct 28, 2016, at 11:43 AM, Dave Crawford <bro at pingtrip.com> wrote:
> 
> Has anyone experienced this crash scenario that I?m in the process of debugging? This just started in the last couple of days on a cluster that has been in production for just shy of two years without issue.
> 
> tcmalloc: large alloc 18446744072956297216 bytes == (nil) @  0x7f08e85b5dcb 0x7f08e85b5f1b 0x7f08e85b6965 0x7f08e85e89c5 0x7f1fff 0x867016 0x86781e 0x7f1a96 0x7f1db6 0x7f121a 0x7efbbe 0x7ed334 0x866eb5 0x56441c 0x600327 0x60124e 0x5cfa14 0x837db3 0x5cfecf 0x52ecb0 0x7f08e759fb45 0x5373ad (nil) out of memory in new.
> 1477631586.955885 fatal error: out of memory in new.
> 
> The server itself still has some ceiling height:
> 
> $ free -h
>              total       used       free     shared    buffers     cached
> Mem:          126G        23G       102G        16M        67M       5.1G
> -/+ buffers/cache:        18G       107G
> Swap:          33G         0B        33G
> 
> But I?m wondering if the processes themselves are hitting a maximum size for memory allocation, as some are nearing 4GB.
> 
>   PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
> 24925 bro        20   0 3964M 3262M  269M S 78.1  2.5  2h47:55 /opt/bro/bin/bro -i eth6 -U .status -p broctl -p broctl-live -p local -p WIN_EXT-5 loc
>   618 bro        20   0 1217M 1160M  269M S 77.0  0.9  1h57:12 /opt/bro/bin/bro -i eth6 -U .status -p broctl -p broctl-live -p local -p WIN_EXT-2 loc
> 26073 bro        20   0 3989M 3156M  269M S 76.4  2.4  2h41:38 /opt/bro/bin/bro -i eth6 -U .status -p broctl -p broctl-live -p local -p WIN_EXT-7 loc
> 23938 bro        20   0 2610M 2318M  269M R 75.9  1.8  2h45:00 /opt/bro/bin/bro -i eth6 -U .status -p broctl -p broctl-live -p local -p WIN_EXT-9 loc
> 27816 bro        20   0 1554M 1455M  269M S 74.8  1.1  2h37:40 /opt/bro/bin/bro -i eth6 -U .status -p broctl -p broctl-live -p local -p WIN_EXT-1 loc
> 27817 bro        20   0 1532M 1435M  269M S 72.1  1.1  2h33:42 /opt/bro/bin/bro -i eth6 -U .status -p broctl -p broctl-live -p local -p WIN_EXT-4 loc
> 26677 bro        20   0 1421M 1372M  269M R 70.5  1.1  2h40:42 /opt/bro/bin/bro -i eth6 -U .status -p broctl -p broctl-live -p local -p WIN_EXT-6 loc
>   622 bro        20   0 1165M 1108M  269M R 70.5  0.9  1h54:24 /opt/bro/bin/bro -i eth6 -U .status -p broctl -p broctl-live -p local -p WIN_EXT-8 loc
>   613 bro        20   0 1231M 1166M  269M R 69.9  0.9  1h55:45 /opt/bro/bin/bro -i eth6 -U .status -p broctl -p broctl-live -p local -p WIN_EXT-10 lo
>   621 bro        20   0 1242M 1177M  269M R 68.9  0.9  1h53:02 /opt/bro/bin/bro -i eth6 -U .status -p broctl -p broctl-live -p local -p WIN_EXT-3 loc
>  1175 bro        20   0  234M  178M 10776 S  2.2  0.1  2h42:07 /opt/bro/bin/bro -U .status -p broctl -p broctl-live -p local -p WIN_EXT_PXY_1 local.b
> 
> 
> -Dave
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From jlay at slave-tothe-box.net  Fri Nov  4 12:57:42 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 04 Nov 2016 13:57:42 -0600
Subject: [Bro] Protosig question, round 2
Message-ID: <88192792624544ba9011f0b815e6d374@localhost>

So not wanting to highjack a thread, here we have the pcap for GRE 
traffic:

https://www.cloudshark.org/captures/000721f1edfb

So per packet #1, src is 10.0.0.1, dst is 10.0.0.2.  Packet #10 in the 
IP header is x2f or 47.  I was hoping one of these would match, but they 
don't:

signature protosig_gre {
    header ip[10] == 47
    event "match"
}

signature protosig_gre {
    header ip[10:1] == 47
    event "match"
}

I tested this, and oddly this didn't match either:

signature protosig_gre {
   header ip[16:4] == 10.0.0.1
   event "match"
}

But THIS did:

signature protosig_gre {
   header ip[16:4] == 1.1.1.1
   event "match"
}

So that tells me that bro is reading the GRE encapsulated IP header, 
which is neat.  Now...how do I tell bro to NOT read the GRE encapsulated 
IP header and read the original IP header?  I also tried matching on GRE 
header proper as a payload of /\x00\x00\x08\x00/:

signature protosig_gre_payload {
   ip-proto == ip
   payload /\x00\x00\x08\x00/
   #payload-size == 4
}

But this didn't match either.  Thank you.

James

From zeolla at gmail.com  Fri Nov  4 13:59:01 2016
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Fri, 04 Nov 2016 20:59:01 +0000
Subject: [Bro] Protosig question, round 2
In-Reply-To: <88192792624544ba9011f0b815e6d374@localhost>
References: <88192792624544ba9011f0b815e6d374@localhost>
Message-ID: <CAM1mTCp30envFLtqi=ajZvkUS9+1ySk1=S0U9OYEQYRJ2aDxMA@mail.gmail.com>

I have the same interests but for vxlan encapsulated traffic.  Last I
heard, no luck doing this with bro.  Have to decap upstream.

On Fri, Nov 4, 2016, 16:05 James Lay <jlay at slave-tothe-box.net> wrote:

> So not wanting to highjack a thread, here we have the pcap for GRE
> traffic:
>
> https://www.cloudshark.org/captures/000721f1edfb
>
> So per packet #1, src is 10.0.0.1, dst is 10.0.0.2.  Packet #10 in the
> IP header is x2f or 47.  I was hoping one of these would match, but they
> don't:
>
> signature protosig_gre {
>     header ip[10] == 47
>     event "match"
> }
>
> signature protosig_gre {
>     header ip[10:1] == 47
>     event "match"
> }
>
> I tested this, and oddly this didn't match either:
>
> signature protosig_gre {
>    header ip[16:4] == 10.0.0.1
>    event "match"
> }
>
> But THIS did:
>
> signature protosig_gre {
>    header ip[16:4] == 1.1.1.1
>    event "match"
> }
>
> So that tells me that bro is reading the GRE encapsulated IP header,
> which is neat.  Now...how do I tell bro to NOT read the GRE encapsulated
> IP header and read the original IP header?  I also tried matching on GRE
> header proper as a payload of /\x00\x00\x08\x00/:
>
> signature protosig_gre_payload {
>    ip-proto == ip
>    payload /\x00\x00\x08\x00/
>    #payload-size == 4
> }
>
> But this didn't match either.  Thank you.
>
> James
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-- 

Jon

Sent from my mobile device
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/d0464db7/attachment.html 

From jazoff at illinois.edu  Fri Nov  4 14:19:11 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Fri, 4 Nov 2016 21:19:11 +0000
Subject: [Bro] Protosig question, round 2
In-Reply-To: <CAM1mTCp30envFLtqi=ajZvkUS9+1ySk1=S0U9OYEQYRJ2aDxMA@mail.gmail.com>
References: <88192792624544ba9011f0b815e6d374@localhost>
	<CAM1mTCp30envFLtqi=ajZvkUS9+1ySk1=S0U9OYEQYRJ2aDxMA@mail.gmail.com>
Message-ID: <1DBA4BAE-D9FE-4875-9644-0960CA8E8159@illinois.edu>


> On Nov 4, 2016, at 4:59 PM, Zeolla at GMail.com <zeolla at gmail.com> wrote:
> 
> I have the same interests but for vxlan encapsulated traffic.  Last I heard, no luck doing this with bro.  Have to decap upstream.

I don't recall anyone ever asking about vxlan before.  I think it's a pretty trivial protocol to decode - look for udp 4789, skip 8 bytes, see if you have what looks like an ethernet frame.

The main issue with that and things like fabric path is the encapsulation into a limited number of outer l3 headers can cause flow hashing to be useless making it hard to load balance the traffic.

-- 
- Justin Azoff



From zeolla at gmail.com  Fri Nov  4 14:38:36 2016
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Fri, 04 Nov 2016 21:38:36 +0000
Subject: [Bro] Protosig question, round 2
In-Reply-To: <1DBA4BAE-D9FE-4875-9644-0960CA8E8159@illinois.edu>
References: <88192792624544ba9011f0b815e6d374@localhost>
	<CAM1mTCp30envFLtqi=ajZvkUS9+1ySk1=S0U9OYEQYRJ2aDxMA@mail.gmail.com>
	<1DBA4BAE-D9FE-4875-9644-0960CA8E8159@illinois.edu>
Message-ID: <CAM1mTCroHFmAHZo7P-+kL9vakRXCa8yFt2kS2UNxT3mKT586FA@mail.gmail.com>

Right, I spoke to arista directly about that too.  On the bro side I asked
via my broala support contract.

Jon

On Fri, Nov 4, 2016, 17:19 Azoff, Justin S <jazoff at illinois.edu> wrote:

>
> > On Nov 4, 2016, at 4:59 PM, Zeolla at GMail.com <zeolla at gmail.com> wrote:
> >
> > I have the same interests but for vxlan encapsulated traffic.  Last I
> heard, no luck doing this with bro.  Have to decap upstream.
>
> I don't recall anyone ever asking about vxlan before.  I think it's a
> pretty trivial protocol to decode - look for udp 4789, skip 8 bytes, see if
> you have what looks like an ethernet frame.
>
> The main issue with that and things like fabric path is the encapsulation
> into a limited number of outer l3 headers can cause flow hashing to be
> useless making it hard to load balance the traffic.
>
> --
> - Justin Azoff
>
> --

Jon

Sent from my mobile device
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/f570f5c8/attachment-0001.html 

From jlay at slave-tothe-box.net  Fri Nov  4 15:06:30 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 04 Nov 2016 16:06:30 -0600
Subject: [Bro] Protosig question, round 2
In-Reply-To: <CAM1mTCroHFmAHZo7P-+kL9vakRXCa8yFt2kS2UNxT3mKT586FA@mail.gmail.com>
References: <88192792624544ba9011f0b815e6d374@localhost>
	<CAM1mTCp30envFLtqi=ajZvkUS9+1ySk1=S0U9OYEQYRJ2aDxMA@mail.gmail.com>
	<1DBA4BAE-D9FE-4875-9644-0960CA8E8159@illinois.edu>
	<CAM1mTCroHFmAHZo7P-+kL9vakRXCa8yFt2kS2UNxT3mKT586FA@mail.gmail.com>
Message-ID: <bcedc7806891d4cf90b70f57b3fbed1b@localhost>

Yea this works:

signature protosig_vxlan {
   ip-proto == udp
   dst-port == 4789
   payload /\x08\x00/
   eval ProtoSig::match
}

Adjust the payload "\x08" for vlanid.  From 
https://surf.cloudshark.org/captures/b6495a4ea5d5.

James

On 2016-11-04 15:38, Zeolla at GMail.com wrote:
> Right, I spoke to arista directly about that too.  On the bro side I
> asked via my broala support contract.
> 
> Jon
> 
> On Fri, Nov 4, 2016, 17:19 Azoff, Justin S <jazoff at illinois.edu>
> wrote:
> 
>>> On Nov 4, 2016, at 4:59 PM, Zeolla at GMail.com <zeolla at gmail.com>
>> wrote:
>>> 
>>> I have the same interests but for vxlan encapsulated traffic.
>> Last I heard, no luck doing this with bro.  Have to decap upstream.
>> 
>> I don't recall anyone ever asking about vxlan before.  I think it's
>> a pretty trivial protocol to decode - look for udp 4789, skip 8
>> bytes, see if you have what looks like an ethernet frame.
>> 
>> The main issue with that and things like fabric path is the
>> encapsulation into a limited number of outer l3 headers can cause
>> flow hashing to be useless making it hard to load balance the
>> traffic.
>> 
>> --
>> - Justin Azoff
> 
> --
> 
> Jon
> 
> Sent from my mobile device

From zeolla at gmail.com  Fri Nov  4 15:32:05 2016
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Fri, 04 Nov 2016 22:32:05 +0000
Subject: [Bro] Protosig question, round 2
In-Reply-To: <bcedc7806891d4cf90b70f57b3fbed1b@localhost>
References: <88192792624544ba9011f0b815e6d374@localhost>
	<CAM1mTCp30envFLtqi=ajZvkUS9+1ySk1=S0U9OYEQYRJ2aDxMA@mail.gmail.com>
	<1DBA4BAE-D9FE-4875-9644-0960CA8E8159@illinois.edu>
	<CAM1mTCroHFmAHZo7P-+kL9vakRXCa8yFt2kS2UNxT3mKT586FA@mail.gmail.com>
	<bcedc7806891d4cf90b70f57b3fbed1b@localhost>
Message-ID: <CAM1mTCq1qV1vA0-GavsSYS+zLZYbHS9-1WrSE4BF9iiKv=HidA@mail.gmail.com>

But that just matches on it, which I think was the original tickets intent
but I missed that.  I'm interested in processing the inner packet/frame - I
thought your initial comments were just your first step towards decap.  I'm
looking to decap and process the inner frame.  Regardless, sorry, don't
mean to hijack.

Jon

On Fri, Nov 4, 2016, 18:09 James Lay <jlay at slave-tothe-box.net> wrote:

> Yea this works:
>
> signature protosig_vxlan {
>    ip-proto == udp
>    dst-port == 4789
>    payload /\x08\x00/
>    eval ProtoSig::match
> }
>
> Adjust the payload "\x08" for vlanid.  From
> https://surf.cloudshark.org/captures/b6495a4ea5d5.
>
> James
>
> On 2016-11-04 15:38, Zeolla at GMail.com wrote:
> > Right, I spoke to arista directly about that too.  On the bro side I
> > asked via my broala support contract.
> >
> > Jon
> >
> > On Fri, Nov 4, 2016, 17:19 Azoff, Justin S <jazoff at illinois.edu>
> > wrote:
> >
> >>> On Nov 4, 2016, at 4:59 PM, Zeolla at GMail.com <zeolla at gmail.com>
> >> wrote:
> >>>
> >>> I have the same interests but for vxlan encapsulated traffic.
> >> Last I heard, no luck doing this with bro.  Have to decap upstream.
> >>
> >> I don't recall anyone ever asking about vxlan before.  I think it's
> >> a pretty trivial protocol to decode - look for udp 4789, skip 8
> >> bytes, see if you have what looks like an ethernet frame.
> >>
> >> The main issue with that and things like fabric path is the
> >> encapsulation into a limited number of outer l3 headers can cause
> >> flow hashing to be useless making it hard to load balance the
> >> traffic.
> >>
> >> --
> >> - Justin Azoff
> >
> > --
> >
> > Jon
> >
> > Sent from my mobile device
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-- 

Jon

Sent from my mobile device
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/a5d540eb/attachment.html 

From jlay at slave-tothe-box.net  Fri Nov  4 15:36:38 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 04 Nov 2016 16:36:38 -0600
Subject: [Bro] Protosig question, round 2
In-Reply-To: <CAM1mTCq1qV1vA0-GavsSYS+zLZYbHS9-1WrSE4BF9iiKv=HidA@mail.gmail.com>
References: <88192792624544ba9011f0b815e6d374@localhost>
	<CAM1mTCp30envFLtqi=ajZvkUS9+1ySk1=S0U9OYEQYRJ2aDxMA@mail.gmail.com>
	<1DBA4BAE-D9FE-4875-9644-0960CA8E8159@illinois.edu>
	<CAM1mTCroHFmAHZo7P-+kL9vakRXCa8yFt2kS2UNxT3mKT586FA@mail.gmail.com>
	<bcedc7806891d4cf90b70f57b3fbed1b@localhost>
	<CAM1mTCq1qV1vA0-GavsSYS+zLZYbHS9-1WrSE4BF9iiKv=HidA@mail.gmail.com>
Message-ID: <0365d6f9093bb16cb85c36c2bdf8db78@localhost>

Ohhhh...gotcha.  We have the reverse problem...it appears that GRE is 
decapsulated, and vxlan isn't :)

James


On 2016-11-04 16:32, Zeolla at GMail.com wrote:
> But that just matches on it, which I think was the original tickets
> intent but I missed that.  I'm interested in processing the inner
> packet/frame - I thought your initial comments were just your first
> step towards decap.  I'm looking to decap and process the inner frame.
>  Regardless, sorry, don't mean to hijack.
> 
> Jon
> 
> On Fri, Nov 4, 2016, 18:09 James Lay <jlay at slave-tothe-box.net> wrote:
> 
>> Yea this works:
>> 
>> signature protosig_vxlan {
>> ip-proto == udp
>> dst-port == 4789
>> payload /\x08\x00/
>> eval ProtoSig::match
>> }
>> 
>> Adjust the payload "\x08" for vlanid.  From
>> https://surf.cloudshark.org/captures/b6495a4ea5d5.
>> 
>> James
>> 
>> On 2016-11-04 15:38, Zeolla at GMail.com wrote:
>>> Right, I spoke to arista directly about that too.  On the bro side
>> I
>>> asked via my broala support contract.
>>> 
>>> Jon
>>> 
>>> On Fri, Nov 4, 2016, 17:19 Azoff, Justin S <jazoff at illinois.edu>
>>> wrote:
>>> 
>>>>> On Nov 4, 2016, at 4:59 PM, Zeolla at GMail.com <zeolla at gmail.com>
>>>> wrote:
>>>>> 
>>>>> I have the same interests but for vxlan encapsulated traffic.
>>>> Last I heard, no luck doing this with bro.  Have to decap
>> upstream.
>>>> 
>>>> I don't recall anyone ever asking about vxlan before.  I think
>> it's
>>>> a pretty trivial protocol to decode - look for udp 4789, skip 8
>>>> bytes, see if you have what looks like an ethernet frame.
>>>> 
>>>> The main issue with that and things like fabric path is the
>>>> encapsulation into a limited number of outer l3 headers can cause
>>>> flow hashing to be useless making it hard to load balance the
>>>> traffic.
>>>> 
>>>> --
>>>> - Justin Azoff
>>> 
>>> --
>>> 
>>> Jon
>>> 
>>> Sent from my mobile device
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> --
> 
> Jon
> 
> Sent from my mobile device

From jan.grashoefer at gmail.com  Fri Nov  4 16:32:47 2016
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Sat, 5 Nov 2016 00:32:47 +0100
Subject: [Bro] Protosig question, round 2
In-Reply-To: <0365d6f9093bb16cb85c36c2bdf8db78@localhost>
References: <88192792624544ba9011f0b815e6d374@localhost>
	<CAM1mTCp30envFLtqi=ajZvkUS9+1ySk1=S0U9OYEQYRJ2aDxMA@mail.gmail.com>
	<1DBA4BAE-D9FE-4875-9644-0960CA8E8159@illinois.edu>
	<CAM1mTCroHFmAHZo7P-+kL9vakRXCa8yFt2kS2UNxT3mKT586FA@mail.gmail.com>
	<bcedc7806891d4cf90b70f57b3fbed1b@localhost>
	<CAM1mTCq1qV1vA0-GavsSYS+zLZYbHS9-1WrSE4BF9iiKv=HidA@mail.gmail.com>
	<0365d6f9093bb16cb85c36c2bdf8db78@localhost>
Message-ID: <b94d91eb-ebc7-2562-a91b-e34eb9d1e79b@gmail.com>

>From https://www.bro.org/sphinx/frameworks/signatures.html:
> Note that the IP-in-IP forms of tunneling are automatically decapsulated by default and signatures apply to only the inner-most packet [...]

>From time to time people want to attach analyzers at layer 2, which
isn't possible at the moment. Maybe once this part of Bro sees an
update, signatures and custom decapsulation analyzers can be integrated.
But that's a question for the devs.

Jan

From jlay at slave-tothe-box.net  Fri Nov  4 18:27:38 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 04 Nov 2016 19:27:38 -0600
Subject: [Bro] Protosig question, round 2
In-Reply-To: <b94d91eb-ebc7-2562-a91b-e34eb9d1e79b@gmail.com>
References: <88192792624544ba9011f0b815e6d374@localhost>
	<CAM1mTCp30envFLtqi=ajZvkUS9+1ySk1=S0U9OYEQYRJ2aDxMA@mail.gmail.com>
	<1DBA4BAE-D9FE-4875-9644-0960CA8E8159@illinois.edu>
	<CAM1mTCroHFmAHZo7P-+kL9vakRXCa8yFt2kS2UNxT3mKT586FA@mail.gmail.com>
	<bcedc7806891d4cf90b70f57b3fbed1b@localhost>
	<CAM1mTCq1qV1vA0-GavsSYS+zLZYbHS9-1WrSE4BF9iiKv=HidA@mail.gmail.com>
	<0365d6f9093bb16cb85c36c2bdf8db78@localhost>
	<b94d91eb-ebc7-2562-a91b-e34eb9d1e79b@gmail.com>
Message-ID: <1478309258.2357.0.camel@slave-tothe-box.net>

On Sat, 2016-11-05 at 00:32 +0100, Jan Grash?fer wrote:
> > 
> > From https://www.bro.org/sphinx/frameworks/signatures.html:
> > Note that the IP-in-IP forms of tunneling are automatically
> > decapsulated by default and signatures apply to only the inner-most 
> > packet [...]
> > 
> > From time to time people want to attach analyzers at layer 2, which
> isn't possible at the moment. Maybe once this part of Bro sees an
> update, signatures and custom decapsulation analyzers can be
> integrated.
> But that's a question for the devs.
> 
> Jan
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
Thanks Jan...I've been looking so long at that page I completely missed
that. ??
James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161104/5aca6cbb/attachment.html 

From xuchen890530 at gmail.com  Mon Nov  7 07:57:07 2016
From: xuchen890530 at gmail.com (Xu Chen)
Date: Mon, 7 Nov 2016 10:57:07 -0500
Subject: [Bro] Get Packet Header for all packets
Message-ID: <7B0CEDEA-A419-45E4-B6B8-624A93454109@gmail.com>

Hello all,

I am now using base/protocols/conn to get the packet header information from packets. It seems this script can only get the packets when the connection is established(the destination is reachable). Is there any other script I can use to capture the packet header no matter whether the destination is reachable  or not?

Thanks,

Chen 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161107/91da5496/attachment.html 

From philosnef at gmail.com  Mon Nov  7 08:02:16 2016
From: philosnef at gmail.com (erik clark)
Date: Mon, 7 Nov 2016 11:02:16 -0500
Subject: [Bro]  Get Packet Header for all packets
Message-ID: <CAK6atxrr-C=weCGZZsvEdvn4mLfwm0bP+q2VTAJk14uw_VZQcg@mail.gmail.com>

I am not sure thats accurate. I was recently troubleshooting a situation
where a printer was sending millions of packets an hour at a remote host.
On the remote destination host, that traffic was never seen, yet bro logged
it just fine. This was confirmed by running tcpdump in the middle (off the
tap) and on the end point (the destination). Tcpdump on the destination
showed zero packets coming from the source....
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161107/9d0d5055/attachment.html 

From xuchen890530 at gmail.com  Mon Nov  7 08:23:20 2016
From: xuchen890530 at gmail.com (Xu Chen)
Date: Mon, 7 Nov 2016 11:23:20 -0500
Subject: [Bro] Get Packet Header for all packets
In-Reply-To: <CAK6atxrr-C=weCGZZsvEdvn4mLfwm0bP+q2VTAJk14uw_VZQcg@mail.gmail.com>
References: <CAK6atxrr-C=weCGZZsvEdvn4mLfwm0bP+q2VTAJk14uw_VZQcg@mail.gmail.com>
Message-ID: <16A76745-9FB6-4EF9-A4A6-7506362018B1@gmail.com>

I use tcpdump on the bro interface. The interface gets APR packets since the destination is unreachable. But these packets can?t not be captured by bro (because the connection is not established) so that bro will not return the src/dst IP of the packets. 

My design is to use bro to capture the src and dst IP from ARP/ICMP request/TCP request and then add rule to an openflow switch to make the destination reachable. Any ideas on this?

Chen 

> On Nov 7, 2016, at 11:02 AM, erik clark <philosnef at gmail.com> wrote:
> 
> I am not sure thats accurate. I was recently troubleshooting a situation where a printer was sending millions of packets an hour at a remote host. On the remote destination host, that traffic was never seen, yet bro logged it just fine. This was confirmed by running tcpdump in the middle (off the tap) and on the end point (the destination). Tcpdump on the destination showed zero packets coming from the source....



From jan.grashoefer at gmail.com  Mon Nov  7 09:05:26 2016
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Mon, 7 Nov 2016 18:05:26 +0100
Subject: [Bro] Get Packet Header for all packets
In-Reply-To: <16A76745-9FB6-4EF9-A4A6-7506362018B1@gmail.com>
References: <CAK6atxrr-C=weCGZZsvEdvn4mLfwm0bP+q2VTAJk14uw_VZQcg@mail.gmail.com>
	<16A76745-9FB6-4EF9-A4A6-7506362018B1@gmail.com>
Message-ID: <5a55cb9e-ab9b-fb29-51d4-191e38bb4768@gmail.com>

Hi Chen,

> Is there any other script I can use to capture the packet header no matter whether the destination is reachable  or not?

in general you can use the raw_packet event (see
https://www.bro.org/sphinx-git/scripts/base/bif/event.bif.bro.html?highlight=new_packet#id-raw_packet)
but that is very expensive.

> My design is to use bro to capture the src and dst IP from ARP/ICMP request/TCP request and then add rule to an openflow switch to make the destination reachable. Any ideas on this?

To avoid the raw_packet event, there are ARP and ICMP analyzers for Bro
you can use. While the ARP analyzer delivers src and dst IP, the ICMP
analyzer uses the origin/responder pattern creating "virtual
connections" by matching eg. echo request and echo reply. With Bro 2.5
(beta2 available) you will be able to retrieve the actual source and
destination IPs of the current packet using get_current_packet_header().
However, depending on the scenario you might just use origin and responder.

Best regards,
Jan

From philosnef at gmail.com  Mon Nov  7 09:13:44 2016
From: philosnef at gmail.com (erik clark)
Date: Mon, 7 Nov 2016 12:13:44 -0500
Subject: [Bro] required ports open for cluster?
Message-ID: <CAK6atxpVYU9Kh3RRSZmssdfsqBxG_11T-gVBtr2MsW8=rMjknQ@mail.gmail.com>

Ok, so I dont see this in any documentation on bro.org. I have a logger
running on the same box as the manager, but I do not see any logs being
generated in /data/bro/logs/current.

I am assuming this is because traffic is being dropped on the floor because
iptables is in a default reject state? Where is the explicit listing of
ports that you need to punch in either firewalld or iptables?

https://www.bro.org/sphinx/components/broctl/README.html

does not have them listed, or any rule to have an entry in node.cfg to set
the port to a specific number... Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161107/3e99e2b1/attachment.html 

From jazoff at illinois.edu  Mon Nov  7 09:24:41 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Mon, 7 Nov 2016 17:24:41 +0000
Subject: [Bro] required ports open for cluster?
In-Reply-To: <CAK6atxpVYU9Kh3RRSZmssdfsqBxG_11T-gVBtr2MsW8=rMjknQ@mail.gmail.com>
References: <CAK6atxpVYU9Kh3RRSZmssdfsqBxG_11T-gVBtr2MsW8=rMjknQ@mail.gmail.com>
Message-ID: <902CB608-B6B9-4E71-97CC-51F3C5CC9C7F@illinois.edu>

It will be in the documentation for 2.5

https://www.bro.org/sphinx-git/components/broctl/README.html#bro-communication

-- 
- Justin Azoff

> On Nov 7, 2016, at 12:13 PM, erik clark <philosnef at gmail.com> wrote:
> 
> Ok, so I dont see this in any documentation on bro.org. I have a logger running on the same box as the manager, but I do not see any logs being generated in /data/bro/logs/current.
> 
> I am assuming this is because traffic is being dropped on the floor because iptables is in a default reject state? Where is the explicit listing of ports that you need to punch in either firewalld or iptables? 
> 
> https://www.bro.org/sphinx/components/broctl/README.html
> 
> does not have them listed, or any rule to have an entry in node.cfg to set the port to a specific number... Thanks!
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From brot212 at googlemail.com  Mon Nov  7 14:56:01 2016
From: brot212 at googlemail.com (Dane Wullen)
Date: Mon, 7 Nov 2016 23:56:01 +0100
Subject: [Bro] BinPAC analyzer name
Message-ID: <58210681.4000508@googlemail.com>

Hi there,

I wrote a new analyzer with BinPAC for a protocol named 'AMS'.
Somehow when I create the analyzer via the binpac python script and name 
the analyzer 'AMS' or 'ams', the analyzer won't work. When I name it 
'TEST' or 'test', it works fine (same protocol specification, C++ Code, 
etc.)

Is there a name convention for new analyzer? Or does anyone know, why 
BinPAC/Bro won't accept the name 'ams'?

Thank you!


From philosnef at gmail.com  Tue Nov  8 04:13:29 2016
From: philosnef at gmail.com (erik clark)
Date: Tue, 8 Nov 2016 07:13:29 -0500
Subject: [Bro] required ports open for cluster?
In-Reply-To: <902CB608-B6B9-4E71-97CC-51F3C5CC9C7F@illinois.edu>
References: <CAK6atxpVYU9Kh3RRSZmssdfsqBxG_11T-gVBtr2MsW8=rMjknQ@mail.gmail.com>
	<902CB608-B6B9-4E71-97CC-51F3C5CC9C7F@illinois.edu>
Message-ID: <CAK6atxrHG_x5Ms0SorwsnFguS9Y4UYUbLwRDjaVZxm8JqGaQ8g@mail.gmail.com>

(Sorry accidentally sent this to just Justin)...

Cool. I had punched the holes after running tcpdump on it for a while and
saw it trying to talk back. However, the one thing I don't understand is
that my logs arent being written back to  the logger host, even though
communication is open.

/data/bro/logs/current

is empty on the logger. All I have there is an stderr.log and an stdout.og.
Neither the workers on the logger machine itself, nor the remote host, are
logging to that directory. Are they being kept somewhere else? I dont see
them anywhere in the /data/bro/(spool/log) directory....

On Mon, Nov 7, 2016 at 12:24 PM, Azoff, Justin S <jazoff at illinois.edu>
wrote:

> It will be in the documentation for 2.5
>
> https://www.bro.org/sphinx-git/components/broctl/README.
> html#bro-communication
>
> --
> - Justin Azoff
>
> > On Nov 7, 2016, at 12:13 PM, erik clark <philosnef at gmail.com> wrote:
> >
> > Ok, so I dont see this in any documentation on bro.org. I have a logger
> running on the same box as the manager, but I do not see any logs being
> generated in /data/bro/logs/current.
> >
> > I am assuming this is because traffic is being dropped on the floor
> because iptables is in a default reject state? Where is the explicit
> listing of ports that you need to punch in either firewalld or iptables?
> >
> > https://www.bro.org/sphinx/components/broctl/README.html
> >
> > does not have them listed, or any rule to have an entry in node.cfg to
> set the port to a specific number... Thanks!
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161108/858b0bec/attachment.html 

From philosnef at gmail.com  Tue Nov  8 05:23:38 2016
From: philosnef at gmail.com (erik clark)
Date: Tue, 8 Nov 2016 08:23:38 -0500
Subject: [Bro] required ports open for cluster?
In-Reply-To: <CAK6atxrHG_x5Ms0SorwsnFguS9Y4UYUbLwRDjaVZxm8JqGaQ8g@mail.gmail.com>
References: <CAK6atxpVYU9Kh3RRSZmssdfsqBxG_11T-gVBtr2MsW8=rMjknQ@mail.gmail.com>
	<902CB608-B6B9-4E71-97CC-51F3C5CC9C7F@illinois.edu>
	<CAK6atxrHG_x5Ms0SorwsnFguS9Y4UYUbLwRDjaVZxm8JqGaQ8g@mail.gmail.com>
Message-ID: <CAK6atxoBV1g+QgBG6EdSD1sCEt2kD+VJyLQqd6yDTjrRZACoHQ@mail.gmail.com>

Just to compound the issue, no matter what host in the cluster I set the
logger destination to, I get zero logs. I am running 2.5beta1. I've
disabled iptables all around to see if that was causing a problem, but it
does not seem to be the case, as I have a pcap showing what appears to be
bro dns logs attempting to go across the wire to the logger. They just
aren't being written....

On Tue, Nov 8, 2016 at 7:13 AM, erik clark <philosnef at gmail.com> wrote:

> (Sorry accidentally sent this to just Justin)...
>
> Cool. I had punched the holes after running tcpdump on it for a while and
> saw it trying to talk back. However, the one thing I don't understand is
> that my logs arent being written back to  the logger host, even though
> communication is open.
>
> /data/bro/logs/current
>
> is empty on the logger. All I have there is an stderr.log and an
> stdout.og. Neither the workers on the logger machine itself, nor the remote
> host, are logging to that directory. Are they being kept somewhere else? I
> dont see them anywhere in the /data/bro/(spool/log) directory....
>
> On Mon, Nov 7, 2016 at 12:24 PM, Azoff, Justin S <jazoff at illinois.edu>
> wrote:
>
>> It will be in the documentation for 2.5
>>
>> https://www.bro.org/sphinx-git/components/broctl/README.html
>> #bro-communication
>>
>> --
>> - Justin Azoff
>>
>> > On Nov 7, 2016, at 12:13 PM, erik clark <philosnef at gmail.com> wrote:
>> >
>> > Ok, so I dont see this in any documentation on bro.org. I have a
>> logger running on the same box as the manager, but I do not see any logs
>> being generated in /data/bro/logs/current.
>> >
>> > I am assuming this is because traffic is being dropped on the floor
>> because iptables is in a default reject state? Where is the explicit
>> listing of ports that you need to punch in either firewalld or iptables?
>> >
>> > https://www.bro.org/sphinx/components/broctl/README.html
>> >
>> > does not have them listed, or any rule to have an entry in node.cfg to
>> set the port to a specific number... Thanks!
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161108/7eea1016/attachment.html 

From philosnef at gmail.com  Tue Nov  8 05:42:54 2016
From: philosnef at gmail.com (erik clark)
Date: Tue, 8 Nov 2016 08:42:54 -0500
Subject: [Bro] compressed file analyzer + docx files
Message-ID: <CAK6atxr7bxpUhMs0Z+znkPBSdrc1VvXrMgjpZRgP+M8cZi=Axg@mail.gmail.com>

Has anyone given any thought as to the possiblity of using a compressed
file analyzer to open and detect embedded flash files in docx files, or
macros in the same? I realize that that means we need a file analyzer
first, but I have been thinking about alternate use cases for the analyzer,
and this one sprung to mind...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161108/b6d4c943/attachment.html 

From philosnef at gmail.com  Tue Nov  8 07:10:29 2016
From: philosnef at gmail.com (erik clark)
Date: Tue, 8 Nov 2016 10:10:29 -0500
Subject: [Bro] required ports open for cluster?
In-Reply-To: <CAK6atxoBV1g+QgBG6EdSD1sCEt2kD+VJyLQqd6yDTjrRZACoHQ@mail.gmail.com>
References: <CAK6atxpVYU9Kh3RRSZmssdfsqBxG_11T-gVBtr2MsW8=rMjknQ@mail.gmail.com>
	<902CB608-B6B9-4E71-97CC-51F3C5CC9C7F@illinois.edu>
	<CAK6atxrHG_x5Ms0SorwsnFguS9Y4UYUbLwRDjaVZxm8JqGaQ8g@mail.gmail.com>
	<CAK6atxoBV1g+QgBG6EdSD1sCEt2kD+VJyLQqd6yDTjrRZACoHQ@mail.gmail.com>
Message-ID: <CAK6atxrAY1GX1v+-Bg2ijP5dSDerUAsTUsHBK38X5i6rGVDCVQ@mail.gmail.com>

Ah, solution found. Use fqdns. I ran into this problem before where I
specified 127.0.0.1 for localhost and things broke. When will Bro support
ip addresses in node.cfg properly? I would have thought that it would be in
2.5 :)

On Tue, Nov 8, 2016 at 8:23 AM, erik clark <philosnef at gmail.com> wrote:

> Just to compound the issue, no matter what host in the cluster I set the
> logger destination to, I get zero logs. I am running 2.5beta1. I've
> disabled iptables all around to see if that was causing a problem, but it
> does not seem to be the case, as I have a pcap showing what appears to be
> bro dns logs attempting to go across the wire to the logger. They just
> aren't being written....
>
> On Tue, Nov 8, 2016 at 7:13 AM, erik clark <philosnef at gmail.com> wrote:
>
>> (Sorry accidentally sent this to just Justin)...
>>
>> Cool. I had punched the holes after running tcpdump on it for a while and
>> saw it trying to talk back. However, the one thing I don't understand is
>> that my logs arent being written back to  the logger host, even though
>> communication is open.
>>
>> /data/bro/logs/current
>>
>> is empty on the logger. All I have there is an stderr.log and an
>> stdout.og. Neither the workers on the logger machine itself, nor the remote
>> host, are logging to that directory. Are they being kept somewhere else? I
>> dont see them anywhere in the /data/bro/(spool/log) directory....
>>
>> On Mon, Nov 7, 2016 at 12:24 PM, Azoff, Justin S <jazoff at illinois.edu>
>> wrote:
>>
>>> It will be in the documentation for 2.5
>>>
>>> https://www.bro.org/sphinx-git/components/broctl/README.html
>>> #bro-communication
>>>
>>> --
>>> - Justin Azoff
>>>
>>> > On Nov 7, 2016, at 12:13 PM, erik clark <philosnef at gmail.com> wrote:
>>> >
>>> > Ok, so I dont see this in any documentation on bro.org. I have a
>>> logger running on the same box as the manager, but I do not see any logs
>>> being generated in /data/bro/logs/current.
>>> >
>>> > I am assuming this is because traffic is being dropped on the floor
>>> because iptables is in a default reject state? Where is the explicit
>>> listing of ports that you need to punch in either firewalld or iptables?
>>> >
>>> > https://www.bro.org/sphinx/components/broctl/README.html
>>> >
>>> > does not have them listed, or any rule to have an entry in node.cfg to
>>> set the port to a specific number... Thanks!
>>> > _______________________________________________
>>> > Bro mailing list
>>> > bro at bro-ids.org
>>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161108/0dc637ee/attachment-0001.html 

From jazoff at illinois.edu  Tue Nov  8 07:23:06 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 8 Nov 2016 15:23:06 +0000
Subject: [Bro] required ports open for cluster?
In-Reply-To: <CAK6atxrAY1GX1v+-Bg2ijP5dSDerUAsTUsHBK38X5i6rGVDCVQ@mail.gmail.com>
References: <CAK6atxpVYU9Kh3RRSZmssdfsqBxG_11T-gVBtr2MsW8=rMjknQ@mail.gmail.com>
	<902CB608-B6B9-4E71-97CC-51F3C5CC9C7F@illinois.edu>
	<CAK6atxrHG_x5Ms0SorwsnFguS9Y4UYUbLwRDjaVZxm8JqGaQ8g@mail.gmail.com>
	<CAK6atxoBV1g+QgBG6EdSD1sCEt2kD+VJyLQqd6yDTjrRZACoHQ@mail.gmail.com>
	<CAK6atxrAY1GX1v+-Bg2ijP5dSDerUAsTUsHBK38X5i6rGVDCVQ@mail.gmail.com>
Message-ID: <1ADC576B-B79B-403A-BE92-D28BB0B13CB9@illinois.edu>


> On Nov 8, 2016, at 10:10 AM, erik clark <philosnef at gmail.com> wrote:
> 
> Ah, solution found. Use fqdns. I ran into this problem before where I specified 127.0.0.1 for localhost and things broke. When will Bro support ip addresses in node.cfg properly? I would have thought that it would be in 2.5 :)

It does.  You can't use an ip address that does not work from all nodes.
If you have evidence otherwise, file a bug.

-- 
- Justin Azoff



From pyrodie18 at gmail.com  Tue Nov  8 12:13:29 2016
From: pyrodie18 at gmail.com (Troy Ward)
Date: Tue, 8 Nov 2016 15:13:29 -0500
Subject: [Bro] Schedule an event
Message-ID: <CAKjK3aT_1s+KLeGbOLvq5hSdhE1A3oFPFaWzZo9Bib3S8_3q5A@mail.gmail.com>

I have a script based on the conn.log events.  As connections are created
it populates some information in a table.  I need to trigger a search of
that table to occur every 30 minutes.  I believe I can use the "schedule"
command but not entirely sure.  So my question is, can I build a function
within my script that does the table search and if so, how do I use the
schedule command to trigger the function?

Thanks in advance,

Troy W
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161108/8e365470/attachment.html 

From dnj0496 at gmail.com  Tue Nov  8 13:52:29 2016
From: dnj0496 at gmail.com (Dk Jack)
Date: Tue, 8 Nov 2016 13:52:29 -0800
Subject: [Bro] script question
Message-ID: <CAGATen755Dc44ovTt-LMr_WLMfkcPeZE6S61F5Ho5X77iziPQA@mail.gmail.com>

Hi,
Found an interesting quirk of bro scripting. Not sure if its a quirk or a
bug.

function myfunction()
{
    if (T) {
        local var = "Hello, World!";
        print var;
    } else {
        local var = "Goodbye, World!";
        print var;
    }
}

For this code, I get the error:

error in ././trybro.bro, line 6 and ././trybro.bro, line 9: already
defined (var)

If I make the local variable names different i.e. var and var2, it
doesn't complain. I think this is a bug. Let me know...

Dk.

PS: Since I couldn't find an ends_with function, I wrote one. Let me
know if this is ok...


function ends_with(input_string: string, match_pattern: string) : bool
{
  local offset = |input_string| - |match_pattern| + 1;
  local sstring = sub_bytes(input_string, offset, |match_pattern|);

  if (sstring == match_pattern)
    return T;
  else
    return F;
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161108/4268a679/attachment.html 

From vern at berkeley.edu  Tue Nov  8 23:09:03 2016
From: vern at berkeley.edu (Vern Paxson)
Date: Tue, 08 Nov 2016 23:09:03 -0800
Subject: [Bro] script question
In-Reply-To: <CAGATen755Dc44ovTt-LMr_WLMfkcPeZE6S61F5Ho5X77iziPQA@mail.gmail.com>
	(Tue, 08 Nov 2016 13:52:29 PST).
Message-ID: <20161109070903.24FD22C41CD@rock.ICSI.Berkeley.EDU>

> Found an interesting quirk of bro scripting. Not sure if its a quirk or a
> bug.

It's a quirk actually, by which I mean known behavior since I implemented
local variables way back 'n the day.  The philosophical view may have been
that it's confusing to have the same variable name mean different things
inside the same function, so good to avoid that; but I might have done it
simply for implementation convenience, I don't recall which at this point.

Given that it's surprising behavior, and arguably not particularly beneficial,
I wouldn't mind changing it.

		Vern


> function myfunction()
> {
>     if (T) {
>         local var = "Hello, World!";
>         print var;
>     } else {
>         local var = "Goodbye, World!";
>         print var;
>     }
> }

From jan.grashoefer at gmail.com  Wed Nov  9 02:25:25 2016
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Wed, 9 Nov 2016 11:25:25 +0100
Subject: [Bro] BroCon slides and videos coming soon
In-Reply-To: <1783D2CA-CA43-4216-B6E8-B3F8F8964614@illinois.edu>
References: <1783D2CA-CA43-4216-B6E8-B3F8F8964614@illinois.edu>
Message-ID: <85d804a0-7b13-b90e-8248-a2d803794767@gmail.com>

Are there any updated estimates?

Thanks,
Jan

Am 21.09.2016 um 21:36 schrieb Dopheide, Jeannette M:
> Bro Community,
> 
> A few people have been asking when the BroCon slides and videos will be posted. We need to do some post-production and other work before this is done. A rough estimate, barring unforeseen interruptions is one month. When they are ready we?ll communicate it on our mailing list and social media outlets. 
> 
> Thanks for your patience. 
> 
> ------
> Jeannette Dopheide
> Training and Outreach Coordinator
> National Center for Supercomputing Applications
> University of Illinois at Urbana-Champaign
>  
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 

From philosnef at gmail.com  Wed Nov  9 04:15:43 2016
From: philosnef at gmail.com (erik clark)
Date: Wed, 9 Nov 2016 07:15:43 -0500
Subject: [Bro] required ports open for cluster?
In-Reply-To: <1ADC576B-B79B-403A-BE92-D28BB0B13CB9@illinois.edu>
References: <CAK6atxpVYU9Kh3RRSZmssdfsqBxG_11T-gVBtr2MsW8=rMjknQ@mail.gmail.com>
	<902CB608-B6B9-4E71-97CC-51F3C5CC9C7F@illinois.edu>
	<CAK6atxrHG_x5Ms0SorwsnFguS9Y4UYUbLwRDjaVZxm8JqGaQ8g@mail.gmail.com>
	<CAK6atxoBV1g+QgBG6EdSD1sCEt2kD+VJyLQqd6yDTjrRZACoHQ@mail.gmail.com>
	<CAK6atxrAY1GX1v+-Bg2ijP5dSDerUAsTUsHBK38X5i6rGVDCVQ@mail.gmail.com>
	<1ADC576B-B79B-403A-BE92-D28BB0B13CB9@illinois.edu>
Message-ID: <CAK6atxqrYE5hu2f49UK=-ThwXbbYRMeFKbg1wXk2jkBohfo5gg@mail.gmail.com>

Ok, so then if I have things segregated into different enclaves that can
only talk to the logger and manager, you are saying this breaks the
cluster?

On Tue, Nov 8, 2016 at 10:23 AM, Azoff, Justin S <jazoff at illinois.edu>
wrote:

>
> > On Nov 8, 2016, at 10:10 AM, erik clark <philosnef at gmail.com> wrote:
> >
> > Ah, solution found. Use fqdns. I ran into this problem before where I
> specified 127.0.0.1 for localhost and things broke. When will Bro support
> ip addresses in node.cfg properly? I would have thought that it would be in
> 2.5 :)
>
> It does.  You can't use an ip address that does not work from all nodes.
> If you have evidence otherwise, file a bug.
>
> --
> - Justin Azoff
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161109/1f545586/attachment.html 

From jazoff at illinois.edu  Wed Nov  9 06:02:54 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Wed, 9 Nov 2016 14:02:54 +0000
Subject: [Bro] required ports open for cluster?
In-Reply-To: <CAK6atxqrYE5hu2f49UK=-ThwXbbYRMeFKbg1wXk2jkBohfo5gg@mail.gmail.com>
References: <CAK6atxpVYU9Kh3RRSZmssdfsqBxG_11T-gVBtr2MsW8=rMjknQ@mail.gmail.com>
	<902CB608-B6B9-4E71-97CC-51F3C5CC9C7F@illinois.edu>
	<CAK6atxrHG_x5Ms0SorwsnFguS9Y4UYUbLwRDjaVZxm8JqGaQ8g@mail.gmail.com>
	<CAK6atxoBV1g+QgBG6EdSD1sCEt2kD+VJyLQqd6yDTjrRZACoHQ@mail.gmail.com>
	<CAK6atxrAY1GX1v+-Bg2ijP5dSDerUAsTUsHBK38X5i6rGVDCVQ@mail.gmail.com>
	<1ADC576B-B79B-403A-BE92-D28BB0B13CB9@illinois.edu>
	<CAK6atxqrYE5hu2f49UK=-ThwXbbYRMeFKbg1wXk2jkBohfo5gg@mail.gmail.com>
Message-ID: <18DAF93E-DE35-4E66-A040-FA2AC3469E98@illinois.edu>


> On Nov 9, 2016, at 7:15 AM, erik clark <philosnef at gmail.com> wrote:
> 
> Ok, so then if I have things segregated into different enclaves that can only talk to the logger and manager, you are saying this breaks the cluster? 


If you're having trouble with cluster communication, that's probably why.  Workers don't need to be able to connect to each other, but the other node types and workers need to be able to see each other.  If you use a hostname it has to resolve properly to an IP that works everywhere.  127.0.0.1 or similar will not work.


-- 
- Justin Azoff



From jdopheid at illinois.edu  Wed Nov  9 09:09:01 2016
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Wed, 9 Nov 2016 17:09:01 +0000
Subject: [Bro] BroCon slides and videos coming soon
In-Reply-To: <85d804a0-7b13-b90e-8248-a2d803794767@gmail.com>
References: <1783D2CA-CA43-4216-B6E8-B3F8F8964614@illinois.edu>,
	<85d804a0-7b13-b90e-8248-a2d803794767@gmail.com>
Message-ID: <7EFD7D614A2BB84ABEA19B2CEDD246581C3AFEAA@CITESMBX5.ad.uillinois.edu>

Hello Jan,

About half of the videos have been edited. I'm still working on the others. I'll make the ones that are ready available tomorrow and will update the list. 

------

Jeannette M. Dopheide
Bro Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign

________________________________________
From: bro-bounces at bro.org [bro-bounces at bro.org] on behalf of Jan Grash?fer [jan.grashoefer at gmail.com]
Sent: Wednesday, November 09, 2016 4:25 AM
To: bro at bro.org
Subject: Re: [Bro] BroCon slides and videos coming soon

Are there any updated estimates?

Thanks,
Jan

Am 21.09.2016 um 21:36 schrieb Dopheide, Jeannette M:
> Bro Community,
>
> A few people have been asking when the BroCon slides and videos will be posted. We need to do some post-production and other work before this is done. A rough estimate, barring unforeseen interruptions is one month. When they are ready we?ll communicate it on our mailing list and social media outlets.
>
> Thanks for your patience.
>
> ------
> Jeannette Dopheide
> Training and Outreach Coordinator
> National Center for Supercomputing Applications
> University of Illinois at Urbana-Champaign
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From philosnef at gmail.com  Wed Nov  9 10:29:11 2016
From: philosnef at gmail.com (erik clark)
Date: Wed, 9 Nov 2016 13:29:11 -0500
Subject: [Bro] lb_procs and af_packet
Message-ID: <CAK6atxrx2i+YqzYrdGzJpsc8DQBM6eGFOeDC6CZwRkf_R9j=2Q@mail.gmail.com>

How do you specify lb_procs for an af_packet::ethX interface? I can't find
any real documentation on using the bro 25 af_packet plugin (or in
general). Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161109/ee1f9d17/attachment.html 

From eshelton at butler.net  Wed Nov  9 10:29:24 2016
From: eshelton at butler.net (eshelton)
Date: Wed, 9 Nov 2016 11:29:24 -0700
Subject: [Bro] 2.5 Intelligence Framework
Message-ID: <CADOaRAmEw9TwPwR+r-r=zBw14rfYGvJJviWM-5OUTgwqdu2wiQ@mail.gmail.com>

I'm trying to familiarize myself with the updates/changes to the 2.5 intel
framework, as well as start leveraging it to greater use. I've come across
a couple of issues I'm not quite clear how to solve yet:

1) Is there a way to expire intel inputs from one input source, but not
another?

For example I have tor data as an input source, and I'd like to set this up
to update several times a day, and expiry would be a wonderful option for
this data to keep it as accurate as possible. I also have some somewhat
static input data that may won't be update regularly via cron, which I'd
doesn't really need expiry at all, save for manual update to this
particular intel file.

2) Is there a way to only send data to the notice framework from particular
sources? Or perhaps this is an issue of suppressing certain emails from the
notice framework?

For example, I want to log my Tor hits in intel.log, and I don't really
mind if they show up in notice.log either, but I don't want to get emails
every time I log a Tor node hit. I'd like to reserve emails sent from the
notice framework to those from particular data sources which aren't Tor
hits, or of my choosing.

Respectfully,

-Erin Shelton

Program Manager: Incident Response and Network Security
Office of Information Technology
University of Colorado Boulder
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161109/76c07714/attachment.html 

From jan.grashoefer at gmail.com  Wed Nov  9 11:35:52 2016
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Wed, 9 Nov 2016 20:35:52 +0100
Subject: [Bro] 2.5 Intelligence Framework
In-Reply-To: <CADOaRAmEw9TwPwR+r-r=zBw14rfYGvJJviWM-5OUTgwqdu2wiQ@mail.gmail.com>
References: <CADOaRAmEw9TwPwR+r-r=zBw14rfYGvJJviWM-5OUTgwqdu2wiQ@mail.gmail.com>
Message-ID: <f01030d4-75fb-ac97-20b8-dd4b8fa95bda@gmail.com>

Hi Erin,

> I'm trying to familiarize myself with the updates/changes to the 2.5 intel
> framework, as well as start leveraging it to greater use. I've come across
> a couple of issues I'm not quite clear how to solve yet:

I have written a blog post that was intended to be published on the Bro
Blog but somehow that was forgotten. You can find my draft for the post
here: https://gist.github.com/J-Gras/3ff4d5308a69e91fb61c65c12ecb818c
The post should help to understand the intelligence framework and the
recent updates.

> 1) Is there a way to expire intel inputs from one input source, but not
> another?

Actually my intention was to allow individual expiration for
intelligence items in the first place. Due to implementation
considerations, there is no "native support" for that feature in the
framework but the design allows to realize this feature. There is a
script (see https://github.com/J-Gras/intel-extensions) that implements
per item expiration. I think it is also mentioned in the blog post. The
script is not well tested and I haven't registered the packet, yet. In
principle it allows to specify expiration for every item using
"meta.expire". Thus you should be able to chose different timeouts for
different sources.

> 2) Is there a way to only send data to the notice framework from particular
> sources? Or perhaps this is an issue of suppressing certain emails from the
> notice framework?

Exactly. Although you should be able to suppress notices based on the
mail_ext vector (see
https://github.com/bro/bro/blob/master/scripts/policy/frameworks/intel/do_notice.bro#L66),
that would be somehow hacky. Maybe it would be better to write your own
version of the do_notice script, which allows suppression by source.

I hope this helps,
Jan

From jan.grashoefer at gmail.com  Wed Nov  9 11:44:21 2016
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Wed, 9 Nov 2016 20:44:21 +0100
Subject: [Bro] lb_procs and af_packet
In-Reply-To: <CAK6atxrx2i+YqzYrdGzJpsc8DQBM6eGFOeDC6CZwRkf_R9j=2Q@mail.gmail.com>
References: <CAK6atxrx2i+YqzYrdGzJpsc8DQBM6eGFOeDC6CZwRkf_R9j=2Q@mail.gmail.com>
Message-ID: <8481a0bf-e662-4c52-32e0-6ddb413cbd12@gmail.com>

> How do you specify lb_procs for an af_packet::ethX interface? I can't find
> any real documentation on using the bro 25 af_packet plugin (or in
> general). Thanks!

lb_method=custom
lb_procs=<number>

However, that will probably change as there are cases that require a
broctl plugin. I will update the readme accordingly.

Best regards,
Jan

From dnj0496 at gmail.com  Wed Nov  9 12:05:12 2016
From: dnj0496 at gmail.com (Dk Jack)
Date: Wed, 9 Nov 2016 12:05:12 -0800
Subject: [Bro] script question
In-Reply-To: <20161109070903.24FD22C41CD@rock.ICSI.Berkeley.EDU>
References: <CAGATen755Dc44ovTt-LMr_WLMfkcPeZE6S61F5Ho5X77iziPQA@mail.gmail.com>
	<20161109070903.24FD22C41CD@rock.ICSI.Berkeley.EDU>
Message-ID: <CAGATen4jJiS9FOXsV1q4BxVENPuApQ8OOLFs=cFnqB6_zrWBmQ@mail.gmail.com>

yeah, was surprised a little since it goes against the scoping rules of
most languages I've dealt with scripting or otherwise...
Perhaps an update to documentation would be helpful...

Bhasker.


On Tue, Nov 8, 2016 at 11:09 PM, Vern Paxson <vern at berkeley.edu> wrote:

> > Found an interesting quirk of bro scripting. Not sure if its a quirk or a
> > bug.
>
> It's a quirk actually, by which I mean known behavior since I implemented
> local variables way back 'n the day.  The philosophical view may have been
> that it's confusing to have the same variable name mean different things
> inside the same function, so good to avoid that; but I might have done it
> simply for implementation convenience, I don't recall which at this point.
>
> Given that it's surprising behavior, and arguably not particularly
> beneficial,
> I wouldn't mind changing it.
>
>                 Vern
>
>
> > function myfunction()
> > {
> >     if (T) {
> >         local var = "Hello, World!";
> >         print var;
> >     } else {
> >         local var = "Goodbye, World!";
> >         print var;
> >     }
> > }
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161109/2e780eb6/attachment.html 

From dnj0496 at gmail.com  Wed Nov  9 12:30:55 2016
From: dnj0496 at gmail.com (Dk Jack)
Date: Wed, 9 Nov 2016 12:30:55 -0800
Subject: [Bro] bro size
In-Reply-To: <9CB8DEE8-B306-4B19-80A6-A541BED0EAA0@gmail.com>
References: <CAGATen6pGLRCVd0qhuMkCBFZW_MqdNh2TiZ_7VXUiaofK6fMJg@mail.gmail.com>
	<E5A2071C6FFFF640B714C224139C9B3327DE9E4A@WP-MBX2B.milky-way.battelle.org>
	<54EE16CB-21FC-44CC-A9E9-F60912313520@icir.org>
	<9CB8DEE8-B306-4B19-80A6-A541BED0EAA0@gmail.com>
Message-ID: <CAGATen5ssuj=Cy-9hC+PLQ=-reetdvxRAtcW-Vs9Dho0cYwwPQ@mail.gmail.com>

Thanks everyone. Yes, my bro binary size reduced to 6M after stripping...

BTW, found an interesting tool...

https://github.com/google/bloaty


On Tue, Oct 25, 2016 at 3:47 PM, Daniel Guerra <daniel.guerra69 at gmail.com>
wrote:

> Same here 6mb stripped.
>
> Check
>
> https://hub.docker.com/r/danielguerra/alpine-bro-build/
>
> The broker and brocolli etc are disabled
>
> Daniel
>
> > On 25 Oct 2016, at 19:56, Johanna Amann <johanna at icir.org> wrote:
> >
> > If for some reason it really is a concern, you also can just call strip
> > on the binary. This brings the binary size down to ~6MB for me.
> >
> > Johanna
> >
> > On 25 Oct 2016, at 10:28, Hosom, Stephen M wrote:
> >
> >> I wouldn?t recommend changing many of the options in configure
> >> unless you truly know what you?re doing. Why is it that you need Bro
> >> to be smaller?
> >>
> >> From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Dk
> >> Jack
> >> Sent: Monday, October 24, 2016 8:43 PM
> >> To: bro at bro.org
> >> Subject: [Bro] bro size
> >>
> >> Hi,
> >> When I compile bro, the bro binary comes out to about 120Mb. Are there
> >> any options I can use to reduce by eliminating some of the features I
> >> don't need? the configure script doesn't seem to have many options.
> >> Thanks.
> >>
> >> Dk.
> >> _______________________________________________
> >> Bro mailing list
> >> bro at bro-ids.org
> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161109/c64975bd/attachment.html 

From jlay at slave-tothe-box.net  Wed Nov  9 12:45:54 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 09 Nov 2016 13:45:54 -0700
Subject: [Bro] bro size
In-Reply-To: <CAGATen5ssuj=Cy-9hC+PLQ=-reetdvxRAtcW-Vs9Dho0cYwwPQ@mail.gmail.com>
References: <CAGATen6pGLRCVd0qhuMkCBFZW_MqdNh2TiZ_7VXUiaofK6fMJg@mail.gmail.com>
	<E5A2071C6FFFF640B714C224139C9B3327DE9E4A@WP-MBX2B.milky-way.battelle.org>
	<54EE16CB-21FC-44CC-A9E9-F60912313520@icir.org>
	<9CB8DEE8-B306-4B19-80A6-A541BED0EAA0@gmail.com>
	<CAGATen5ssuj=Cy-9hC+PLQ=-reetdvxRAtcW-Vs9Dho0cYwwPQ@mail.gmail.com>
Message-ID: <5671ae7f40b3c2d67d76db40410fdc3a@localhost>

Did you notice any impact when running?

James

On 2016-11-09 13:30, Dk Jack wrote:
> Thanks everyone. Yes, my bro binary size reduced to 6M after
> stripping...
> 
> BTW, found an interesting tool...
> 
> https://github.com/google/bloaty
> 
> On Tue, Oct 25, 2016 at 3:47 PM, Daniel Guerra
> <daniel.guerra69 at gmail.com> wrote:
> 
>> Same here 6mb stripped.
>> 
>> Check
>> 
>> https://hub.docker.com/r/danielguerra/alpine-bro-build/ [1]
>> 
>> The broker and brocolli etc are disabled
>> 
>> Daniel
>> 
>>> On 25 Oct 2016, at 19:56, Johanna Amann <johanna at icir.org> wrote:
>>> 
>>> If for some reason it really is a concern, you also can just call
>> strip
>>> on the binary. This brings the binary size down to ~6MB for me.
>>> 
>>> Johanna
>>> 
>>> On 25 Oct 2016, at 10:28, Hosom, Stephen M wrote:
>>> 
>>>> I wouldn?t recommend changing many of the options in configure
>>>> unless you truly know what you?re doing. Why is it that you
>> need Bro
>>>> to be smaller?
>>>> 
>>>> From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf
>> Of Dk
>>>> Jack
>>>> Sent: Monday, October 24, 2016 8:43 PM
>>>> To: bro at bro.org
>>>> Subject: [Bro] bro size
>>>> 
>>>> Hi,
>>>> When I compile bro, the bro binary comes out to about 120Mb. Are
>> there
>>>> any options I can use to reduce by eliminating some of the
>> features I
>>>> don't need? the configure script doesn't seem to have many
>> options.
>>>> Thanks.
>>>> 
>>>> Dk.
>>>> _______________________________________________
>>>> Bro mailing list
>>>> bro at bro-ids.org
>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro [2]
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro [2]
> 
> 
> 
> Links:
> ------
> [1] https://hub.docker.com/r/danielguerra/alpine-bro-build/
> [2] http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

From philosnef at gmail.com  Thu Nov 10 06:17:35 2016
From: philosnef at gmail.com (erik clark)
Date: Thu, 10 Nov 2016 09:17:35 -0500
Subject: [Bro] capstats doesnt work with af_packet
Message-ID: <CAK6atxqSL+V6dWaoz6PxqM11Ar7VPzFUpqseFOropGGRyaJMeg@mail.gmail.com>

Subject says it all. When I run interface=af_packet::em3, broctl capstats
reports no statistics.

How can I fix this, as I rely on this information for traffic profiling of
the system. Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161110/07151def/attachment.html 

From philosnef at gmail.com  Thu Nov 10 07:15:21 2016
From: philosnef at gmail.com (erik clark)
Date: Thu, 10 Nov 2016 10:15:21 -0500
Subject: [Bro] capstats doesnt work with af_packet
In-Reply-To: <CAK6atxqSL+V6dWaoz6PxqM11Ar7VPzFUpqseFOropGGRyaJMeg@mail.gmail.com>
References: <CAK6atxqSL+V6dWaoz6PxqM11Ar7VPzFUpqseFOropGGRyaJMeg@mail.gmail.com>
Message-ID: <CAK6atxqbAh=CCOZD3nokXRV+rRzBX9vz2aFGqB2kF6eXrUv-ZQ@mail.gmail.com>

Hm, after investigating, I think this might be a parsing issue of node.cfg?
If I set the interface to em3 by itself with bro already running, capstats
works with broctl. This seems to indicate to me that maybe node.cfg isn't
parsing out interfaces using plugins? I can't find the broctl function that
handles the capstats call.

On Thu, Nov 10, 2016 at 9:17 AM, erik clark <philosnef at gmail.com> wrote:

> Subject says it all. When I run interface=af_packet::em3, broctl capstats
> reports no statistics.
>
> How can I fix this, as I rely on this information for traffic profiling of
> the system. Thanks!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161110/3cded798/attachment.html 

From jazoff at illinois.edu  Thu Nov 10 07:27:50 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Thu, 10 Nov 2016 15:27:50 +0000
Subject: [Bro] capstats doesnt work with af_packet
In-Reply-To: <CAK6atxqSL+V6dWaoz6PxqM11Ar7VPzFUpqseFOropGGRyaJMeg@mail.gmail.com>
References: <CAK6atxqSL+V6dWaoz6PxqM11Ar7VPzFUpqseFOropGGRyaJMeg@mail.gmail.com>
Message-ID: <181966F6-2075-4856-AF70-FDABE30348E5@illinois.edu>


> On Nov 10, 2016, at 9:17 AM, erik clark <philosnef at gmail.com> wrote:
> 
> Subject says it all. When I run interface=af_packet::em3, broctl capstats reports no statistics. 
> How can I fix this, as I rely on this information for traffic profiling of the system. Thanks!
> 


It doesn't work right, and it can't really work right.  I think the short answer is that capstats is going away.  As a standalone tool it is ok, but running it on a schedule is not a great feature. It generates stats by actually capturing the packets and reporting on what it saw.  On a heavily loaded worker this is the absolute last thing you want to do.

The stats.log will contain the same data split out by worker in the fields like bytes_recv, pkts_proc,pkts_dropped,pkts_link.  You should be able to do the profiling you need using this data.

-- 
- Justin Azoff



From philosnef at gmail.com  Thu Nov 10 07:48:30 2016
From: philosnef at gmail.com (erik clark)
Date: Thu, 10 Nov 2016 10:48:30 -0500
Subject: [Bro] capstats doesnt work with af_packet
In-Reply-To: <181966F6-2075-4856-AF70-FDABE30348E5@illinois.edu>
References: <CAK6atxqSL+V6dWaoz6PxqM11Ar7VPzFUpqseFOropGGRyaJMeg@mail.gmail.com>
	<181966F6-2075-4856-AF70-FDABE30348E5@illinois.edu>
Message-ID: <CAK6atxqMfY+vYd6jjH_gnqPbApytdjZ20FX-DLp-JeOn_S70_Q@mail.gmail.com>

Hm, ok. Previously I was using pfcount, since we were using pf_ring, but
since moving to af_packet, pfcount is obviously no longer an option, and I
was hoping to use capstats as a standalone. Thanks for the quick response!
Will probably just massage this into splunk with a timechart.

On Thu, Nov 10, 2016 at 10:27 AM, Azoff, Justin S <jazoff at illinois.edu>
wrote:

>
> > On Nov 10, 2016, at 9:17 AM, erik clark <philosnef at gmail.com> wrote:
> >
> > Subject says it all. When I run interface=af_packet::em3, broctl
> capstats reports no statistics.
> > How can I fix this, as I rely on this information for traffic profiling
> of the system. Thanks!
> >
>
>
> It doesn't work right, and it can't really work right.  I think the short
> answer is that capstats is going away.  As a standalone tool it is ok, but
> running it on a schedule is not a great feature. It generates stats by
> actually capturing the packets and reporting on what it saw.  On a heavily
> loaded worker this is the absolute last thing you want to do.
>
> The stats.log will contain the same data split out by worker in the fields
> like bytes_recv, pkts_proc,pkts_dropped,pkts_link.  You should be able to
> do the profiling you need using this data.
>
> --
> - Justin Azoff
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161110/2cb502df/attachment.html 

From jdopheid at illinois.edu  Thu Nov 10 08:22:18 2016
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Thu, 10 Nov 2016 16:22:18 +0000
Subject: [Bro] BroCon 2016, some videos are posted
Message-ID: <725D92BB-7655-4E6D-B4DF-F95BB86BA2E8@illinois.edu>

Bro Community,

I?ve posted five videos to the BroCon 2016 Event page next to the corresponding agenda item: 

https://www.bro.org/community/brocon2016.html#agenda

Not all the videos are edited yet and I?m waiting for permission to post others, so please check back on this page periodically to see more updates. Or, subscribe to our YouTube channel: 

https://www.youtube.com/user/BroPlatform

Thanks,
Jeannette Dopheide

------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
 



From abdulrahmanmusallam at gmail.com  Fri Nov 11 11:19:12 2016
From: abdulrahmanmusallam at gmail.com (abdulrahman musallam)
Date: Fri, 11 Nov 2016 21:19:12 +0200
Subject: [Bro] Conn Log
Message-ID: <CAMZgFn--rfW+Z223WjTf+Dj3z3V35ZA-RdyOO5wkbUskmnGGOQ@mail.gmail.com>

Hello,
The connection log generated by Bro provide a services field which declare
the application layer protocol which was used in that connection, I've
noticed that it sometimes uses ' - '  instead of known protocol, could you
please tell what does this sign stand for?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161111/eadf742c/attachment.html 

From seth at icir.org  Fri Nov 11 12:00:13 2016
From: seth at icir.org (Seth Hall)
Date: Fri, 11 Nov 2016 15:00:13 -0500
Subject: [Bro] Conn Log
In-Reply-To: <CAMZgFn--rfW+Z223WjTf+Dj3z3V35ZA-RdyOO5wkbUskmnGGOQ@mail.gmail.com>
References: <CAMZgFn--rfW+Z223WjTf+Dj3z3V35ZA-RdyOO5wkbUskmnGGOQ@mail.gmail.com>
Message-ID: <003C3E3C-10F1-4B80-8650-957BE0BFBCDD@icir.org>


> On Nov 11, 2016, at 2:19 PM, abdulrahman musallam <abdulrahmanmusallam at gmail.com> wrote:
> 
> The connection log generated by Bro provide a services field which declare the application layer protocol which was used in that connection, I've noticed that it sometimes uses ' - '  instead of known protocol, could you please tell what does this sign stand for?

That is just an indicator for NULL.  It means that no analyzer was attached to the connection.

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From dopheide at gmail.com  Sun Nov 13 13:54:45 2016
From: dopheide at gmail.com (Mike Dopheide)
Date: Sun, 13 Nov 2016 15:54:45 -0600
Subject: [Bro] bad record initializer (between Bro 2.3 and 2.4)
Message-ID: <CAPy2kFa6DiSoQ1J24v-XBRw+REL6bHGhqM8Gx2MdhgmQiEbz8g@mail.gmail.com>

I've isolated the problem to using an 'int' in a record type as seen
below.  According to try.bro.org the it started to fail between versions
2.3 and 2.4.  If you change the boothnum to a string and put quotes around
it, everything is fine.

Was there a syntax change that requires something around the int when it's
initialized?

-Dop


@load base/frameworks/input
@load base/frameworks/notice

# add some stuff to generate notices from our test traffic:
@load misc/scan
@load misc/detect-traceroute
@load protocols/ssh/detect-bruteforcing

module Conn;

export {

    type vlandata: record {
        booth: string &log &optional;
        boothnum: int &log &optional;
    };

    global vlanlist: table[int] of vlandata = table() &redef;

}

redef vlanlist += {
    [11] = [$booth="darkspace"],
    [18] = [$booth="ASDF",$boothnum=1743]
};
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161113/44feecc3/attachment.html 

From bengen--bro at hilluzination.de  Mon Nov 14 04:16:38 2016
From: bengen--bro at hilluzination.de (Hilko Bengen)
Date: Mon, 14 Nov 2016 13:16:38 +0100
Subject: [Bro] Bro and OpenSSL 1.1
Message-ID: <87twbaxnmh.fsf@msgid.hilluzination.de>

TL;DR: Good news, Bro is going to be part of Debian 9 "stretch", but we
need some advice.

Hi,

as Debian is transitioning to using OpenSSL 1.1 in the upcoming release
(9.x "stretch"), we are forced to deal with widespread API breakage
because many data structures that had previously been considered part of
the API have been made opaque. Many of these changes are fairly easy to
implement by using getter/setter functions instead. (The main time-sink
for me was locating those functions in the OpenSSL sources.)

For the bro package, some work-in-progress patches can be found in our
bug tracking system[1].

One missing piece (apart from running tests with real packet trace data)
is that some OCSP details cannot yet be accessed through OpenSSL 1.1's
current set of API functions. Specifically, the function

    X509* x509_get_ocsp_signer(STACK_OF(X509) *certs, OCSP_RESPID *rid)

from src/file_analysis/analyzer/x509/functions.bif cannot currently be
ported. There's ongoing work to fix that[2] in upstream OpenSSL, but we
don't know yet whether this change will be ready in time for the freeze
leading to the next Debian release. So, we are thinking that we may have
to disable the x509_ocsp_verify function and anything that uses it.

Does anyone have any advice on what to look for when disabling that
functionality? Or is there maybe a less intrusive alternative that we
haven't discovered yet?

Cheers,
-Hilko

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=828254
[2] https://github.com/openssl/openssl/pull/1876

From jazoff at illinois.edu  Mon Nov 14 06:27:03 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Mon, 14 Nov 2016 14:27:03 +0000
Subject: [Bro] bad record initializer (between Bro 2.3 and 2.4)
In-Reply-To: <CAPy2kFa6DiSoQ1J24v-XBRw+REL6bHGhqM8Gx2MdhgmQiEbz8g@mail.gmail.com>
References: <CAPy2kFa6DiSoQ1J24v-XBRw+REL6bHGhqM8Gx2MdhgmQiEbz8g@mail.gmail.com>
Message-ID: <973713FC-02E1-4935-B587-D4DF00EBC14F@illinois.edu>

Does it work if you use 'count' instead of 'int' ?
-- 
- Justin Azoff

> On Nov 13, 2016, at 4:54 PM, Mike Dopheide <dopheide at gmail.com> wrote:
> 
> I've isolated the problem to using an 'int' in a record type as seen below.  According to try.bro.org the it started to fail between versions 2.3 and 2.4.  If you change the boothnum to a string and put quotes around it, everything is fine.
> 
> Was there a syntax change that requires something around the int when it's initialized?
> 
> -Dop
> 
> 
> @load base/frameworks/input
> @load base/frameworks/notice
> 
> # add some stuff to generate notices from our test traffic:
> @load misc/scan
> @load misc/detect-traceroute
> @load protocols/ssh/detect-bruteforcing
> 
> module Conn;
> 
> export {
> 
>     type vlandata: record {
>         booth: string &log &optional;
>         boothnum: int &log &optional;
>     };
> 
>     global vlanlist: table[int] of vlandata = table() &redef;
> 
> }
> 
> redef vlanlist += {
>     [11] = [$booth="darkspace"],
>     [18] = [$booth="ASDF",$boothnum=1743]
> };
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From philosnef at gmail.com  Mon Nov 14 06:35:18 2016
From: philosnef at gmail.com (erik clark)
Date: Mon, 14 Nov 2016 09:35:18 -0500
Subject: [Bro] logging locally and to remote logger
Message-ID: <CAK6atxpSP4XUkiUjnXg6kc1mPpdBEr7OmHQ6i52UE05bypx-_w@mail.gmail.com>

So, if I use:

redef Log::enable_local_logging

in a bro worker cluster, what I find is that all the logs go to
/data/bro/spool/worker-1-X instead of all in /data/bro/logs/current on the
local machine... Is there a way to fix this?

Also, I would want to rotate logs out on the workers that are doing
additional local logging to have a much more constrained timeframe for
logging, specifically 1 week for local nodes, and 3 months for the logger
host.

Is the best way to do this just with a cron rm -rf /data/bro/logs/$date ?
It seems this would run into a conflict with broctlconfig....

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161114/a18810d2/attachment.html 

From obdnanr at gmail.com  Mon Nov 14 07:04:16 2016
From: obdnanr at gmail.com (Obndnar smith)
Date: Mon, 14 Nov 2016 15:04:16 +0000
Subject: [Bro] JustinAzoff/bro-pdns
Message-ID: <CALCRDTR=qMnZGp3ZjKTex8g7HxA+prpaoXHv6rjTvQzfhQ1YGQ@mail.gmail.com>

Has anyone had any luck getting this passive dns script to work?  I can't
seem to get any of the data from Bro to get into the mysql server.  Has
anyone gotten this to work and have any tips or tricks to get it working?

https://github.com/JustinAzoff/bro-pdns
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161114/198a351b/attachment.html 

From puntogtg at tiscali.it  Mon Nov 14 07:19:07 2016
From: puntogtg at tiscali.it (puntogtg at tiscali.it)
Date: Mon, 14 Nov 2016 16:19:07 +0100
Subject: [Bro] JustinAzoff/bro-pdns
In-Reply-To: <CALCRDTR=qMnZGp3ZjKTex8g7HxA+prpaoXHv6rjTvQzfhQ1YGQ@mail.gmail.com>
References: <CALCRDTR=qMnZGp3ZjKTex8g7HxA+prpaoXHv6rjTvQzfhQ1YGQ@mail.gmail.com>
Message-ID: <0a7d748dbfbf6526427c2b4a4b58bd6d@tiscali.it>

  I was using it with mysql, but performances was not so good.
Time ago
I Wrote to Justin and he told me he was rewriting code:
https://github.com/JustinAzoff/bro-pdns/tree/go-rewrite

Il 14.11.2016
16:04 Obndnar smith ha scritto: 

> Has anyone had any luck getting this
passive dns script to work? I can't seem to get any of the data from Bro
to get into the mysql server. Has anyone gotten this to work and have
any tips or tricks to get it working? 
>
https://github.com/JustinAzoff/bro-pdns [1]
  


Con Smart 3 Giga a 9 euro/4 sett navighi veloce, chiami e invii SMS dal tuo smartphone verso tutti i fissi e mobili in Italia. Passa a Tiscali Mobile! http://casa.tiscali.it/mobile/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161114/42fa3523/attachment.html 

From jazoff at illinois.edu  Mon Nov 14 07:57:34 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Mon, 14 Nov 2016 15:57:34 +0000
Subject: [Bro] JustinAzoff/bro-pdns
In-Reply-To: <0a7d748dbfbf6526427c2b4a4b58bd6d@tiscali.it>
References: <CALCRDTR=qMnZGp3ZjKTex8g7HxA+prpaoXHv6rjTvQzfhQ1YGQ@mail.gmail.com>
	<0a7d748dbfbf6526427c2b4a4b58bd6d@tiscali.it>
Message-ID: <574B547B-A94C-46CE-A435-531933D0955D@illinois.edu>

Hi :-)

Yes.. the python version worked, but the performance when using remote databases was not that good.  It was also really hard for people to install correctly.

Also, it turned out that using the log postprocessor to execute the tool during log rotation was really fragile.

I re-wrote it as a simpler tool that can just read existing bro log archives instead of needing to be ran from bro.

I implemented a sqlite and a postgresql backend, but haven't done mysql yet.  It's fully functional, but it needs some polishing.

$ ./bro-pdns index big.log
2016/11/14 10:44:11 big.log: Aggregation: Duration=3.9 TotalRecords=1058400 SkippedRecords=0 Tuples=1496 Individual=1962
2016/11/14 10:44:11 batch: Store: Duration=0.1 Inserted=3458 Updated=0
$ ./bro-pdns index big.log
2016/11/14 10:55:35 big.log: Already indexed
$ ./bro-pdns like tuple google.com|head
Query	Type	Answer	Count	TTL	First	Last
clients3.google.com	A	173.194.46.64	144	70	2014-03-14 14:31:06	2014-03-14 14:31:06
clients3.google.com	A	173.194.46.65	144	70	2014-03-14 14:31:06	2014-03-14 14:31:06
clients3.google.com	A	173.194.46.66	144	70	2014-03-14 14:31:06	2014-03-14 14:31:06

$ ./bro-pdns web --listen :8081&
[1] 7449
2016/11/14 10:51:59 Listening on ":8081"

$ curl -s localhost:8081/dns/like/tuples/173.194.46.64  | jq . | head
[
  {
    "Last": "2014-03-14 14:31:20",
    "First": "2014-03-14 14:31:20",
    "TTL": 300,
    "Count": 288,
    "Answer": "173.194.46.64",
    "Type": "A",
    "Query": "maps.google.nl"
  },

Indexing all bro logs currently involves a find -name 'dns.*' | xargs -n 50 bro-pdns index

-- 
- Justin Azoff

> On Nov 14, 2016, at 10:19 AM, puntogtg at tiscali.it wrote:
> 
> I was using it with mysql, but performances was not so good.
> Time ago I Wrote to Justin and he told me he was rewriting code: https://github.com/JustinAzoff/bro-pdns/tree/go-rewrite
> 
> 
> Il 14.11.2016 16:04 Obndnar smith ha scritto:
> 
>> Has anyone had any luck getting this passive dns script to work?  I can't seem to get any of the data from Bro to get into the mysql server.  Has anyone gotten this to work and have any tips or tricks to get it working?
>> https://github.com/JustinAzoff/bro-pdns
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From dopheide at gmail.com  Mon Nov 14 08:30:26 2016
From: dopheide at gmail.com (Mike Dopheide)
Date: Mon, 14 Nov 2016 10:30:26 -0600
Subject: [Bro] bad record initializer (between Bro 2.3 and 2.4)
In-Reply-To: <973713FC-02E1-4935-B587-D4DF00EBC14F@illinois.edu>
References: <CAPy2kFa6DiSoQ1J24v-XBRw+REL6bHGhqM8Gx2MdhgmQiEbz8g@mail.gmail.com>
	<973713FC-02E1-4935-B587-D4DF00EBC14F@illinois.edu>
Message-ID: <CAPy2kFactqyjTNE+eqjfOY5WZ23zTjTHLjKssE8wRDgLKX75-w@mail.gmail.com>

*sigh*

That works, and int works if I prepend a + to the number.

Thanks,
Dop

On Mon, Nov 14, 2016 at 8:27 AM, Azoff, Justin S <jazoff at illinois.edu>
wrote:

> Does it work if you use 'count' instead of 'int' ?
> --
> - Justin Azoff
>
> > On Nov 13, 2016, at 4:54 PM, Mike Dopheide <dopheide at gmail.com> wrote:
> >
> > I've isolated the problem to using an 'int' in a record type as seen
> below.  According to try.bro.org the it started to fail between versions
> 2.3 and 2.4.  If you change the boothnum to a string and put quotes around
> it, everything is fine.
> >
> > Was there a syntax change that requires something around the int when
> it's initialized?
> >
> > -Dop
> >
> >
> > @load base/frameworks/input
> > @load base/frameworks/notice
> >
> > # add some stuff to generate notices from our test traffic:
> > @load misc/scan
> > @load misc/detect-traceroute
> > @load protocols/ssh/detect-bruteforcing
> >
> > module Conn;
> >
> > export {
> >
> >     type vlandata: record {
> >         booth: string &log &optional;
> >         boothnum: int &log &optional;
> >     };
> >
> >     global vlanlist: table[int] of vlandata = table() &redef;
> >
> > }
> >
> > redef vlanlist += {
> >     [11] = [$booth="darkspace"],
> >     [18] = [$booth="ASDF",$boothnum=1743]
> > };
> >
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161114/c8d6b213/attachment.html 

From seth at icir.org  Mon Nov 14 13:08:05 2016
From: seth at icir.org (Seth Hall)
Date: Mon, 14 Nov 2016 16:08:05 -0500
Subject: [Bro] bad record initializer (between Bro 2.3 and 2.4)
In-Reply-To: <CAPy2kFactqyjTNE+eqjfOY5WZ23zTjTHLjKssE8wRDgLKX75-w@mail.gmail.com>
References: <CAPy2kFa6DiSoQ1J24v-XBRw+REL6bHGhqM8Gx2MdhgmQiEbz8g@mail.gmail.com>
	<973713FC-02E1-4935-B587-D4DF00EBC14F@illinois.edu>
	<CAPy2kFactqyjTNE+eqjfOY5WZ23zTjTHLjKssE8wRDgLKX75-w@mail.gmail.com>
Message-ID: <3BB5D83A-C568-42B3-BADC-59C0121F5C6C@icir.org>


> On Nov 14, 2016, at 11:30 AM, Mike Dopheide <dopheide at gmail.com> wrote:
> 
> That works, and int works if I prepend a + to the number.   

That makes sense.  The typecasting has some trouble promoting numbers assigned that way sometimes.  If you create a minimal test case that fails in a way that you think it should work, it would help to have a ticket filed.

Thanks!
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From dnthayer at illinois.edu  Mon Nov 14 14:19:54 2016
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Mon, 14 Nov 2016 16:19:54 -0600
Subject: [Bro] bad record initializer (between Bro 2.3 and 2.4)
In-Reply-To: <3BB5D83A-C568-42B3-BADC-59C0121F5C6C@icir.org>
References: <CAPy2kFa6DiSoQ1J24v-XBRw+REL6bHGhqM8Gx2MdhgmQiEbz8g@mail.gmail.com>
	<973713FC-02E1-4935-B587-D4DF00EBC14F@illinois.edu>
	<CAPy2kFactqyjTNE+eqjfOY5WZ23zTjTHLjKssE8wRDgLKX75-w@mail.gmail.com>
	<3BB5D83A-C568-42B3-BADC-59C0121F5C6C@icir.org>
Message-ID: <31efb329-0219-727c-8c74-8367dee62189@illinois.edu>

On 11/14/16 3:08 PM, Seth Hall wrote:
>
>> On Nov 14, 2016, at 11:30 AM, Mike Dopheide <dopheide at gmail.com> wrote:
>>
>> That works, and int works if I prepend a + to the number.
>
> That makes sense.  The typecasting has some trouble promoting numbers assigned that way sometimes.  If you create a minimal test case that fails in a way that you think it should work, it would help to have a ticket filed.
>
> Thanks!
>   .Seth

Here is a minimal test that works, but if you remove the "+" sign, then 
it fails:

type myrecord: record {
     ii: int;
};

# The "+" sign is required here:
global rr1 = myrecord($ii = +3);

# But "+" is not required here:
global rr2: myrecord;
rr2$ii = 3;

print rr1;
print rr2;

From bill.de.ping at gmail.com  Tue Nov 15 03:31:25 2016
From: bill.de.ping at gmail.com (william de ping)
Date: Tue, 15 Nov 2016 13:31:25 +0200
Subject: [Bro] logging locally and to remote logger
In-Reply-To: <CAK6atxpSP4XUkiUjnXg6kc1mPpdBEr7OmHQ6i52UE05bypx-_w@mail.gmail.com>
References: <CAK6atxpSP4XUkiUjnXg6kc1mPpdBEr7OmHQ6i52UE05bypx-_w@mail.gmail.com>
Message-ID: <CAKZA8xi7Kiscc88VKwBuTEK_wGnz+bS46rJJ6aW==DSPTF2NFQ@mail.gmail.com>

Hi,

If you wish to log locally and you care about the worker-id who produce
this logged event :

   - to know what is the worker-id you can add a field "worker" to your
   logs and populate it from bro script using : get_event_peer()$descr
   - to change the rotation for each log (here, rotate every 200 minutes)
   you need to use
   - LOG::remove_default_filter(SSH::LOG);
      - and then add LOG::add_filter(SSH::LOG, [$name="ssh",$path="ssh",
      *$interv=200min*, $include=("field1","field2") ]
      - btw, you can set $path to be a mounted dir
   - to save the log to another machine simultaneously :
      - use bro, add a new writer (https://www.bro.org/sphinx/
      scripts/base/frameworks/logging/main.bro.html#id-Log::default_writer
      <https://www.bro.org/sphinx/scripts/base/frameworks/logging/main.bro.html#id-Log::default_writer>)
      and then add_filter to ssh and ask it to use the new writer
      - use syslog, just monitor this main local log and transmit it to
      another machine

Hope it helps

On Mon, Nov 14, 2016 at 4:35 PM, erik clark <philosnef at gmail.com> wrote:

> So, if I use:
>
> redef Log::enable_local_logging
>
> in a bro worker cluster, what I find is that all the logs go to
> /data/bro/spool/worker-1-X instead of all in /data/bro/logs/current on the
> local machine... Is there a way to fix this?
>
> Also, I would want to rotate logs out on the workers that are doing
> additional local logging to have a much more constrained timeframe for
> logging, specifically 1 week for local nodes, and 3 months for the logger
> host.
>
> Is the best way to do this just with a cron rm -rf /data/bro/logs/$date ?
> It seems this would run into a conflict with broctlconfig....
>
> Thanks!
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161115/f54ec67e/attachment.html 

From philosnef at gmail.com  Tue Nov 15 04:15:55 2016
From: philosnef at gmail.com (erik clark)
Date: Tue, 15 Nov 2016 07:15:55 -0500
Subject: [Bro] logging locally and to remote logger
In-Reply-To: <CAKZA8xidBC1RByqr=jAeeum1kzHt8nMLo51zBC+rCcxHYmhGyA@mail.gmail.com>
References: <CAK6atxpSP4XUkiUjnXg6kc1mPpdBEr7OmHQ6i52UE05bypx-_w@mail.gmail.com>
	<CAKZA8xidBC1RByqr=jAeeum1kzHt8nMLo51zBC+rCcxHYmhGyA@mail.gmail.com>
Message-ID: <CAK6atxqKKTfR8h-Scpu=Kaob2UEh=3wgOsrr_92VQVLAc1cRUA@mail.gmail.com>

Ah, I think there is some confusion. Out of the box if you log locally as
well as using a remote logger (2.5), the logs locally get shoved into
worker buckets. I was hoping to see how it would be possible to get
standard cluster behavior, where all workers log locally to one bucket
instead of each worker having its own bucket.

Anyone know why this logs to separate buckets in the first place?

On Tue, Nov 15, 2016 at 6:31 AM, william de ping <bill.de.ping at gmail.com>
wrote:

> Hi,
>
> If you wish to log locally and you care about the worker-id who produce
> this logged event :
>
>    - to know what is the worker-id you can add a field "worker" to your
>    logs and populate it from bro script using : get_event_peer()$descr
>    - to change the rotation for each log (here, rotate every 200 minutes)
>    you need to use
>    - LOG::remove_default_filter(SSH::LOG);
>       - and then add LOG::add_filter(SSH::LOG, [$name="ssh",$path="ssh",
>       *$interv=200min*, $include=("field1","field2") ]
>       - btw, you can set $path to be a mounted dir
>    - to save the log to another machine simultaneously :
>       - use bro, add a new writer (https://www.bro.org/sphinx/
>       scripts/base/frameworks/logging/main.bro.html#id-Log::default_writer
>       <https://www.bro.org/sphinx/scripts/base/frameworks/logging/main.bro.html#id-Log::default_writer>)
>       and then add_filter to ssh and ask it to use the new writer
>       - use syslog, just monitor this main local log and transmit it to
>       another machine
>
> Hope it helps
>
> On Mon, Nov 14, 2016 at 4:35 PM, erik clark <philosnef at gmail.com> wrote:
>
>> So, if I use:
>>
>> redef Log::enable_local_logging
>>
>> in a bro worker cluster, what I find is that all the logs go to
>> /data/bro/spool/worker-1-X instead of all in /data/bro/logs/current on the
>> local machine... Is there a way to fix this?
>>
>> Also, I would want to rotate logs out on the workers that are doing
>> additional local logging to have a much more constrained timeframe for
>> logging, specifically 1 week for local nodes, and 3 months for the logger
>> host.
>>
>> Is the best way to do this just with a cron rm -rf /data/bro/logs/$date ?
>> It seems this would run into a conflict with broctlconfig....
>>
>> Thanks!
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161115/b881f828/attachment.html 

From philosnef at gmail.com  Tue Nov 15 04:36:54 2016
From: philosnef at gmail.com (erik clark)
Date: Tue, 15 Nov 2016 07:36:54 -0500
Subject: [Bro] hyperscan and bro
Message-ID: <CAK6atxog+3Cqxq6LpNxsu=PKCq0Q_1zh-DeW6_5s9WU3TVYTFA@mail.gmail.com>

Anyone looked into doing hyperscan for pattern matching in Bro? The current
pattern matching is very MEH, and it makes me wonder if it might be
possible to use hyperscan and a plugin to do more intense pattern matching.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161115/fbab31d3/attachment.html 

From francois.pennaneach at free.fr  Tue Nov 15 08:20:29 2016
From: francois.pennaneach at free.fr (francois.pennaneach at free.fr)
Date: Tue, 15 Nov 2016 17:20:29 +0100 (CET)
Subject: [Bro] Questions about bro module declarations
Message-ID: <145537267.13041818.1479226829985.JavaMail.root@zimbra8-e1.priv.proxad.net>

Hi Bro people,

I have a few questions about Bro Module declarations (more specifically the GLOBAL module), see below.


I tried this script :

export {
    const var : string = "GLOBAL::var_1";
}

module GLOBAL;

export {
    const var : string = "GLOBAL::var_2";
}

Bro returns an error, which is what I was expecting. Great.
error in ././trybro.bro, line 3 and ././trybro.bro, line 9: already defined (var)
internal warning in ././trybro.bro, line 9: Duplicate identifier documentation: var

So I guess I have defined a variable with scope GLOBAL::var in both cases ?


Then I tried this second script :

export {
    const var : string = "GLOBAL::var_1";
}
 
module Module1;

export {
    function test_func() { print var; }
}


event bro_init() {
    Module1::test_func();
}


It prints "GLOBAL::var_1". Good. So Module1::test_func() has "default" visibility on GLOBAL module.

Third try :

export {
    const var : string = "GLOBAL::var_1";
}

 
module Module1;
    const var : string = "Module1::var";
    function test_func() { print var; }
}

It prints "Module1::var". Why not.
However, I was expecting Bro to report a conflict between using GLOBAL::var or Module1::var, as both are visible...


Fourth test : try to force the use of GLOBAL::var in test_func().

export {
    const var : string = "GLOBAL::var_1";
}

 
module Module1;
    const var : string = "Module1::var";
    function test_func() { print GLOBAL::var; }
}

Bro reports an error : 
 line 15: unknown identifier GLOBAL::var, at or near "GLOBAL::var"


Did I miss something ?
What exactly is GLOBAL and what are its visibility rules and how does Bro search for identifiers in modules ?
How to explicitly make a reference to an identifier in GLOBAL module, as GLOBAL:: does not work ?


Thank you for all helpful tips !

Fran?ois







From rgentz at asu.edu  Tue Nov 15 12:43:57 2016
From: rgentz at asu.edu (Reinhard Gentz)
Date: Tue, 15 Nov 2016 13:43:57 -0700
Subject: [Bro] select element of set of records
Message-ID: <CALtrmTebXaOXsnqExY1SuP0e18fLM7pYm0eE15KjrNvEsd=1Pw@mail.gmail.com>

Hi

I want to send one element of a set of records to a remote event via broker
to python. But i can only set the complete set of records

The code below works, but i receive all elements of the set of records, not
just the one i want to select. When I execute the code below on the other
side i receive  {(whatever, 1), (whatever, 2)} but i only want to receive
{(whatever, 1)}. I know i can filter out the unnecessary data in python,
but it seems wasteful of the bandwidth and computation needed.

type mytest: record{
        a: string &default = "whatever";
        b: string &default= "inhere";
};
type myrecordset: set[mytest];
local myrecord2 = myrecordset([$b="1"],[$b="2"]);

global my_event3: event(msg: myrecordset);

Broker::send_event("bro/events/my_event", Broker::event_args(my_event3,m
yrecord2[mytest($b="1")]));
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161115/bf225ed4/attachment.html 

From jazoff at illinois.edu  Tue Nov 15 13:33:15 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 15 Nov 2016 21:33:15 +0000
Subject: [Bro] select element of set of records
In-Reply-To: <CALtrmTebXaOXsnqExY1SuP0e18fLM7pYm0eE15KjrNvEsd=1Pw@mail.gmail.com>
References: <CALtrmTebXaOXsnqExY1SuP0e18fLM7pYm0eE15KjrNvEsd=1Pw@mail.gmail.com>
Message-ID: <6A7CF17B-61DF-435A-8CCE-5F178EEA7DEF@illinois.edu>


> On Nov 15, 2016, at 3:43 PM, Reinhard Gentz <rgentz at asu.edu> wrote:
> 
> Hi
> 
> I want to send one element of a set of records to a remote event via broker to python. But i can only set the complete set of records
> 
> The code below works, but i receive all elements of the set of records, not just the one i want to select. When I execute the code below on the other side i receive  {(whatever, 1), (whatever, 2)} but i only want to receive {(whatever, 1)}. I know i can filter out the unnecessary data in python, but it seems wasteful of the bandwidth and computation needed.
> 
> type mytest: record{
>         a: string &default = "whatever";
>         b: string &default= "inhere";
> };
> type myrecordset: set[mytest];
> local myrecord2 = myrecordset([$b="1"],[$b="2"]);
> 
> global my_event3: event(msg: myrecordset);
> 
> Broker::send_event("bro/events/my_event", Broker::event_args(my_event3,myrecord2[mytest($b="1")]));

I don't really follow.. why aren't you just doing

global my_event3: event(msg: mytest);
Broker::send_event("bro/events/my_event", Broker::event_args(my_event3, mytest($b="1")));

myrecord2 is a set of two records.  If you only want to send one of the records, just send one of the records, not the set.

It might make more sense if you describe what you're trying to do here.

-- 
- Justin Azoff




From rgentz at asu.edu  Tue Nov 15 13:46:01 2016
From: rgentz at asu.edu (Reinhard Gentz)
Date: Tue, 15 Nov 2016 14:46:01 -0700
Subject: [Bro] select element of set of records
In-Reply-To: <6A7CF17B-61DF-435A-8CCE-5F178EEA7DEF@illinois.edu>
References: <CALtrmTebXaOXsnqExY1SuP0e18fLM7pYm0eE15KjrNvEsd=1Pw@mail.gmail.com>
	<6A7CF17B-61DF-435A-8CCE-5F178EEA7DEF@illinois.edu>
Message-ID: <CALtrmTeO362xC81ST3MKrgQPx9xgjvgGueo97+05qMjqt3CCGQ@mail.gmail.com>

The reason is that the creation of the set elements and sending them out
might not happen at the same time and i do not know how how many elements I
will have.
The overall idea is that i make one element in the set for each ip address
observed, that will have each the corresponding subelements a,b,c saved.
If a critical condition occurs then send the record of that single ip (with
the corresponding elements a,b,c) out to python for handling.


Second from that I thought i can access the elements the following way but
it does not work as expected, tell me what i am doing wrong:
myrecord2[mytest($b="1")]$a   #from myrecord2 take the set element record
where b is "1" and from that return the content of a.



On Tue, Nov 15, 2016 at 2:33 PM, Azoff, Justin S <jazoff at illinois.edu>
wrote:

>
> > On Nov 15, 2016, at 3:43 PM, Reinhard Gentz <rgentz at asu.edu> wrote:
> >
> > Hi
> >
> > I want to send one element of a set of records to a remote event via
> broker to python. But i can only set the complete set of records
> >
> > The code below works, but i receive all elements of the set of records,
> not just the one i want to select. When I execute the code below on the
> other side i receive  {(whatever, 1), (whatever, 2)} but i only want to
> receive {(whatever, 1)}. I know i can filter out the unnecessary data in
> python, but it seems wasteful of the bandwidth and computation needed.
> >
> > type mytest: record{
> >         a: string &default = "whatever";
> >         b: string &default= "inhere";
> > };
> > type myrecordset: set[mytest];
> > local myrecord2 = myrecordset([$b="1"],[$b="2"]);
> >
> > global my_event3: event(msg: myrecordset);
> >
> > Broker::send_event("bro/events/my_event", Broker::event_args(my_event3,
> myrecord2[mytest($b="1")]));
>
> I don't really follow.. why aren't you just doing
>
> global my_event3: event(msg: mytest);
> Broker::send_event("bro/events/my_event", Broker::event_args(my_event3,
> mytest($b="1")));
>
> myrecord2 is a set of two records.  If you only want to send one of the
> records, just send one of the records, not the set.
>
> It might make more sense if you describe what you're trying to do here.
>
> --
> - Justin Azoff
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161115/cf6a4d89/attachment.html 

From jazoff at illinois.edu  Tue Nov 15 13:55:50 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 15 Nov 2016 21:55:50 +0000
Subject: [Bro] select element of set of records
In-Reply-To: <CALtrmTeO362xC81ST3MKrgQPx9xgjvgGueo97+05qMjqt3CCGQ@mail.gmail.com>
References: <CALtrmTebXaOXsnqExY1SuP0e18fLM7pYm0eE15KjrNvEsd=1Pw@mail.gmail.com>
	<6A7CF17B-61DF-435A-8CCE-5F178EEA7DEF@illinois.edu>
	<CALtrmTeO362xC81ST3MKrgQPx9xgjvgGueo97+05qMjqt3CCGQ@mail.gmail.com>
Message-ID: <50547D8E-9A21-4275-8FF9-78CFC235A123@illinois.edu>


> On Nov 15, 2016, at 4:46 PM, Reinhard Gentz <rgentz at asu.edu> wrote:
> 
> The reason is that the creation of the set elements and sending them out might not happen at the same time and i do not know how how many elements I will have. 
> The overall idea is that i make one element in the set for each ip address observed, that will have each the corresponding subelements a,b,c saved.
> If a critical condition occurs then send the record of that single ip (with the corresponding elements a,b,c) out to python for handling. 
> 
> 
> Second from that I thought i can access the elements the following way but it does not work as expected, tell me what i am doing wrong:
> myrecord2[mytest($b="1")]$a   #from myrecord2 take the set element record where b is "1" and from that return the content of a.

You don't want a set then, you want a table[string] of mytest and

mytable["1"] = mytest($b="1", a="2");
mytable["2"] = mytest($b="2", a="4");
...
mytable["1"]$a

or something similar.. It's hard to say without more information.. but you definitely do not want a set.

-- 
- Justin Azoff



From rgentz at asu.edu  Tue Nov 15 15:27:09 2016
From: rgentz at asu.edu (Reinhard Gentz)
Date: Tue, 15 Nov 2016 16:27:09 -0700
Subject: [Bro] select element of set of records
In-Reply-To: <50547D8E-9A21-4275-8FF9-78CFC235A123@illinois.edu>
References: <CALtrmTebXaOXsnqExY1SuP0e18fLM7pYm0eE15KjrNvEsd=1Pw@mail.gmail.com>
	<6A7CF17B-61DF-435A-8CCE-5F178EEA7DEF@illinois.edu>
	<CALtrmTeO362xC81ST3MKrgQPx9xgjvgGueo97+05qMjqt3CCGQ@mail.gmail.com>
	<50547D8E-9A21-4275-8FF9-78CFC235A123@illinois.edu>
Message-ID: <CALtrmTeFs+9vEueAycEXLvEWV9+fE8H2aJg7MkHB6wmvTynoEg@mail.gmail.com>

Thank you for your input. The conversion to tables did what I wanted. Thanks

As a side effort to this project I made a bro2rabbitmq script that can take
any data from bro and send it out to rabbitmq via broker. Once fully
finished I will upload it to github...

On Tue, Nov 15, 2016 at 2:55 PM, Azoff, Justin S <jazoff at illinois.edu>
wrote:

>
> > On Nov 15, 2016, at 4:46 PM, Reinhard Gentz <rgentz at asu.edu> wrote:
> >
> > The reason is that the creation of the set elements and sending them out
> might not happen at the same time and i do not know how how many elements I
> will have.
> > The overall idea is that i make one element in the set for each ip
> address observed, that will have each the corresponding subelements a,b,c
> saved.
> > If a critical condition occurs then send the record of that single ip
> (with the corresponding elements a,b,c) out to python for handling.
> >
> >
> > Second from that I thought i can access the elements the following way
> but it does not work as expected, tell me what i am doing wrong:
> > myrecord2[mytest($b="1")]$a   #from myrecord2 take the set element
> record where b is "1" and from that return the content of a.
>
> You don't want a set then, you want a table[string] of mytest and
>
> mytable["1"] = mytest($b="1", a="2");
> mytable["2"] = mytest($b="2", a="4");
> ...
> mytable["1"]$a
>
> or something similar.. It's hard to say without more information.. but you
> definitely do not want a set.
>
> --
> - Justin Azoff
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161115/4af8c19d/attachment-0001.html 

From tgdesrochers at gmail.com  Thu Nov 17 05:29:54 2016
From: tgdesrochers at gmail.com (Tim Desrochers)
Date: Thu, 17 Nov 2016 08:29:54 -0500
Subject: [Bro] [bro] conn-summary
Message-ID: <CANrcqc07QECWnogjZjn1a-=W77484xjCgKpzKn1Qv6Cn8OAXyw@mail.gmail.com>

Is there a way, when logging in JSON, to get a readable connection summary
log.  When logging in JSON the log is unusable and the tables included in
the log do not get populated.  I like the log because it gives a great
overview of the sensors.

Thanks
Tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161117/9e8809cc/attachment.html 

From seth at icir.org  Thu Nov 17 06:18:57 2016
From: seth at icir.org (Seth Hall)
Date: Thu, 17 Nov 2016 06:18:57 -0800
Subject: [Bro] hyperscan and bro
In-Reply-To: <CAK6atxog+3Cqxq6LpNxsu=PKCq0Q_1zh-DeW6_5s9WU3TVYTFA@mail.gmail.com>
References: <CAK6atxog+3Cqxq6LpNxsu=PKCq0Q_1zh-DeW6_5s9WU3TVYTFA@mail.gmail.com>
Message-ID: <D2F289A4-879A-4619-9331-930E2262097C@icir.org>


> On Nov 15, 2016, at 4:36 AM, erik clark <philosnef at gmail.com> wrote:
> 
> Anyone looked into doing hyperscan for pattern matching in Bro? The current pattern matching is very MEH, and it makes me wonder if it might be possible to use hyperscan and a plugin to do more intense pattern matching.

Is there something in particular that you are unable to do that hyperscan would enable?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Thu Nov 17 06:33:13 2016
From: seth at icir.org (Seth Hall)
Date: Thu, 17 Nov 2016 06:33:13 -0800
Subject: [Bro] Questions about bro module declarations
In-Reply-To: <145537267.13041818.1479226829985.JavaMail.root@zimbra8-e1.priv.proxad.net>
References: <145537267.13041818.1479226829985.JavaMail.root@zimbra8-e1.priv.proxad.net>
Message-ID: <25C64972-F44D-4C54-8DE4-6DD30FC6105D@icir.org>


> On Nov 15, 2016, at 8:20 AM, francois.pennaneach at free.fr wrote:
> 
> Then I tried this second script :
> 
> export {
>    const var : string = "GLOBAL::var_1";
> }
> 
> module Module1;
> 
> export {
>    function test_func() { print var; }
> }
> 
> 
> event bro_init() {
>    Module1::test_func();
> }
> 
> 
> It prints "GLOBAL::var_1". Good. So Module1::test_func() has "default" visibility on GLOBAL module.

It's just looking to the global scope since there is no var_1 in the local scope.


> Third try :
> 
> export {
>    const var : string = "GLOBAL::var_1";
> }
> 
> 
> module Module1;
> export {
>    const var : string = "Module1::var";
>    function test_func() { print var; }
> }
> 
> It prints "Module1::var". Why not.
> However, I was expecting Bro to report a conflict between using GLOBAL::var or Module1::var, as both are visible...

This is the same as your second script.  It looks for the id in the local scope and then tries the global module if the local module doesn't have one with that name.


> Fourth test : try to force the use of GLOBAL::var in test_func().
> 
> export {
>    const var : string = "GLOBAL::var_1";
> }
> 
> module Module1;
> export {
>    const var : string = "Module1::var";
>    function test_func() { print GLOBAL::var; }
> }
> 
> Bro reports an error : 
> line 15: unknown identifier GLOBAL::var, at or near "GLOBAL::var"
> 
> 
> Did I miss something ?
> What exactly is GLOBAL and what are its visibility rules and how does Bro search for identifiers in modules ?
> How to explicitly make a reference to an identifier in GLOBAL module, as GLOBAL:: does not work ?

I'm pretty sure you just encountered a bug.  I see the same behavior and it definitely shouldn't be working that way.  I filed a bug ticket:
	https://bro-tracker.atlassian.net/browse/BIT-1758

Thanks!
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From philosnef at gmail.com  Thu Nov 17 07:07:33 2016
From: philosnef at gmail.com (erik clark)
Date: Thu, 17 Nov 2016 10:07:33 -0500
Subject: [Bro] hyperscan and bro
In-Reply-To: <D2F289A4-879A-4619-9331-930E2262097C@icir.org>
References: <CAK6atxog+3Cqxq6LpNxsu=PKCq0Q_1zh-DeW6_5s9WU3TVYTFA@mail.gmail.com>
	<D2F289A4-879A-4619-9331-930E2262097C@icir.org>
Message-ID: <CAK6atxoFpgirwN9p5EcrrcOWwmy9Jzbg_+cPfg5icnNxjbrK3Q@mail.gmail.com>

Matching acceleration. Currently, we do not have groups in our regexes in
Bro. From a practical point of view, neither does hyperscan. Moving to an
accelerated regex matching engine with roughly the same features as the
current implementation through flex in bro would allow for more expensive
expressions to be rolled out with less overhead. We have seen issues with
some moderately intense regular expressions causing giant spikes in load,
and pushing it through hyperscan would help alleviate it. I realize this
constitutes a major departure from how they are handled currently, but
thought if there was interest, we might be able to push it into 2.6 or even
as far out as 2.7....

On Thu, Nov 17, 2016 at 9:18 AM, Seth Hall <seth at icir.org> wrote:

>
> > On Nov 15, 2016, at 4:36 AM, erik clark <philosnef at gmail.com> wrote:
> >
> > Anyone looked into doing hyperscan for pattern matching in Bro? The
> current pattern matching is very MEH, and it makes me wonder if it might be
> possible to use hyperscan and a plugin to do more intense pattern matching.
>
> Is there something in particular that you are unable to do that hyperscan
> would enable?
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161117/e86a7f4b/attachment.html 

From edautz at gmail.com  Thu Nov 17 08:39:14 2016
From: edautz at gmail.com (=?utf-8?Q?Eug=C3=A8ne_Dautzenberg?=)
Date: Thu, 17 Nov 2016 17:39:14 +0100
Subject: [Bro] Two questions
Message-ID: <9B3941F2-5AD7-4AFE-9282-C942F6E46813@gmail.com>

I setup a Bro instance on a Raspberry Pi3 with an WLAN monitor interface, for IDS home use.

I got notices and with the 

hook Notice::policy(n: Notice::Info)
         {
         add n$actions[Notice::ACTION_EMAIL];
         }

config (example from the mailing list) into my local.bro.I got notices by mail. Works fine.

I also installed critical stack intel feeds, and when I see an Intel file created when I test a banned ip address.

I am new to Bro and have no knowlegde about Bro configuration and scripting language. But I want to make a quickstart

I have two questions:

1: How can the intel also get mailed, when an intel event occurs? 
I tried 

redef Notice::emailed_types += {
         HTTP::IN_HOST_HEADER,
};

Config check is ok but after triggering an intel event I got no mail.

2: I want to incorporate a Bash curl script send alerts to other systems when a notice or an intel event occurs. How to accomplish this?

Thanks in advance.

From elhijo at 0lim.net  Thu Nov 17 01:57:47 2016
From: elhijo at 0lim.net (David)
Date: Thu, 17 Nov 2016 10:57:47 +0100
Subject: [Bro] Json output
Message-ID: <582D7F1B.6010103@0lim.net>

Hi,

I'm probably missing something somewhere but when output log in json 
format I'm missing some information.

Here is an ascii output:
479376326.037159       CAehBQ1VNmICCPUhGk      X.X.X.X       36211 
212.27.48.10    80      1       GET     free.fr /       - 
Lynx/2.8.7rel.2 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/1.0.1i 0       
154     302     Moved Temporarily       -       - -       (empty) 
-       -       -       -       - F6XUb56IvHftrZKH6       text/html

Here is the json one:
{"ts":"2016-11-17T09:52:40.953982Z","uid":"CPRQ0t2QzUecwZtHn4","id.orig_h":"X.X.X.X","id.orig_p":55750,"id.resp_h":"212.27.48.10","id.resp_p":80,"trans_depth":1,"version":"1.1","request_body_len":0,"response_body_len":154,"status_code":302,"status_msg":"Moved 
Temporarily","tags":[],"resp_fuids":["Fh69hd1zG4Giojep18"],"resp_mime_types":["text/html"]}


method, host, uri, referrer, user_agent  and others are missing in json.

Is there a way to add them ?

Thanks,

David


From jazoff at illinois.edu  Thu Nov 17 10:04:12 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Thu, 17 Nov 2016 18:04:12 +0000
Subject: [Bro] Json output
In-Reply-To: <582D7F1B.6010103@0lim.net>
References: <582D7F1B.6010103@0lim.net>
Message-ID: <8224C41A-C810-4CB6-8BC1-BD4890C8E5D1@illinois.edu>

https://www.bro.org/documentation/faq.html#why-isn-t-bro-producing-the-logs-i-expect-a-note-about-checksums
-- 
- Justin Azoff

> On Nov 17, 2016, at 4:57 AM, David <elhijo at 0lim.net> wrote:
> 
> Hi,
> 
> I'm probably missing something somewhere but when output log in json 
> format I'm missing some information.
> 
> Here is an ascii output:
> 479376326.037159       CAehBQ1VNmICCPUhGk      X.X.X.X       36211 
> 212.27.48.10    80      1       GET     free.fr /       - 
> Lynx/2.8.7rel.2 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/1.0.1i 0       
> 154     302     Moved Temporarily       -       - -       (empty) 
> -       -       -       -       - F6XUb56IvHftrZKH6       text/html
> 
> Here is the json one:
> {"ts":"2016-11-17T09:52:40.953982Z","uid":"CPRQ0t2QzUecwZtHn4","id.orig_h":"X.X.X.X","id.orig_p":55750,"id.resp_h":"212.27.48.10","id.resp_p":80,"trans_depth":1,"version":"1.1","request_body_len":0,"response_body_len":154,"status_code":302,"status_msg":"Moved 
> Temporarily","tags":[],"resp_fuids":["Fh69hd1zG4Giojep18"],"resp_mime_types":["text/html"]}
> 
> 
> method, host, uri, referrer, user_agent  and others are missing in json.
> 
> Is there a way to add them ?
> 
> Thanks,
> 
> David
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From elhijo at 0lim.net  Thu Nov 17 10:10:28 2016
From: elhijo at 0lim.net (David)
Date: Thu, 17 Nov 2016 19:10:28 +0100
Subject: [Bro] Json output
In-Reply-To: <8224C41A-C810-4CB6-8BC1-BD4890C8E5D1@illinois.edu>
References: <582D7F1B.6010103@0lim.net>
	<8224C41A-C810-4CB6-8BC1-BD4890C8E5D1@illinois.edu>
Message-ID: <582DF294.3010109@0lim.net>

Yes of course...
Shame on me, forgot this one....

Thanks Justin,
David

On 11/17/2016 07:04 PM, Azoff, Justin S wrote:
> https://www.bro.org/documentation/faq.html#why-isn-t-bro-producing-the-logs-i-expect-a-note-about-checksums


From philosnef at gmail.com  Thu Nov 17 10:40:26 2016
From: philosnef at gmail.com (erik clark)
Date: Thu, 17 Nov 2016 13:40:26 -0500
Subject: [Bro]  Two questions
Message-ID: <CAK6atxqyMO+cVBG9b=s26VfqTDOBgcoC+fKSW+J9n6mtCSnDug@mail.gmail.com>

There is a dirty way you can do it without TOO much effort. Grep your
notice out of notice.log, store the conn_id in a flat file, iterate over it
periodically. For any conn_id not in your flat file, process it, store the
conn_id in the flatfile, and continue. This way you can just run a grep
driven script every X minutes to do this without much effort.

On a big link, this just isn't going to work. You might be grepping a
notice.log file hundreds of megs in size every X minutes, and thats just no
bueno. If you have a small link.... then thats different.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161117/7a9789eb/attachment.html 

From abdulrahmanmusallam at gmail.com  Thu Nov 17 14:09:05 2016
From: abdulrahmanmusallam at gmail.com (abdulrahman musallam)
Date: Fri, 18 Nov 2016 00:09:05 +0200
Subject: [Bro] NOTICES
Message-ID: <CAMZgFn9nTdOBOzHLnKDp+n6Rkss_SQ_qW68qBSSizoqw-O-QYg@mail.gmail.com>

Dear Mrs/Ms,

when i execute a TCP port scanning the notice.log is updated to inform me
about the scanning, on the other hand when i execute an FTP brute force
attack, or an SSH password guessing  attack nothing is updated (no
indication about those attacks), what i should i do to get info about such
attacks?

Regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161118/05716cde/attachment.html 

From robin at icir.org  Thu Nov 17 14:45:47 2016
From: robin at icir.org (Robin Sommer)
Date: Thu, 17 Nov 2016 14:45:47 -0800
Subject: [Bro] Bro 2.5 released
Message-ID: <20161117224547.GG96609@icir.org>

We are very happy to announce the release of Bro v2.5. The new version
is now available for download, see the blog posting for more:
http://blog.bro.org/2016/11/bro-25-released.html

Thanks to everybody providing feedback during the beta period!

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From edautz at gmail.com  Fri Nov 18 00:57:28 2016
From: edautz at gmail.com (Eugene Dautzenberg)
Date: Fri, 18 Nov 2016 09:57:28 +0100
Subject: [Bro] Two questions
In-Reply-To: <CAK6atxqyMO+cVBG9b=s26VfqTDOBgcoC+fKSW+J9n6mtCSnDug@mail.gmail.com>
References: <CAK6atxqyMO+cVBG9b=s26VfqTDOBgcoC+fKSW+J9n6mtCSnDug@mail.gmail.com>
Message-ID: <5429DF09-54AB-4AA9-9E93-26FDD59E46B7@gmail.com>

Eric,

Thnx for your reply. But I am looking for an alerting solution within the Bro framework based on triggerd events. 

Verstuurd vanaf mijn iPhone

> Op 17 nov. 2016 om 19:40 heeft erik clark <philosnef at gmail.com> het volgende geschreven:
> 
> 
> There is a dirty way you can do it without TOO much effort. Grep your notice out of notice.log, store the conn_id in a flat file, iterate over it periodically. For any conn_id not in your flat file, process it, store the conn_id in the flatfile, and continue. This way you can just run a grep driven script every X minutes to do this without much effort.
> 
> On a big link, this just isn't going to work. You might be grepping a notice.log file hundreds of megs in size every X minutes, and thats just no bueno. If you have a small link.... then thats different.


From francois.pennaneach at free.fr  Fri Nov 18 02:15:49 2016
From: francois.pennaneach at free.fr (francois.pennaneach at free.fr)
Date: Fri, 18 Nov 2016 11:15:49 +0100 (CET)
Subject: [Bro] Questions about bro module declarations
In-Reply-To: <25C64972-F44D-4C54-8DE4-6DD30FC6105D@icir.org>
Message-ID: <1950466182.23321364.1479464149630.JavaMail.root@zimbra8-e1.priv.proxad.net>

Hi,

Thank you for your answer.
I took a glimpse at "Scope" C++ code but nothing obvious  :(

Btw, what do you think of the following test case :


module Module1;
export {
    const var : string = "Module1::var";
     
    function test_func1(var : string) { print var, Module1::var; }
    
    function test_func2(            ) { print var, Module1::var; }
}

event bro_init()
	{
	test_func1("Argument");
	test_func2();
	}

Output for this test is :

Argument Argument
Module1::var Module1::var

I think it's a bit disturbing ? I was expecting a warning on variable name clashes, at least.
(I get this one while renaming a local variable in one of my big bro script, suddently everything was broken... ).

Thank you.

----- Mail original -----
De: "Seth Hall" <seth at icir.org>
?: "francois pennaneach" <francois.pennaneach at free.fr>
Cc: bro at bro.org
Envoy?: Jeudi 17 Novembre 2016 15:33:13
Objet: Re: [Bro] Questions about bro module declarations


> On Nov 15, 2016, at 8:20 AM, francois.pennaneach at free.fr wrote:
> 
> Then I tried this second script :
> 
> export {
>    const var : string = "GLOBAL::var_1";
> }
> 
> module Module1;
> 
> export {
>    function test_func() { print var; }
> }
> 
> 
> event bro_init() {
>    Module1::test_func();
> }
> 
> 
> It prints "GLOBAL::var_1". Good. So Module1::test_func() has "default" visibility on GLOBAL module.

It's just looking to the global scope since there is no var_1 in the local scope.


> Third try :
> 
> export {
>    const var : string = "GLOBAL::var_1";
> }
> 
> 
> module Module1;
> export {
>    const var : string = "Module1::var";
>    function test_func() { print var; }
> }
> 
> It prints "Module1::var". Why not.
> However, I was expecting Bro to report a conflict between using GLOBAL::var or Module1::var, as both are visible...

This is the same as your second script.  It looks for the id in the local scope and then tries the global module if the local module doesn't have one with that name.


> Fourth test : try to force the use of GLOBAL::var in test_func().
> 
> export {
>    const var : string = "GLOBAL::var_1";
> }
> 
> module Module1;
> export {
>    const var : string = "Module1::var";
>    function test_func() { print GLOBAL::var; }
> }
> 
> Bro reports an error : 
> line 15: unknown identifier GLOBAL::var, at or near "GLOBAL::var"
> 
> 
> Did I miss something ?
> What exactly is GLOBAL and what are its visibility rules and how does Bro search for identifiers in modules ?
> How to explicitly make a reference to an identifier in GLOBAL module, as GLOBAL:: does not work ?

I'm pretty sure you just encountered a bug.  I see the same behavior and it definitely shouldn't be working that way.  I filed a bug ticket:
	https://bro-tracker.atlassian.net/browse/BIT-1758

Thanks!
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From mfernandez at mitre.org  Fri Nov 18 04:44:12 2016
From: mfernandez at mitre.org (Fernandez, Mark I)
Date: Fri, 18 Nov 2016 12:44:12 +0000
Subject: [Bro] Warning: "Bro node ... possibly still running"
Message-ID: <MWHPR09MB15333715BFF2576C5935D725CFB00@MWHPR09MB1533.namprd09.prod.outlook.com>

Issue #1: My node.cfg file specifies "type=standalone", but I get a BroCtl warning that "Bro node 'worker-1' possibly still running on host...".

Operating on Bro 2.4.1 and BroControl 1.4.

Background:
I configured a local cluster with one manager, one proxy, and two workers.  Worker-1 is monitoring eth1, and worker-2 is monitoring eth2.  The host was suffering too much packet loss, as indicated in the notice.log with the messages "PacketFilter::Dropped_Packets" and "CaptureLoss::Too_Much_Loss".  Therefore, I backed down from a local cluster, to just a standalone configuration in node.cfg.  First, monitored only eth1 for a few days to observe packet loss, and then changed to monitor only eth2 for a few days.  When I edit node.cfg and then run broctl, I get the following warnings:

Warning: broctl node config has changed (run the broctl "deploy" command)
Warning: Bro node "worker-1" possibly still running on host "localhost" (PID www)
Warning: Bro node "worker-2" possibly still running on host "localhost" (PID xxx)
Warning: Bro node "proxy" possibly still running on host "localhost" (PID yyy)
Warning: Bro node "manager" possibly still running on host "localhost" (PID zzz)

This is very curious that broctl "remembers" the previous node.cfg settings.  Of course, none of the PIDs are valid anymore, because those processes were terminated when I changed from a cluster to standalone.  But for some reason, broctl believes these processes might still be running.  Where does BroCtl store this information?

Issue #2: Originally, when I changed node.cfg back to standalone, and then ran BroCtl "deploy" to implement the new configuration, the original manager, proxy, and worker processes were not terminated.  BroCtl left these processes running, and then started a new set of processes for the new config.  I discovered this a few days later because the notice.logs had entries from "bro" (standalone), and still was getting entries from "worker-1" and "worker-2" even though the cluster configuration was removed two days prior.  I would run BroCtl "nodes" and it would correctly show that Bro is standalone monitoring eth1 only.  I was confused.  Finally, I ran process list on the host, and it revealed the original manager, proxy, and workers were all still running.  To clear the situation, I ran BroCtl "stop", then ran "kill -9" on every Bro-related PID, and then ran BroCtl "deploy".  This cleared away the issue of "worker-1" and "worker-2" from writing to the notice.logs; however, I still observe Issue #1, where BroCtl gives the warning messages that "Warning: Bro node ... possibly still running".

I have a crontab to run BroCtl "cron" every five minutes.  Does BroCtl "cron" affect how various configs are "remembered"?  Should I disable that crontab item before making any changes to node.cfg and/or before running BroCtl "deploy"?


Thanks!
Mark I. Fernandez
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161118/c25a9c36/attachment-0001.html 

From francois.pennaneach at free.fr  Fri Nov 18 05:01:05 2016
From: francois.pennaneach at free.fr (francois.pennaneach at free.fr)
Date: Fri, 18 Nov 2016 14:01:05 +0100 (CET)
Subject: [Bro] Questions about bro module declarations
In-Reply-To: <1950466182.23321364.1479464149630.JavaMail.root@zimbra8-e1.priv.proxad.net>
Message-ID: <996602932.23949015.1479474065324.JavaMail.root@zimbra8-e1.priv.proxad.net>

Hi again,


Maybe this example is better to illustrate my problem :


global var : string = "global::var";

function fun1(var : string) {
    print var;                # OK, no error, no warning, function parameter overrides global variable. It prints "parameter var"
}

function fun2(param: string) {
    local var : string = "local var";   # KO, error in ././trybro.bro, line 2: already a global identifier (var)
                                        # error in ././trybro.bro, line 2 and ././trybro.bro, line 9: already defined (var)
    print var;
}


event bro_init()
{
    fun1("parameter var");
    fun2("parameter var");
}

To summarize :
- a function parameter can overrides a global variable with the same name
- a local variable in a function can not override a global variable with the same name


I think this is inconsistent and function parameter and local variables should have the very same behaviour.


What do you think ?

Thanks,
Fran?ois

----- Mail original -----
De: "francois pennaneach" <francois.pennaneach at free.fr>
?: bro at bro.org
Envoy?: Vendredi 18 Novembre 2016 11:15:49
Objet: Re: [Bro] Questions about bro module declarations

Hi,

Thank you for your answer.
I took a glimpse at "Scope" C++ code but nothing obvious  :(

Btw, what do you think of the following test case :


module Module1;
export {
    const var : string = "Module1::var";
     
    function test_func1(var : string) { print var, Module1::var; }
    
    function test_func2(            ) { print var, Module1::var; }
}

event bro_init()
	{
	test_func1("Argument");
	test_func2();
	}

Output for this test is :

Argument Argument
Module1::var Module1::var

I think it's a bit disturbing ? I was expecting a warning on variable name clashes, at least.
(I get this one while renaming a local variable in one of my big bro script, suddently everything was broken... ).

Thank you.

----- Mail original -----
De: "Seth Hall" <seth at icir.org>
?: "francois pennaneach" <francois.pennaneach at free.fr>
Cc: bro at bro.org
Envoy?: Jeudi 17 Novembre 2016 15:33:13
Objet: Re: [Bro] Questions about bro module declarations


> On Nov 15, 2016, at 8:20 AM, francois.pennaneach at free.fr wrote:
> 
> Then I tried this second script :
> 
> export {
>    const var : string = "GLOBAL::var_1";
> }
> 
> module Module1;
> 
> export {
>    function test_func() { print var; }
> }
> 
> 
> event bro_init() {
>    Module1::test_func();
> }
> 
> 
> It prints "GLOBAL::var_1". Good. So Module1::test_func() has "default" visibility on GLOBAL module.

It's just looking to the global scope since there is no var_1 in the local scope.


> Third try :
> 
> export {
>    const var : string = "GLOBAL::var_1";
> }
> 
> 
> module Module1;
> export {
>    const var : string = "Module1::var";
>    function test_func() { print var; }
> }
> 
> It prints "Module1::var". Why not.
> However, I was expecting Bro to report a conflict between using GLOBAL::var or Module1::var, as both are visible...

This is the same as your second script.  It looks for the id in the local scope and then tries the global module if the local module doesn't have one with that name.


> Fourth test : try to force the use of GLOBAL::var in test_func().
> 
> export {
>    const var : string = "GLOBAL::var_1";
> }
> 
> module Module1;
> export {
>    const var : string = "Module1::var";
>    function test_func() { print GLOBAL::var; }
> }
> 
> Bro reports an error : 
> line 15: unknown identifier GLOBAL::var, at or near "GLOBAL::var"
> 
> 
> Did I miss something ?
> What exactly is GLOBAL and what are its visibility rules and how does Bro search for identifiers in modules ?
> How to explicitly make a reference to an identifier in GLOBAL module, as GLOBAL:: does not work ?

I'm pretty sure you just encountered a bug.  I see the same behavior and it definitely shouldn't be working that way.  I filed a bug ticket:
	https://bro-tracker.atlassian.net/browse/BIT-1758

Thanks!
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/


_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From dnthayer at illinois.edu  Fri Nov 18 06:01:45 2016
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Fri, 18 Nov 2016 08:01:45 -0600
Subject: [Bro] Warning: "Bro node ... possibly still running"
In-Reply-To: <MWHPR09MB15333715BFF2576C5935D725CFB00@MWHPR09MB1533.namprd09.prod.outlook.com>
References: <MWHPR09MB15333715BFF2576C5935D725CFB00@MWHPR09MB1533.namprd09.prod.outlook.com>
Message-ID: <ecb41cba-a89d-1f6c-0e75-ee66403898c9@illinois.edu>

In order to prevent this problem, you should run "broctl stop"
before removing (or renaming) any nodes in your node.cfg.


On 11/18/16 6:44 AM, Fernandez, Mark I wrote:
> *_Issue #1_*: My node.cfg file specifies ?type=standalone?, but I get a
> BroCtl warning that ?Bro node ?worker-1? possibly still running on host??.
>
>
>
> Operating on Bro 2.4.1 and BroControl 1.4.
>
>
>
> *_Background_*:
>
> I configured a local cluster with one manager, one proxy, and two
> workers.  Worker-1 is monitoring eth1, and worker-2 is monitoring eth2.
> The host was suffering too much packet loss, as indicated in the
> notice.log with the messages ?PacketFilter::Dropped_Packets? and
> ?CaptureLoss::Too_Much_Loss?.  Therefore, I backed down from a local
> cluster, to just a standalone configuration in node.cfg.  First,
> monitored only eth1 for a few days to observe packet loss, and then
> changed to monitor only eth2 for a few days.  When I edit node.cfg and
> then run broctl, I get the following warnings:
>
>
>
> Warning: broctl node config has changed (run the broctl ?deploy? command)
>
> Warning: Bro node ?worker-1? possibly still running on host ?localhost?
> (PID www)
>
> Warning: Bro node ?worker-2? possibly still running on host ?localhost?
> (PID xxx)
>
> Warning: Bro node ?proxy? possibly still running on host ?localhost?
> (PID yyy)
>
> Warning: Bro node ?manager? possibly still running on host ?localhost?
> (PID zzz)
>
>
>
> This is very curious that broctl ?remembers? the previous node.cfg
> settings.  Of course, none of the PIDs are valid anymore, because those
> processes were terminated when I changed from a cluster to standalone.
>  But for some reason, broctl believes these processes might still be
> running.  Where does BroCtl store this information?
>
>
>
> *_Issue #2_*: Originally, when I changed node.cfg back to standalone,
> and then ran BroCtl ?deploy? to implement the new configuration, the
> original manager, proxy, and worker processes were not terminated.
> BroCtl left these processes running, and then started a new set of
> processes for the new config.  I discovered this a few days later
> because the notice.logs had entries from ?bro? (standalone), and still
> was getting entries from ?worker-1? and ?worker-2? even though the
> cluster configuration was removed two days prior.  I would run BroCtl
> ?nodes? and it would correctly show that Bro is standalone monitoring
> eth1 only.  I was confused.  Finally, I ran process list on the host,
> and it revealed the original manager, proxy, and workers were all still
> running.  To clear the situation, I ran BroCtl ?stop?, then ran ?kill
> -9? on every Bro-related PID, and then ran BroCtl ?deploy?.  This
> cleared away the issue of ?worker-1? and ?worker-2? from writing to the
> notice.logs; however, I still observe *_Issue #1_*, where BroCtl gives
> the warning messages that ?Warning: Bro node ... possibly still running".
>
>
>
> I have a crontab to run BroCtl ?cron? every five minutes.  Does BroCtl
> ?cron? affect how various configs are ?remembered??  Should I disable
> that crontab item before making any changes to node.cfg and/or before
> running BroCtl ?deploy??
>
>
>
>
>
> Thanks!
>
> *Mark I. Fernandez*
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>

From mfernandez at mitre.org  Fri Nov 18 06:44:31 2016
From: mfernandez at mitre.org (Fernandez, Mark I)
Date: Fri, 18 Nov 2016 14:44:31 +0000
Subject: [Bro] Warning: "Bro node ... possibly still running"
In-Reply-To: <ecb41cba-a89d-1f6c-0e75-ee66403898c9@illinois.edu>
References: <MWHPR09MB15333715BFF2576C5935D725CFB00@MWHPR09MB1533.namprd09.prod.outlook.com>
	<ecb41cba-a89d-1f6c-0e75-ee66403898c9@illinois.edu>
Message-ID: <CY4PR09MB15281F747D09C966BB72B30ECFB00@CY4PR09MB1528.namprd09.prod.outlook.com>

Daniel,

Thank you.  To clarify, I should run broctl stop before I even edit the node.cfg file?  I did not do so the first time.  Bro was still running, I edited node.cfg, then ran broctl deploy.  Indeed, while I was troubleshooting this issue, I tried every variation.  I would run broctl stop, then edit node.cfg, then broctl deploy.  This had no effect on the original manager, proxy and worker processes; and the only way to terminate these processes was to run "kill -9".  Even earlier this morning, I ran broctl stop, edited node.cfg, and when I ran broctl, it gave the warnings.  

Now that the damage is done, how do I undo this condition?  I believe the system is monitoring and logging as intended, but for trust and confidence in the system state, I would like to clear away these warnings.  Any advice on how to do so?

Mark


-----Original Message-----
From: Daniel Thayer [mailto:dnthayer at illinois.edu] 
Sent: Friday, November 18, 2016 9:02 AM
To: Fernandez, Mark I <mfernandez at mitre.org>; bro at bro.org
Subject: Re: [Bro] Warning: "Bro node ... possibly still running"

In order to prevent this problem, you should run "broctl stop"
before removing (or renaming) any nodes in your node.cfg.


On 11/18/16 6:44 AM, Fernandez, Mark I wrote:
> *_Issue #1_*: My node.cfg file specifies "type=standalone", but I get a
> BroCtl warning that "Bro node 'worker-1' possibly still running on host...".
>
>
>
> Operating on Bro 2.4.1 and BroControl 1.4.
>
>
>
> *_Background_*:
>
> I configured a local cluster with one manager, one proxy, and two
> workers.  Worker-1 is monitoring eth1, and worker-2 is monitoring eth2.
> The host was suffering too much packet loss, as indicated in the
> notice.log with the messages "PacketFilter::Dropped_Packets" and
> "CaptureLoss::Too_Much_Loss".  Therefore, I backed down from a local
> cluster, to just a standalone configuration in node.cfg.  First,
> monitored only eth1 for a few days to observe packet loss, and then
> changed to monitor only eth2 for a few days.  When I edit node.cfg and
> then run broctl, I get the following warnings:
>
>
>
> Warning: broctl node config has changed (run the broctl "deploy" command)
>
> Warning: Bro node "worker-1" possibly still running on host "localhost"
> (PID www)
>
> Warning: Bro node "worker-2" possibly still running on host "localhost"
> (PID xxx)
>
> Warning: Bro node "proxy" possibly still running on host "localhost"
> (PID yyy)
>
> Warning: Bro node "manager" possibly still running on host "localhost"
> (PID zzz)
>
>
>
> This is very curious that broctl "remembers" the previous node.cfg
> settings.  Of course, none of the PIDs are valid anymore, because those
> processes were terminated when I changed from a cluster to standalone.
>  But for some reason, broctl believes these processes might still be
> running.  Where does BroCtl store this information?
>
>
>
> *_Issue #2_*: Originally, when I changed node.cfg back to standalone,
> and then ran BroCtl "deploy" to implement the new configuration, the
> original manager, proxy, and worker processes were not terminated.
> BroCtl left these processes running, and then started a new set of
> processes for the new config.  I discovered this a few days later
> because the notice.logs had entries from "bro" (standalone), and still
> was getting entries from "worker-1" and "worker-2" even though the
> cluster configuration was removed two days prior.  I would run BroCtl
> "nodes" and it would correctly show that Bro is standalone monitoring
> eth1 only.  I was confused.  Finally, I ran process list on the host,
> and it revealed the original manager, proxy, and workers were all still
> running.  To clear the situation, I ran BroCtl "stop", then ran "kill
> -9" on every Bro-related PID, and then ran BroCtl "deploy".  This
> cleared away the issue of "worker-1" and "worker-2" from writing to the
> notice.logs; however, I still observe *_Issue #1_*, where BroCtl gives
> the warning messages that "Warning: Bro node ... possibly still running".
>
>
>
> I have a crontab to run BroCtl "cron" every five minutes.  Does BroCtl
> "cron" affect how various configs are "remembered"?  Should I disable
> that crontab item before making any changes to node.cfg and/or before
> running BroCtl "deploy"?
>
>
>
>
>
> Thanks!
>
> *Mark I. Fernandez*
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>


From philosnef at gmail.com  Fri Nov 18 09:04:25 2016
From: philosnef at gmail.com (erik clark)
Date: Fri, 18 Nov 2016 12:04:25 -0500
Subject: [Bro] af_packet plugin doesnt compile in 2.5
Message-ID: <CAK6atxo=CpVxG__RS35H7ifa1mC2C6ACiLtaHuU+nOX6dTVMTA@mail.gmail.com>

In the 2.5 release, the af_packet plugin is not compiling on rh6.8. I am
getting"

.../RX_Ring.h:27:21: error: 'tpacket_hdr' has not been declared
          bool GetNextPacket(tpacket3_hdr** hdr);


.../RX_Ring.h:35:22: error: field 'layout' has incomplete type
         struct tpacket_req3 layout;

Thanks!

Please note this compiles on rh7. Just not rh6.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161118/4487559d/attachment.html 

From jan.grashoefer at gmail.com  Fri Nov 18 10:28:39 2016
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Fri, 18 Nov 2016 19:28:39 +0100
Subject: [Bro] af_packet plugin doesnt compile in 2.5
In-Reply-To: <CAK6atxo=CpVxG__RS35H7ifa1mC2C6ACiLtaHuU+nOX6dTVMTA@mail.gmail.com>
References: <CAK6atxo=CpVxG__RS35H7ifa1mC2C6ACiLtaHuU+nOX6dTVMTA@mail.gmail.com>
Message-ID: <87bf1776-6203-4efd-d2e6-ea125766290e@gmail.com>

Hi Erik,

> In the 2.5 release, the af_packet plugin is not compiling on rh6.8.

"Make sure the kernel headers are installed and your kernel supports
PACKET_FANOUT and TPACKET_V3."

As far as I know, RHEL 6.8 is based on a 2.6 kernel, which does not
support TPACKET_V3. I think something around the 3.10 kernel is a
minimum requirement. Furthermore there have been issues with later
kernel versions that have been fixed for 4.4.16, 4.6.5 and 4.7. I don't
know whether the fix made it into the 3.X kernels.

Best regards,
Jan

From mgill6 at student.concordia.ab.ca  Fri Nov 18 21:35:09 2016
From: mgill6 at student.concordia.ab.ca (Manmeet Gill)
Date: Fri, 18 Nov 2016 22:35:09 -0700
Subject: [Bro] Is this type of script is possible to create ?
Message-ID: <CAMFH7YRGCWirkUg4KEGHkEraeEtW8LB7d8-5uQZ+=vkxWNLr9g@mail.gmail.com>

is it possible that below described statement can be crafted into a bro
script ?
Plz help me if it is possible, let me know what i need to do, to make this
possible.

If my incoming traffic rate exceeds 44Mbps and the average incoming traffic
rate over the last 504seconds exceeds the average incoming traffic rate
over the last 965seconds by more than 70%, send an alert

Thank you Everyone.
MeetGill
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161118/02d40b13/attachment.html 

From bobharrelsons at gmail.com  Sat Nov 19 13:44:11 2016
From: bobharrelsons at gmail.com (Robert Harrelson)
Date: Sat, 19 Nov 2016 16:44:11 -0500
Subject: [Bro] Does x509.log contain the raw certificate?
Message-ID: <CAHQ_EYQuzY66DdOxke3VcgCsWTeC+Sgq5fDO_pBvfX2i_e2sFg@mail.gmail.com>

The log file x509.log contains parsed information from the X.509
certificate. However, I would like to know if the x509.log file contains
the raw X.509 certificate itself. If yes, how do I extract the certificate
from the log, not in real-time? Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161119/38596618/attachment.html 

From anthony.kasza at gmail.com  Sat Nov 19 15:06:21 2016
From: anthony.kasza at gmail.com (anthony kasza)
Date: Sat, 19 Nov 2016 16:06:21 -0700
Subject: [Bro] Does x509.log contain the raw certificate?
In-Reply-To: <CAHQ_EYQuzY66DdOxke3VcgCsWTeC+Sgq5fDO_pBvfX2i_e2sFg@mail.gmail.com>
References: <CAHQ_EYQuzY66DdOxke3VcgCsWTeC+Sgq5fDO_pBvfX2i_e2sFg@mail.gmail.com>
Message-ID: <CAEZw2bxhmeFsQav-NoMn7D80G+1ZYh309LiMWDRAN8+Y+XBotw@mail.gmail.com>

The certificates are not contained in any log file, just certificate meta
data. To enable certificate extraction you need to enable the files
framework which will write certificates to disk.

-AK

On Nov 19, 2016 2:53 PM, "Robert Harrelson" <bobharrelsons at gmail.com> wrote:

> The log file x509.log contains parsed information from the X.509
> certificate. However, I would like to know if the x509.log file contains
> the raw X.509 certificate itself. If yes, how do I extract the certificate
> from the log, not in real-time? Thanks
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161119/f1cec3bb/attachment.html 

From abdulrahmanmusallam at gmail.com  Sun Nov 20 14:28:16 2016
From: abdulrahmanmusallam at gmail.com (abdulrahman musallam)
Date: Mon, 21 Nov 2016 00:28:16 +0200
Subject: [Bro] Bro detection scripts
Message-ID: <CAMZgFn-VKY6TZbu-yOwvT3PHSBpxgVaDLGkkimG7eSXSnntTPQ@mail.gmail.com>

Hi,
when i perform an TCP port scanning on my machine Bro raises a notice
immediately  to notice.log and this notice is raised by scan.bro script
that detect scanning, such scripts exist for FTP brute forcing  and SSH
password guessing but when i perform any of these attacks (FTP brute
forcing  and SSH password guessing)  it won't show anything in notice log
that indicates any occurrence of them!! could someone please help me with
this problem! HOW TO INVOKE BRO DETECTION SCRIPTS??

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161121/8e156b4d/attachment.html 

From pachinko.tw at gmail.com  Sun Nov 20 19:17:51 2016
From: pachinko.tw at gmail.com (Po-Ching Lin)
Date: Mon, 21 Nov 2016 11:17:51 +0800
Subject: [Bro] Building Bro with PF_RING
Message-ID: <ebd39239-0a87-64ed-e5f1-23caf57a9e7d@gmail.com>

Dear all,

           I tried the method introduced in the following link:

https://www.bro.org/sphinx-git/components/bro-plugins/pf_ring/README.html

,but the output of bro -N Bro::PF_RING is not as expected, given PF_RING is installed

in /usr/local. That web page suggests the use of --with-pfring in configure, but it seems

that configure does not support that option. There is another way to build with PF_RING:

https://www.bro.org/documentation/load-balancing.html

This method works. Should the first document be updated?


From jdopheid at illinois.edu  Mon Nov 21 06:38:22 2016
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Mon, 21 Nov 2016 14:38:22 +0000
Subject: [Bro] Announcing Bro4Pros 2017, Feb 2nd in San Francisco
Message-ID: <97308EC1-C456-4CCE-95FA-722857811B03@illinois.edu>

Mark your calendars! Bro4Pros 2017<https://www.bro.org/community/bro4pros2017.html> will be on Thursday, February 2nd in San Francisco, CA at Salesforce<https://www.salesforce.com/>'s Spear St. office (map<https://goo.gl/maps/rChxDcQ24r62>).



Bro4Pros is a one-day workshop for advanced Bro users (i.e., those who use Bro on a daily basis, feel comfortable customizing its configuration, and have written scripts on their own).



This is a joined community effort and get-together, and the program will depend to a large degree on what people want to talk about. Attendance is limited to ensure an interactive and productive atmosphere.



We scheduled this year's workshop immediately after the Usenix Enigma<https://www.usenix.org/conference/enigma2017> conference to make travel a little more convenient for out-of-towners.



Registration<https://www.eventbrite.com/e/bro4pros-2017-tickets-29303802462> is free and will open at 11am PST on Thursday, December 1st. Seats are limited and are first-come, first serve. If you have to cancel your registration, please contact us to release your seat.



Call for presentations:



We have a few spots available for community presentations.



Send abstracts (max 500 words) to: info at bro.org

Subject: Bro4Pros 2017 Call for Presentations

Submission due date: January 6th, 2017



More details about the event can be found on our event page<https://www.bro.org/community/bro4pros2017.html>.



Thank you to Salesforce for sponsoring this event.





------

Jeannette Dopheide

Training and Outreach Coordinator

National Center for Supercomputing Applications

University of Illinois at Urbana-Champaign
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161121/a9f00e36/attachment.html 

From philosnef at gmail.com  Mon Nov 21 08:03:59 2016
From: philosnef at gmail.com (erik clark)
Date: Mon, 21 Nov 2016 11:03:59 -0500
Subject: [Bro]  Building Bro with PF_RING
Message-ID: <CAK6atxrChyduHpftTXp=R9K00AEsk_0s3gzPR6CWb_QAPhFuSw@mail.gmail.com>

Are you building bro on 2.5 or 241? If you are building on 25, its in
aux/plugins/pf_ring and you need to specify where the headers are for
pfring.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161121/2eb8871d/attachment.html 

From vladg at illinois.edu  Mon Nov 21 08:12:21 2016
From: vladg at illinois.edu (Vlad Grigorescu)
Date: Mon, 21 Nov 2016 10:12:21 -0600
Subject: [Bro] binPAC : more than one &require attribute on a field
In-Reply-To: <86842898-2d45-b079-e029-97b243cfdaad@free.fr>
References: <86842898-2d45-b079-e029-97b243cfdaad@free.fr>
Message-ID: <m0bmx8revu.fsf@vladg.net>

Hi Fran?ois,

Thanks for reporting this. I'll need to think through some of the
implications a bit more closely.

If you'd be willing to share your patch, I'd be happy to take a look.
Thanks,

  --Vlad




Fran?ois Pennaneach <francois.pennaneach at free.fr> writes:

> Hi all,
>
>
> I'm a Bro beginner. I got a small problem when writing binPAC. See below.
>
> I'm using master branch of binPAC.
>
>
> In the binPAC grammar, nothing prevents from applying many &requires 
> attributes to the same field.
> However,  in such a case the generated C++ code is incorrect.
>
> type MyArray = record {
>      a: uint16 &requires(c) &requires(d);
>      b: uint16;
> } &let {
>      c : uint16 = b * 2;
>      d : uint16 = b * 3;
> };
>
> The generated code is :
>      // Parse "a"
>      // Parse "b"
>      b_ = FixByteOrder(t_byteorder, *((uint16 const *) ((t_begin_of_data 
> + 2))));
>      // Evaluate 'let' and 'withinput' fields
>      d_ = b() * 3;
>      a_ = FixByteOrder(t_byteorder, *((uint16 const *) (t_begin_of_data)));
>      // Evaluate 'let' and 'withinput' fields
>
>      // Evaluate 'let' and 'withinput' fields
>      c_ = b() * 2;
>
>
> In pac_types.cc, only the last &requires attribute is kept, the previous 
> ones are forgotten.
> Replacing attr_requires_ of type Expr with a ListExpr solves the problem 
> and produces the (expected) C++ code below :
>
>      // Parse "a"
>      // Parse "b"
>      b_ = FixByteOrder(t_byteorder, *((uint16 const *) ((t_begin_of_data 
> + 2))));
>      // Evaluate 'let' and 'withinput' fields
>      c_ = b() * 2;
>      d_ = b() * 3;
>      a_ = FixByteOrder(t_byteorder, *((uint16 const *) (t_begin_of_data)));
>
>
> I have written a small patch for this problem. I can submit it if you 
> agree with my changes.
>
>
> Thank you.
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 800 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161121/0f8de2f0/attachment.bin 

From jlay at slave-tothe-box.net  Mon Nov 21 08:14:19 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Mon, 21 Nov 2016 09:14:19 -0700
Subject: [Bro] Building Bro with PF_RING
In-Reply-To: <CAK6atxrChyduHpftTXp=R9K00AEsk_0s3gzPR6CWb_QAPhFuSw@mail.gmail.com>
References: <CAK6atxrChyduHpftTXp=R9K00AEsk_0s3gzPR6CWb_QAPhFuSw@mail.gmail.com>
Message-ID: <e304d1f3f89764e9332caa40ed464151@localhost>

On 2016-11-21 09:03, erik clark wrote:
> Are you building bro on 2.5 or 241? If you are building on 25, its in
> aux/plugins/pf_ring and you need to specify where the headers are for
> pfring.
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

My notes, building pf_ring into /opt:

git clone https://github.com/ntop/PF_RING.git
cd PF_RING/kernel
make
sudo make install

cd ../userland/lib
./configure --prefix=/opt/pfring
sudo make install

cd ../libpcap
./configure --prefix=/opt/pfring
sudo make install

cd ../tcpdump
./configure --prefix=/opt/pfring
sudo make install

cd bro-2.5
./configure --with-pcap=/opt/pfring
make
sudo make install

pf_ring plugin
cd aux/plugins/pf_ring/
./configure --bro-dist=../../.. --with-pfring=/opt/pfring 
--install-root=/opt/bro/lib/bro/plugins
make
sudo make install

Should get you up and going..if someone sees any errors please let me 
know.

James

From jedwards2728 at gmail.com  Wed Nov 23 00:23:58 2016
From: jedwards2728 at gmail.com (John Edwards)
Date: Wed, 23 Nov 2016 08:23:58 +0000
Subject: [Bro] New Bro cluster
Message-ID: <CAAcg0e+AjwieyTFfmVD_e8x-k5hkqQEmHaYQq71mvTGrEMNx3w@mail.gmail.com>

Hi

I currently have two bro stand alone workers analysing traffic. I am
building 6 more systems so in total i will have 1 Manager/Proxy and 3
workers per leg of my gateway i need to have visibility on.

I was going to spec out the workers the most and then im not 100% on the
front end worker options. Do i place the PF_RING on each worker and the
manager? Or just the workers? And i am not familiar with the other pieces
of hardware mentioned in the clustering guide as a front end option if I
don't use PF_RING . Would a F5 load balancer that had a VIP for the workers
sending traffic to the manager work just as good? Or not a good option?

Thanks
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161123/97af1963/attachment.html 

From philosnef at gmail.com  Wed Nov 23 04:27:25 2016
From: philosnef at gmail.com (erik clark)
Date: Wed, 23 Nov 2016 07:27:25 -0500
Subject: [Bro]  New Bro cluster
Message-ID: <CAK6atxrTcC4xkPwzPcVx0u_gEtrEpS2zF8ayXutPJRG2mTSmuw@mail.gmail.com>

Re bro and pf_ring, I would recommend af_packet over pfring, if you are
running a recent OS that supports it in Bro (see earlier). This is because
af_packet comes built-in with your distro, and pf_ring is an addon. This
makes it easier to manage imo.

If you build pf_ring, you will need the kernel module and shared objects on
each box. Bro isn't going to put those there for you....

Moreover, I would highly recommend you build pf_ring as a module vrs
compiled into bro itself. Personal opinion though.

Pf_ring doesnt do loadbalancing on a link (it does it on the card between
threads), so if you want to balance over multiple bro boxes, you definitely
need something like a load balancing tap, a passive load balancer, or your
f5 (which I believe does 5 tuple balancing). Cue the
pleaselookatthelblpaperonloadbalancinga100giglink paper comments. :D


Hope this helps.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161123/66a4544b/attachment.html 

From ysrivas at ncsu.edu  Wed Nov 23 12:25:33 2016
From: ysrivas at ncsu.edu (Yagyesh Srivastava)
Date: Wed, 23 Nov 2016 15:25:33 -0500
Subject: [Bro] help required in logs with bro
Message-ID: <CAP4Z=hRv9KCnYBckf_pEWc69c4b2W+8p3qw1F_ZB-onWKD0oBA@mail.gmail.com>

Hi,

I have downloaded bro and built it on a VM, using configure, make and make
install.
Then i ran broctl install and deploy.
when i run broctl using "sudo broctl start" and subsequently issue "sudo
broctl status", it shows bro running as standalone on localhost.

my /nsm/bro/etc/nod.cfg file has
type = standalone
host = localhost
interface = eth0

Now when i try to connect to internet using my vm browser
or i curl to localhost (which has apache server running and after making
node.cfg file to hear on interface loopback) in either of the cases i
cannot see any logs getting generated.


*can someone please help me with this issue?*I dont think bro is sniffing
on the correct interface , there is something trivial i am guessing which
is going wrong. Please provide any pointers if possible.

Thanks,
Yagyesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161123/502df060/attachment.html 

From anthony.kasza at gmail.com  Wed Nov 23 13:54:50 2016
From: anthony.kasza at gmail.com (anthony kasza)
Date: Wed, 23 Nov 2016 14:54:50 -0700
Subject: [Bro] help required in logs with bro
In-Reply-To: <CAP4Z=hRv9KCnYBckf_pEWc69c4b2W+8p3qw1F_ZB-onWKD0oBA@mail.gmail.com>
References: <CAP4Z=hRv9KCnYBckf_pEWc69c4b2W+8p3qw1F_ZB-onWKD0oBA@mail.gmail.com>
Message-ID: <CAEZw2bwnbWkDzpwywM3NC5D7cExmZUfsVyV9dd0x=5y288u_Tg@mail.gmail.com>

Your VM may be using its loopback address for the connection to the local
Apache server. If Bro is listening on eth0 (not the loopback interface) it
won't see that traffic.

As for the curl'ing of external sites, have you tried something basic like
tcpdump just to make sure packets are moving? I'd also try running the Bro
binary, without broctl, on an interface just to make sure Bro is compiled,
happy, and seeing packets move.

-AK

On Nov 23, 2016 1:33 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu> wrote:

> Hi,
>
> I have downloaded bro and built it on a VM, using configure, make and make
> install.
> Then i ran broctl install and deploy.
> when i run broctl using "sudo broctl start" and subsequently issue "sudo
> broctl status", it shows bro running as standalone on localhost.
>
> my /nsm/bro/etc/nod.cfg file has
> type = standalone
> host = localhost
> interface = eth0
>
> Now when i try to connect to internet using my vm browser
> or i curl to localhost (which has apache server running and after making
> node.cfg file to hear on interface loopback) in either of the cases i
> cannot see any logs getting generated.
>
>
> *can someone please help me with this issue?*I dont think bro is sniffing
> on the correct interface , there is something trivial i am guessing which
> is going wrong. Please provide any pointers if possible.
>
> Thanks,
> Yagyesh
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161123/c1dec42c/attachment.html 

From ysrivas at ncsu.edu  Wed Nov 23 14:01:48 2016
From: ysrivas at ncsu.edu (Yagyesh Srivastava)
Date: Wed, 23 Nov 2016 17:01:48 -0500
Subject: [Bro] help required in logs with bro
In-Reply-To: <CAEZw2bwnbWkDzpwywM3NC5D7cExmZUfsVyV9dd0x=5y288u_Tg@mail.gmail.com>
References: <CAP4Z=hRv9KCnYBckf_pEWc69c4b2W+8p3qw1F_ZB-onWKD0oBA@mail.gmail.com>
	<CAEZw2bwnbWkDzpwywM3NC5D7cExmZUfsVyV9dd0x=5y288u_Tg@mail.gmail.com>
Message-ID: <CAP4Z=hS8aKP1ihpmU1SU3f49--T2WgDyGAVZKQiWpACXA2=imA@mail.gmail.com>

By bro binary you mean " bro -i eth0" command?
I can see that when I give this command it's listening on eth0 interface.
It initially gave me a warning saying due to NIC checksum it is receiving
bad checksum packets so it will discard it.
So I ran the above command with -C option.
Is this what you were referring to?
Could you please help me understand what's the difference between this
command and broctl?

Thanks and regards

On Nov 23, 2016 4:54 PM, "anthony kasza" <anthony.kasza at gmail.com> wrote:

> Your VM may be using its loopback address for the connection to the local
> Apache server. If Bro is listening on eth0 (not the loopback interface) it
> won't see that traffic.
>
> As for the curl'ing of external sites, have you tried something basic like
> tcpdump just to make sure packets are moving? I'd also try running the Bro
> binary, without broctl, on an interface just to make sure Bro is compiled,
> happy, and seeing packets move.
>
> -AK
>
> On Nov 23, 2016 1:33 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu> wrote:
>
>> Hi,
>>
>> I have downloaded bro and built it on a VM, using configure, make and
>> make install.
>> Then i ran broctl install and deploy.
>> when i run broctl using "sudo broctl start" and subsequently issue "sudo
>> broctl status", it shows bro running as standalone on localhost.
>>
>> my /nsm/bro/etc/nod.cfg file has
>> type = standalone
>> host = localhost
>> interface = eth0
>>
>> Now when i try to connect to internet using my vm browser
>> or i curl to localhost (which has apache server running and after making
>> node.cfg file to hear on interface loopback) in either of the cases i
>> cannot see any logs getting generated.
>>
>>
>> *can someone please help me with this issue?*I dont think bro is
>> sniffing on the correct interface , there is something trivial i am
>> guessing which is going wrong. Please provide any pointers if possible.
>>
>> Thanks,
>> Yagyesh
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161123/62167bda/attachment.html 

From anthony.kasza at gmail.com  Wed Nov 23 14:22:01 2016
From: anthony.kasza at gmail.com (anthony kasza)
Date: Wed, 23 Nov 2016 15:22:01 -0700
Subject: [Bro] help required in logs with bro
In-Reply-To: <CAP4Z=hS8aKP1ihpmU1SU3f49--T2WgDyGAVZKQiWpACXA2=imA@mail.gmail.com>
References: <CAP4Z=hRv9KCnYBckf_pEWc69c4b2W+8p3qw1F_ZB-onWKD0oBA@mail.gmail.com>
	<CAEZw2bwnbWkDzpwywM3NC5D7cExmZUfsVyV9dd0x=5y288u_Tg@mail.gmail.com>
	<CAP4Z=hS8aKP1ihpmU1SU3f49--T2WgDyGAVZKQiWpACXA2=imA@mail.gmail.com>
Message-ID: <CAEZw2bwBJUxJOdyXyciv-cQ+iKP3my0CN67gfhNNmEP+MpTbKw@mail.gmail.com>

Broctl wraps the Bro binary and adds some niceties like config management,
worker management, log rotation, etc. The Bro binary is what processes
packets, interprets scripts, and writes logs.

If you run 'bro -Ci eth0' and browse a webserver over eth0, bro should spit
out logs in your current working directory. If not, Bro is either not
seeing packets or something else is wrong.

-AK

On Nov 23, 2016 3:01 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu> wrote:

> By bro binary you mean " bro -i eth0" command?
> I can see that when I give this command it's listening on eth0 interface.
> It initially gave me a warning saying due to NIC checksum it is receiving
> bad checksum packets so it will discard it.
> So I ran the above command with -C option.
> Is this what you were referring to?
> Could you please help me understand what's the difference between this
> command and broctl?
>
> Thanks and regards
>
> On Nov 23, 2016 4:54 PM, "anthony kasza" <anthony.kasza at gmail.com> wrote:
>
>> Your VM may be using its loopback address for the connection to the local
>> Apache server. If Bro is listening on eth0 (not the loopback interface) it
>> won't see that traffic.
>>
>> As for the curl'ing of external sites, have you tried something basic
>> like tcpdump just to make sure packets are moving? I'd also try running the
>> Bro binary, without broctl, on an interface just to make sure Bro is
>> compiled, happy, and seeing packets move.
>>
>> -AK
>>
>> On Nov 23, 2016 1:33 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu> wrote:
>>
>>> Hi,
>>>
>>> I have downloaded bro and built it on a VM, using configure, make and
>>> make install.
>>> Then i ran broctl install and deploy.
>>> when i run broctl using "sudo broctl start" and subsequently issue "sudo
>>> broctl status", it shows bro running as standalone on localhost.
>>>
>>> my /nsm/bro/etc/nod.cfg file has
>>> type = standalone
>>> host = localhost
>>> interface = eth0
>>>
>>> Now when i try to connect to internet using my vm browser
>>> or i curl to localhost (which has apache server running and after making
>>> node.cfg file to hear on interface loopback) in either of the cases i
>>> cannot see any logs getting generated.
>>>
>>>
>>> *can someone please help me with this issue?*I dont think bro is
>>> sniffing on the correct interface , there is something trivial i am
>>> guessing which is going wrong. Please provide any pointers if possible.
>>>
>>> Thanks,
>>> Yagyesh
>>>
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161123/ebc85203/attachment-0001.html 

From anthony.kasza at gmail.com  Wed Nov 23 14:30:26 2016
From: anthony.kasza at gmail.com (anthony kasza)
Date: Wed, 23 Nov 2016 15:30:26 -0700
Subject: [Bro] help required in logs with bro
In-Reply-To: <CAP4Z=hRMcbkc=ToHvHK0z0rT7Ld=sLvA3UGkvifVPQpoUAUH5w@mail.gmail.com>
References: <CAP4Z=hRv9KCnYBckf_pEWc69c4b2W+8p3qw1F_ZB-onWKD0oBA@mail.gmail.com>
	<CAEZw2bwnbWkDzpwywM3NC5D7cExmZUfsVyV9dd0x=5y288u_Tg@mail.gmail.com>
	<CAP4Z=hS8aKP1ihpmU1SU3f49--T2WgDyGAVZKQiWpACXA2=imA@mail.gmail.com>
	<CAP4Z=hRJ6mkqqyrjGTuYAnzt-XvE4Zt0TAQxdP7OQg9w9jGjaA@mail.gmail.com>
	<CAEZw2bwyMKig2kMHxYcx4xJFQDD7aUg0kuEG8h=EnD3pEApMxA@mail.gmail.com>
	<CAP4Z=hRMcbkc=ToHvHK0z0rT7Ld=sLvA3UGkvifVPQpoUAUH5w@mail.gmail.com>
Message-ID: <CAEZw2bw6Cz_NAxFPqtVhTRkD=wiwLxL-McEzdQrVsaU6smoPeQ@mail.gmail.com>

I've put the list back on CC.

Broctl will write logs wherever it's configured to. I'm not familiar with
the Security Onion distribution enough to troubleshoot it. Bro seems to be
working correctly. My guess is you need to fiddle with the broctl
configuration security onion is using.

-AK

On Nov 23, 2016 3:25 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu> wrote:

> If you run 'bro -Ci eth0' and browse a webserver over eth0, bro should
> spit out logs in your current working directory. If not, Bro is either not
> seeing packets or something else is wrong.
> *I am getting this correctly.*
>
> Is there a reason why broctl would not work in this case(when bro binary
> is)?
> Also just to make sure, broctl will always send logs in
> /nsm/bro/logs/current right?
>
> On Wed, Nov 23, 2016 at 5:22 PM, anthony kasza <anthony.kasza at gmail.com>
> wrote:
>
>> Try writing a trace file to disk with tcpdump and reading it with Bro
>> using the -r option.
>>
>> On Nov 23, 2016 3:11 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu> wrote:
>>
>>> also anthony,
>>> I did try tcpdump:
>>> yagyesh at yagyesh-virtual-machine:/nsm/bro/share/bro/base/protocols/http$
>>> sudo tcpdump -nS
>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>> decode
>>> listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
>>> 17:03:30.300262 IP 192.168.170.135.45554 > 31.13.69.228.443: Flags [P.],
>>> seq 1936267894:1936268148, ack 2692707210, win 64240, length 254
>>> 17:03:30.300484 IP 31.13.69.228.443 > 192.168.170.135.45554: Flags [.],
>>> ack 1936268148, win 64240, length 0
>>>
>>>
>>> it is showing me the movement of packets.
>>>
>>> When i am doing a curl to localhost, then i am changing the node.cfg
>>> file so as to reflect interface=lo and not eth0.
>>> Thats the only change that we need if we need to monitor loopback port
>>> with bro instead of eth0?
>>>
>>> Also someone suggested that the interface might not be in monitor mode,
>>> had this been the case would i have received the tcpdump like mentioned
>>> above?
>>>
>>> On Wed, Nov 23, 2016 at 5:01 PM, Yagyesh Srivastava <ysrivas at ncsu.edu>
>>> wrote:
>>>
>>>> By bro binary you mean " bro -i eth0" command?
>>>> I can see that when I give this command it's listening on eth0
>>>> interface. It initially gave me a warning saying due to NIC checksum it is
>>>> receiving bad checksum packets so it will discard it.
>>>> So I ran the above command with -C option.
>>>> Is this what you were referring to?
>>>> Could you please help me understand what's the difference between this
>>>> command and broctl?
>>>>
>>>> Thanks and regards
>>>>
>>>> On Nov 23, 2016 4:54 PM, "anthony kasza" <anthony.kasza at gmail.com>
>>>> wrote:
>>>>
>>>>> Your VM may be using its loopback address for the connection to the
>>>>> local Apache server. If Bro is listening on eth0 (not the loopback
>>>>> interface) it won't see that traffic.
>>>>>
>>>>> As for the curl'ing of external sites, have you tried something basic
>>>>> like tcpdump just to make sure packets are moving? I'd also try running the
>>>>> Bro binary, without broctl, on an interface just to make sure Bro is
>>>>> compiled, happy, and seeing packets move.
>>>>>
>>>>> -AK
>>>>>
>>>>> On Nov 23, 2016 1:33 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have downloaded bro and built it on a VM, using configure, make and
>>>>>> make install.
>>>>>> Then i ran broctl install and deploy.
>>>>>> when i run broctl using "sudo broctl start" and subsequently issue
>>>>>> "sudo broctl status", it shows bro running as standalone on localhost.
>>>>>>
>>>>>> my /nsm/bro/etc/nod.cfg file has
>>>>>> type = standalone
>>>>>> host = localhost
>>>>>> interface = eth0
>>>>>>
>>>>>> Now when i try to connect to internet using my vm browser
>>>>>> or i curl to localhost (which has apache server running and after
>>>>>> making node.cfg file to hear on interface loopback) in either of the cases
>>>>>> i cannot see any logs getting generated.
>>>>>>
>>>>>>
>>>>>> *can someone please help me with this issue?*I dont think bro is
>>>>>> sniffing on the correct interface , there is something trivial i am
>>>>>> guessing which is going wrong. Please provide any pointers if possible.
>>>>>>
>>>>>> Thanks,
>>>>>> Yagyesh
>>>>>>
>>>>>> _______________________________________________
>>>>>> Bro mailing list
>>>>>> bro at bro-ids.org
>>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>>>>
>>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161123/dfaf8814/attachment.html 

From ysrivas at ncsu.edu  Wed Nov 23 15:26:04 2016
From: ysrivas at ncsu.edu (Yagyesh Srivastava)
Date: Wed, 23 Nov 2016 18:26:04 -0500
Subject: [Bro] help required in logs with bro
In-Reply-To: <CAEZw2bwaJ5FUJBxfmSHNx9RnU-s+wRJ+Q6Fw3yfs7Do302czoA@mail.gmail.com>
References: <CAP4Z=hRv9KCnYBckf_pEWc69c4b2W+8p3qw1F_ZB-onWKD0oBA@mail.gmail.com>
	<CAEZw2bwnbWkDzpwywM3NC5D7cExmZUfsVyV9dd0x=5y288u_Tg@mail.gmail.com>
	<CAP4Z=hS8aKP1ihpmU1SU3f49--T2WgDyGAVZKQiWpACXA2=imA@mail.gmail.com>
	<CAP4Z=hRJ6mkqqyrjGTuYAnzt-XvE4Zt0TAQxdP7OQg9w9jGjaA@mail.gmail.com>
	<CAEZw2bwyMKig2kMHxYcx4xJFQDD7aUg0kuEG8h=EnD3pEApMxA@mail.gmail.com>
	<CAP4Z=hRMcbkc=ToHvHK0z0rT7Ld=sLvA3UGkvifVPQpoUAUH5w@mail.gmail.com>
	<CAP4Z=hTn+=BZJ24wgt+OdFhipWzbpd5PTPeUe62Fe0CT4eJ6+A@mail.gmail.com>
	<CAEZw2bwaJ5FUJBxfmSHNx9RnU-s+wRJ+Q6Fw3yfs7Do302czoA@mail.gmail.com>
Message-ID: <CAP4Z=hSurXueK9+Cc_PEQNdvMvDn0K74jc1Tkpp=rxiC25NiOg@mail.gmail.com>

Sure, I am sorry,  I dont quite understand what you mean by raising an
event and defining an event.

As i understand the incoming packets are picked by bro and then some sort
of stream(Delivered Stream) is formed (not sure here)and then go through
the process of anaylzer tree which then figures out based on the signature
that a particular packet is HTTP or not(lets say). Then an HTTP event is
generated and if the corresponding event handler is defined then the event
is put in events queue.
When the event reaches head of line in the queue then it is processed and
the event handler feeds corresponding data structures which will be used by
scripts as well, and the script is notified by the event handler of the
event having occurred.

so with respect to HTTP what function does HTTP.cc perform here, and what
does events.bif.cc perform?

It would be great if you can give some idea here.

I am trying to put some debug logs in each of the functions in events.bif.cc
and HTTP.cc (the debug logs are just opening a file and printing in that
file). But all i can see printed is bro_init. Any idea as to why?

Thanks and regards,


On Wed, Nov 23, 2016 at 5:32 PM, anthony kasza <anthony.kasza at gmail.com>
wrote:

> HTTP.cc raises events. Events.bif defines events. If you have more
> questions please include the mailing list.
>
> -AK
>
> On Nov 23, 2016 3:30 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu> wrote:
>
>> One more quick question Anthony, i did follow your blog but i couldnt
>> understand whats the difference between the code present in
>> bro/build/src/analyzer/protocol/http and just
>> bro/src/analyzer/protocol/http, one has HTTP.cc file the other has
>> events.bif.cc file Both seem to be generating events, but i dont
>> understand the context.
>> Like when will functions in events.bif.cc file be called and when will
>> the HTTP.cc functions be called.
>> Could you please explain briefly?
>>
>> On Wed, Nov 23, 2016 at 5:25 PM, Yagyesh Srivastava <ysrivas at ncsu.edu>
>> wrote:
>>
>>> If you run 'bro -Ci eth0' and browse a webserver over eth0, bro should
>>> spit out logs in your current working directory. If not, Bro is either not
>>> seeing packets or something else is wrong.
>>> *I am getting this correctly.*
>>>
>>> Is there a reason why broctl would not work in this case(when bro binary
>>> is)?
>>> Also just to make sure, broctl will always send logs in
>>> /nsm/bro/logs/current right?
>>>
>>> On Wed, Nov 23, 2016 at 5:22 PM, anthony kasza <anthony.kasza at gmail.com>
>>> wrote:
>>>
>>>> Try writing a trace file to disk with tcpdump and reading it with Bro
>>>> using the -r option.
>>>>
>>>> On Nov 23, 2016 3:11 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu> wrote:
>>>>
>>>>> also anthony,
>>>>> I did try tcpdump:
>>>>> yagyesh at yagyesh-virtual-machine:/nsm/bro/share/bro/base/protocols/http$
>>>>> sudo tcpdump -nS
>>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>>>> decode
>>>>> listening on eth0, link-type EN10MB (Ethernet), capture size 262144
>>>>> bytes
>>>>> 17:03:30.300262 IP 192.168.170.135.45554 > 31.13.69.228.443: Flags
>>>>> [P.], seq 1936267894:1936268148, ack 2692707210, win 64240, length 254
>>>>> 17:03:30.300484 IP 31.13.69.228.443 > 192.168.170.135.45554: Flags
>>>>> [.], ack 1936268148, win 64240, length 0
>>>>>
>>>>>
>>>>> it is showing me the movement of packets.
>>>>>
>>>>> When i am doing a curl to localhost, then i am changing the node.cfg
>>>>> file so as to reflect interface=lo and not eth0.
>>>>> Thats the only change that we need if we need to monitor loopback port
>>>>> with bro instead of eth0?
>>>>>
>>>>> Also someone suggested that the interface might not be in monitor
>>>>> mode, had this been the case would i have received the tcpdump like
>>>>> mentioned above?
>>>>>
>>>>> On Wed, Nov 23, 2016 at 5:01 PM, Yagyesh Srivastava <ysrivas at ncsu.edu>
>>>>> wrote:
>>>>>
>>>>>> By bro binary you mean " bro -i eth0" command?
>>>>>> I can see that when I give this command it's listening on eth0
>>>>>> interface. It initially gave me a warning saying due to NIC checksum it is
>>>>>> receiving bad checksum packets so it will discard it.
>>>>>> So I ran the above command with -C option.
>>>>>> Is this what you were referring to?
>>>>>> Could you please help me understand what's the difference between
>>>>>> this command and broctl?
>>>>>>
>>>>>> Thanks and regards
>>>>>>
>>>>>> On Nov 23, 2016 4:54 PM, "anthony kasza" <anthony.kasza at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Your VM may be using its loopback address for the connection to the
>>>>>>> local Apache server. If Bro is listening on eth0 (not the loopback
>>>>>>> interface) it won't see that traffic.
>>>>>>>
>>>>>>> As for the curl'ing of external sites, have you tried something
>>>>>>> basic like tcpdump just to make sure packets are moving? I'd also try
>>>>>>> running the Bro binary, without broctl, on an interface just to make sure
>>>>>>> Bro is compiled, happy, and seeing packets move.
>>>>>>>
>>>>>>> -AK
>>>>>>>
>>>>>>> On Nov 23, 2016 1:33 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I have downloaded bro and built it on a VM, using configure, make
>>>>>>>> and make install.
>>>>>>>> Then i ran broctl install and deploy.
>>>>>>>> when i run broctl using "sudo broctl start" and subsequently issue
>>>>>>>> "sudo broctl status", it shows bro running as standalone on localhost.
>>>>>>>>
>>>>>>>> my /nsm/bro/etc/nod.cfg file has
>>>>>>>> type = standalone
>>>>>>>> host = localhost
>>>>>>>> interface = eth0
>>>>>>>>
>>>>>>>> Now when i try to connect to internet using my vm browser
>>>>>>>> or i curl to localhost (which has apache server running and after
>>>>>>>> making node.cfg file to hear on interface loopback) in either of the cases
>>>>>>>> i cannot see any logs getting generated.
>>>>>>>>
>>>>>>>>
>>>>>>>> *can someone please help me with this issue?*I dont think bro is
>>>>>>>> sniffing on the correct interface , there is something trivial i am
>>>>>>>> guessing which is going wrong. Please provide any pointers if possible.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Yagyesh
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Bro mailing list
>>>>>>>> bro at bro-ids.org
>>>>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>>>>>>
>>>>>>>
>>>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161123/98371a1d/attachment-0001.html 

From anthony.kasza at gmail.com  Wed Nov 23 22:32:54 2016
From: anthony.kasza at gmail.com (anthony kasza)
Date: Wed, 23 Nov 2016 23:32:54 -0700
Subject: [Bro] Help with bro
In-Reply-To: <CAP4Z=hTEhB_DJeU+T06zipFUyG_4NKcwNX5_HMieGzOc5LkyXw@mail.gmail.com>
References: <CAP4Z=hSx5DbKH6yCyAcAGgED0p_TKbk8bmAj6-vQSHfYCSeb+g@mail.gmail.com>
	<CAP4Z=hSsuvu+VL0SU4es6zPi=gv6aYCWQHnxAmAT6Dy8PcPcCA@mail.gmail.com>
	<CAP4Z=hR0nFFqXJr7NGBGbHprLGvudp=BC9rHze-ci6ctmW7EbQ@mail.gmail.com>
	<CAP4Z=hQ6A=_yEJqaAbPL7ioO5SHaL_2HAJ_Zi=ZsuP_HENj3xQ@mail.gmail.com>
	<CAP4Z=hQpDPx3qFDnmToxkm_tFcaPVmW978JpEcxZ+GY0nv3O0g@mail.gmail.com>
	<CAP4Z=hRiYyePNfdHK62tGx2nOEdoUMomByb_Po0Y9PxuuLep6A@mail.gmail.com>
	<CAP4Z=hTEhB_DJeU+T06zipFUyG_4NKcwNX5_HMieGzOc5LkyXw@mail.gmail.com>
Message-ID: <CAEZw2byTj3VFh3AmOXDhdLpDCct6F_DKKK4XRmtvVFYXv4gKAw@mail.gmail.com>

Again, I'm adding the bro list for others edification.

HTTP.cc is part of "the core". It parses connection streams passed to it,
from the analyzer tree, and raises events defined in events.bif. Such
events include those around http headers, http requests, http responses,
etc. Then, the script "layer" is called based on those events. Such events
have handler code in scripts that do things such as logging. I hope this
helps.

-AK

On Nov 23, 2016 9:30 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu> wrote:

> Hi Anthony,
>
> I included this question with the bro mailing list also, by any chance if
> you know the answers please let me know. Would be great help. I am stuck on
> this since a couple of days.
> I dont quite understand what you mean by raising an event and defining an
> event.
>
> As i understand the incoming packets are picked by bro and then some sort
> of stream(Delivered Stream) is formed (not sure here if you could please
> elaborate this)and then go through the process of anaylzer tree which then
> figures out based on the signature that a particular packet is HTTP or
> not(lets say). Then an HTTP event is generated and if the corresponding
> event handler is defined then the event is put in events queue.
> When the event reaches head of line in the queue then it is processed and
> the event handler feeds corresponding data structures which will be used by
> scripts as well, and the script is notified by the event handler of the
> event having occurred.
>
> so with respect to HTTP what function does HTTP.cc perform here, and what
> does events.bif.cc perform?
> Present in bro/build/src/analyzer/protocol/http
>
> It would be great if you can give some idea here.
>
> I am trying to put some debug logs in each of the functions in
> events.bif.cc and HTTP.cc (the debug logs are just opening a file and
> printing in that file). But all i can see printed is bro_init. Any idea as
> to why?
>
> Thanks and regards,
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161123/3c3b8b41/attachment.html 

From anthony.kasza at gmail.com  Thu Nov 24 00:03:47 2016
From: anthony.kasza at gmail.com (anthony kasza)
Date: Thu, 24 Nov 2016 01:03:47 -0700
Subject: [Bro] Help with bro
In-Reply-To: <CAEZw2byTj3VFh3AmOXDhdLpDCct6F_DKKK4XRmtvVFYXv4gKAw@mail.gmail.com>
References: <CAP4Z=hSx5DbKH6yCyAcAGgED0p_TKbk8bmAj6-vQSHfYCSeb+g@mail.gmail.com>
	<CAP4Z=hSsuvu+VL0SU4es6zPi=gv6aYCWQHnxAmAT6Dy8PcPcCA@mail.gmail.com>
	<CAP4Z=hR0nFFqXJr7NGBGbHprLGvudp=BC9rHze-ci6ctmW7EbQ@mail.gmail.com>
	<CAP4Z=hQ6A=_yEJqaAbPL7ioO5SHaL_2HAJ_Zi=ZsuP_HENj3xQ@mail.gmail.com>
	<CAP4Z=hQpDPx3qFDnmToxkm_tFcaPVmW978JpEcxZ+GY0nv3O0g@mail.gmail.com>
	<CAP4Z=hRiYyePNfdHK62tGx2nOEdoUMomByb_Po0Y9PxuuLep6A@mail.gmail.com>
	<CAP4Z=hTEhB_DJeU+T06zipFUyG_4NKcwNX5_HMieGzOc5LkyXw@mail.gmail.com>
	<CAEZw2byTj3VFh3AmOXDhdLpDCct6F_DKKK4XRmtvVFYXv4gKAw@mail.gmail.com>
Message-ID: <CAEZw2bxDP7n9yyE35HLXJ6wXS53S+icx32c05xYW1K8_DV5SvQ@mail.gmail.com>

init_bro is raised as Bro executes and HTTP events are rased as Bro sees
HTTP connections. Usually the Bro binary needs to begin execution before it
begins parsing network traffic. Here are some ideas for you to test:

- You could be terminating Bro before it recognizes an HTTP connection
- You could have no traffic going to Bro (do you see a conn.log file? Are
we sure about interfaces at this point?)
- You may be generating HTTP traffic Bro doesn't recognize.

Collect a pcap and attach to this thread. Best case scenario, you found a
protocol parsing issue and the Bro devs can fix. Worst case scenario, your
SO distribution need tweaking and you may need to ask another list for
directions.

-AK

On Nov 23, 2016 11:32 PM, "anthony kasza" <anthony.kasza at gmail.com> wrote:

> Again, I'm adding the bro list for others edification.
>
> HTTP.cc is part of "the core". It parses connection streams passed to it,
> from the analyzer tree, and raises events defined in events.bif. Such
> events include those around http headers, http requests, http responses,
> etc. Then, the script "layer" is called based on those events. Such events
> have handler code in scripts that do things such as logging. I hope this
> helps.
>
> -AK
>
> On Nov 23, 2016 9:30 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu> wrote:
>
>> Hi Anthony,
>>
>> I included this question with the bro mailing list also, by any chance if
>> you know the answers please let me know. Would be great help. I am stuck on
>> this since a couple of days.
>> I dont quite understand what you mean by raising an event and defining an
>> event.
>>
>> As i understand the incoming packets are picked by bro and then some sort
>> of stream(Delivered Stream) is formed (not sure here if you could please
>> elaborate this)and then go through the process of anaylzer tree which then
>> figures out based on the signature that a particular packet is HTTP or
>> not(lets say). Then an HTTP event is generated and if the corresponding
>> event handler is defined then the event is put in events queue.
>> When the event reaches head of line in the queue then it is processed and
>> the event handler feeds corresponding data structures which will be used by
>> scripts as well, and the script is notified by the event handler of the
>> event having occurred.
>>
>> so with respect to HTTP what function does HTTP.cc perform here, and what
>> does events.bif.cc perform?
>> Present in bro/build/src/analyzer/protocol/http
>>
>> It would be great if you can give some idea here.
>>
>> I am trying to put some debug logs in each of the functions in
>> events.bif.cc and HTTP.cc (the debug logs are just opening a file and
>> printing in that file). But all i can see printed is bro_init. Any idea as
>> to why?
>>
>> Thanks and regards,
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161124/bdcd5c25/attachment.html 

From jedwards2728 at gmail.com  Thu Nov 24 00:40:51 2016
From: jedwards2728 at gmail.com (John Edwards)
Date: Thu, 24 Nov 2016 19:40:51 +1100
Subject: [Bro] Adding more bro workers
Message-ID: <CAAcg0eJ09bkFhS46Pa5n_nAP3XcNn18fXQW6smWCEEMHkr-=Mw@mail.gmail.com>

Hi all,


As i have a standalone bro worker at the moment i am wanting to branch out
and add more workers to the same link because of a high rate of packet loss.

the worker has the fibre card receiving the direct packet stream from our
TAP. When i upgrade to cluster mode i will be building two more systems,
one for the management/proxy/logger and then another worker.

As one worker will have the fibre card how does the worker number 2 or 3 or
4 in the future receive traffic to assist in processing the data feed? is
the load shared between workers that reside on the same subnet? as the feed
from the tap is just a passive fibre feed with no routing configured on it.

i hope each worker doesnt need to have a fibre card added into the same tap
point.

Cheers
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161124/b7682bad/attachment.html 

From ysrivas at ncsu.edu  Thu Nov 24 01:44:55 2016
From: ysrivas at ncsu.edu (Yagyesh Srivastava)
Date: Thu, 24 Nov 2016 04:44:55 -0500
Subject: [Bro] Help with bro
In-Reply-To: <CAEZw2bxDP7n9yyE35HLXJ6wXS53S+icx32c05xYW1K8_DV5SvQ@mail.gmail.com>
References: <CAP4Z=hSx5DbKH6yCyAcAGgED0p_TKbk8bmAj6-vQSHfYCSeb+g@mail.gmail.com>
	<CAP4Z=hSsuvu+VL0SU4es6zPi=gv6aYCWQHnxAmAT6Dy8PcPcCA@mail.gmail.com>
	<CAP4Z=hR0nFFqXJr7NGBGbHprLGvudp=BC9rHze-ci6ctmW7EbQ@mail.gmail.com>
	<CAP4Z=hQ6A=_yEJqaAbPL7ioO5SHaL_2HAJ_Zi=ZsuP_HENj3xQ@mail.gmail.com>
	<CAP4Z=hQpDPx3qFDnmToxkm_tFcaPVmW978JpEcxZ+GY0nv3O0g@mail.gmail.com>
	<CAP4Z=hRiYyePNfdHK62tGx2nOEdoUMomByb_Po0Y9PxuuLep6A@mail.gmail.com>
	<CAP4Z=hTEhB_DJeU+T06zipFUyG_4NKcwNX5_HMieGzOc5LkyXw@mail.gmail.com>
	<CAEZw2byTj3VFh3AmOXDhdLpDCct6F_DKKK4XRmtvVFYXv4gKAw@mail.gmail.com>
	<CAEZw2bxDP7n9yyE35HLXJ6wXS53S+icx32c05xYW1K8_DV5SvQ@mail.gmail.com>
Message-ID: <CAP4Z=hQNHBa=Z1vbG0OR-+8T+cMCb8Jhy5FtFrWrDPOZp=o4Aw@mail.gmail.com>

*so you mean to say that generate_http_request is a definition of
HTTP_request event?*

void BifEvent::generate_http_request(analyzer::Analyzer* analyzer,
Connection* c, StringVal* method, StringVal* original_URI, StringVal*
unescaped_URI, StringVal* version)
{
// Note that it is intentional that here we do not
// check if ::http_request is NULL, which should happen *before*
// BifEvent::generate_http_request is called to avoid unnecessary Val
// allocation.

val_list* vl = new val_list;

vl->append(c->BuildConnVal());
vl->append(method);
vl->append(original_URI);
vl->append(unescaped_URI);
vl->append(version);

mgr.QueueEvent(::http_request, vl, SOURCE_LOCAL, analyzer->GetID(),
timer_mgr, c);
}


*if thats the case, why does it have mgr.QueueEvent, that gets me confused.*
*And the HTTP_request event is raised by:*

void HTTP_Analyzer::HTTP_Request() { ProtocolConfirmation(); const char*
method = (const char*) request_method->AsString()->Bytes(); int method_len =
request_method->AsString()->Len(); if ( strncasecmp(method, "CONNECT",
method_len) == 0 ) connect_request = true; if ( http_request ) { val_list*
vl = new val_list; vl->append(BuildConnVal()); Ref(request_method); vl->
append(request_method); vl->append(TruncateURI(request_URI->AsStringVal()));
vl->append(TruncateURI(unescaped_URI->AsStringVal())); vl->append(new
StringVal(fmt("%.1f", request_version))); // DEBUG_MSG("%.6f
http_request\n", network_time); ConnectionEvent(http_request, vl); } }

I have attached the responses inline to what asked to test:

Here are some ideas for you to test:

- You could be terminating Bro before it recognizes an HTTP connection

When i use bro -Ci eth0 (i.e. the binary ), then i can see all the
http.log, weird.log,conn.log files getting dumped in my current working
directory. They have the correct information. So I dont think this is the
case. Still can't get why Broctl wouldnt give me the logs
- You could have no traffic going to Bro (do you see a conn.log file? Are
we sure about interfaces at this point?)

As mentioned traffic is going to bro, only then i am able to see those logs.
- You may be generating HTTP traffic Bro doesn't recognize.

Doubt This.


Having said the above, what I did as means of a small experiment was to
write a small code to open a file and print a line having function name in
each of the event definitions in events.bif.cc and each of the events in
HTTP.cc, just to get a trace of the function calls, but it just shows me
bro_init as the printed message.
*Again I am confused as to how its able to detect the HTTP packets without
passing through either of these functions?*

Thanks,
Yagyesh



On Thu, Nov 24, 2016 at 3:03 AM, anthony kasza <anthony.kasza at gmail.com>
wrote:

> init_bro is raised as Bro executes and HTTP events are rased as Bro sees
> HTTP connections. Usually the Bro binary needs to begin execution before it
> begins parsing network traffic. Here are some ideas for you to test:
>
> - You could be terminating Bro before it recognizes an HTTP connection
> - You could have no traffic going to Bro (do you see a conn.log file? Are
> we sure about interfaces at this point?)
> - You may be generating HTTP traffic Bro doesn't recognize.
>
> Collect a pcap and attach to this thread. Best case scenario, you found a
> protocol parsing issue and the Bro devs can fix. Worst case scenario, your
> SO distribution need tweaking and you may need to ask another list for
> directions.
>
> -AK
>
> On Nov 23, 2016 11:32 PM, "anthony kasza" <anthony.kasza at gmail.com> wrote:
>
>> Again, I'm adding the bro list for others edification.
>>
>> HTTP.cc is part of "the core". It parses connection streams passed to it,
>> from the analyzer tree, and raises events defined in events.bif. Such
>> events include those around http headers, http requests, http responses,
>> etc. Then, the script "layer" is called based on those events. Such events
>> have handler code in scripts that do things such as logging. I hope this
>> helps.
>>
>> -AK
>>
>> On Nov 23, 2016 9:30 PM, "Yagyesh Srivastava" <ysrivas at ncsu.edu> wrote:
>>
>>> Hi Anthony,
>>>
>>> I included this question with the bro mailing list also, by any chance
>>> if you know the answers please let me know. Would be great help. I am stuck
>>> on this since a couple of days.
>>> I dont quite understand what you mean by raising an event and defining
>>> an event.
>>>
>>> As i understand the incoming packets are picked by bro and then some
>>> sort of stream(Delivered Stream) is formed (not sure here if you could
>>> please elaborate this)and then go through the process of anaylzer tree
>>> which then figures out based on the signature that a particular packet is
>>> HTTP or not(lets say). Then an HTTP event is generated and if the
>>> corresponding event handler is defined then the event is put in events
>>> queue.
>>> When the event reaches head of line in the queue then it is processed
>>> and the event handler feeds corresponding data structures which will be
>>> used by scripts as well, and the script is notified by the event handler of
>>> the event having occurred.
>>>
>>> so with respect to HTTP what function does HTTP.cc perform here, and
>>> what does events.bif.cc perform?
>>> Present in bro/build/src/analyzer/protocol/http
>>>
>>> It would be great if you can give some idea here.
>>>
>>> I am trying to put some debug logs in each of the functions in
>>> events.bif.cc and HTTP.cc (the debug logs are just opening a file and
>>> printing in that file). But all i can see printed is bro_init. Any idea as
>>> to why?
>>>
>>> Thanks and regards,
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161124/ffb1e023/attachment-0001.html 

From zeolla at gmail.com  Thu Nov 24 02:52:30 2016
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Thu, 24 Nov 2016 10:52:30 +0000
Subject: [Bro] Adding more bro workers
In-Reply-To: <CAAcg0eJ09bkFhS46Pa5n_nAP3XcNn18fXQW6smWCEEMHkr-=Mw@mail.gmail.com>
References: <CAAcg0eJ09bkFhS46Pa5n_nAP3XcNn18fXQW6smWCEEMHkr-=Mw@mail.gmail.com>
Message-ID: <CAM1mTCqSh8qJxE8w4pyrZqrR_EkXcb3m2rdKnVRg3sY0BANnzw@mail.gmail.com>

In my scenario we use a switch (specifically an arista 7150 with a Z
licence) to do symmetric load balancing to a cluster of bro sensors.  That
requires that all of my sensors have the appropriate NICs (myricoms for me).

Check out what the architecture documentation refers to as the frontend.
https://www.bro.org/sphinx/cluster/index.html#frontend

Jon

On Thu, Nov 24, 2016, 03:49 John Edwards <jedwards2728 at gmail.com> wrote:

> Hi all,
>
>
> As i have a standalone bro worker at the moment i am wanting to branch out
> and add more workers to the same link because of a high rate of packet loss.
>
> the worker has the fibre card receiving the direct packet stream from our
> TAP. When i upgrade to cluster mode i will be building two more systems,
> one for the management/proxy/logger and then another worker.
>
> As one worker will have the fibre card how does the worker number 2 or 3
> or 4 in the future receive traffic to assist in processing the data feed?
> is the load shared between workers that reside on the same subnet? as the
> feed from the tap is just a passive fibre feed with no routing configured
> on it.
>
> i hope each worker doesnt need to have a fibre card added into the same
> tap point.
>
> Cheers
> John
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-- 

Jon

Sent from my mobile device
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161124/41219aa9/attachment.html 

From pachinko.tw at gmail.com  Thu Nov 24 04:52:24 2016
From: pachinko.tw at gmail.com (Po-Ching Lin)
Date: Thu, 24 Nov 2016 20:52:24 +0800
Subject: [Bro] Building Bro with PF_RING
In-Reply-To: <e304d1f3f89764e9332caa40ed464151@localhost>
References: <CAK6atxrChyduHpftTXp=R9K00AEsk_0s3gzPR6CWb_QAPhFuSw@mail.gmail.com>
	<e304d1f3f89764e9332caa40ed464151@localhost>
Message-ID: <b3673b33-622e-2fea-6553-16867cc782d8@gmail.com>

James, I followed the following steps to build, but have one more question to ask.

Since bro is built before the PF_RING plugin, how can bro find the plugin in the right path?

I tested bro with -N Bro::PF_RING, but failed.

$ /usr/local/bro/bin/bro -N Bro::PF_RING
error in /usr/local/bro/share/bro/base/init-bare.bro, line 1: plugin Bro::PF_RING is not available
fatal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1: Failed to activate requested dynamic plugin(s).

Po-Ching

James Lay On 2016/11/22 12:14AM wrote:
> On 2016-11-21 09:03, erik clark wrote:
>> Are you building bro on 2.5 or 241? If you are building on 25, its in
>> aux/plugins/pf_ring and you need to specify where the headers are for
>> pfring.
>>
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> My notes, building pf_ring into /opt:
>
> git clone https://github.com/ntop/PF_RING.git
> cd PF_RING/kernel
> make
> sudo make install
>
> cd ../userland/lib
> ./configure --prefix=/opt/pfring
> sudo make install
>
> cd ../libpcap
> ./configure --prefix=/opt/pfring
> sudo make install
>
> cd ../tcpdump
> ./configure --prefix=/opt/pfring
> sudo make install
>
> cd bro-2.5
> ./configure --with-pcap=/opt/pfring
> make
> sudo make install
>
> pf_ring plugin
> cd aux/plugins/pf_ring/
> ./configure --bro-dist=../../.. --with-pfring=/opt/pfring
> --install-root=/opt/bro/lib/bro/plugins
> make
> sudo make install
>
> Should get you up and going..if someone sees any errors please let me
> know.
>
> James
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From jazoff at illinois.edu  Thu Nov 24 05:18:24 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Thu, 24 Nov 2016 13:18:24 +0000
Subject: [Bro] Help with bro
In-Reply-To: <CAP4Z=hQNHBa=Z1vbG0OR-+8T+cMCb8Jhy5FtFrWrDPOZp=o4Aw@mail.gmail.com>
References: <CAP4Z=hSx5DbKH6yCyAcAGgED0p_TKbk8bmAj6-vQSHfYCSeb+g@mail.gmail.com>
	<CAP4Z=hSsuvu+VL0SU4es6zPi=gv6aYCWQHnxAmAT6Dy8PcPcCA@mail.gmail.com>
	<CAP4Z=hR0nFFqXJr7NGBGbHprLGvudp=BC9rHze-ci6ctmW7EbQ@mail.gmail.com>
	<CAP4Z=hQ6A=_yEJqaAbPL7ioO5SHaL_2HAJ_Zi=ZsuP_HENj3xQ@mail.gmail.com>
	<CAP4Z=hQpDPx3qFDnmToxkm_tFcaPVmW978JpEcxZ+GY0nv3O0g@mail.gmail.com>
	<CAP4Z=hRiYyePNfdHK62tGx2nOEdoUMomByb_Po0Y9PxuuLep6A@mail.gmail.com>
	<CAP4Z=hTEhB_DJeU+T06zipFUyG_4NKcwNX5_HMieGzOc5LkyXw@mail.gmail.com>
	<CAEZw2byTj3VFh3AmOXDhdLpDCct6F_DKKK4XRmtvVFYXv4gKAw@mail.gmail.com>
	<CAEZw2bxDP7n9yyE35HLXJ6wXS53S+icx32c05xYW1K8_DV5SvQ@mail.gmail.com>
	<CAP4Z=hQNHBa=Z1vbG0OR-+8T+cMCb8Jhy5FtFrWrDPOZp=o4Aw@mail.gmail.com>
Message-ID: <FB627F25-C8B5-4CEF-86B8-8F1747594B61@illinois.edu>



> On Nov 24, 2016, at 4:44 AM, Yagyesh Srivastava <ysrivas at ncsu.edu> wrote:
> 
> When i use bro -Ci eth0 (i.e. the binary ), then i can see all the http.log, weird.log,conn.log files getting dumped in my current working directory. They have the correct information. So I dont think this is the case. Still can't get why Broctl wouldnt give me the logs

Because you didn't tell broctl to use -C.

https://www.bro.org/documentation/faq.html#why-isn-t-bro-producing-the-logs-i-expect-a-note-about-checksums

-- 
- Justin Azoff

From jlay at slave-tothe-box.net  Thu Nov 24 06:21:11 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 24 Nov 2016 07:21:11 -0700
Subject: [Bro] Building Bro with PF_RING
In-Reply-To: <b3673b33-622e-2fea-6553-16867cc782d8@gmail.com>
References: <CAK6atxrChyduHpftTXp=R9K00AEsk_0s3gzPR6CWb_QAPhFuSw@mail.gmail.com>
	<e304d1f3f89764e9332caa40ed464151@localhost>
	<b3673b33-622e-2fea-6553-16867cc782d8@gmail.com>
Message-ID: <1479997271.2291.2.camel@slave-tothe-box.net>

Verify that the pfring plugin is installed in the right spot:
[07:17:27 :~$] locate PF_RING | grep usr
/usr/local/bro/lib/bro/plugins/Bro_PF_RING
/usr/local/bro/lib/bro/plugins/Bro_PF_RING/__bro_plugin__
/usr/local/bro/lib/bro/plugins/Bro_PF_RING/lib
/usr/local/bro/lib/bro/plugins/Bro_PF_RING/lib/Bro-PF_RING.linux-
x86_64.so
[07:17:36 :~$] /usr/local/bro/bin/bro -N Bro::PF_RING
Bro::PF_RING - Packet acquisition via PF_RING (dynamic, version 1.0)
The instructions below installed into /opt as I have on on other
machines....so you'll want to adjust that configure line.
James
On Thu, 2016-11-24 at 20:52 +0800, Po-Ching Lin wrote:
> James, I followed the following steps to build, but have one more
> question to ask.
> 
> Since bro is built before the PF_RING plugin, how can bro find the
> plugin in the right path?
> 
> I tested bro with -N Bro::PF_RING, but failed.
> 
> $ /usr/local/bro/bin/bro -N Bro::PF_RING
> error in /usr/local/bro/share/bro/base/init-bare.bro, line 1: plugin
> Bro::PF_RING is not available
> fatal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1:
> Failed to activate requested dynamic plugin(s).
> 
> Po-Ching
> 
> James Lay On 2016/11/22 12:14AM wrote:
> > 
> > On 2016-11-21 09:03, erik clark wrote:
> > > 
> > > Are you building bro on 2.5 or 241? If you are building on 25,
> > > its in
> > > aux/plugins/pf_ring and you need to specify where the headers are
> > > for
> > > pfring.
> > > 
> > > 
> > > _______________________________________________
> > > Bro mailing list
> > > bro at bro-ids.org
> > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> > My notes, building pf_ring into /opt:
> > 
> > git clone https://github.com/ntop/PF_RING.git
> > cd PF_RING/kernel
> > make
> > sudo make install
> > 
> > cd ../userland/lib
> > ./configure --prefix=/opt/pfring
> > sudo make install
> > 
> > cd ../libpcap
> > ./configure --prefix=/opt/pfring
> > sudo make install
> > 
> > cd ../tcpdump
> > ./configure --prefix=/opt/pfring
> > sudo make install
> > 
> > cd bro-2.5
> > ./configure --with-pcap=/opt/pfring
> > make
> > sudo make install
> > 
> > pf_ring plugin
> > cd aux/plugins/pf_ring/
> > ./configure --bro-dist=../../.. --with-pfring=/opt/pfring
> > --install-root=/opt/bro/lib/bro/plugins
> > make
> > sudo make install
> > 
> > Should get you up and going..if someone sees any errors please let
> > me
> > know.
> > 
> > James
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161124/50ed7d35/attachment.html 

From edautz at gmail.com  Thu Nov 24 09:03:48 2016
From: edautz at gmail.com (Eugene Dautzenberg)
Date: Thu, 24 Nov 2016 18:03:48 +0100
Subject: [Bro] [Bro type clash]
Message-ID: <EB5EF48B-6E1B-4B04-998D-7B7237CA9C53@gmail.com>

I want to check if 

n$id$orig_h 

contains a valid ip address. 

But when I use and if comparison something like. 

If ( n$id$orig_h = "-" )

I got a type clash (string and cmd) error. 

How to solve this?

Thx

Verstuurd vanaf mijn iPhone

From zeolla at gmail.com  Thu Nov 24 10:23:45 2016
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Thu, 24 Nov 2016 18:23:45 +0000
Subject: [Bro] [Bro type clash]
In-Reply-To: <EB5EF48B-6E1B-4B04-998D-7B7237CA9C53@gmail.com>
References: <EB5EF48B-6E1B-4B04-998D-7B7237CA9C53@gmail.com>
Message-ID: <CAM1mTCop9bkL0cTuSke5Byua_2AiroUZOvw_SSxGfexRHSe7wA@mail.gmail.com>

Have you tried n$id?$orig_h ?  Should return true if it is set.  Are you
concerned that it may contain something but it isn't a valid IP?

Also, I believe "-" is just a representation of an unset field for the log
output.
https://www.bro.org/sphinx/scripts/base/frameworks/logging/writers/ascii.bro.html#id-LogAscii::unset_field

On Thu, Nov 24, 2016, 13:06 Eugene Dautzenberg <edautz at gmail.com> wrote:

> I want to check if
>
> n$id$orig_h
>
> contains a valid ip address.
>
> But when I use and if comparison something like.
>
> If ( n$id$orig_h = "-" )
>
> I got a type clash (string and cmd) error.
>
> How to solve this?
>
> Thx
>
> Verstuurd vanaf mijn iPhone
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-- 

Jon

Sent from my mobile device
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161124/f834f62f/attachment-0001.html 

From daniel.guerra69 at gmail.com  Thu Nov 24 10:35:39 2016
From: daniel.guerra69 at gmail.com (Daniel Guerra)
Date: Thu, 24 Nov 2016 19:35:39 +0100
Subject: [Bro] [Bro type clash]
In-Reply-To: <CAM1mTCop9bkL0cTuSke5Byua_2AiroUZOvw_SSxGfexRHSe7wA@mail.gmail.com>
References: <EB5EF48B-6E1B-4B04-998D-7B7237CA9C53@gmail.com>
	<CAM1mTCop9bkL0cTuSke5Byua_2AiroUZOvw_SSxGfexRHSe7wA@mail.gmail.com>
Message-ID: <6837A3B4-F9A4-49A5-A09B-2D91BCC9C7C5@gmail.com>

Check if present first.
if (n$id?orig_h)
 if (n$id$orig_h =

Regards,

Daniel
> On 24 Nov 2016, at 19:23, Zeolla at GMail.com <zeolla at gmail.com> wrote:
> 
> Have you tried n$id?$orig_h ?  Should return true if it is set.  Are you concerned that it may contain something but it isn't a valid IP? 
> 
> Also, I believe "-" is just a representation of an unset field for the log output. https://www.bro.org/sphinx/scripts/base/frameworks/logging/writers/ascii.bro.html#id-LogAscii::unset_field <https://www.bro.org/sphinx/scripts/base/frameworks/logging/writers/ascii.bro.html#id-LogAscii::unset_field>
> On Thu, Nov 24, 2016, 13:06 Eugene Dautzenberg <edautz at gmail.com <mailto:edautz at gmail.com>> wrote:
> I want to check if
> 
> n$id$orig_h
> 
> contains a valid ip address.
> 
> But when I use and if comparison something like.
> 
> If ( n$id$orig_h = "-" )
> 
> I got a type clash (string and cmd) error.
> 
> How to solve this?
> 
> Thx
> 
> Verstuurd vanaf mijn iPhone
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org <mailto:bro at bro-ids.org>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro <http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
> -- 
> Jon
> 
> Sent from my mobile device
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161124/03e513bb/attachment.html 

From philosnef at gmail.com  Thu Nov 24 11:48:24 2016
From: philosnef at gmail.com (erik clark)
Date: Thu, 24 Nov 2016 14:48:24 -0500
Subject: [Bro] [Bro type clash]
Message-ID: <CAK6atxpXC0Mm12hD=R_EBmgR=w6w+jhSQAsAFYtx+t3z_fhi8A@mail.gmail.com>

---
I want to check if

n$id$orig_h

contains a valid ip address.
---

In the framework there is already something that does this....

https://www.bro.org/sphinx/scripts/base/utils/addrs.bro.html

Specifically:

is_valid_ip
<https://www.bro.org/sphinx/scripts/base/utils/addrs.bro.html#id-is_valid_ip>
: function
<https://www.bro.org/sphinx/script-reference/types.html#type-function>
<https://www.bro.org/sphinx/script-reference/types.html#type-function>

This is a VERY useful function, as it validates both ipv4 and ipv6....
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161124/3c87b7f7/attachment.html 

From philosnef at gmail.com  Thu Nov 24 12:33:35 2016
From: philosnef at gmail.com (erik clark)
Date: Thu, 24 Nov 2016 15:33:35 -0500
Subject: [Bro] [Bro type clash]
In-Reply-To: <CAK6atxpXC0Mm12hD=R_EBmgR=w6w+jhSQAsAFYtx+t3z_fhi8A@mail.gmail.com>
References: <CAK6atxpXC0Mm12hD=R_EBmgR=w6w+jhSQAsAFYtx+t3z_fhi8A@mail.gmail.com>
Message-ID: <CAK6atxov1ekNDg3cyk9ktNDt8j2s7p4AMKX8fDRxpdLOxGWg6A@mail.gmail.com>

Just for clarification, is_valid_ip actually does more than run a basic
regex against the string. For ipv4 it slices the address and then runs an
evaluation against each piece, which is faster than the regex as far as I
can tell from my testing. I don't recall offhand how it determines valid
ipv6 addresses offhand, as I don't have the addrs.bro script handy.

On Thu, Nov 24, 2016 at 2:48 PM, erik clark <philosnef at gmail.com> wrote:

> ---
> I want to check if
>
> n$id$orig_h
>
> contains a valid ip address.
> ---
>
> In the framework there is already something that does this....
>
> https://www.bro.org/sphinx/scripts/base/utils/addrs.bro.html
>
> Specifically:
>
> is_valid_ip
> <https://www.bro.org/sphinx/scripts/base/utils/addrs.bro.html#id-is_valid_ip>
> : function
> <https://www.bro.org/sphinx/script-reference/types.html#type-function>
> <https://www.bro.org/sphinx/script-reference/types.html#type-function>
>
> This is a VERY useful function, as it validates both ipv4 and ipv6....
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161124/becc8434/attachment.html 

From edautz at gmail.com  Thu Nov 24 13:39:26 2016
From: edautz at gmail.com (Eugene Dautzenberg)
Date: Thu, 24 Nov 2016 22:39:26 +0100
Subject: [Bro] [Bro type clash]
In-Reply-To: <6837A3B4-F9A4-49A5-A09B-2D91BCC9C7C5@gmail.com>
References: <EB5EF48B-6E1B-4B04-998D-7B7237CA9C53@gmail.com>
	<CAM1mTCop9bkL0cTuSke5Byua_2AiroUZOvw_SSxGfexRHSe7wA@mail.gmail.com>
	<6837A3B4-F9A4-49A5-A09B-2D91BCC9C7C5@gmail.com>
Message-ID: <01ef01d2469b$3b147f80$b13d7e80$@com>

Thnx,

 

Your reply solves my syntax error, but I want to use an external script to
push a message to my Phone when a notice occur.

 

When I have an Intel hit and an port scan I see the notice.log filled. 

 

 

#separator \x09

#set_separator ,

#empty_field   (empty)

#unset_field   -

#path   notice

#open   2016-11-24-22-26-05

#fields ts      uid     id.orig_h      id.orig_p      id.resp_h
id.resp_p      fuid        file_mime_type file_desc      proto   note    msg
sub     src     dst     p       n        peer_descr     actions suppress_for
dropped remote_location.country_code        remote_location.region
remote_location.city   remote_location.latitude
remote_location.longitude

#types  time    string  addr    port    addr    port    string  string
string  enum    enum        string  string  addr    addr    port    count
string  set[enum]      interval        bool    string  string  string
double  double

1480022763.168490      Cim1y02Kw1ySXSCjFb     192.168.2.2    43632
185.78.29.33   80      -        -       -       tcp     Intel::Notice  Intel
hit on 185.78.29.33 at HTTP::IN_HOST_HEADER   185.78.29.33   192.168.2.2
185.78.29.33   80      -       bro
Notice::ACTION_EMAIL,Notice::ACTION_LOG       0.000000       F       -
-       -        -       -

1480022784.174025      -       -       -       -       -       -       -
-       -        Scan::Port_Scan 192.168.2.2 scanned at least 15 unique
ports of host 192.168.2.254 in 0m1s    local   192.168.2.2    192.168.2.254
-       -       bro        Notice::ACTION_EMAIL,Notice::ACTION_LOG
0.000000       F       -       -       -        -       -

 

In my local.bro I do an hook in the notice function

 

hook Notice::policy(n: Notice::Info)

            {

            add n$actions[Notice::ACTION_EMAIL];

           if (n$id?$orig_h)

 

               {

               local cmd = fmt(cat("/home/pi/scripts/pushover_notify
Bro_alert src: ",n$id$orig_h," ",n$msg));

                }

 

           else

 

               {

            cmd = fmt(cat("/home/pi/scripts/pushover_notify Bro_alert
",n$msg));

               }

 

           system(cmd);

 

But the else part is not executed with the port scan despite the id.orig_h
in the notice.log contains an - so the then should be false on an portscan.

 

Can you help me out? 

 

 

 

Van: Daniel Guerra [mailto:daniel.guerra69 at gmail.com] 
Verzonden: donderdag 24 november 2016 19:36
Aan: Zeolla at GMail.com
CC: Eugene Dautzenberg; bro at bro.org
Onderwerp: Re: [Bro] [Bro type clash]

 

Check if present first.

if (n$id?orig_h)

 if (n$id$orig_h =

 

Regards,

 

Daniel

On 24 Nov 2016, at 19:23, Zeolla at GMail.com <zeolla at gmail.com> wrote:

 

Have you tried n$id?$orig_h ?  Should return true if it is set.  Are you
concerned that it may contain something but it isn't a valid IP?  

Also, I believe "-" is just a representation of an unset field for the log
output.
https://www.bro.org/sphinx/scripts/base/frameworks/logging/writers/ascii.bro
.html#id-LogAscii::unset_field

 

On Thu, Nov 24, 2016, 13:06 Eugene Dautzenberg <edautz at gmail.com> wrote:

I want to check if

n$id$orig_h

contains a valid ip address.

But when I use and if comparison something like.

If ( n$id$orig_h = "-" )

I got a type clash (string and cmd) error.

How to solve this?

Thx

Verstuurd vanaf mijn iPhone
_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-- 

Jon

Sent from my mobile device

_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

 



---
Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware.
https://www.avast.com/antivirus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161124/b28bb396/attachment-0001.html 

From zeolla at gmail.com  Thu Nov 24 14:02:58 2016
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Thu, 24 Nov 2016 22:02:58 +0000
Subject: [Bro] [Bro type clash]
In-Reply-To: <01ef01d2469b$3b147f80$b13d7e80$@com>
References: <EB5EF48B-6E1B-4B04-998D-7B7237CA9C53@gmail.com>
	<CAM1mTCop9bkL0cTuSke5Byua_2AiroUZOvw_SSxGfexRHSe7wA@mail.gmail.com>
	<6837A3B4-F9A4-49A5-A09B-2D91BCC9C7C5@gmail.com>
	<01ef01d2469b$3b147f80$b13d7e80$@com>
Message-ID: <CAM1mTCr2b=gD3pHyahzbs0u6d80B6iW=qzjWFoR3Kf499-w+Xw@mail.gmail.com>

Typically the way I would do something like that is I add a custom notice
(like NOTICE::DO_SOMETHING) for things to take that action, then make a
hook notice to look for NOTICE::DO_SOMETHING and take a specific action.

On Thu, Nov 24, 2016, 16:39 Eugene Dautzenberg <edautz at gmail.com> wrote:

> Thnx,
>
>
>
> Your reply solves my syntax error, but I want to use an external script to
> push a message to my Phone when a notice occur.
>
>
>
> When I have an Intel hit and an port scan I see the notice.log filled.
>
>
>
>
>
> #separator \x09
>
> #set_separator ,
>
> #empty_field   (empty)
>
> #unset_field   -
>
> #path   notice
>
> #open   2016-11-24-22-26-05
>
> #fields ts      uid     id.orig_h      id.orig_p      id.resp_h
> id.resp_p      fuid        file_mime_type file_desc      proto   note
> msg     sub     src     dst     p       n        peer_descr     actions
> suppress_for   dropped remote_location.country_code
> remote_location.region remote_location.city
> remote_location.latitude        remote_location.longitude
>
> #types  time    string  addr    port    addr    port    string  string
> string  enum    enum        string  string  addr    addr    port    count
> string  set[enum]      interval        bool    string  string  string
> double  double
>
> 1480022763.168490      Cim1y02Kw1ySXSCjFb     192.168.2.2    43632
> 185.78.29.33   80      -        -       -       tcp     Intel::Notice
> Intel hit on 185.78.29.33 at HTTP::IN_HOST_HEADER   185.78.29.33
> 192.168.2.2    185.78.29.33   80      -       bro
> Notice::ACTION_EMAIL,Notice::ACTION_LOG       0.000000       F
> -       -       -        -       -
>
> 1480022784.174025      -       -       -       -       -       -
> -       -       -        Scan::Port_Scan 192.168.2.2 scanned at least 15
> unique ports of host 192.168.2.254 in 0m1s    local   192.168.2.2
> 192.168.2.254  -       -       bro
> Notice::ACTION_EMAIL,Notice::ACTION_LOG       0.000000       F
> -       -       -        -       -
>
>
>
> In my local.bro I do an hook in the notice function
>
>
>
> hook Notice::policy(n: Notice::Info)
>
>             {
>
>             add n$actions[Notice::ACTION_EMAIL];
>
>            if (n$id?$orig_h)
>
>
>
>                {
>
>                local cmd = fmt(cat("/home/pi/scripts/pushover_notify
> Bro_alert src: ",n$id$orig_h," ",n$msg));
>
>                 }
>
>
>
>            else
>
>
>
>                {
>
>             cmd = fmt(cat("/home/pi/scripts/pushover_notify Bro_alert
> ",n$msg));
>
>                }
>
>
>
>            system(cmd);
>
>
>
> But the else part is not executed with the port scan despite the id.orig_h
> in the notice.log contains an ? so the then should be false on an portscan.
>
>
>
> Can you help me out?
>
>
>
>
>
>
>
> *Van:* Daniel Guerra [mailto:daniel.guerra69 at gmail.com]
> *Verzonden:* donderdag 24 november 2016 19:36
> *Aan:* Zeolla at GMail.com
> *CC:* Eugene Dautzenberg; bro at bro.org
> *Onderwerp:* Re: [Bro] [Bro type clash]
>
>
>
> Check if present first.
>
> if (n$id?orig_h)
>
>  if (n$id$orig_h =
>
>
>
> Regards,
>
>
>
> Daniel
>
> On 24 Nov 2016, at 19:23, Zeolla at GMail.com <Zeolla at gmail.com> <
> zeolla at gmail.com> wrote:
>
>
>
> Have you tried n$id?$orig_h ?  Should return true if it is set.  Are you
> concerned that it may contain something but it isn't a valid IP?
>
> Also, I believe "-" is just a representation of an unset field for the log
> output.
> https://www.bro.org/sphinx/scripts/base/frameworks/logging/writers/ascii.bro.html#id-LogAscii::unset_field
>
>
>
> On Thu, Nov 24, 2016, 13:06 Eugene Dautzenberg <edautz at gmail.com> wrote:
>
> I want to check if
>
> n$id$orig_h
>
> contains a valid ip address.
>
> But when I use and if comparison something like.
>
> If ( n$id$orig_h = "-" )
>
> I got a type clash (string and cmd) error.
>
> How to solve this?
>
> Thx
>
> Verstuurd vanaf mijn iPhone
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> <http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
>
> --
>
> Jon
>
> Sent from my mobile device
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
>
>
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> Virusvrij.
> www.avast.com
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
>
-- 

Jon

Sent from my mobile device
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161124/ee889212/attachment-0001.html 

From bro at pingtrip.com  Fri Nov 25 08:45:09 2016
From: bro at pingtrip.com (Dave Crawford)
Date: Fri, 25 Nov 2016 11:45:09 -0500
Subject: [Bro] Bro 2.5 CPU usage
Message-ID: <1CD497CC-2AED-43E0-95ED-1B898B7C6D49@pingtrip.com>

I finally had an opportunity to install a Bro 2.5 cluster in the lab for review and was surprised to see a higher CPU usage than 2.4 deployments.

A clean install with (w/ PF_RING)  never drops below 25% CPU per worker at idle, meaning I?ve disabled the SPAN traffic and Bro stays at 25%.

I then went as far as disabling every default script except for the following:

@load misc/loaded-scripts
@load tuning/defaults
@load misc/capture-loss
@load misc/profiling.bro
@load misc/stats

And the CPU remains at 25%. 

Has anyone experienced similar results with 2.5?

-Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161125/5a17a775/attachment.html 

From jazoff at illinois.edu  Fri Nov 25 08:59:39 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Fri, 25 Nov 2016 16:59:39 +0000
Subject: [Bro] Bro 2.5 CPU usage
In-Reply-To: <1CD497CC-2AED-43E0-95ED-1B898B7C6D49@pingtrip.com>
References: <1CD497CC-2AED-43E0-95ED-1B898B7C6D49@pingtrip.com>
Message-ID: <D60FB163-F001-403F-80AB-5F956ED575B3@illinois.edu>

Bro doesn't do a great job of using low amounts of CPU at low data rates - it's more tweaked for a constant packet rate.

I use the following patch at home (it applies on 2.4-2.5) which reduces cpu quite a bit when traffic rates are extremely low

--- a/bro-2.4/src/iosource/Manager.cc
+++ b/bro-2.4/src/iosource/Manager.cc
@@ -137,7 +137,7 @@
 		// decrease CPU load. I guess that's because it allows
 		// the kernel's packet buffers to fill. - Robin
 		timeout.tv_sec = 0;
-		timeout.tv_usec = 20; // SELECT_TIMEOUT;
+		timeout.tv_usec = 2000; // SELECT_TIMEOUT;
 		select(0, 0, 0, 0, &timeout);
 		}
 
--- a/bro-2.4/src/Net.cc
+++ b/bro-2.4/src/Net.cc
@@ -359,7 +359,7 @@
 			if ( ! communication_enabled )
 				usleep(100000);
 			else
-				usleep(1000);
+				usleep(10000);
 
 			// Flawfinder says about usleep:
 			//
--- a/bro-2.4/src/threading/MsgThread.cc
+++ b/bro-2.4/src/threading/MsgThread.cc
@@ -234,7 +234,7 @@
 			}
 
 		if ( ! Killed() )
-			usleep(1000);
+			usleep(10000);
 		}
 
 	signal_val = old_signal_val;


-- 
- Justin Azoff

> On Nov 25, 2016, at 11:45 AM, Dave Crawford <bro at pingtrip.com> wrote:
> 
> I finally had an opportunity to install a Bro 2.5 cluster in the lab for review and was surprised to see a higher CPU usage than 2.4 deployments.
> 
> A clean install with (w/ PF_RING)  never drops below 25% CPU per worker at idle, meaning I?ve disabled the SPAN traffic and Bro stays at 25%.
> 
> I then went as far as disabling every default script except for the following:
> 
> @load misc/loaded-scripts
> @load tuning/defaults
> @load misc/capture-loss
> @load misc/profiling.bro
> @load misc/stats
> 
> And the CPU remains at 25%. 
> 
> Has anyone experienced similar results with 2.5?
> 
> -Dave
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From bro at pingtrip.com  Fri Nov 25 09:02:02 2016
From: bro at pingtrip.com (Dave Crawford)
Date: Fri, 25 Nov 2016 12:02:02 -0500
Subject: [Bro] Bro 2.5 CPU usage
In-Reply-To: <D60FB163-F001-403F-80AB-5F956ED575B3@illinois.edu>
References: <1CD497CC-2AED-43E0-95ED-1B898B7C6D49@pingtrip.com>
	<D60FB163-F001-403F-80AB-5F956ED575B3@illinois.edu>
Message-ID: <81A1176D-CC1B-4F4F-A734-D00394BDD8E7@pingtrip.com>

Thanks Justin, The patch should be perfect for home/lab deployments. A tight ?select loop? looks like the exact culprit:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 63.26    0.053786           0    307412           select
 36.74    0.031237           0    153591           nanosleep
  0.00    0.000000           0       157       127 read
  0.00    0.000000           0       153           write
  0.00    0.000000           0         2           stat
  0.00    0.000000           0        16           kill
  0.00    0.000000           0         8           getrusage
  0.00    0.000000           0         1           restart_syscall
------ ----------- ----------- --------- --------- ----------------
100.00    0.085023                461340       127 total

-Dave

> On Nov 25, 2016, at 11:59 AM, Azoff, Justin S <jazoff at illinois.edu> wrote:
> 
> Bro doesn't do a great job of using low amounts of CPU at low data rates - it's more tweaked for a constant packet rate.
> 
> I use the following patch at home (it applies on 2.4-2.5) which reduces cpu quite a bit when traffic rates are extremely low
> 
> --- a/bro-2.4/src/iosource/Manager.cc
> +++ b/bro-2.4/src/iosource/Manager.cc
> @@ -137,7 +137,7 @@
> 		// decrease CPU load. I guess that's because it allows
> 		// the kernel's packet buffers to fill. - Robin
> 		timeout.tv_sec = 0;
> -		timeout.tv_usec = 20; // SELECT_TIMEOUT;
> +		timeout.tv_usec = 2000; // SELECT_TIMEOUT;
> 		select(0, 0, 0, 0, &timeout);
> 		}
> 
> --- a/bro-2.4/src/Net.cc
> +++ b/bro-2.4/src/Net.cc
> @@ -359,7 +359,7 @@
> 			if ( ! communication_enabled )
> 				usleep(100000);
> 			else
> -				usleep(1000);
> +				usleep(10000);
> 
> 			// Flawfinder says about usleep:
> 			//
> --- a/bro-2.4/src/threading/MsgThread.cc
> +++ b/bro-2.4/src/threading/MsgThread.cc
> @@ -234,7 +234,7 @@
> 			}
> 
> 		if ( ! Killed() )
> -			usleep(1000);
> +			usleep(10000);
> 		}
> 
> 	signal_val = old_signal_val;
> 
> 
> -- 
> - Justin Azoff
> 
>> On Nov 25, 2016, at 11:45 AM, Dave Crawford <bro at pingtrip.com> wrote:
>> 
>> I finally had an opportunity to install a Bro 2.5 cluster in the lab for review and was surprised to see a higher CPU usage than 2.4 deployments.
>> 
>> A clean install with (w/ PF_RING)  never drops below 25% CPU per worker at idle, meaning I?ve disabled the SPAN traffic and Bro stays at 25%.
>> 
>> I then went as far as disabling every default script except for the following:
>> 
>> @load misc/loaded-scripts
>> @load tuning/defaults
>> @load misc/capture-loss
>> @load misc/profiling.bro
>> @load misc/stats
>> 
>> And the CPU remains at 25%. 
>> 
>> Has anyone experienced similar results with 2.5?
>> 
>> -Dave
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161125/61b4504f/attachment.html 

From bro at pingtrip.com  Fri Nov 25 09:34:43 2016
From: bro at pingtrip.com (Dave Crawford)
Date: Fri, 25 Nov 2016 12:34:43 -0500
Subject: [Bro] Bro 2.5 CPU usage
In-Reply-To: <D60FB163-F001-403F-80AB-5F956ED575B3@illinois.edu>
References: <1CD497CC-2AED-43E0-95ED-1B898B7C6D49@pingtrip.com>
	<D60FB163-F001-403F-80AB-5F956ED575B3@illinois.edu>
Message-ID: <62CF81BE-CD85-4B71-80A3-132E725F2892@pingtrip.com>

Thanks again Justin, the CPU now sits down around 3% on extremely low traffic links.
 
> On Nov 25, 2016, at 11:59 AM, Azoff, Justin S <jazoff at illinois.edu> wrote:
> 
> Bro doesn't do a great job of using low amounts of CPU at low data rates - it's more tweaked for a constant packet rate.
> 
> I use the following patch at home (it applies on 2.4-2.5) which reduces cpu quite a bit when traffic rates are extremely low
> 
> --- a/bro-2.4/src/iosource/Manager.cc
> +++ b/bro-2.4/src/iosource/Manager.cc
> @@ -137,7 +137,7 @@
> 		// decrease CPU load. I guess that's because it allows
> 		// the kernel's packet buffers to fill. - Robin
> 		timeout.tv_sec = 0;
> -		timeout.tv_usec = 20; // SELECT_TIMEOUT;
> +		timeout.tv_usec = 2000; // SELECT_TIMEOUT;
> 		select(0, 0, 0, 0, &timeout);
> 		}
> 
> --- a/bro-2.4/src/Net.cc
> +++ b/bro-2.4/src/Net.cc
> @@ -359,7 +359,7 @@
> 			if ( ! communication_enabled )
> 				usleep(100000);
> 			else
> -				usleep(1000);
> +				usleep(10000);
> 
> 			// Flawfinder says about usleep:
> 			//
> --- a/bro-2.4/src/threading/MsgThread.cc
> +++ b/bro-2.4/src/threading/MsgThread.cc
> @@ -234,7 +234,7 @@
> 			}
> 
> 		if ( ! Killed() )
> -			usleep(1000);
> +			usleep(10000);
> 		}
> 
> 	signal_val = old_signal_val;
> 
> 
> -- 
> - Justin Azoff
> 
>> On Nov 25, 2016, at 11:45 AM, Dave Crawford <bro at pingtrip.com> wrote:
>> 
>> I finally had an opportunity to install a Bro 2.5 cluster in the lab for review and was surprised to see a higher CPU usage than 2.4 deployments.
>> 
>> A clean install with (w/ PF_RING)  never drops below 25% CPU per worker at idle, meaning I?ve disabled the SPAN traffic and Bro stays at 25%.
>> 
>> I then went as far as disabling every default script except for the following:
>> 
>> @load misc/loaded-scripts
>> @load tuning/defaults
>> @load misc/capture-loss
>> @load misc/profiling.bro
>> @load misc/stats
>> 
>> And the CPU remains at 25%. 
>> 
>> Has anyone experienced similar results with 2.5?
>> 
>> -Dave
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 



From shirkdog.bsd at gmail.com  Fri Nov 25 09:43:48 2016
From: shirkdog.bsd at gmail.com (Michael Shirk)
Date: Fri, 25 Nov 2016 12:43:48 -0500
Subject: [Bro] Bro 2.5 CPU usage
In-Reply-To: <81A1176D-CC1B-4F4F-A734-D00394BDD8E7@pingtrip.com>
References: <1CD497CC-2AED-43E0-95ED-1B898B7C6D49@pingtrip.com>
	<D60FB163-F001-403F-80AB-5F956ED575B3@illinois.edu>
	<81A1176D-CC1B-4F4F-A734-D00394BDD8E7@pingtrip.com>
Message-ID: <CAL8PkUWx-b0je1_nTbuP5M=Z_Di9rS0NVHM+M1vOCFmv_GBJ0Q@mail.gmail.com>

Is this something worthy of a feature request for low bandwidth setups?

In addition to something like this, I have to do a patch for very low
network traffic with bro cron reporting network traffic has stopped on the
monitoring interface.

--
Michael Shirk
Daemon Security, Inc.
http://www.daemon-security.com

On Nov 25, 2016 12:17 PM, "Dave Crawford" <bro at pingtrip.com> wrote:

> Thanks Justin, The patch should be perfect for home/lab deployments. A
> tight ?select loop? looks like the exact culprit:
>
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  63.26    0.053786           0    307412           select
>  36.74    0.031237           0    153591           nanosleep
>   0.00    0.000000           0       157       127 read
>   0.00    0.000000           0       153           write
>   0.00    0.000000           0         2           stat
>   0.00    0.000000           0        16           kill
>   0.00    0.000000           0         8           getrusage
>   0.00    0.000000           0         1           restart_syscall
> ------ ----------- ----------- --------- --------- ----------------
> 100.00    0.085023                461340       127 total
>
> -Dave
>
> On Nov 25, 2016, at 11:59 AM, Azoff, Justin S <jazoff at illinois.edu> wrote:
>
> Bro doesn't do a great job of using low amounts of CPU at low data rates -
> it's more tweaked for a constant packet rate.
>
> I use the following patch at home (it applies on 2.4-2.5) which reduces
> cpu quite a bit when traffic rates are extremely low
>
> --- a/bro-2.4/src/iosource/Manager.cc
> +++ b/bro-2.4/src/iosource/Manager.cc
> @@ -137,7 +137,7 @@
> // decrease CPU load. I guess that's because it allows
> // the kernel's packet buffers to fill. - Robin
> timeout.tv_sec = 0;
> - timeout.tv_usec = 20; // SELECT_TIMEOUT;
> + timeout.tv_usec = 2000; // SELECT_TIMEOUT;
> select(0, 0, 0, 0, &timeout);
> }
>
> --- a/bro-2.4/src/Net.cc
> +++ b/bro-2.4/src/Net.cc
> @@ -359,7 +359,7 @@
> if ( ! communication_enabled )
> usleep(100000);
> else
> - usleep(1000);
> + usleep(10000);
>
> // Flawfinder says about usleep:
> //
> --- a/bro-2.4/src/threading/MsgThread.cc
> +++ b/bro-2.4/src/threading/MsgThread.cc
> @@ -234,7 +234,7 @@
> }
>
> if ( ! Killed() )
> - usleep(1000);
> + usleep(10000);
> }
>
> signal_val = old_signal_val;
>
>
> --
> - Justin Azoff
>
> On Nov 25, 2016, at 11:45 AM, Dave Crawford <bro at pingtrip.com> wrote:
>
> I finally had an opportunity to install a Bro 2.5 cluster in the lab for
> review and was surprised to see a higher CPU usage than 2.4 deployments.
>
> A clean install with (w/ PF_RING)  never drops below 25% CPU per worker at
> idle, meaning I?ve disabled the SPAN traffic and Bro stays at 25%.
>
> I then went as far as disabling every default script except for the
> following:
>
> @load misc/loaded-scripts
> @load tuning/defaults
> @load misc/capture-loss
> @load misc/profiling.bro
> @load misc/stats
>
> And the CPU remains at 25%.
>
> Has anyone experienced similar results with 2.5?
>
> -Dave
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161125/e7403200/attachment.html 

From dnthayer at illinois.edu  Fri Nov 25 10:54:16 2016
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Fri, 25 Nov 2016 12:54:16 -0600
Subject: [Bro] Bro 2.5 CPU usage
In-Reply-To: <CAL8PkUWx-b0je1_nTbuP5M=Z_Di9rS0NVHM+M1vOCFmv_GBJ0Q@mail.gmail.com>
References: <1CD497CC-2AED-43E0-95ED-1B898B7C6D49@pingtrip.com>
	<D60FB163-F001-403F-80AB-5F956ED575B3@illinois.edu>
	<81A1176D-CC1B-4F4F-A734-D00394BDD8E7@pingtrip.com>
	<CAL8PkUWx-b0je1_nTbuP5M=Z_Di9rS0NVHM+M1vOCFmv_GBJ0Q@mail.gmail.com>
Message-ID: <0c66d80d-ed39-6789-9560-edcb89329ada@illinois.edu>

Regarding broctl, you can disable the "not seeing any packets"
warnings if you set this in your etc/broctl.cfg:
StatsLogEnable = 0

Doing so will also disable logging to broctl's stats.log (note:
this is NOT the stats.log that Bro itself logs), which I'm
guessing most people don't need anyway.


On 11/25/16 11:43 AM, Michael Shirk wrote:
> Is this something worthy of a feature request for low bandwidth setups?
>
> In addition to something like this, I have to do a patch for very low
> network traffic with bro cron reporting network traffic has stopped on
> the monitoring interface.
>
> --
> Michael Shirk
> Daemon Security, Inc.
> http://www.daemon-security.com

From jdopheid at illinois.edu  Mon Nov 28 07:59:24 2016
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Mon, 28 Nov 2016 15:59:24 +0000
Subject: [Bro] New additions to the Bro Leadership Team
Message-ID: <BD96B551-F707-4D76-ADB4-CA42A99117E3@illinois.edu>

Click here for original blog post<http://blog.bro.org/2016/11/new-additions-to-bro-leadership-team.html>.



Last year when we announced The Bro Project had joined Software Freedom Conservancy<https://sfconservancy.org/> we also announced the formation of Bro Leadership Team. The team consists of key contributors and community representatives working with SFC to set the direction of the project.

The Team has recently added two more members to the group: Johanna Amann and Martin van Hensbergen. As you may know, Johanna is a Bro developer and works on ICSI SSL Notary Service<https://notary.icsi.berkeley.edu/>. Martin is a Threat and Malware analyst and the creator of the Bro (RFB)VNC parser<https://www.youtube.com/watch?v=Wc4lKu-0YMA>.

Welcome Johanna and Martin, thank you for your contributions to the Bro Project!

The complete Leadership Team is now:

?         Johanna Amann<http://www.icir.org/johanna>, International Computer Science Institute

?         Seth Hall<http://www.icir.org/seth>, International Computer Science Institute

?         Keith Lehigh, Indiana University

?         Vern Paxson<http://www.icir.org/vern>, University of California at Berkeley

?         Michal Purzynski, Mozilla Foundation

?         Aashish Sharma, Lawrence Berkeley Lab

?         Adam Slagell<http://www.slagell.info/>, National Center for Supercomputing Applications

?         Robin Sommer<http://www.icir.org/robin>, International Computer Science Institute

?         Martin van Hensbergen, Fox-IT





------

Jeannette Dopheide

Training and Outreach Coordinator

National Center for Supercomputing Applications

University of Illinois at Urbana-Champaign
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161128/78e48368/attachment.html 

From slagell at illinois.edu  Mon Nov 28 08:56:51 2016
From: slagell at illinois.edu (Slagell, Adam J)
Date: Mon, 28 Nov 2016 16:56:51 +0000
Subject: [Bro] New additions to the Bro Leadership Team
In-Reply-To: <BD96B551-F707-4D76-ADB4-CA42A99117E3@illinois.edu>
References: <BD96B551-F707-4D76-ADB4-CA42A99117E3@illinois.edu>
Message-ID: <10AD6C9E-E4BD-430F-905D-4257D8DCB7F2@illinois.edu>

I would say "Martin is a Threat and Malware analyst at FOX IT"
On Nov 28, 2016, at 9:59 AM, Dopheide, Jeannette M <jdopheid at illinois.edu<mailto:jdopheid at illinois.edu>> wrote:

Click here for original blog post<https://urldefense.proofpoint.com/v2/url?u=http-3A__blog.bro.org_2016_11_new-2Dadditions-2Dto-2Dbro-2Dleadership-2Dteam.html&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=-Fz7LHAfm-F1ngmSqRASAa5AnS-wYWlwXm_wjwibnlg&e=>.

Last year when we announced The Bro Project had joined Software Freedom Conservancy<https://urldefense.proofpoint.com/v2/url?u=https-3A__sfconservancy.org_&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=T6s4Re7dLsTDBS6EU38Y12jQ5JrMIpbmtJ3xnj_H7xE&e=> we also announced the formation of Bro Leadership Team. The team consists of key contributors and community representatives working with SFC to set the direction of the project.

The Team has recently added two more members to the group: Johanna Amann and Martin van Hensbergen. As you may know, Johanna is a Bro developer and works on ICSI SSL Notary Service<https://urldefense.proofpoint.com/v2/url?u=https-3A__notary.icsi.berkeley.edu_&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=wQ4R0eaD2S2lgyNy3o9MBWP8ylaAbd75Uhre1fCv8fA&e=>. Martin is a Threat and Malware analyst and the creator of the Bro (RFB)VNC parser<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3DWc4lKu-2D0YMA&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=GG1CDSwPoIg-F0GYceuN3HaEKrTWyxiZ3JIh0FL1Qew&e=>.

Welcome Johanna and Martin, thank you for your contributions to the Bro Project!

The complete Leadership Team is now:
?         Johanna Amann<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.icir.org_johanna&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=a1f79qtvmYWzFn0KbJkiIW7NWu99SE0hau0uHY69pIg&e=>, International Computer Science Institute
?         Seth Hall<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.icir.org_seth&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=i0ppIMUZzQZbDkla4naQGSCTRyGR-reQ2HhdSxnbzpQ&e=>, International Computer Science Institute
?         Keith Lehigh, Indiana University
?         Vern Paxson<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.icir.org_vern&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=TRZhY2Kjh2QsNECiFmlGGJIEa8Mm4WbsK3ZtDtG54uw&e=>, University of California at Berkeley
?         Michal Purzynski, Mozilla Foundation
?         Aashish Sharma, Lawrence Berkeley Lab
?         Adam Slagell<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.slagell.info_&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=w_fpwV99RtvZS7IsTKryUdfr7PJHW4lWZDJEEYA_-rs&e=>, National Center for Supercomputing Applications
?         Robin Sommer<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.icir.org_robin&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=NP0tbk59ShE1hI8N2PBIubEta4jU14cdhlDuK0_x4pA&e=>, International Computer Science Institute
?         Martin van Hensbergen, Fox-IT


------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
_______________________________________________
Bro mailing list
bro at bro-ids.org<mailto:bro at bro-ids.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro<http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>

------

Adam J. Slagell
Chief Information Security Officer
Director, Cybersecurity Division
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info<http://www.slagell.info>

"Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure."








-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161128/940418e3/attachment-0001.html 

From slagell at illinois.edu  Mon Nov 28 10:00:44 2016
From: slagell at illinois.edu (Slagell, Adam J)
Date: Mon, 28 Nov 2016 18:00:44 +0000
Subject: [Bro] New additions to the Bro Leadership Team
In-Reply-To: <10AD6C9E-E4BD-430F-905D-4257D8DCB7F2@illinois.edu>
References: <BD96B551-F707-4D76-ADB4-CA42A99117E3@illinois.edu>
	<10AD6C9E-E4BD-430F-905D-4257D8DCB7F2@illinois.edu>
Message-ID: <2AB9642D-AB4A-4641-AA07-600F4FB43E7C@illinois.edu>

Sorry folks, didn?t realize I did a reply all. :-) Anyway, welcome Martin and Johanna!

:Adam

On Nov 28, 2016, at 10:56 AM, Slagell, Adam J <slagell at ILLINOIS.EDU<mailto:slagell at ILLINOIS.EDU>> wrote:

I would say "Martin is a Threat and Malware analyst at FOX IT"
On Nov 28, 2016, at 9:59 AM, Dopheide, Jeannette M <jdopheid at illinois.edu<mailto:jdopheid at illinois.edu>> wrote:

Click here for original blog post<https://urldefense.proofpoint.com/v2/url?u=http-3A__blog.bro.org_2016_11_new-2Dadditions-2Dto-2Dbro-2Dleadership-2Dteam.html&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=-Fz7LHAfm-F1ngmSqRASAa5AnS-wYWlwXm_wjwibnlg&e=>.

Last year when we announced The Bro Project had joined Software Freedom Conservancy<https://urldefense.proofpoint.com/v2/url?u=https-3A__sfconservancy.org_&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=T6s4Re7dLsTDBS6EU38Y12jQ5JrMIpbmtJ3xnj_H7xE&e=> we also announced the formation of Bro Leadership Team. The team consists of key contributors and community representatives working with SFC to set the direction of the project.

The Team has recently added two more members to the group: Johanna Amann and Martin van Hensbergen. As you may know, Johanna is a Bro developer and works on ICSI SSL Notary Service<https://urldefense.proofpoint.com/v2/url?u=https-3A__notary.icsi.berkeley.edu_&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=wQ4R0eaD2S2lgyNy3o9MBWP8ylaAbd75Uhre1fCv8fA&e=>. Martin is a Threat and Malware analyst and the creator of the Bro (RFB)VNC parser<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3DWc4lKu-2D0YMA&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=GG1CDSwPoIg-F0GYceuN3HaEKrTWyxiZ3JIh0FL1Qew&e=>.

Welcome Johanna and Martin, thank you for your contributions to the Bro Project!

The complete Leadership Team is now:
?         Johanna Amann<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.icir.org_johanna&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=a1f79qtvmYWzFn0KbJkiIW7NWu99SE0hau0uHY69pIg&e=>, International Computer Science Institute
?         Seth Hall<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.icir.org_seth&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=i0ppIMUZzQZbDkla4naQGSCTRyGR-reQ2HhdSxnbzpQ&e=>, International Computer Science Institute
?         Keith Lehigh, Indiana University
?         Vern Paxson<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.icir.org_vern&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=TRZhY2Kjh2QsNECiFmlGGJIEa8Mm4WbsK3ZtDtG54uw&e=>, University of California at Berkeley
?         Michal Purzynski, Mozilla Foundation
?         Aashish Sharma, Lawrence Berkeley Lab
?         Adam Slagell<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.slagell.info_&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=w_fpwV99RtvZS7IsTKryUdfr7PJHW4lWZDJEEYA_-rs&e=>, National Center for Supercomputing Applications
?         Robin Sommer<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.icir.org_robin&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=cdrA_FBv8I8UQhkh6A1710NLel-oT8qNDDfil9BiU_o&s=NP0tbk59ShE1hI8N2PBIubEta4jU14cdhlDuK0_x4pA&e=>, International Computer Science Institute
?         Martin van Hensbergen, Fox-IT


------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
_______________________________________________
Bro mailing list
bro at bro-ids.org<mailto:bro at bro-ids.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro<https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.icsi.berkeley.edu_mailman_listinfo_bro&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=QRL6LFbLFeJ4eb4Qac7Cw-PzNyUZtSnkfbTqiCVqvjU&s=f680ud3qGdazBKOSak5zxF4P3Ko6KSlky0ol_kRupM0&e=>

------

Adam J. Slagell
Chief Information Security Officer
Director, Cybersecurity Division
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.slagell.info&d=DQMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=gMEsgy9kNQo7aTfyIJsOSuw4Z57hfQyz6uV2H4S9PvE&m=QRL6LFbLFeJ4eb4Qac7Cw-PzNyUZtSnkfbTqiCVqvjU&s=o1oO-0ALkyatBwhoNO-jbk6BJEeNld02vPC_ImtUuEg&e=>

"Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure."








_______________________________________________
Bro mailing list
bro at bro-ids.org<mailto:bro at bro-ids.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

------

Adam J. Slagell
Chief Information Security Officer
Director, Cybersecurity Division
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info<http://www.slagell.info>

"Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure."








-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161128/dae35f17/attachment-0001.html 

From dwdixon at umich.edu  Mon Nov 28 10:19:47 2016
From: dwdixon at umich.edu (Drew Dixon)
Date: Mon, 28 Nov 2016 13:19:47 -0500
Subject: [Bro] Bro 2.5 CPU usage
In-Reply-To: <0c66d80d-ed39-6789-9560-edcb89329ada@illinois.edu>
References: <1CD497CC-2AED-43E0-95ED-1B898B7C6D49@pingtrip.com>
	<D60FB163-F001-403F-80AB-5F956ED575B3@illinois.edu>
	<81A1176D-CC1B-4F4F-A734-D00394BDD8E7@pingtrip.com>
	<CAL8PkUWx-b0je1_nTbuP5M=Z_Di9rS0NVHM+M1vOCFmv_GBJ0Q@mail.gmail.com>
	<0c66d80d-ed39-6789-9560-edcb89329ada@illinois.edu>
Message-ID: <CA+pNqTOFXBseTFHdQqKM2r=FyZEGUH9Gj4O7i=Sn9PoZLbU-3w@mail.gmail.com>

Would it be possible for someone quantify what a low bandwidth/low traffic
setup might be in terms of a bandwidth unit of measurement range where
Justin's patch would be advised to be used? I.E. Kbps/Mbps etc.  What would
be a cut-off bandwidth/traffic rate value where it would not be advisable
that this patch be used?

On Fri, Nov 25, 2016 at 1:54 PM, Daniel Thayer <dnthayer at illinois.edu>
wrote:

> Regarding broctl, you can disable the "not seeing any packets"
> warnings if you set this in your etc/broctl.cfg:
> StatsLogEnable = 0
>
> Doing so will also disable logging to broctl's stats.log (note:
> this is NOT the stats.log that Bro itself logs), which I'm
> guessing most people don't need anyway.
>
>
> On 11/25/16 11:43 AM, Michael Shirk wrote:
> > Is this something worthy of a feature request for low bandwidth setups?
> >
> > In addition to something like this, I have to do a patch for very low
> > network traffic with bro cron reporting network traffic has stopped on
> > the monitoring interface.
> >
> > --
> > Michael Shirk
> > Daemon Security, Inc.
> > http://www.daemon-security.com
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161128/03d37f8d/attachment.html 

From fatema.bannatwala at gmail.com  Mon Nov 28 11:14:37 2016
From: fatema.bannatwala at gmail.com (fatema bannatwala)
Date: Mon, 28 Nov 2016 14:14:37 -0500
Subject: [Bro] File extraction in different directories (maybe day vise)
Message-ID: <CACX0rUSYj_8JnSwS7ZKjef=HNsKDnX6z+oxFo-VXw7mO0xZ4WA@mail.gmail.com>

HI,

Just wanted to check-in, so that I don't re-invent the wheel, is there any
way, or if somebody has tried extracting the files in different
directories,i.e maybe in daily directory (just like bro logs the events in
the day vise directory)?
Right now we have over thousands of files extracted in a single directory
and it's getting harder to manage the one single directory to access the
extracted files, hence was looking into the Bro logging framework so that I
can steal some code from the event logging and rotation part for the file
extraction script.
Any other way around to it?

Appreciate the help.

Thanks,
Fatema.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161128/0453d91c/attachment.html 

From hosom at battelle.org  Mon Nov 28 11:46:20 2016
From: hosom at battelle.org (Hosom, Stephen M)
Date: Mon, 28 Nov 2016 19:46:20 +0000
Subject: [Bro] File extraction in different directories (maybe day vise)
In-Reply-To: <CACX0rUSYj_8JnSwS7ZKjef=HNsKDnX6z+oxFo-VXw7mO0xZ4WA@mail.gmail.com>
References: <CACX0rUSYj_8JnSwS7ZKjef=HNsKDnX6z+oxFo-VXw7mO0xZ4WA@mail.gmail.com>
Message-ID: <E5A2071C6FFFF640B714C224139C9B3327DEFC0F@WP-MBX2B.milky-way.battelle.org>

One of the arguments for attaching the file extraction analyzer is the filename that you want it to extract to. So long as you?re building this filename on the fly every time you attach the analyzer, you should be able to specify a different directory for every file?if you wished for such a thing.

Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);

Where I have specified ?fname?, just specify the string of the filename/path that you would like to store the file.

From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of fatema bannatwala
Sent: Monday, November 28, 2016 2:15 PM
To: bro at bro.org
Subject: [Bro] File extraction in different directories (maybe day vise)

HI,

Just wanted to check-in, so that I don't re-invent the wheel, is there any way, or if somebody has tried extracting the files in different directories,i.e maybe in daily directory (just like bro logs the events in the day vise directory)?
Right now we have over thousands of files extracted in a single directory and it's getting harder to manage the one single directory to access the extracted files, hence was looking into the Bro logging framework so that I can steal some code from the event logging and rotation part for the file extraction script.
Any other way around to it?

Appreciate the help.

Thanks,
Fatema.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161128/061b9e37/attachment.html 

From johanna at icir.org  Mon Nov 28 14:54:37 2016
From: johanna at icir.org (Johanna Amann)
Date: Mon, 28 Nov 2016 14:54:37 -0800
Subject: [Bro] Bro detection scripts
In-Reply-To: <CAMZgFn-VKY6TZbu-yOwvT3PHSBpxgVaDLGkkimG7eSXSnntTPQ@mail.gmail.com>
References: <CAMZgFn-VKY6TZbu-yOwvT3PHSBpxgVaDLGkkimG7eSXSnntTPQ@mail.gmail.com>
Message-ID: <20161128225437.mzo4g3nvi3zivclh@Beezling.local>

Hi,

since the ftp bruteforcing / ssh password guessing scripts are policy
scripts, they are not loaded by default.

If you invoke bro via command-line, just add
protocols/ssh/detect-bruteforcing.bro and
protocols/ftp/detect-bruteforcing.bro to your command line. If you use
broctl, the ssh bruteforce detector should be loaded by default; you have
to add the ftp one to local.bro.

If the notices still do not show up afterwards, you might need to tweak
the thresholds of the different scripts.

I hope this helps,
 Johanna


On Mon, Nov 21, 2016 at 12:28:16AM +0200, abdulrahman musallam wrote:
> Hi,
> when i perform an TCP port scanning on my machine Bro raises a notice
> immediately  to notice.log and this notice is raised by scan.bro script
> that detect scanning, such scripts exist for FTP brute forcing  and SSH
> password guessing but when i perform any of these attacks (FTP brute
> forcing  and SSH password guessing)  it won't show anything in notice log
> that indicates any occurrence of them!! could someone please help me with
> this problem! HOW TO INVOKE BRO DETECTION SCRIPTS??
> 
> Thanks.

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From johanna at icir.org  Mon Nov 28 14:55:46 2016
From: johanna at icir.org (Johanna Amann)
Date: Mon, 28 Nov 2016 14:55:46 -0800
Subject: [Bro] Does x509.log contain the raw certificate?
In-Reply-To: <CAEZw2bxhmeFsQav-NoMn7D80G+1ZYh309LiMWDRAN8+Y+XBotw@mail.gmail.com>
References: <CAHQ_EYQuzY66DdOxke3VcgCsWTeC+Sgq5fDO_pBvfX2i_e2sFg@mail.gmail.com>
	<CAEZw2bxhmeFsQav-NoMn7D80G+1ZYh309LiMWDRAN8+Y+XBotw@mail.gmail.com>
Message-ID: <20161128225546.4qvnwritdax4lkbr@Beezling.local>

Just to expand on this a bit - if you want the certificates dumped in pem
format, there also is a policy script for this that ships with Bro; you
can just load protocols/ssl/extract-certs-pem.bro.

Johanna

On Sat, Nov 19, 2016 at 04:06:21PM -0700, anthony kasza wrote:
> The certificates are not contained in any log file, just certificate meta
> data. To enable certificate extraction you need to enable the files
> framework which will write certificates to disk.
> 
> -AK
> 
> On Nov 19, 2016 2:53 PM, "Robert Harrelson" <bobharrelsons at gmail.com> wrote:
> 
> > The log file x509.log contains parsed information from the X.509
> > certificate. However, I would like to know if the x509.log file contains
> > the raw X.509 certificate itself. If yes, how do I extract the certificate
> > from the log, not in real-time? Thanks
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From johanna at icir.org  Mon Nov 28 15:02:54 2016
From: johanna at icir.org (Johanna Amann)
Date: Mon, 28 Nov 2016 15:02:54 -0800
Subject: [Bro] Is this type of script is possible to create ?
In-Reply-To: <CAMFH7YRGCWirkUg4KEGHkEraeEtW8LB7d8-5uQZ+=vkxWNLr9g@mail.gmail.com>
References: <CAMFH7YRGCWirkUg4KEGHkEraeEtW8LB7d8-5uQZ+=vkxWNLr9g@mail.gmail.com>
Message-ID: <20161128230254.xpexksf2y6i37zz4@Beezling.local>

Hi,

this is actually a bit difficult - there is a function that you can call
regularly to get you information about the number of packets/bytes that
Bro received (get_net_stats); if you call this every second or so, you can
determine traffic rates. However, it does not split things out by
incoming/outgoing connections.

Apart from that the only other idea I have is to use the packet-level
events and count things manually - however, this will have quite a
performance impact.

I might be missing an obvious solution I am not thinking about here
though.

Johanna

On Fri, Nov 18, 2016 at 10:35:09PM -0700, Manmeet Gill wrote:
> is it possible that below described statement can be crafted into a bro
> script ?
> Plz help me if it is possible, let me know what i need to do, to make this
> possible.
> 
> If my incoming traffic rate exceeds 44Mbps and the average incoming traffic
> rate over the last 504seconds exceeds the average incoming traffic rate
> over the last 965seconds by more than 70%, send an alert
> 
> Thank you Everyone.
> MeetGill

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From johanna at icir.org  Mon Nov 28 15:05:55 2016
From: johanna at icir.org (Johanna Amann)
Date: Mon, 28 Nov 2016 15:05:55 -0800
Subject: [Bro] BinPAC analyzer name
In-Reply-To: <58210681.4000508@googlemail.com>
References: <58210681.4000508@googlemail.com>
Message-ID: <20161128230555.bkkqrgeuedtebexa@Beezling.local>

By don't work - do you mean that it doesn't compile? Or does it not get
any traffic? Or does it not raise events?

Johanna

On Mon, Nov 07, 2016 at 11:56:01PM +0100, Dane Wullen wrote:
> Hi there,
> 
> I wrote a new analyzer with BinPAC for a protocol named 'AMS'.
> Somehow when I create the analyzer via the binpac python script and name 
> the analyzer 'AMS' or 'ams', the analyzer won't work. When I name it 
> 'TEST' or 'test', it works fine (same protocol specification, C++ Code, 
> etc.)
> 
> Is there a name convention for new analyzer? Or does anyone know, why 
> BinPAC/Bro won't accept the name 'ams'?
> 
> Thank you!
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 

From johanna at icir.org  Mon Nov 28 15:08:06 2016
From: johanna at icir.org (Johanna Amann)
Date: Mon, 28 Nov 2016 15:08:06 -0800
Subject: [Bro] Schedule an event
In-Reply-To: <CAKjK3aT_1s+KLeGbOLvq5hSdhE1A3oFPFaWzZo9Bib3S8_3q5A@mail.gmail.com>
References: <CAKjK3aT_1s+KLeGbOLvq5hSdhE1A3oFPFaWzZo9Bib3S8_3q5A@mail.gmail.com>
Message-ID: <20161128230806.tip24tqtk6j43332@Beezling.local>

Hello Troy,

yes, you can indeed use schedule for that. Just create an event that does
the searching, schedule it once and then re-schedule it inside the event.

So something along the lines of:

event search()
	{
	# do the searching

	schedule 30min { search() };
	}

event bro_init()
	{
	schedule 30min { search() };
	}

Johanna

On Tue, Nov 08, 2016 at 03:13:29PM -0500, Troy Ward wrote:
> I have a script based on the conn.log events.  As connections are created
> it populates some information in a table.  I need to trigger a search of
> that table to occur every 30 minutes.  I believe I can use the "schedule"
> command but not entirely sure.  So my question is, can I build a function
> within my script that does the table search and if so, how do I use the
> schedule command to trigger the function?
> 
> Thanks in advance,
> 
> Troy W

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From brot212 at googlemail.com  Mon Nov 28 15:11:00 2016
From: brot212 at googlemail.com (Dane Wullen)
Date: Tue, 29 Nov 2016 00:11:00 +0100
Subject: [Bro] BinPAC analyzer name
In-Reply-To: <20161128230555.bkkqrgeuedtebexa@Beezling.local>
References: <58210681.4000508@googlemail.com>
	<20161128230555.bkkqrgeuedtebexa@Beezling.local>
Message-ID: <1ab7114a-2307-62d6-ddda-d79374db8044@googlemail.com>

Hey,

thanks for your reply.

"Don't work" means that it doesn't raise any event nor executes the 
(C++) code in the analyser.pac file. It's like it can't read the traffic 
or something.

Like I said, when I name it Test or PROTO-AMS or something like that, it 
works fine.

Dane

Am 29.11.2016 um 00:05 schrieb Johanna Amann:
> By don't work - do you mean that it doesn't compile? Or does it not get
> any traffic? Or does it not raise events?
>
> Johanna
>
> On Mon, Nov 07, 2016 at 11:56:01PM +0100, Dane Wullen wrote:
>> Hi there,
>>
>> I wrote a new analyzer with BinPAC for a protocol named 'AMS'.
>> Somehow when I create the analyzer via the binpac python script and name
>> the analyzer 'AMS' or 'ams', the analyzer won't work. When I name it
>> 'TEST' or 'test', it works fine (same protocol specification, C++ Code,
>> etc.)
>>
>> Is there a name convention for new analyzer? Or does anyone know, why
>> BinPAC/Bro won't accept the name 'ams'?
>>
>> Thank you!
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>


From johanna at icir.org  Mon Nov 28 15:12:26 2016
From: johanna at icir.org (Johanna Amann)
Date: Mon, 28 Nov 2016 15:12:26 -0800
Subject: [Bro] logging locally and to remote logger
In-Reply-To: <CAK6atxqKKTfR8h-Scpu=Kaob2UEh=3wgOsrr_92VQVLAc1cRUA@mail.gmail.com>
References: <CAK6atxpSP4XUkiUjnXg6kc1mPpdBEr7OmHQ6i52UE05bypx-_w@mail.gmail.com>
	<CAKZA8xidBC1RByqr=jAeeum1kzHt8nMLo51zBC+rCcxHYmhGyA@mail.gmail.com>
	<CAK6atxqKKTfR8h-Scpu=Kaob2UEh=3wgOsrr_92VQVLAc1cRUA@mail.gmail.com>
Message-ID: <20161128231226.pb44dffvczcgokjy@Beezling.local>

Hi Erik,

the workers log to one directory each, to not conflict with each other. If
you have several active workers on one machine, they cannot local-log to
the same directory/file because they would conflict with each other and
you would get files where different workers might write into lines of
other workers.

As soon as you want merged logs from more than one Bro instance, you need
remote-logging (even if the manager/logger for the workers is on the same
machine).

I hope this helps :)
 Johanna

On Tue, Nov 15, 2016 at 07:15:55AM -0500, erik clark wrote:
> Ah, I think there is some confusion. Out of the box if you log locally as
> well as using a remote logger (2.5), the logs locally get shoved into
> worker buckets. I was hoping to see how it would be possible to get
> standard cluster behavior, where all workers log locally to one bucket
> instead of each worker having its own bucket.
> 
> Anyone know why this logs to separate buckets in the first place?
> 
> On Tue, Nov 15, 2016 at 6:31 AM, william de ping <bill.de.ping at gmail.com>
> wrote:
> 
> > Hi,
> >
> > If you wish to log locally and you care about the worker-id who produce
> > this logged event :
> >
> >    - to know what is the worker-id you can add a field "worker" to your
> >    logs and populate it from bro script using : get_event_peer()$descr
> >    - to change the rotation for each log (here, rotate every 200 minutes)
> >    you need to use
> >    - LOG::remove_default_filter(SSH::LOG);
> >       - and then add LOG::add_filter(SSH::LOG, [$name="ssh",$path="ssh",
> >       *$interv=200min*, $include=("field1","field2") ]
> >       - btw, you can set $path to be a mounted dir
> >    - to save the log to another machine simultaneously :
> >       - use bro, add a new writer (https://www.bro.org/sphinx/
> >       scripts/base/frameworks/logging/main.bro.html#id-Log::default_writer
> >       <https://www.bro.org/sphinx/scripts/base/frameworks/logging/main.bro.html#id-Log::default_writer>)
> >       and then add_filter to ssh and ask it to use the new writer
> >       - use syslog, just monitor this main local log and transmit it to
> >       another machine
> >
> > Hope it helps
> >
> > On Mon, Nov 14, 2016 at 4:35 PM, erik clark <philosnef at gmail.com> wrote:
> >
> >> So, if I use:
> >>
> >> redef Log::enable_local_logging
> >>
> >> in a bro worker cluster, what I find is that all the logs go to
> >> /data/bro/spool/worker-1-X instead of all in /data/bro/logs/current on the
> >> local machine... Is there a way to fix this?
> >>
> >> Also, I would want to rotate logs out on the workers that are doing
> >> additional local logging to have a much more constrained timeframe for
> >> logging, specifically 1 week for local nodes, and 3 months for the logger
> >> host.
> >>
> >> Is the best way to do this just with a cron rm -rf /data/bro/logs/$date ?
> >> It seems this would run into a conflict with broctlconfig....
> >>
> >> Thanks!
> >>
> >> _______________________________________________
> >> Bro mailing list
> >> bro at bro-ids.org
> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >>
> >
> >

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From johanna at icir.org  Mon Nov 28 15:15:25 2016
From: johanna at icir.org (Johanna Amann)
Date: Mon, 28 Nov 2016 15:15:25 -0800
Subject: [Bro] BinPAC analyzer name
In-Reply-To: <1ab7114a-2307-62d6-ddda-d79374db8044@googlemail.com>
References: <58210681.4000508@googlemail.com>
	<20161128230555.bkkqrgeuedtebexa@Beezling.local>
	<1ab7114a-2307-62d6-ddda-d79374db8044@googlemail.com>
Message-ID: <20161128231525.otgkhyf3pnccemcj@Beezling.local>

Interesting, I am not really aware of any reason why just using AMS should
not work.

Do you appen to have your code up on github (or somewhere else)? Then I
could take a look.

If you want to investigate for a bit yourself, build bro with
--enable-debug, start it with -B dpd and look at debug.log. There you
should see if data is sent to your analyzer - that might already give you
pointers if something is going wrong at/before/after this step.

Johanna

On Tue, Nov 29, 2016 at 12:11:00AM +0100, Dane Wullen wrote:
> Hey,
> 
> thanks for your reply.
> 
> "Don't work" means that it doesn't raise any event nor executes the (C++)
> code in the analyser.pac file. It's like it can't read the traffic or
> something.
> 
> Like I said, when I name it Test or PROTO-AMS or something like that, it
> works fine.
> 
> Dane
> 
> Am 29.11.2016 um 00:05 schrieb Johanna Amann:
> > By don't work - do you mean that it doesn't compile? Or does it not get
> > any traffic? Or does it not raise events?
> > 
> > Johanna
> > 
> > On Mon, Nov 07, 2016 at 11:56:01PM +0100, Dane Wullen wrote:
> > > Hi there,
> > > 
> > > I wrote a new analyzer with BinPAC for a protocol named 'AMS'.
> > > Somehow when I create the analyzer via the binpac python script and name
> > > the analyzer 'AMS' or 'ams', the analyzer won't work. When I name it
> > > 'TEST' or 'test', it works fine (same protocol specification, C++ Code,
> > > etc.)
> > > 
> > > Is there a name convention for new analyzer? Or does anyone know, why
> > > BinPAC/Bro won't accept the name 'ams'?
> > > 
> > > Thank you!
> > > 
> > > _______________________________________________
> > > Bro mailing list
> > > bro at bro-ids.org
> > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> > > 
> 

From jedwards2728 at gmail.com  Mon Nov 28 15:27:38 2016
From: jedwards2728 at gmail.com (John Edwards)
Date: Tue, 29 Nov 2016 10:27:38 +1100
Subject: [Bro] Bro Digest, Vol 127, Issue 46
In-Reply-To: <mailman.1.1480363201.3153.bro@bro.org>
References: <mailman.1.1480363201.3153.bro@bro.org>
Message-ID: <CAAcg0eJoM5Zt5tVYzyhNDVV7uiT=q9iYMPkAJw+_9MQ_WehvUA@mail.gmail.com>

Where is this line defined? what file would i define this once i create sub
folders for file types?  I wish to get cron to compress specific folders to
save disk space.

Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);

Cheers,
John

On Tue, Nov 29, 2016 at 7:00 AM, <bro-request at bro.org> wrote:

> Send Bro mailing list submissions to
>         bro at bro.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> or, via email, send a message with subject or body 'help' to
>         bro-request at bro.org
>
> You can reach the person managing the list at
>         bro-owner at bro.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Bro digest..."
>
>
> Today's Topics:
>
>    1. Re: Bro 2.5 CPU usage (Drew Dixon)
>    2. File extraction in different directories (maybe day vise)
>       (fatema bannatwala)
>    3. Re: File extraction in different directories (maybe day vise)
>       (Hosom, Stephen M)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 28 Nov 2016 13:19:47 -0500
> From: Drew Dixon <dwdixon at umich.edu>
> Subject: Re: [Bro] Bro 2.5 CPU usage
> To: Daniel Thayer <dnthayer at illinois.edu>
> Cc: bro at bro.org
> Message-ID:
>         <CA+pNqTOFXBseTFHdQqKM2r=FyZEGUH9Gj4O7i=Sn9PoZLbU-3w@
> mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Would it be possible for someone quantify what a low bandwidth/low traffic
> setup might be in terms of a bandwidth unit of measurement range where
> Justin's patch would be advised to be used? I.E. Kbps/Mbps etc.  What would
> be a cut-off bandwidth/traffic rate value where it would not be advisable
> that this patch be used?
>
> On Fri, Nov 25, 2016 at 1:54 PM, Daniel Thayer <dnthayer at illinois.edu>
> wrote:
>
> > Regarding broctl, you can disable the "not seeing any packets"
> > warnings if you set this in your etc/broctl.cfg:
> > StatsLogEnable = 0
> >
> > Doing so will also disable logging to broctl's stats.log (note:
> > this is NOT the stats.log that Bro itself logs), which I'm
> > guessing most people don't need anyway.
> >
> >
> > On 11/25/16 11:43 AM, Michael Shirk wrote:
> > > Is this something worthy of a feature request for low bandwidth setups?
> > >
> > > In addition to something like this, I have to do a patch for very low
> > > network traffic with bro cron reporting network traffic has stopped on
> > > the monitoring interface.
> > >
> > > --
> > > Michael Shirk
> > > Daemon Security, Inc.
> > > http://www.daemon-security.com
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161128/03d37f8d/attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Mon, 28 Nov 2016 14:14:37 -0500
> From: fatema bannatwala <fatema.bannatwala at gmail.com>
> Subject: [Bro] File extraction in different directories (maybe day
>         vise)
> To: bro at bro.org
> Message-ID:
>         <CACX0rUSYj_8JnSwS7ZKjef=HNsKDnX6z+oxFo-VXw7mO0xZ4WA at mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> HI,
>
> Just wanted to check-in, so that I don't re-invent the wheel, is there any
> way, or if somebody has tried extracting the files in different
> directories,i.e maybe in daily directory (just like bro logs the events in
> the day vise directory)?
> Right now we have over thousands of files extracted in a single directory
> and it's getting harder to manage the one single directory to access the
> extracted files, hence was looking into the Bro logging framework so that I
> can steal some code from the event logging and rotation part for the file
> extraction script.
> Any other way around to it?
>
> Appreciate the help.
>
> Thanks,
> Fatema.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161128/0453d91c/attachment-0001.html
>
> ------------------------------
>
> Message: 3
> Date: Mon, 28 Nov 2016 19:46:20 +0000
> From: "Hosom, Stephen M" <hosom at battelle.org>
> Subject: Re: [Bro] File extraction in different directories (maybe day
>         vise)
> To: fatema bannatwala <fatema.bannatwala at gmail.com>, "bro at bro.org"
>         <bro at bro.org>
> Message-ID:
>         <E5A2071C6FFFF640B714C224139C9B3327DEFC0F at WP-MBX2B.milky-
> way.battelle.org>
>
> Content-Type: text/plain; charset="utf-8"
>
> One of the arguments for attaching the file extraction analyzer is the
> filename that you want it to extract to. So long as you?re building this
> filename on the fly every time you attach the analyzer, you should be able
> to specify a different directory for every file?if you wished for such a
> thing.
>
> Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
>
> Where I have specified ?fname?, just specify the string of the
> filename/path that you would like to store the file.
>
> From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of
> fatema bannatwala
> Sent: Monday, November 28, 2016 2:15 PM
> To: bro at bro.org
> Subject: [Bro] File extraction in different directories (maybe day vise)
>
> HI,
>
> Just wanted to check-in, so that I don't re-invent the wheel, is there any
> way, or if somebody has tried extracting the files in different
> directories,i.e maybe in daily directory (just like bro logs the events in
> the day vise directory)?
> Right now we have over thousands of files extracted in a single directory
> and it's getting harder to manage the one single directory to access the
> extracted files, hence was looking into the Bro logging framework so that I
> can steal some code from the event logging and rotation part for the file
> extraction script.
> Any other way around to it?
>
> Appreciate the help.
>
> Thanks,
> Fatema.
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161128/061b9e37/attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> Bro mailing list
> Bro at bro.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> End of Bro Digest, Vol 127, Issue 46
> ************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161129/8521b854/attachment.html 

From brot212 at googlemail.com  Mon Nov 28 15:45:12 2016
From: brot212 at googlemail.com (Dane Wullen)
Date: Tue, 29 Nov 2016 00:45:12 +0100
Subject: [Bro] BinPAC analyzer name
In-Reply-To: <20161128231525.otgkhyf3pnccemcj@Beezling.local>
References: <58210681.4000508@googlemail.com>
	<20161128230555.bkkqrgeuedtebexa@Beezling.local>
	<1ab7114a-2307-62d6-ddda-d79374db8044@googlemail.com>
	<20161128231525.otgkhyf3pnccemcj@Beezling.local>
Message-ID: <124db383-ead5-a669-d7a4-54ff95ea9619@googlemail.com>

Well, I tested this behavior with the "standard code" generated by the 
binpac python script. I named one analyzer Test and the other AMS.

Both have the same PDU record type (except for the name of course)

type NAME_PDU(is_orig: bool) = record {
     data: bytestring &restofdata;
} &byteorder=bigendian;

and the same analyzer.pac (except for the name again) with the same 
"proc" function:

function proc_NAME_message(msg: NAME_PDU) : bool
...
     BifEvent::generate_NAME_event(...);
     std::cout <<  "Name PDU" << endl; # for debugging
...

Both analyzers are enabled (checked it with -B dpd and -NN)

When I run it with some .pcap file, I only get the "Test PDU" output. 
Tested it with several .pcap files, everytime the same result.

Dane

Am 29.11.2016 um 00:15 schrieb Johanna Amann:
> Interesting, I am not really aware of any reason why just using AMS should
> not work.
>
> Do you appen to have your code up on github (or somewhere else)? Then I
> could take a look.
>
> If you want to investigate for a bit yourself, build bro with
> --enable-debug, start it with -B dpd and look at debug.log. There you
> should see if data is sent to your analyzer - that might already give you
> pointers if something is going wrong at/before/after this step.
>
> Johanna
>
> On Tue, Nov 29, 2016 at 12:11:00AM +0100, Dane Wullen wrote:
>> Hey,
>>
>> thanks for your reply.
>>
>> "Don't work" means that it doesn't raise any event nor executes the (C++)
>> code in the analyser.pac file. It's like it can't read the traffic or
>> something.
>>
>> Like I said, when I name it Test or PROTO-AMS or something like that, it
>> works fine.
>>
>> Dane
>>
>> Am 29.11.2016 um 00:05 schrieb Johanna Amann:
>>> By don't work - do you mean that it doesn't compile? Or does it not get
>>> any traffic? Or does it not raise events?
>>>
>>> Johanna
>>>
>>> On Mon, Nov 07, 2016 at 11:56:01PM +0100, Dane Wullen wrote:
>>>> Hi there,
>>>>
>>>> I wrote a new analyzer with BinPAC for a protocol named 'AMS'.
>>>> Somehow when I create the analyzer via the binpac python script and name
>>>> the analyzer 'AMS' or 'ams', the analyzer won't work. When I name it
>>>> 'TEST' or 'test', it works fine (same protocol specification, C++ Code,
>>>> etc.)
>>>>
>>>> Is there a name convention for new analyzer? Or does anyone know, why
>>>> BinPAC/Bro won't accept the name 'ams'?
>>>>
>>>> Thank you!
>>>>
>>>> _______________________________________________
>>>> Bro mailing list
>>>> bro at bro-ids.org
>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161129/138244ee/attachment.html 

From johanna at icir.org  Mon Nov 28 16:28:02 2016
From: johanna at icir.org (Johanna Amann)
Date: Mon, 28 Nov 2016 16:28:02 -0800
Subject: [Bro] Bro and OpenSSL 1.1
In-Reply-To: <87twbaxnmh.fsf@msgid.hilluzination.de>
References: <87twbaxnmh.fsf@msgid.hilluzination.de>
Message-ID: <20161129002802.pwxjrlr5dbb6r7pi@Beezling.local>

Hello Hilko,

> as Debian is transitioning to using OpenSSL 1.1 in the upcoming release
> (9.x "stretch"), we are forced to deal with widespread API breakage
> because many data structures that had previously been considered part of
> the API have been made opaque. Many of these changes are fairly easy to
> implement by using getter/setter functions instead. (The main time-sink
> for me was locating those functions in the OpenSSL sources.)

Thanks a lot for doing this; I was aware that we will have to do that at
some point of time, but I have not really looked into this myself.

Just to make sure - the OpenSSL 1.1 API is incompatible to the older API?
(If the answer is yes - as I assume - this will mean quite a few
ifdefs...)

> For the bro package, some work-in-progress patches can be found in our
> bug tracking system[1].

Thanks. Can we just use the patches as a starting point when we add
support to Bro itself?

> One missing piece (apart from running tests with real packet trace data)
> is that some OCSP details cannot yet be accessed through OpenSSL 1.1's
> current set of API functions. Specifically, the function
> 
>     X509* x509_get_ocsp_signer(STACK_OF(X509) *certs, OCSP_RESPID *rid)

Heh. Yes, that was actually one of the things that I already worried about
when writing that specific code - it required messing around way too deep
the internal data structures of OpenSSL; it makes validating an OCSP reply
at an arbitrary time really hard.

> from src/file_analysis/analyzer/x509/functions.bif cannot currently be
> ported. There's ongoing work to fix that[2] in upstream OpenSSL, but we
> don't know yet whether this change will be ready in time for the freeze
> leading to the next Debian release. So, we are thinking that we may have
> to disable the x509_ocsp_verify function and anything that uses it.
> 
> Does anyone have any advice on what to look for when disabling that
> functionality? Or is there maybe a less intrusive alternative that we
> haven't discovered yet?

After thinkina about this for a bit - I think the best way in that case is
to disable this functionality; when looking at the code just now, I found
a small bug in it and I think solving it will actually increase the
reliance on x509_get_ocsp_signer.

In theory, one could also choose to just patch out everything in the
current code that uses signer (it is not that much), and rely completely
on the validation logic inside of OpenSSL -- however, that will fail on
traces, especially once they are older (you can't pass a validation time
to the ocsp validation function).

So - yes - disabling this seems like the better way to me at the moment.

I think all you need to do to disable this is remove the function and also
remove policy/protocols/ssl/validate-ocsp.bro. Alternatively, you could
make the function always return X509_V_ERR_APPLICATION_VERIFICATION with
an error string describing that the functionality is disabled in your
build. This might actually be the better solution since it won't break
custom scripts that rely on this functionality.

Thanks again for looking into this :)
 Johanna

From bengen--bro at hilluzination.de  Tue Nov 29 00:30:41 2016
From: bengen--bro at hilluzination.de (Hilko Bengen)
Date: Tue, 29 Nov 2016 09:30:41 +0100
Subject: [Bro] Bro and OpenSSL 1.1
In-Reply-To: <20161129002802.pwxjrlr5dbb6r7pi@Beezling.local> (Johanna Amann's
	message of "Mon, 28 Nov 2016 16:28:02 -0800")
References: <87twbaxnmh.fsf@msgid.hilluzination.de>
	<20161129002802.pwxjrlr5dbb6r7pi@Beezling.local>
Message-ID: <87wpfmpu1a.fsf@msgid.hilluzination.de>

* Johanna Amann:

> Hello Hilko,
>
>> as Debian is transitioning to using OpenSSL 1.1 in the upcoming release
>> (9.x "stretch"), we are forced to deal with widespread API breakage
>> because many data structures that had previously been considered part of
>> the API have been made opaque. Many of these changes are fairly easy to
>> implement by using getter/setter functions instead. (The main time-sink
>> for me was locating those functions in the OpenSSL sources.)
>
> Thanks a lot for doing this; I was aware that we will have to do that at
> some point of time, but I have not really looked into this myself.

I initially though that Debian would ship its next release without
OpenSSL 1.0, but this is not the case, so I have disabled the patches
for the package for the time being.

> Just to make sure - the OpenSSL 1.1 API is incompatible to the older API?
> (If the answer is yes - as I assume - this will mean quite a few
> ifdefs...)

Yes. A bunch of structs have been made opaque and can now only be
accessed through getter/setter calls. Which in general is a Good Thing.

>> For the bro package, some work-in-progress patches can be found in our
>> bug tracking system[1].
>
> Thanks. Can we just use the patches as a starting point when we add
> support to Bro itself?

Of course. Should I open a PR on Github?

Cheers,
-Hilko

From lagoon7 at gmail.com  Tue Nov 29 06:36:05 2016
From: lagoon7 at gmail.com (Ludwig Goon)
Date: Tue, 29 Nov 2016 09:36:05 -0500
Subject: [Bro] BRO PKG scripts not in version 2.5
Message-ID: <CAOHA7OkCPTpAzu-X4EDjzb+wW5DraGh7sciveNs5Lb1KzFzdEg@mail.gmail.com>

Are there any plans to include the pkg scripts with BRO IDS version 2.5? I
have modified the version 2.4 pkg scripts since they were not in the 2.5
version source code.

The 2.4 version for debian based systems does not produce a .deb file or
set of files.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161129/7c27d372/attachment.html 

From fatema.bannatwala at gmail.com  Tue Nov 29 07:18:22 2016
From: fatema.bannatwala at gmail.com (fatema bannatwala)
Date: Tue, 29 Nov 2016 10:18:22 -0500
Subject: [Bro] File extraction in different directories (maybe day vise)
In-Reply-To: <E5A2071C6FFFF640B714C224139C9B3327DEFC0F@WP-MBX2B.milky-way.battelle.org>
References: <CACX0rUSYj_8JnSwS7ZKjef=HNsKDnX6z+oxFo-VXw7mO0xZ4WA@mail.gmail.com>
	<E5A2071C6FFFF640B714C224139C9B3327DEFC0F@WP-MBX2B.milky-way.battelle.org>
Message-ID: <CACX0rUSRkORGesFFizCEwz+9JZ0oPPgpios8jUTD_SWJLud_AA@mail.gmail.com>

Thanks Stephen for the solution, finally got it working.

Fatema.

On Mon, Nov 28, 2016 at 2:46 PM, Hosom, Stephen M <hosom at battelle.org>
wrote:

> One of the arguments for attaching the file extraction analyzer is the
> filename that you want it to extract to. So long as you?re building this
> filename on the fly every time you attach the analyzer, you should be able
> to specify a different directory for every file?if you wished for such a
> thing.
>
>
>
> Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=
> fname]);
>
> Where I have specified ?fname?, just specify the string of the
> filename/path that you would like to store the file.
>
>
>
> *From:* bro-bounces at bro.org [mailto:bro-bounces at bro.org] *On Behalf Of *fatema
> bannatwala
> *Sent:* Monday, November 28, 2016 2:15 PM
> *To:* bro at bro.org
> *Subject:* [Bro] File extraction in different directories (maybe day vise)
>
>
>
> HI,
>
>
>
> Just wanted to check-in, so that I don't re-invent the wheel, is there any
> way, or if somebody has tried extracting the files in different
> directories,i.e maybe in daily directory (just like bro logs the events in
> the day vise directory)?
>
> Right now we have over thousands of files extracted in a single directory
> and it's getting harder to manage the one single directory to access the
> extracted files, hence was looking into the Bro logging framework so that I
> can steal some code from the event logging and rotation part for the file
> extraction script.
>
> Any other way around to it?
>
>
>
> Appreciate the help.
>
>
>
> Thanks,
>
> Fatema.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161129/d47af5dc/attachment.html 

From jdopheid at illinois.edu  Tue Nov 29 08:44:00 2016
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Tue, 29 Nov 2016 16:44:00 +0000
Subject: [Bro] Donate to The Bro Project
Message-ID: <A4CAD214-7AD5-4693-B584-A5FBEA5DEFCF@illinois.edu>

Bro Community,

As 2016 comes to a close, please consider adding The Bro Project to your list of charitable donations. We are managed by Software Freedom Conservancy, which is a 501(c)(3) organization and therefore exempt from US taxes. To donate via credit card, click on the "Donate" button in the side panel of our website, or go here for more payment options:

https://www.bro.org/donate/index.html

If your commercial organization is considering sponsoring the Bro Project, you can find more information about it here:

https://www.bro.org/donate/sponsorship.html

Thank you. And have a happy new year.


The Bro Project

------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
 



From johanna at icir.org  Tue Nov 29 11:04:32 2016
From: johanna at icir.org (Johanna Amann)
Date: Tue, 29 Nov 2016 11:04:32 -0800
Subject: [Bro] BRO PKG scripts not in version 2.5
In-Reply-To: <CAOHA7OkCPTpAzu-X4EDjzb+wW5DraGh7sciveNs5Lb1KzFzdEg@mail.gmail.com>
References: <CAOHA7OkCPTpAzu-X4EDjzb+wW5DraGh7sciveNs5Lb1KzFzdEg@mail.gmail.com>
Message-ID: <20161129190432.gukxgzyxighsx3vz@wifi242.sys.ICSI.Berkeley.EDU>

Hi,

On Tue, Nov 29, 2016 at 09:36:05AM -0500, Ludwig Goon wrote:
> Are there any plans to include the pkg scripts with BRO IDS version 2.5? I
> have modified the version 2.4 pkg scripts since they were not in the 2.5
> version source code.

No, there are no such plans. The scripts were removed because we do not
use them anymore to create the binary packages ourselves, they are
untested and I think, they pretty much did not work correctly anymore at
least for some systems.

The build files that are used to create our binary packages are available
at https://build.opensuse.org/package/show/network:bro/bro. If you need
the old behavior, you can try to just use the 2.4 files (as you apparently
already did) - I don't think that any of us know how good/bad they work.

Johanna

From johanna at icir.org  Tue Nov 29 11:07:00 2016
From: johanna at icir.org (Johanna Amann)
Date: Tue, 29 Nov 2016 11:07:00 -0800
Subject: [Bro] Bro and OpenSSL 1.1
In-Reply-To: <87wpfmpu1a.fsf@msgid.hilluzination.de>
References: <87twbaxnmh.fsf@msgid.hilluzination.de>
	<20161129002802.pwxjrlr5dbb6r7pi@Beezling.local>
	<87wpfmpu1a.fsf@msgid.hilluzination.de>
Message-ID: <20161129190700.n7emtb7f73hzgmi5@wifi242.sys.ICSI.Berkeley.EDU>

On Tue, Nov 29, 2016 at 09:30:41AM +0100, Hilko Bengen wrote:
> I initially though that Debian would ship its next release without
> OpenSSL 1.0, but this is not the case, so I have disabled the patches
> for the package for the time being.

Great, that makes things easier in that regard. By the time of the next
debian release, we should be ready too :)

> >> For the bro package, some work-in-progress patches can be found in our
> >> bug tracking system[1].
> >
> > Thanks. Can we just use the patches as a starting point when we add
> > support to Bro itself?
> 
> Of course. Should I open a PR on Github?

Either that, or (what I actually would slightly prefer) - if you could
just create a ticket at tracker.bro.org, and attach the patch to it, that
would be great.

Thanks again,
 Johanna

From hosom at battelle.org  Tue Nov 29 12:12:16 2016
From: hosom at battelle.org (Hosom, Stephen M)
Date: Tue, 29 Nov 2016 20:12:16 +0000
Subject: [Bro] Is this type of script is possible to create ?
In-Reply-To: <20161128230254.xpexksf2y6i37zz4@Beezling.local>
References: <CAMFH7YRGCWirkUg4KEGHkEraeEtW8LB7d8-5uQZ+=vkxWNLr9g@mail.gmail.com>
	<20161128230254.xpexksf2y6i37zz4@Beezling.local>
Message-ID: <E5A2071C6FFFF640B714C224139C9B3327DEFEFA@WP-MBX2B.milky-way.battelle.org>

You could do this with sumstats... you just have to do a bunch of math... and be happy with an average over a longer period of time. Since you only have to observe two counts, it actually wouldn't be that bad. Just observe the sum of the ip bytes based on which direction the traffic is in.... I could probably write an example script sometime tonight. 

-----Original Message-----
From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Johanna Amann
Sent: Monday, November 28, 2016 6:03 PM
To: Manmeet Gill <mgill6 at student.concordia.ab.ca>
Cc: bro at bro.org
Subject: Re: [Bro] Is this type of script is possible to create ?

Hi,

this is actually a bit difficult - there is a function that you can call regularly to get you information about the number of packets/bytes that Bro received (get_net_stats); if you call this every second or so, you can determine traffic rates. However, it does not split things out by incoming/outgoing connections.

Apart from that the only other idea I have is to use the packet-level events and count things manually - however, this will have quite a performance impact.

I might be missing an obvious solution I am not thinking about here though.

Johanna

On Fri, Nov 18, 2016 at 10:35:09PM -0700, Manmeet Gill wrote:
> is it possible that below described statement can be crafted into a 
> bro script ?
> Plz help me if it is possible, let me know what i need to do, to make 
> this possible.
> 
> If my incoming traffic rate exceeds 44Mbps and the average incoming 
> traffic rate over the last 504seconds exceeds the average incoming 
> traffic rate over the last 965seconds by more than 70%, send an alert
> 
> Thank you Everyone.
> MeetGill

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From hosom at battelle.org  Tue Nov 29 12:24:28 2016
From: hosom at battelle.org (Hosom, Stephen M)
Date: Tue, 29 Nov 2016 20:24:28 +0000
Subject: [Bro] Is this type of script is possible to create ?
In-Reply-To: <E5A2071C6FFFF640B714C224139C9B3327DEFEFA@WP-MBX2B.milky-way.battelle.org>
References: <CAMFH7YRGCWirkUg4KEGHkEraeEtW8LB7d8-5uQZ+=vkxWNLr9g@mail.gmail.com>
	<20161128230254.xpexksf2y6i37zz4@Beezling.local>
	<E5A2071C6FFFF640B714C224139C9B3327DEFEFA@WP-MBX2B.milky-way.battelle.org>
Message-ID: <E5A2071C6FFFF640B714C224139C9B3327DEFF15@WP-MBX2B.milky-way.battelle.org>

Alternatively--and I have no idea what the performance impact of this would be... you could use connection polling: https://www.bro.org/sphinx/scripts/base/protocols/conn/polling.bro.html

-----Original Message-----
From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Hosom, Stephen M
Sent: Tuesday, November 29, 2016 3:12 PM
To: Johanna Amann <johanna at icir.org>; Manmeet Gill <mgill6 at student.concordia.ab.ca>
Cc: bro at bro.org
Subject: Re: [Bro] Is this type of script is possible to create ?

You could do this with sumstats... you just have to do a bunch of math... and be happy with an average over a longer period of time. Since you only have to observe two counts, it actually wouldn't be that bad. Just observe the sum of the ip bytes based on which direction the traffic is in.... I could probably write an example script sometime tonight. 

-----Original Message-----
From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Johanna Amann
Sent: Monday, November 28, 2016 6:03 PM
To: Manmeet Gill <mgill6 at student.concordia.ab.ca>
Cc: bro at bro.org
Subject: Re: [Bro] Is this type of script is possible to create ?

Hi,

this is actually a bit difficult - there is a function that you can call regularly to get you information about the number of packets/bytes that Bro received (get_net_stats); if you call this every second or so, you can determine traffic rates. However, it does not split things out by incoming/outgoing connections.

Apart from that the only other idea I have is to use the packet-level events and count things manually - however, this will have quite a performance impact.

I might be missing an obvious solution I am not thinking about here though.

Johanna

On Fri, Nov 18, 2016 at 10:35:09PM -0700, Manmeet Gill wrote:
> is it possible that below described statement can be crafted into a 
> bro script ?
> Plz help me if it is possible, let me know what i need to do, to make 
> this possible.
> 
> If my incoming traffic rate exceeds 44Mbps and the average incoming 
> traffic rate over the last 504seconds exceeds the average incoming 
> traffic rate over the last 965seconds by more than 70%, send an alert
> 
> Thank you Everyone.
> MeetGill

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From dave.a.florek at gmail.com  Tue Nov 29 14:33:34 2016
From: dave.a.florek at gmail.com (Dave Florek)
Date: Tue, 29 Nov 2016 17:33:34 -0500
Subject: [Bro] Confimation on Bro 2.5 MHR feature
Message-ID: <CA+M9YC=sdbHpTvFuUeHVRacsNvwx-Jo-_TEDN5mSsPrYye3m=g@mail.gmail.com>

Good afternoon,

To confirm, with the Team Cymru MHR feature enabled, the warnings and
VirusTotal URL outputs would appear in the Bro files.log?

Thanks,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161129/dbaa3be0/attachment.html 

From johanna at icir.org  Tue Nov 29 15:35:05 2016
From: johanna at icir.org (Johanna Amann)
Date: Tue, 29 Nov 2016 15:35:05 -0800
Subject: [Bro] Two questions
In-Reply-To: <9B3941F2-5AD7-4AFE-9282-C942F6E46813@gmail.com>
References: <9B3941F2-5AD7-4AFE-9282-C942F6E46813@gmail.com>
Message-ID: <20161129233501.673tuic77cmp5gwd@wifi242.sys.ICSI.Berkeley.EDU>

Hi,

> 1: How can the intel also get mailed, when an intel event occurs? 
> I tried 
> 
> redef Notice::emailed_types += {
>          HTTP::IN_HOST_HEADER,
> };

HTTP::IN_HOST_HEADER actually is not a notice type; it is a location of
the Intel framework. Try using Intel::Notice instead, that should work.

> 2: I want to incorporate a Bash curl script send alerts to other systems when a notice or an intel event occurs. How to accomplish this?

You probably want to use the exec framework -
https://www.bro.org/sphinx/scripts/base/utils/exec.bro.html.

I hope this helps,
 Johanna

From johanna at icir.org  Tue Nov 29 15:38:36 2016
From: johanna at icir.org (Johanna Amann)
Date: Tue, 29 Nov 2016 15:38:36 -0800
Subject: [Bro] [bro] conn-summary
In-Reply-To: <CANrcqc07QECWnogjZjn1a-=W77484xjCgKpzKn1Qv6Cn8OAXyw@mail.gmail.com>
References: <CANrcqc07QECWnogjZjn1a-=W77484xjCgKpzKn1Qv6Cn8OAXyw@mail.gmail.com>
Message-ID: <20161129233836.qcgum3bx2gilqpyy@wifi242.sys.ICSI.Berkeley.EDU>

Hello Tim,

since trace-summary (the tool that generates the connection summaries)
only supports the standard Bro log file syntax, there currently is no way
to get a usable output when only logging in json.

You could log in json and in the standard format simultaneously, as one
solution. Adding json support to trace-summary also should not be that
hard - but I don't think that that is currently on anyones plate.

Johanna

On Thu, Nov 17, 2016 at 08:29:54AM -0500, Tim Desrochers wrote:
> Is there a way, when logging in JSON, to get a readable connection summary
> log.  When logging in JSON the log is unusable and the tables included in
> the log do not get populated.  I like the log because it gives a great
> overview of the sensors.
> 
> Thanks
> Tim

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From johanna at icir.org  Tue Nov 29 15:41:37 2016
From: johanna at icir.org (Johanna Amann)
Date: Tue, 29 Nov 2016 15:41:37 -0800
Subject: [Bro] Confimation on Bro 2.5 MHR feature
In-Reply-To: <CA+M9YC=sdbHpTvFuUeHVRacsNvwx-Jo-_TEDN5mSsPrYye3m=g@mail.gmail.com>
References: <CA+M9YC=sdbHpTvFuUeHVRacsNvwx-Jo-_TEDN5mSsPrYye3m=g@mail.gmail.com>
Message-ID: <20161129234137.56u5yld3c6s7gzdw@wifi242.sys.ICSI.Berkeley.EDU>

Hi Dave,

> To confirm, with the Team Cymru MHR feature enabled, the warnings and
> VirusTotal URL outputs would appear in the Bro files.log?

No, that information gets added to notice.log, not to files.log.

Johanna

From eshelton at butler.net  Wed Nov 30 08:42:16 2016
From: eshelton at butler.net (eshelton)
Date: Wed, 30 Nov 2016 09:42:16 -0700
Subject: [Bro] Weird log rotation issues in 2.5
Message-ID: <CADOaRAkVXiKqefdYWs_4JUrTwa4DJfYq7UanE_JPAw95R+Sk9A@mail.gmail.com>

My site was an early tester of 2.5_beta, and during most of our testing, we
did not experience much in the way of issues related to log rotation. As
soon as we started using 2.5_beta we were utilizing logger mode, as we run
an extremely busy cluster, and had been experiencing memory utilization
exhaustion issues with the manager being single threaded At one point near
the end of the 2.5 beta, we started seeing some irregular rotation
intervals, which we noted were potentially related to us starting to use
the Intel framework.

I have since moved to 2.5 stable, and I'm still seeing irregular rotation
issues which appear to be related to how busy the manager is at the time of
log rotation. I do notice that the main local-manager process seems to stay
loaded at or near 100% CPU utilization when we are seeing a lot of traffic.
As an example, I rotated logs perfectly from midnight up to 07:00-08:00.
After that, my current logs in spool/logger are running well over into 9
o'clocks data set. I frequently see .rotated files well after a log
rotation interval has run (the following was observed 09:41 today):

-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.capture_loss
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.communication
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.conn
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.conn-summary
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.dce_rpc
-rw-r--r--  1 root root          18 Nov 30 03:00 .rotated.dhcp
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.dns
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.dpd
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.files
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.ftp
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.http
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.intel
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.irc
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.kerberos
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.known_certs
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.known_hosts
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.known_services
-rw-r--r--  1 root root          18 Nov 29 22:00 .rotated.loaded_scripts
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.mysql
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.notice
-rw-r--r--  1 root root          18 Nov 29 22:00 .rotated.packet_filter
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.pe
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.radius
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.rdp
-rw-r--r--  1 root root          18 Nov 29 23:00 .rotated.reporter
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.rfb
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.sip
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.smtp
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.snmp
-rw-r--r--  1 root root          18 Nov 30 01:00 .rotated.socks
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.software
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.ssh
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.ssl
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.stats
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.syslog
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.tunnel
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.weird
-rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.x509

Any tips or pointers on how to track down the culprit here and get things
back to normally scheduled log rotation intervals?

FWIW, I'm running on a brand new R730xd with SSD's for the OS, an NVMe
drive for the Bro installation, and RAID 10 & RAID 5 respectively for the
60 day archive and long term storage archive volumes. I'm linting my intel
files to make sure they are properly formatted, and there doesn't seem to
be an issue here...

Respectfully,

-Erin Shelton

Program Manager: Incident Response and Network Security
Office of Information Technology
University of Colorado Boulder
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161130/918d470a/attachment-0001.html 

From gfaulkner.nsm at gmail.com  Wed Nov 30 09:22:40 2016
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Wed, 30 Nov 2016 11:22:40 -0600
Subject: [Bro] Weird log rotation issues in 2.5
In-Reply-To: <CADOaRAkVXiKqefdYWs_4JUrTwa4DJfYq7UanE_JPAw95R+Sk9A@mail.gmail.com>
References: <CADOaRAkVXiKqefdYWs_4JUrTwa4DJfYq7UanE_JPAw95R+Sk9A@mail.gmail.com>
Message-ID: <302f0947-ab0e-4755-13ad-8ff106f33d85@gmail.com>

Not sure if your case is going to be the same, but I used to experience 
a similar issue on pre-2.5 versions of Bro. Upon checking my process 
list I usually found that gzip was choking on a relatively large 
multi-gigabyte log file and would not finish compressing fast enough if 
at all. This seemed to cause a sort of chain reaction that broke log 
rotation. Usual culprits were http.log, dns.log, or conn.log. The 
solution I found at the time was to decrease the log rotation interval 
to keep the file sizes down. I still seem to recall at least some of the 
logs would rotate before it got stuck, but I'm not sure that is 
happening based on your example.

~Gary

On 11/30/16 10:42 AM, eshelton wrote:
> My site was an early tester of 2.5_beta, and during most of our testing, we
> did not experience much in the way of issues related to log rotation. As
> soon as we started using 2.5_beta we were utilizing logger mode, as we run
> an extremely busy cluster, and had been experiencing memory utilization
> exhaustion issues with the manager being single threaded At one point near
> the end of the 2.5 beta, we started seeing some irregular rotation
> intervals, which we noted were potentially related to us starting to use
> the Intel framework.
>
> I have since moved to 2.5 stable, and I'm still seeing irregular rotation
> issues which appear to be related to how busy the manager is at the time of
> log rotation. I do notice that the main local-manager process seems to stay
> loaded at or near 100% CPU utilization when we are seeing a lot of traffic.
> As an example, I rotated logs perfectly from midnight up to 07:00-08:00.
> After that, my current logs in spool/logger are running well over into 9
> o'clocks data set. I frequently see .rotated files well after a log
> rotation interval has run (the following was observed 09:41 today):
>
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.capture_loss
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.communication
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.conn
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.conn-summary
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.dce_rpc
> -rw-r--r--  1 root root          18 Nov 30 03:00 .rotated.dhcp
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.dns
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.dpd
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.files
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.ftp
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.http
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.intel
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.irc
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.kerberos
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.known_certs
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.known_hosts
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.known_services
> -rw-r--r--  1 root root          18 Nov 29 22:00 .rotated.loaded_scripts
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.mysql
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.notice
> -rw-r--r--  1 root root          18 Nov 29 22:00 .rotated.packet_filter
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.pe
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.radius
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.rdp
> -rw-r--r--  1 root root          18 Nov 29 23:00 .rotated.reporter
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.rfb
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.sip
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.smtp
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.snmp
> -rw-r--r--  1 root root          18 Nov 30 01:00 .rotated.socks
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.software
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.ssh
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.ssl
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.stats
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.syslog
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.tunnel
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.weird
> -rw-r--r--  1 root root          18 Nov 30 08:00 .rotated.x509
>
> Any tips or pointers on how to track down the culprit here and get things
> back to normally scheduled log rotation intervals?
>
> FWIW, I'm running on a brand new R730xd with SSD's for the OS, an NVMe
> drive for the Bro installation, and RAID 10 & RAID 5 respectively for the
> 60 day archive and long term storage archive volumes. I'm linting my intel
> files to make sure they are properly formatted, and there doesn't seem to
> be an issue here...
>
> Respectfully,
>
> -Erin Shelton
>
> Program Manager: Incident Response and Network Security
> Office of Information Technology
> University of Colorado Boulder
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161130/844bcedd/attachment.html 

From johanna at icir.org  Wed Nov 30 10:38:47 2016
From: johanna at icir.org (Johanna Amann)
Date: Wed, 30 Nov 2016 10:38:47 -0800
Subject: [Bro] Bro Digest, Vol 127, Issue 46
In-Reply-To: <CAAcg0eJoM5Zt5tVYzyhNDVV7uiT=q9iYMPkAJw+_9MQ_WehvUA@mail.gmail.com>
References: <mailman.1.1480363201.3153.bro@bro.org>
	<CAAcg0eJoM5Zt5tVYzyhNDVV7uiT=q9iYMPkAJw+_9MQ_WehvUA@mail.gmail.com>
Message-ID: <20161130183847.rrmecpeseiw7kdop@wifi242.sys.ICSI.Berkeley.EDU>

You call add_analyzer in one of your scripts, typically in file_new or in
file_sniff.

https://www.bro.org/sphinx-git/frameworks/file-analysis.html gives a lot
more detail on how and where to use the function.

Johanna

On Tue, Nov 29, 2016 at 10:27:38AM +1100, John Edwards wrote:
> Where is this line defined? what file would i define this once i create sub
> folders for file types?  I wish to get cron to compress specific folders to
> save disk space.
> 
> Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
> 
> Cheers,
> John
> 
> On Tue, Nov 29, 2016 at 7:00 AM, <bro-request at bro.org> wrote:
> 
> > Send Bro mailing list submissions to
> >         bro at bro.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >         http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> > or, via email, send a message with subject or body 'help' to
> >         bro-request at bro.org
> >
> > You can reach the person managing the list at
> >         bro-owner at bro.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of Bro digest..."
> >
> >
> > Today's Topics:
> >
> >    1. Re: Bro 2.5 CPU usage (Drew Dixon)
> >    2. File extraction in different directories (maybe day vise)
> >       (fatema bannatwala)
> >    3. Re: File extraction in different directories (maybe day vise)
> >       (Hosom, Stephen M)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Mon, 28 Nov 2016 13:19:47 -0500
> > From: Drew Dixon <dwdixon at umich.edu>
> > Subject: Re: [Bro] Bro 2.5 CPU usage
> > To: Daniel Thayer <dnthayer at illinois.edu>
> > Cc: bro at bro.org
> > Message-ID:
> >         <CA+pNqTOFXBseTFHdQqKM2r=FyZEGUH9Gj4O7i=Sn9PoZLbU-3w@
> > mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Would it be possible for someone quantify what a low bandwidth/low traffic
> > setup might be in terms of a bandwidth unit of measurement range where
> > Justin's patch would be advised to be used? I.E. Kbps/Mbps etc.  What would
> > be a cut-off bandwidth/traffic rate value where it would not be advisable
> > that this patch be used?
> >
> > On Fri, Nov 25, 2016 at 1:54 PM, Daniel Thayer <dnthayer at illinois.edu>
> > wrote:
> >
> > > Regarding broctl, you can disable the "not seeing any packets"
> > > warnings if you set this in your etc/broctl.cfg:
> > > StatsLogEnable = 0
> > >
> > > Doing so will also disable logging to broctl's stats.log (note:
> > > this is NOT the stats.log that Bro itself logs), which I'm
> > > guessing most people don't need anyway.
> > >
> > >
> > > On 11/25/16 11:43 AM, Michael Shirk wrote:
> > > > Is this something worthy of a feature request for low bandwidth setups?
> > > >
> > > > In addition to something like this, I have to do a patch for very low
> > > > network traffic with bro cron reporting network traffic has stopped on
> > > > the monitoring interface.
> > > >
> > > > --
> > > > Michael Shirk
> > > > Daemon Security, Inc.
> > > > http://www.daemon-security.com
> > > _______________________________________________
> > > Bro mailing list
> > > bro at bro-ids.org
> > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> > >
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> > 20161128/03d37f8d/attachment-0001.html
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Mon, 28 Nov 2016 14:14:37 -0500
> > From: fatema bannatwala <fatema.bannatwala at gmail.com>
> > Subject: [Bro] File extraction in different directories (maybe day
> >         vise)
> > To: bro at bro.org
> > Message-ID:
> >         <CACX0rUSYj_8JnSwS7ZKjef=HNsKDnX6z+oxFo-VXw7mO0xZ4WA at mail.
> > gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > HI,
> >
> > Just wanted to check-in, so that I don't re-invent the wheel, is there any
> > way, or if somebody has tried extracting the files in different
> > directories,i.e maybe in daily directory (just like bro logs the events in
> > the day vise directory)?
> > Right now we have over thousands of files extracted in a single directory
> > and it's getting harder to manage the one single directory to access the
> > extracted files, hence was looking into the Bro logging framework so that I
> > can steal some code from the event logging and rotation part for the file
> > extraction script.
> > Any other way around to it?
> >
> > Appreciate the help.
> >
> > Thanks,
> > Fatema.
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> > 20161128/0453d91c/attachment-0001.html
> >
> > ------------------------------
> >
> > Message: 3
> > Date: Mon, 28 Nov 2016 19:46:20 +0000
> > From: "Hosom, Stephen M" <hosom at battelle.org>
> > Subject: Re: [Bro] File extraction in different directories (maybe day
> >         vise)
> > To: fatema bannatwala <fatema.bannatwala at gmail.com>, "bro at bro.org"
> >         <bro at bro.org>
> > Message-ID:
> >         <E5A2071C6FFFF640B714C224139C9B3327DEFC0F at WP-MBX2B.milky-
> > way.battelle.org>
> >
> > Content-Type: text/plain; charset="utf-8"
> >
> > One of the arguments for attaching the file extraction analyzer is the
> > filename that you want it to extract to. So long as you?re building this
> > filename on the fly every time you attach the analyzer, you should be able
> > to specify a different directory for every file?if you wished for such a
> > thing.
> >
> > Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
> >
> > Where I have specified ?fname?, just specify the string of the
> > filename/path that you would like to store the file.
> >
> > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of
> > fatema bannatwala
> > Sent: Monday, November 28, 2016 2:15 PM
> > To: bro at bro.org
> > Subject: [Bro] File extraction in different directories (maybe day vise)
> >
> > HI,
> >
> > Just wanted to check-in, so that I don't re-invent the wheel, is there any
> > way, or if somebody has tried extracting the files in different
> > directories,i.e maybe in daily directory (just like bro logs the events in
> > the day vise directory)?
> > Right now we have over thousands of files extracted in a single directory
> > and it's getting harder to manage the one single directory to access the
> > extracted files, hence was looking into the Bro logging framework so that I
> > can steal some code from the event logging and rotation part for the file
> > extraction script.
> > Any other way around to it?
> >
> > Appreciate the help.
> >
> > Thanks,
> > Fatema.
> >
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> > 20161128/061b9e37/attachment-0001.html
> >
> > ------------------------------
> >
> > _______________________________________________
> > Bro mailing list
> > Bro at bro.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >
> >
> > End of Bro Digest, Vol 127, Issue 46
> > ************************************
> >

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From nbblrr at gmail.com  Wed Nov 30 21:34:03 2016
From: nbblrr at gmail.com (Nibbler nib)
Date: Thu, 1 Dec 2016 00:34:03 -0500
Subject: [Bro] Intelligence framework and bro logs
Message-ID: <CAJb7T4Kk7Gb=LpnXPof+fuEYa8BMtnZ_1B_zc2LMb79QWKPaZw@mail.gmail.com>

Hi Bro list,

I am starting to use Bro to check some IOCs on my network using the Bro
Intelligence Framework, and I have few questions regarding my configuration
:
-I am updating the IOCs regularly and the only way I found to reload IOCs
in bro is to restart the service with broctl, is there any better way?
(like just reloading the configuration and not restarting everything)
-When restarting bro with broctl, Bro is having a weird behaviour with
logs, they are stored in directory with weird names (like 2039-01-
 2039-02-  2039-10-  2046-49-  2050-58-  2051-03-...), have you already
seen such case? Is it a due to a bad configuration? Or a bug? Is there a
way to restart bro without rotating logs?

(all this with bro 2.5 compiled from sources)

Thanks
N
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161201/72d47809/attachment.html