From hiren.panchasara at gmail.com  Fri Nov  1 08:36:49 2013
From: hiren.panchasara at gmail.com (hiren panchasara)
Date: Fri, 1 Nov 2013 08:36:49 -0700
Subject: [Bro] Cluster setup
In-Reply-To: <7E6CC23C-25D6-4DF0-BCE1-F213B51D9198@icir.org>
References: <CALCpEUFP0+iasa_kxjP-OsSpfiS9M7U10WmshDhYmkoxU3b2_A@mail.gmail.com>
	<526F5ACB.4080404@illinois.edu>
	<CALCpEUHJF3kDa59UH=btczYXZmeWSN8vQM3S1QA1n8_gM_vNSA@mail.gmail.com>
	<CALCpEUHWyGm18RwfQPXaDjG3mveAhdkN-2Le6GQVBFT0DMW00w@mail.gmail.com>
	<CALCpEUGKzcx6vD7fMsYmpZe1AR8ej+zz-cgUJJ0yjMWO4-gEXg@mail.gmail.com>
	<20131029182142.GC4436@datacomm.albany.edu>
	<576FD060-B258-4877-B5F0-91B4FDE39602@icir.org>
	<CALCpEUHEquaQVjr1iG4JBG71phhJBR75=HdOu0Mawr9AQkqtsQ@mail.gmail.com>
	<FFAF558E-2D73-4B37-8F53-5D3C0DC7079D@icir.org>
	<CALCpEUGjsSVruy-Zt8msRXyfZ0T4Pmn42NBbiGDUpK3=n9oH4w@mail.gmail.com>
	<AD185128-BC02-4F4F-B2E7-7581A6726EED@icir.org>
	<CALCpEUGTU7yoOj4UUMOOt5yQSNeZMRpw8WLdSpnEu3o=QgVtUA@mail.gmail.com>
	<7E6CC23C-25D6-4DF0-BCE1-F213B51D9198@icir.org>
Message-ID: <CALCpEUE94TPdW3o-Rg4MSiv3g34NSkVex-FQ_+Li+QiZMBFoTg@mail.gmail.com>

On Thu, Oct 31, 2013 at 5:51 PM, Seth Hall <seth at icir.org> wrote:
>
> On Oct 31, 2013, at 3:20 PM, hiren panchasara <hiren.panchasara at gmail.com> wrote:
>
>> Right. So (afaik) in FreeBSD we do not have PF_RING like functionality
>> where there is an PF_RING application sdk and applications can choose
>
> Ah, generally right now people are only doing load balancing on FreeBSD with Myricom NICs and the Myricom Sniffer driver.

This is not an option for me but I will surely see how they are doing it.
>
>> which queue it wants to listen to. Intel NIC (that I am using)
>> definitely can distribute traffic in 8 queues it has but question for
>> me is, how do I distribute it to the application/workers.
>
> In FreeBSD at the moment you don't.  It's possible that if you have netmap enabled you might be able to use that in some fashion, but generally those FlowDirector based queues on the high end Intel NICs aren't actually exposed in userland.  If you are talking about RSS (receive side scaling), then that's insufficient unless you have RX and TX RSS (I'm a little confused about this, but I read something recently that seemed to indicate this might be a thing on some NICs) because both directions of each connection need to go to each process.

Yeah, tricky part is the userland association. But I am also not too
clear on RSS detail. That looks like the only option I have. I need to
dig deeper.
>
>> Do they have PF_RING setup which blindly ports queue:1 traffic to
>> worker:1 and bro (using PF_RING's sdk) will do the parsing?
>
> Typically people run PF_Ring in mode 0 which is actually not exposing hardware load balanced traffic.  It's collecting all of the traffic and load balancing it in the core.

Here, core == bro's core?

I really appreciate you taking time out and responding to my questions, Seth :-)

cheers,
Hiren



From seth at icir.org  Fri Nov  1 10:34:39 2013
From: seth at icir.org (Seth Hall)
Date: Fri, 1 Nov 2013 13:34:39 -0400
Subject: [Bro] Cluster setup
In-Reply-To: <CALCpEUE94TPdW3o-Rg4MSiv3g34NSkVex-FQ_+Li+QiZMBFoTg@mail.gmail.com>
References: <CALCpEUFP0+iasa_kxjP-OsSpfiS9M7U10WmshDhYmkoxU3b2_A@mail.gmail.com>
	<526F5ACB.4080404@illinois.edu>
	<CALCpEUHJF3kDa59UH=btczYXZmeWSN8vQM3S1QA1n8_gM_vNSA@mail.gmail.com>
	<CALCpEUHWyGm18RwfQPXaDjG3mveAhdkN-2Le6GQVBFT0DMW00w@mail.gmail.com>
	<CALCpEUGKzcx6vD7fMsYmpZe1AR8ej+zz-cgUJJ0yjMWO4-gEXg@mail.gmail.com>
	<20131029182142.GC4436@datacomm.albany.edu>
	<576FD060-B258-4877-B5F0-91B4FDE39602@icir.org>
	<CALCpEUHEquaQVjr1iG4JBG71phhJBR75=HdOu0Mawr9AQkqtsQ@mail.gmail.com>
	<FFAF558E-2D73-4B37-8F53-5D3C0DC7079D@icir.org>
	<CALCpEUGjsSVruy-Zt8msRXyfZ0T4Pmn42NBbiGDUpK3=n9oH4w@mail.gmail.com>
	<AD185128-BC02-4F4F-B2E7-7581A6726EED@icir.org>
	<CALCpEUGTU7yoOj4UUMOOt5yQSNeZMRpw8WLdSpnEu3o=QgVtUA@mail.gmail.com>
	<7E6CC23C-25D6-4DF0-BCE1-F213B51D9198@icir.org>
	<CALCpEUE94TPdW3o-Rg4MSiv3g34NSkVex-FQ_+Li+QiZMBFoTg@mail.gmail.com>
Message-ID: <31B1202B-FE8A-4A3F-A430-FB3610C6FE9C@icir.org>


On Nov 1, 2013, at 11:36 AM, hiren panchasara <hiren.panchasara at gmail.com> wrote:

>> Typically people run PF_Ring in mode 0 which is actually not exposing hardware load balanced traffic.  It's collecting all of the traffic and load balancing it in the core.
> 
> Here, core == bro's core?


Sorry, totally wrong word.  I meant kernel. :)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131101/f3be3302/attachment.bin 

From hiren.panchasara at gmail.com  Fri Nov  1 11:36:35 2013
From: hiren.panchasara at gmail.com (hiren panchasara)
Date: Fri, 1 Nov 2013 11:36:35 -0700
Subject: [Bro] Cluster setup
In-Reply-To: <31B1202B-FE8A-4A3F-A430-FB3610C6FE9C@icir.org>
References: <CALCpEUFP0+iasa_kxjP-OsSpfiS9M7U10WmshDhYmkoxU3b2_A@mail.gmail.com>
	<526F5ACB.4080404@illinois.edu>
	<CALCpEUHJF3kDa59UH=btczYXZmeWSN8vQM3S1QA1n8_gM_vNSA@mail.gmail.com>
	<CALCpEUHWyGm18RwfQPXaDjG3mveAhdkN-2Le6GQVBFT0DMW00w@mail.gmail.com>
	<CALCpEUGKzcx6vD7fMsYmpZe1AR8ej+zz-cgUJJ0yjMWO4-gEXg@mail.gmail.com>
	<20131029182142.GC4436@datacomm.albany.edu>
	<576FD060-B258-4877-B5F0-91B4FDE39602@icir.org>
	<CALCpEUHEquaQVjr1iG4JBG71phhJBR75=HdOu0Mawr9AQkqtsQ@mail.gmail.com>
	<FFAF558E-2D73-4B37-8F53-5D3C0DC7079D@icir.org>
	<CALCpEUGjsSVruy-Zt8msRXyfZ0T4Pmn42NBbiGDUpK3=n9oH4w@mail.gmail.com>
	<AD185128-BC02-4F4F-B2E7-7581A6726EED@icir.org>
	<CALCpEUGTU7yoOj4UUMOOt5yQSNeZMRpw8WLdSpnEu3o=QgVtUA@mail.gmail.com>
	<7E6CC23C-25D6-4DF0-BCE1-F213B51D9198@icir.org>
	<CALCpEUE94TPdW3o-Rg4MSiv3g34NSkVex-FQ_+Li+QiZMBFoTg@mail.gmail.com>
	<31B1202B-FE8A-4A3F-A430-FB3610C6FE9C@icir.org>
Message-ID: <CALCpEUEoui5uukoG2bvd=tQ34dSAMYGDdoM29USv7VZKO4f5Bg@mail.gmail.com>

On Fri, Nov 1, 2013 at 10:34 AM, Seth Hall <seth at icir.org> wrote:
>
> On Nov 1, 2013, at 11:36 AM, hiren panchasara <hiren.panchasara at gmail.com> wrote:
>
>>> Typically people run PF_Ring in mode 0 which is actually not exposing hardware load balanced traffic.  It's collecting all of the traffic and load balancing it in the core.
>>
>> Here, core == bro's core?
>
>
> Sorry, totally wrong word.  I meant kernel. :)

Ah, okay. So instead of card, let kernel do all the load balancing
(probably with assistance from PF_RING).

Thank you,
Hiren


From knrd at rogers.com  Sun Nov  3 19:34:43 2013
From: knrd at rogers.com (Konrad Weglowski)
Date: Sun, 3 Nov 2013 22:34:43 -0500
Subject: [Bro] BRO conn.log - connection flow direction wrong - non
	standard telnet port connection
In-Reply-To: <4F534409-ECE8-4B1E-B87E-1C8A326A632E@icir.org>
References: <003301ced4f7$48c406a0$da4c13e0$@com>
	<4F534409-ECE8-4B1E-B87E-1C8A326A632E@icir.org>
Message-ID: <000c01ced90e$ce3432e0$6a9c98a0$@com>

Hello Seth,

Thanks for looking into this.

I do not have notice.log file created in that particular timeframe. Also I
see that capture_loss.log files are there...see output below:

Session reported wrong direction:

zcat conn.00\:00\:00-01\:00\:00.log.gz | bro-cut -d ts uid proto conn_state
history | grep BuR4quUCRKe
2013-11-03T00:41:24+0000        BuR4quUCRKe     tcp     SH      Fa

Capture_loss log file for the same timeframe as above:

zcat capture_loss.00\:00\:00-01\:00\:00.log.gz | bro-cut -d
2013-11-03T00:11:59+0000        900.000034      bro     0       669214
0.000%
2013-11-03T00:26:59+0000        900.000020      bro     0       675273
0.000%
2013-11-03T00:41:59+0000        900.000052      bro     0       672973
0.000%
2013-11-03T00:56:59+0000        900.000032      bro     0       681093
0.000%


I had to look at a data from different timeframe as my BRO logs got deleted
from the time frame I referenced in the original email.

Thanks,

Konrad



-----Original Message-----
From: Seth Hall [mailto:seth at icir.org] 
Sent: October-30-13 10:21 AM
To: Konrad Weglowski
Cc: bro at bro.org
Subject: Re: [Bro] BRO conn.log - connection flow direction wrong - non
standard telnet port connection


On Oct 29, 2013, at 6:36 PM, Konrad Weglowski <knrd at rogers.com> wrote:

> Just to give some context, we have a script running which telnets to
multiple devices and polls certain variables and exits on a non-standard
telnet ports.

Are you dropping a lot of packets?  It looks like Bro isn't seeing the
beginning of these connections (syn packets) which makes it nearly
impossible to determine the direction without guessing.  Bro's current
strategy for "fixing" reversed connections like this is by consulting the
likely_server_ports variable but since it sounds like you are using
non-standard ports it's unlikely that this would work.

I think the big question we need to answer is why you aren't seeing the SYN
packets.  Check for PacketFilter::Dropped_Packets notices in your notice.log
and add "@load misc/capture-loss" to your local.bro script so that you will
have a capture_loss.log which will give a holistic measurement of packet
loss.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2014.0.4158 / Virus Database: 3615/6806 - Release Date: 11/03/13



From seth at icir.org  Mon Nov  4 05:50:24 2013
From: seth at icir.org (Seth Hall)
Date: Mon, 4 Nov 2013 08:50:24 -0500
Subject: [Bro] BRO conn.log - connection flow direction wrong - non
	standard telnet port connection
In-Reply-To: <000c01ced90e$ce3432e0$6a9c98a0$@com>
References: <003301ced4f7$48c406a0$da4c13e0$@com>
	<4F534409-ECE8-4B1E-B87E-1C8A326A632E@icir.org>
	<000c01ced90e$ce3432e0$6a9c98a0$@com>
Message-ID: <681677E7-11E2-4B06-A55A-F3B4795E058E@icir.org>


On Nov 3, 2013, at 10:34 PM, Konrad Weglowski <knrd at rogers.com> wrote:

> zcat conn.00\:00\:00-01\:00\:00.log.gz | bro-cut -d ts uid proto conn_state
> history | grep BuR4quUCRKe
> 2013-11-03T00:41:24+0000        BuR4quUCRKe     tcp     SH      Fa

Could you capture some packets from one of these connections and send it to me?  There must be something else going on here.

> 2013-11-03T00:11:59+0000        900.000034      bro     0       669214
> 0.000%
> 2013-11-03T00:26:59+0000        900.000020      bro     0       675273

This looks good at least.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131104/8f5127f8/attachment.bin 

From mattchess50 at gmail.com  Mon Nov  4 11:09:30 2013
From: mattchess50 at gmail.com (Matt Stucky)
Date: Mon, 4 Nov 2013 13:09:30 -0600
Subject: [Bro] pf_ring on RHEL/CENTOS 6?
In-Reply-To: <CAMM5JYoALoCUhXUP8UiwYHZzp45FU7GcHEtHTpycpU44-P_ntA@mail.gmail.com>
References: <CAMM5JYoALoCUhXUP8UiwYHZzp45FU7GcHEtHTpycpU44-P_ntA@mail.gmail.com>
Message-ID: <CAMM5JYoq1mW1dWGhL81zRyYcZLqBqEDLvLApmFtykAS4=QFE_g@mail.gmail.com>

In case anyone is interested, I ended up installing PF_RING from source,
then rebuilding the Bro RPM with PF_RING support.  It would be nice if the
native libpcap and tcpdump already had support for PF_RING, but that's not
currently the case.  I'd rather install everything from RPMs, but having
Bro at least installed from a package should make updates a little easier.
Here are the basic steps:

Install Prerequisites

1.    Add the EPEL repo to the system but leave it disabled:
/etc/yum.repos.d/epel.repo
2.    Remove conflicting packages: libpcap, tcpdump, cmake.
3.    Install prerequisites:  mpfr cpp ppl cloog-ppl gcc kernel-devel
pcre-devel libpcap-devel yum-plugin-priorities libnet flex bison gcc-c++
swig rpm-build
4.    Install prerequisites from EPEL:  libyaml libyaml-devel cmake28
5.    Create a softlink for cmake pointing to the newer version from EPEL.

Build and Install PF_RING

1.    Download the source from
http://sourceforge.net/projects/ntop/files/PF_RING/
2.    Configure, make, and install the kernel module, libpcap, and tcpdump
3.    Create an /etc/modprobe.d/pfring.conf entry to load the kernel module
at boot
4.    Manually load the pf_ring module for now
5.    Create an ldconfig file /etc/ld.so.conf.d/pfring.conf that contains
the path to the libpcap dynamic libraries
6.    Run ?ldconfig? to load the new config for now

Build the Bro RPM with PF_RING Support

1.    Download the source from http://www.bro.org/download/index.html and
unpack it with a non-root user.
2.    As that non-root user, go into the bro-2.1/pkg directory and edit the
check-cmake file so that the cmake check matches the version you have.
3.    As the non-root user edit the make-rpm-packages file and add the
--with-pcap=/usr/local/pfring (or wherever you installed PF_RING) option to
the configure lines.
4.    As the non-root user execute the make-rpm-packages script; the
packages will end up in the bro-2.1/build/  directory.

Install Bro from the newly built RPM package

It's running now with PF_RING and very few dropped packet notices.

# cat /proc/net/pf_ring/info
PF_RING Version          : 5.6.1 ($Revision: exported$)
Total rings              : 4

Standard (non DNA) Options
Ring slots               : 4096
Slot version             : 15
Capture TX               : No [RX only]
IP Defragment            : No
Socket Mode              : Standard
Transparent mode         : Yes [mode 0]
Total plugins            : 0
Cluster Fragment Queue   : 3464
Cluster Fragment Discard : 1036837

-matt

 On Wed, Oct 30, 2013 at 10:33 AM, Matt Stucky <mattchess50 at gmail.com>wrote:

> I've set up a Bro 2.1 instance with a network tap, but keep getting notice
> log entries of "PacketFilter::Dropped_Packets".  I'm assuming this is
> because Bro is single threaded and it needs more workers to keep up with
> the traffic, so I'm trying to implement pf_ring to distribute the traffic
> across multiple workers.  I've installed the pf_ring RPM package from ntop (
> http://www.nmon.net/packages/rpm/x86_64/PF_RING/) and that gets the
> kernel module loaded but seems to be lacking something still - probably
> linking libpcap to pf_ring?  That's what I'm not sure about.  After
> installing pf_ring from the RPM package and configuring Bro for multiple
> workers it starts up ok but is still dropping packets (all of the workers,
> per the notice log) and pf_ring doesn't appear to be used:
>
> # cat /proc/net/pf_ring/info
> PF_RING Version          : 5.6.2 ($Revision: 6910$)
> Total rings              : 0
>
> Standard (non DNA) Options
> Ring slots               : 4096
> Slot version             : 15
> Capture TX               : No [RX only]
> IP Defragment            : No
> Socket Mode              : Standard
> Transparent mode         : Yes [mode 0]
> Total plugins            : 0
> Cluster Fragment Queue   : 0
> Cluster Fragment Discard : 0
>
> Has anyone had any success with clustered Bro with pf_ring on RHEL/CENTOS,
> and did you have to compile it from source and re-compile libpcap?  I'd
> prefer to stick with the RPM packages since it tends to make updating less
> problematic.  I installed Bro 2.1 as an RPM package as well.
>
> Thanks,
> Matt
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131104/5c5fc16e/attachment.html 

From connar.rosebraugh at egov.com  Mon Nov  4 15:18:32 2013
From: connar.rosebraugh at egov.com (Rosebraugh, Connar)
Date: Mon, 4 Nov 2013 23:18:32 +0000
Subject: [Bro] Problem with broccoli-python
Message-ID: <088ED11BA811374BACE1259396F485E3DA8CD7@VADC-MBX02.ad.cdc.nicusa.com>

Hello All,

I went through the install for broccoli-python, and when I try and do "from broccoli import *" from my home directory, I get the error :
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/broccoli.py", line 6, in <module>
    from _broccoli_intern import *
ImportError: libbroccoli.so.5: cannot open shared object file: No such file or directory

However, when I run this command from /usr/local/lib/python, the command works just fine. Does anyone know what files are supposed to end up in what directories?

Thanks,
Connar Rosebraugh
Intern, Security Operations
NICUSA, Inc.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131104/d5c71b22/attachment.html 

From omer007security at walla.co.il  Tue Nov  5 01:22:25 2013
From: omer007security at walla.co.il (=?UTF-8?Q?=D7=A2=D7=95=D7=9E=D7=A8=20=D7=A2=D7=95=D7=9E=D7=A8?=)
Date: Tue, 5 Nov 2013 11:22:25 +0200
Subject: [Bro] =?utf-8?q?Customize_Bro=27s_Http_log?=
Message-ID: <1383643345.333000-16048540-8634@walla.co.il>

An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131105/aadb781b/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: main.bro
Type: application/octet-stream
Size: 11437 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131105/aadb781b/attachment.obj 

From jsiwek at illinois.edu  Tue Nov  5 08:18:46 2013
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Tue, 5 Nov 2013 16:18:46 +0000
Subject: [Bro] Problem with broccoli-python
In-Reply-To: <088ED11BA811374BACE1259396F485E3DA8CD7@VADC-MBX02.ad.cdc.nicusa.com>
References: <088ED11BA811374BACE1259396F485E3DA8CD7@VADC-MBX02.ad.cdc.nicusa.com>
Message-ID: <A73AB677-1DAE-4A9C-984C-48FD27493150@illinois.edu>


On Nov 4, 2013, at 5:18 PM, Rosebraugh, Connar <connar.rosebraugh at egov.com> wrote:

> I went through the install for broccoli-python, and when I try and do ?from broccoli import *? from my home directory, I get the error :
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>  File "/usr/local/lib/python2.7/dist-packages/broccoli.py", line 6, in <module>
>    from _broccoli_intern import *
> ImportError: libbroccoli.so.5: cannot open shared object file: No such file or directory
> 
> However, when I run this command from /usr/local/lib/python, the command works just fine. Does anyone know what files are supposed to end up in what directories?

The default install prefix for broccoli if installed on its own is /usr/local.  But if broccoli is installed as part of the full bro package, the default install prefix is /usr/local/bro.  If you don?t see the shared library indicated in the error message in the lib subdir of either place, please give more details on how you configured/built/installed broccoli and the python bindings.

If you do find libbroccoli.so in your manual search for it, then it seems like the run-time linker just needs help finding it.  And that is typically resolved by either setting the LD_LIBRARY_PATH env var or doing stuff with /etc/ld.so.conf and `ldconfig` if you?re on linux ? the man pages for ?ld-linux.so? and ?ldconfig? are good to review here.

- Jon


From sconzo at visiblerisk.com  Tue Nov  5 09:06:43 2013
From: sconzo at visiblerisk.com (Mike Sconzo)
Date: Tue, 5 Nov 2013 11:06:43 -0600
Subject: [Bro] Problem with broccoli-python
In-Reply-To: <A73AB677-1DAE-4A9C-984C-48FD27493150@illinois.edu>
References: <088ED11BA811374BACE1259396F485E3DA8CD7@VADC-MBX02.ad.cdc.nicusa.com>
	<A73AB677-1DAE-4A9C-984C-48FD27493150@illinois.edu>
Message-ID: <CAGZoo6FY-52FZD4D9o8dyFAvz7K=krrjQMtP-M9vF5FzimCm9Q@mail.gmail.com>

Showing our setup to verify what Jon said.

On ubuntu (built for install in /opt):
root at host:# cat /etc/ld.so.conf.d/bro.conf
/opt/bro/lib/

(don't forget to run ldconfig) after the bro.conf file

The other thing we did was add a .pth file to our python venv:
root at host:# cat /opt/python/venv/lib/python2.7/site-packages/bro.pth
/opt/bro/lib/broctl/

On Tue, Nov 5, 2013 at 10:18 AM, Siwek, Jonathan Luke
<jsiwek at illinois.edu> wrote:
>
> On Nov 4, 2013, at 5:18 PM, Rosebraugh, Connar <connar.rosebraugh at egov.com> wrote:
>
>> I went through the install for broccoli-python, and when I try and do ?from broccoli import *? from my home directory, I get the error :
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in <module>
>>  File "/usr/local/lib/python2.7/dist-packages/broccoli.py", line 6, in <module>
>>    from _broccoli_intern import *
>> ImportError: libbroccoli.so.5: cannot open shared object file: No such file or directory
>>
>> However, when I run this command from /usr/local/lib/python, the command works just fine. Does anyone know what files are supposed to end up in what directories?
>
> The default install prefix for broccoli if installed on its own is /usr/local.  But if broccoli is installed as part of the full bro package, the default install prefix is /usr/local/bro.  If you don?t see the shared library indicated in the error message in the lib subdir of either place, please give more details on how you configured/built/installed broccoli and the python bindings.
>
> If you do find libbroccoli.so in your manual search for it, then it seems like the run-time linker just needs help finding it.  And that is typically resolved by either setting the LD_LIBRARY_PATH env var or doing stuff with /etc/ld.so.conf and `ldconfig` if you?re on linux ? the man pages for ?ld-linux.so? and ?ldconfig? are good to review here.
>
> - Jon
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



-- 
cat ~/.bash_history > documentation.txt



From gary at doit.wisc.edu  Tue Nov  5 14:16:27 2013
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Tue, 05 Nov 2013 16:16:27 -0600
Subject: [Bro] Broctl pf_ring_DNA support / Bro at 100G
In-Reply-To: <5271447D.3060909@illinois.edu>
References: <52713748.1020908@doit.wisc.edu> <5271447D.3060909@illinois.edu>
Message-ID: <52796E3B.5050806@doit.wisc.edu>

First off, I'll admit I'm new to both pf_ring and bro cluster set-up, so 
quite possibly I've made some rookie mistakes, but I've been trying to 
read documentation, source comments, and lists to try to fill in the 
gaps as best I can with a full helping of trial an error. I also 
understand that I'm attempting to test some features that are in 
development and not necessarily ready for prime-time.

I've been experimenting with the broctl with DNA support 
(topic/dnthayer/ticket845) on a single node to start.  I have tried 
testing this with various RSS settings (0,1 and 4) as well as 
transparent mode 0 and 2 by tweaking the shell script load_dna_driver.sh 
that comes with pf_ring, but I could be horribly misconfiguring 
something somewhere. What seems to happen based on the output from 
running diag within an interactive broctl  (and I may be misinterpreting 
things) is that every worker process tries to listen on the same cluster 
ID(21). pfdnacluster_master appears to run and then crash and then the 
workers seem to start in a non-DNA mode. Running capstats from within 
broctl usually returns an error that cluster ID 21 does not exist at 
this point, and attempting to run the stop command typically results in 
one or more worker process being hung up and having to be killed or 
crashing brotctl in some way. I thought I ran across a previous issue 
for vanilla pf_ring where there was another bug ID related to needing to 
spawn each process with a different cluster id, but can't recall. Maybe 
there are two different branches addressing different issues related to 
what I'm trying to do.

Here is what my node.cfg looks like (where xx.xx.xx.xx is currently the 
same IP for manager/proxy/worker):

[manager]
type=manager
host=xx.xx.xx.xx

[proxy-1]
type=proxy
host=xx.xx.xx.xx

[worker-1]
type=worker
host=xx.xx.xx.xx
interface=dna0
lb_procs=4
lb_method=pf_ring_dna

Typically what I end up seeing in /proc/net/pf_ring/ is something like 
this where processid-none.xx matches each bro worker process:

30194-dna0.12  30319-none.13  30320-none.14  30321-none.16 30322-none.15

and then after some time has passed:

30319-none.13  30320-none.14  30321-none.16  30322-none.15

Output from each looks a such:

# cat 30194-dna0.12
Bound Device(s)    :
Active             : 1
Breed              : DNA
Sampling Rate      : 1
Capture Direction  : RX+TX
Socket Mode        : RX only
Appl. Name         : pfdnacluster_master-cluster-21-
IP Defragment      : No
BPF Filtering      : Disabled
# Sw Filt. Rules   : 0
# Hw Filt. Rules   : 0
Poll Pkt Watermark : 128
Num Poll Calls     : 0
Channel Id         : 0
Num RX Slots       : 8192
Num TX Slots       : 8192
Tot Memory         : 25952256 bytes
Cluster: Tot Recvd : 2217888
Cluster: Tot Sent  : 0

# cat 30319-none.13
Bound Device(s)    :
Active             : 1
Breed              : Non-DNA
Sampling Rate      : 1
Capture Direction  : RX+TX
Socket Mode        : RX+TX
Appl. Name         : <unknown>
IP Defragment      : No
BPF Filtering      : Disabled
# Sw Filt. Rules   : 0
# Hw Filt. Rules   : 0
Poll Pkt Watermark : 1
Num Poll Calls     : 600262

# cat 30320-none.14
Bound Device(s)    :
Active             : 1
Breed              : Non-DNA
Sampling Rate      : 1
Capture Direction  : RX+TX
Socket Mode        : RX+TX
Appl. Name         : <unknown>
IP Defragment      : No
BPF Filtering      : Disabled
# Sw Filt. Rules   : 0
# Hw Filt. Rules   : 0
Poll Pkt Watermark : 1
Num Poll Calls     : 706408

cat 30321-none.16
Bound Device(s)    :
Active             : 1
Breed              : Non-DNA
Sampling Rate      : 1
Capture Direction  : RX+TX
Socket Mode        : RX+TX
Appl. Name         : <unknown>
IP Defragment      : No
BPF Filtering      : Disabled
# Sw Filt. Rules   : 0
# Hw Filt. Rules   : 0
Poll Pkt Watermark : 1
Num Poll Calls     : 775591

# cat 30322-none.15
Bound Device(s)    :
Active             : 1
Breed              : Non-DNA
Sampling Rate      : 1
Capture Direction  : RX+TX
Socket Mode        : RX+TX
Appl. Name         : <unknown>
IP Defragment      : No
BPF Filtering      : Disabled
# Sw Filt. Rules   : 0
# Hw Filt. Rules   : 0
Poll Pkt Watermark : 1
Num Poll Calls     : 886131

Any thoughts? Is anything I've said at all useful in seeing where I may 
be failing or where bro might not do what it is I'm trying to get it to do?

Regards,

Gary Faulkner
UW Madison
Office of Campus Information Security
608-262-8591

On 10/30/2013 12:40 PM, Daniel Thayer wrote:
>
> If you want to test the PF_RING/DNA plugin, then you'll need to use
> the BroControl in the branch "topic/dnthayer/ticket845" (in the broctl
> git repo), but I'm not sure if anyone has successfully used it yet.
>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131105/936fbb67/attachment.bin 

From tyler.schoenke at colorado.edu  Wed Nov  6 07:19:54 2013
From: tyler.schoenke at colorado.edu (Tyler T. Schoenke)
Date: Wed, 6 Nov 2013 08:19:54 -0700
Subject: [Bro] DNS alert for CryptoLocker?
Message-ID: <0AA5D924DE90AF48BBD563CCD296B8FBF6FF14CD3B@EXC2.ad.colorado.edu>

So I don't have to reinvent the wheel, does anyone have a script to alert when a bunch of DNS  nxdomain response codes are returned?  We had a CryptoLocker infected system.  Here is a snippet of the DNS queries it was performing.  I assume the script will be fairly trivial to write with the new metrics framework.

1382548938.833528       GMCxsRbK0Ai     128.x.y.z 58872   128.a.b.c   53      udp     11849   ndqycnknvoouv.net       1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382548944.705308       gNc8acns5pe     128.x.y.z 57136   128.a.b.c   53      udp     29248   hcanlyoattqnk.info      1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382548947.922531       2wQ3L1SjO2i     128.x.y.z 55438   128.a.b.c   53      udp     37701   pggqvjlpjuvfj.biz       1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382548950.164884       K6SBCLsCeHd     128.x.y.z 62257   128.a.b.c   53      udp     27109   rkvrpstomducl.org       1       C_INTERNET      1       A       -       -       F       F       T       F       0       -       -       F
1382548952.804004       A3cpzxeprDd     128.x.y.z 62188   128.a.b.c   53      udp     19436   xdlmipcfinsnx.info      1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382548953.848624       oFpUoyQaeT6     128.x.y.z 58160   128.a.b.c   53      udp     64315   yskkfkmsvjyjh.com       1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382548956.153981       42MqOejLeC7     128.x.y.z 61254   128.a.b.c   53      udp     25859   bwalyturyrxgh.biz       1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382548960.964978       iwlngihsWR2     128.x.y.z 59060   128.a.b.c   53      udp     49446   wfffkyemceall.info      1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382548965.228544       BSHfNWkQmN2     128.x.y.z 50542   128.a.b.c   53      udp     64599   gxfbvapxgjhhwir.ru      1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382548966.392850       AL4jDt0K4Bl     128.x.y.z 65068   128.a.b.c   53      udp     60778   pbxksllrmivxhjc.org     1       C_INTERNET      1       A       -       -       F       F       T       F       0       -       -       F
1382548998.923970       hvrkgMU1nj9     128.x.y.z 64366   128.a.b.c   53      udp     58017   -       -       -       -       -       0       NOERROR F       F       F       T       0       212.71.250.4,212.71.250.4       0.000000,0.000000       F
1382549001.210921       F0wHtNhVKQj     128.x.y.z 53692   128.a.b.c   53      udp     18268   eijwmsocubkbifr.com     1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382549004.587866       dupMP8ecnh9     128.x.y.z 65102   128.a.b.c   53      udp     55272   -       -       -       -       -       3       NXDOMAIN        F       F       F       F       0       -       -       F
1382549005.590564       8hHrrWK3ySg     128.x.y.z 53233   128.a.b.c   53      udp     49644   csnrwkgpneybfdw.org     1       C_INTERNET      1       A       -       -       F       F       T       F       0       -       -       F
1382549008.355729       2zHHnrpDv94     128.x.y.z 49268   128.a.b.c   53      udp     48578   yxhlnnrvnxwhvjb.info    1       C_INTERNET      1       A       -       -       F       F       T       F       0       -       -       F
1382549009.401946       XGYKkM7TJHb     128.x.y.z 58084   128.a.b.c   53      udp     21374   ypqijlryiuibvra.com     1       C_INTERNET      1       A       -       -       F       F       T       F       0       -       -       F
1382549011.483780       jPbHypWQKyh     128.x.y.z 56556   128.a.b.c   53      udp     38615   gfidmpcvtbjipor.biz     1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382549014.515443       ndy7OcvfED      128.x.y.z 49785   128.a.b.c   53      udp     11355   -       -       -       -       -       3       NXDOMAIN        F       F       F       F       0       -       -       F
1382549015.564495       qkrQfYjmd8g     128.x.y.z 64433   128.a.b.c   53      udp     45      -       -       -       -       -       0       NOERROR F       F       F       T       0       212.71.250.4,212.71.250.4       0.000000,0.000000       F
1382549017.104583       bQbmeVq6PSl     128.x.y.z 60956   128.a.b.c   53      udp     21595   epmydibaismctwn.info    1       C_INTERNET      1       A       -       -       F       F       T       F       0       -       -       F
1382549020.276359       ZyCXQrFDUie     128.x.y.z 58936   128.a.b.c   53      udp     45237   taxkcsutphxwues.biz     1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382549021.295831       DDxa09moudg     128.x.y.z 51396   128.a.b.c   53      udp     14981   ooqydautbpucsxk.ru      1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382549024.077917       utOUlYH43La     128.x.y.z 61588   128.a.b.c   53      udp     33615   -       -       -       -       -       0       NOERROR F       F       F       T       0       212.71.250.4    0.000000        F
1382549026.376626       7NYXLG3zOJ4     128.x.y.z 52200   128.a.b.c   53      udp     30833   myuutstxphxvlmn.com     1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382549028.599961       MBxVPKOcOl3     128.x.y.z 58592   128.a.b.c   53      udp     49290   ohfvyihiguvwuxp.biz     1       C_INTERNET      1       A       -       -       F       F       T       F       0       -       -       F
1382549031.847178       vD02D08eII4     128.x.y.z 61924   128.a.b.c   53      udp     23377   shocdnhyfmdfsoj.co.uk   1       C_INTERNET      1       A       -       -       F       F       T       F       0       -       -       F
1382549034.478314       n3WCj7AlLU2     128.x.y.z 60108   128.a.b.c   53      udp     33753   tmyedwcqvvykcjj.com     1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382549036.575201       caR4StggyDa     128.x.y.z 52132   128.a.b.c   53      udp     4039    oxsaegepxdvieuh.biz     1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382549037.595521       OgiZzasfva3     128.x.y.z 52622   128.a.b.c   53      udp     49144   cbcrkxjuurixfpe.ru      1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382549038.784184       fbHvNBwyQr6     128.x.y.z 65484   128.a.b.c   53      udp     51376   pddcepyhomrngqq.org     1       C_INTERNET      1       A       3       NXDOMAIN        F       F       T       F       0       -       -       F
1382549039.995781       MdZxaa06IYh     128.x.y.z 56073   128.a.b.c   53      udp     1505    novnagkvsgbfbvv.co.uk   1       C_INTERNET      1       A       0       NOERROR F       F       T       T       0       212.71.250.4,212.71.250.4       0.000000,0.00000


Thanks,

Tyler


--
--
Tyler Schoenke
Network Security Program Manager
IT Security Office
University of Colorado at Boulder

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131106/1269e87b/attachment.html 

From anthony.kasza at gmail.com  Wed Nov  6 08:02:14 2013
From: anthony.kasza at gmail.com (anthony kasza)
Date: Wed, 6 Nov 2013 08:02:14 -0800
Subject: [Bro] DNS alert for CryptoLocker?
In-Reply-To: <0AA5D924DE90AF48BBD563CCD296B8FBF6FF14CD3B@EXC2.ad.colorado.edu>
References: <0AA5D924DE90AF48BBD563CCD296B8FBF6FF14CD3B@EXC2.ad.colorado.edu>
Message-ID: <CAEZw2byJQ1fsbwzaQ5Ne5eUeQkQxhUahgOocaHXX7QfxNpBqLg@mail.gmail.com>

I wrote this: https://github.com/anthonykasza/nxes

It's not exactly what you're looking to do, as it doesn't make use of the
SumStats framework. Hopefully you still find it helpful.

-AK
On Nov 6, 2013 7:41 AM, "Tyler T. Schoenke" <tyler.schoenke at colorado.edu>
wrote:

> So I don?t have to reinvent the wheel, does anyone have a script to alert
> when a bunch of DNS  nxdomain response codes are returned?  We had a
> CryptoLocker infected system.  Here is a snippet of the DNS queries it was
> performing.  I assume the script will be fairly trivial to write with the
> new metrics framework.
>
>
>
> 1382548938.833528       GMCxsRbK0Ai     128.x.y.z 58872   128.a.b.c
> 53      udp     11849   ndqycnknvoouv.net       1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382548944.705308       gNc8acns5pe     128.x.y.z 57136   128.a.b.c
> 53      udp     29248   hcanlyoattqnk.info      1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382548947.922531       2wQ3L1SjO2i     128.x.y.z 55438   128.a.b.c
> 53      udp     37701   pggqvjlpjuvfj.biz       1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382548950.164884       K6SBCLsCeHd     128.x.y.z 62257   128.a.b.c
> 53      udp     27109   rkvrpstomducl.org       1       C_INTERNET
> 1       A       -       -       F       F       T       F       0
> -       -       F
>
> 1382548952.804004       A3cpzxeprDd     128.x.y.z 62188   128.a.b.c
> 53      udp     19436   xdlmipcfinsnx.info      1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382548953.848624       oFpUoyQaeT6     128.x.y.z 58160   128.a.b.c
> 53      udp     64315   yskkfkmsvjyjh.com       1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382548956.153981       42MqOejLeC7     128.x.y.z 61254   128.a.b.c
> 53      udp     25859   bwalyturyrxgh.biz       1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382548960.964978       iwlngihsWR2     128.x.y.z 59060   128.a.b.c
> 53      udp     49446   wfffkyemceall.info      1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382548965.228544       BSHfNWkQmN2     128.x.y.z 50542   128.a.b.c
> 53      udp     64599   gxfbvapxgjhhwir.ru      1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382548966.392850       AL4jDt0K4Bl     128.x.y.z 65068   128.a.b.c
> 53      udp     60778   pbxksllrmivxhjc.org     1       C_INTERNET
> 1       A       -       -       F       F       T       F       0
> -       -       F
>
> 1382548998.923970       hvrkgMU1nj9     128.x.y.z 64366   128.a.b.c
> 53      udp     58017   -       -       -       -       -       0
> NOERROR F       F       F       T       0
> 212.71.250.4,212.71.250.4       0.000000,0.000000       F
>
> 1382549001.210921       F0wHtNhVKQj     128.x.y.z 53692   128.a.b.c
> 53      udp     18268   eijwmsocubkbifr.com     1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382549004.587866       dupMP8ecnh9     128.x.y.z 65102   128.a.b.c
> 53      udp     55272   -       -       -       -       -       3
> NXDOMAIN        F       F       F       F       0       -       -       F
>
> 1382549005.590564       8hHrrWK3ySg     128.x.y.z 53233   128.a.b.c
> 53      udp     49644   csnrwkgpneybfdw.org     1       C_INTERNET
> 1       A       -       -       F       F       T       F       0
>       -       -       F
>
> 1382549008.355729       2zHHnrpDv94     128.x.y.z 49268   128.a.b.c
> 53      udp     48578   yxhlnnrvnxwhvjb.info    1       C_INTERNET
> 1       A       -       -       F       F       T       F       0
> -       -       F
>
> 1382549009.401946       XGYKkM7TJHb     128.x.y.z 58084   128.a.b.c
> 53      udp     21374   ypqijlryiuibvra.com     1       C_INTERNET
> 1       A       -       -       F       F       T       F       0
> -       -       F
>
> 1382549011.483780       jPbHypWQKyh     128.x.y.z 56556   128.a.b.c
> 53      udp     38615   gfidmpcvtbjipor.biz     1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382549014.515443       ndy7OcvfED      128.x.y.z 49785   128.a.b.c
> 53      udp     11355   -       -       -       -       -       3
> NXDOMAIN        F       F       F       F       0       -       -       F
>
> 1382549015.564495       qkrQfYjmd8g     128.x.y.z 64433   128.a.b.c
> 53      udp     45      -       -       -       -       -       0
> NOERROR F       F       F       T       0
> 212.71.250.4,212.71.250.4       0.000000,0.000000       F
>
> 1382549017.104583       bQbmeVq6PSl     128.x.y.z 60956   128.a.b.c
> 53      udp     21595   epmydibaismctwn.info    1       C_INTERNET
> 1       A       -       -       F       F       T       F       0
> -       -       F
>
> 1382549020.276359       ZyCXQrFDUie     128.x.y.z 58936   128.a.b.c
> 53      udp     45237   taxkcsutphxwues.biz     1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382549021.295831       DDxa09moudg     128.x.y.z 51396   128.a.b.c
> 53      udp     14981   ooqydautbpucsxk.ru      1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382549024.077917       utOUlYH43La     128.x.y.z 61588   128.a.b.c
> 53      udp     33615   -       -       -       -       -       0
> NOERROR F       F       F       T       0       212.71.250.4
> 0.000000        F
>
> 1382549026.376626       7NYXLG3zOJ4     128.x.y.z 52200   128.a.b.c
> 53      udp     30833   myuutstxphxvlmn.com     1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382549028.599961       MBxVPKOcOl3     128.x.y.z 58592   128.a.b.c
> 53      udp     49290   ohfvyihiguvwuxp.biz     1       C_INTERNET
> 1       A       -       -       F       F       T       F       0
> -       -       F
>
> 1382549031.847178       vD02D08eII4     128.x.y.z 61924   128.a.b.c
> 53      udp     23377   shocdnhyfmdfsoj.co.uk   1       C_INTERNET
> 1       A       -       -       F       F       T       F       0       -
>       -       F
>
> 1382549034.478314       n3WCj7AlLU2     128.x.y.z 60108   128.a.b.c
> 53      udp     33753   tmyedwcqvvykcjj.com     1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382549036.575201       caR4StggyDa     128.x.y.z 52132   128.a.b.c
> 53      udp     4039    oxsaegepxdvieuh.biz     1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382549037.595521       OgiZzasfva3     128.x.y.z 52622   128.a.b.c
> 53      udp     49144   cbcrkxjuurixfpe.ru      1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382549038.784184       fbHvNBwyQr6     128.x.y.z 65484   128.a.b.c
> 53      udp     51376   pddcepyhomrngqq.org     1       C_INTERNET
> 1       A       3       NXDOMAIN        F       F       T       F
> 0       -       -       F
>
> 1382549039.995781       MdZxaa06IYh     128.x.y.z 56073   128.a.b.c
> 53      udp     1505    novnagkvsgbfbvv.co.uk   1       C_INTERNET
> 1       A       0       NOERROR F       F       T       T       0
> 212.71.250.4,212.71.250.4       0.000000,0.00000
>
>
>
>
>
> Thanks,
>
>
>
> Tyler
>
>
>
>
>
> --
>
> --
>
> Tyler Schoenke
>
> Network Security Program Manager
>
> IT Security Office
>
> University of Colorado at Boulder
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131106/b6231a5c/attachment.html 

From liam at broala.com  Wed Nov  6 08:56:13 2013
From: liam at broala.com (Liam Randall)
Date: Wed, 6 Nov 2013 11:56:13 -0500
Subject: [Bro] DNS alert for CryptoLocker?
In-Reply-To: <CAEZw2byJQ1fsbwzaQ5Ne5eUeQkQxhUahgOocaHXX7QfxNpBqLg@mail.gmail.com>
References: <0AA5D924DE90AF48BBD563CCD296B8FBF6FF14CD3B@EXC2.ad.colorado.edu>
	<CAEZw2byJQ1fsbwzaQ5Ne5eUeQkQxhUahgOocaHXX7QfxNpBqLg@mail.gmail.com>
Message-ID: <CAGGyr=7PuKA5c9jGVE5mVZe3WEn7wWKQ7Fq4=_0B9qrtKSBzQA@mail.gmail.com>

I have a whole crap load of DNS & Recon scripts I did for bsides DC I just
haven't had time to post yet.

Too many NXDomains:

https://gist.github.com/LiamRandall/7339749

Tune as you see fit.  Important note- if you are only instrumented at the
ingress/egress point you will most likely only be seeing your recursive
resolver.

Liam


On Wed, Nov 6, 2013 at 11:02 AM, anthony kasza <anthony.kasza at gmail.com>wrote:

> I wrote this: https://github.com/anthonykasza/nxes
>
> It's not exactly what you're looking to do, as it doesn't make use of the
> SumStats framework. Hopefully you still find it helpful.
>
> -AK
> On Nov 6, 2013 7:41 AM, "Tyler T. Schoenke" <tyler.schoenke at colorado.edu>
> wrote:
>
>> So I don?t have to reinvent the wheel, does anyone have a script to alert
>> when a bunch of DNS  nxdomain response codes are returned?  We had a
>> CryptoLocker infected system.  Here is a snippet of the DNS queries it was
>> performing.  I assume the script will be fairly trivial to write with the
>> new metrics framework.
>>
>>
>>
>> 1382548938.833528       GMCxsRbK0Ai     128.x.y.z 58872   128.a.b.c
>> 53      udp     11849   ndqycnknvoouv.net       1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382548944.705308       gNc8acns5pe     128.x.y.z 57136   128.a.b.c
>> 53      udp     29248   hcanlyoattqnk.info      1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382548947.922531       2wQ3L1SjO2i     128.x.y.z 55438   128.a.b.c
>> 53      udp     37701   pggqvjlpjuvfj.biz       1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382548950.164884       K6SBCLsCeHd     128.x.y.z 62257   128.a.b.c
>> 53      udp     27109   rkvrpstomducl.org       1       C_INTERNET
>> 1       A       -       -       F       F       T       F       0
>> -       -       F
>>
>> 1382548952.804004       A3cpzxeprDd     128.x.y.z 62188   128.a.b.c
>> 53      udp     19436   xdlmipcfinsnx.info      1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382548953.848624       oFpUoyQaeT6     128.x.y.z 58160   128.a.b.c
>> 53      udp     64315   yskkfkmsvjyjh.com       1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382548956.153981       42MqOejLeC7     128.x.y.z 61254   128.a.b.c
>> 53      udp     25859   bwalyturyrxgh.biz       1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382548960.964978       iwlngihsWR2     128.x.y.z 59060   128.a.b.c
>> 53      udp     49446   wfffkyemceall.info      1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382548965.228544       BSHfNWkQmN2     128.x.y.z 50542   128.a.b.c
>> 53      udp     64599   gxfbvapxgjhhwir.ru      1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382548966.392850       AL4jDt0K4Bl     128.x.y.z 65068   128.a.b.c
>> 53      udp     60778   pbxksllrmivxhjc.org     1       C_INTERNET
>> 1       A       -       -       F       F       T       F       0
>> -       -       F
>>
>> 1382548998.923970       hvrkgMU1nj9     128.x.y.z 64366   128.a.b.c
>> 53      udp     58017   -       -       -       -       -       0
>> NOERROR F       F       F       T       0
>> 212.71.250.4,212.71.250.4       0.000000,0.000000       F
>>
>> 1382549001.210921       F0wHtNhVKQj     128.x.y.z 53692   128.a.b.c
>> 53      udp     18268   eijwmsocubkbifr.com     1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382549004.587866       dupMP8ecnh9     128.x.y.z 65102   128.a.b.c
>> 53      udp     55272   -       -       -       -       -       3
>> NXDOMAIN        F       F       F       F       0       -       -       F
>>
>> 1382549005.590564       8hHrrWK3ySg     128.x.y.z 53233   128.a.b.c
>> 53      udp     49644   csnrwkgpneybfdw.org     1       C_INTERNET
>> 1       A       -       -       F       F       T       F       0
>>       -       -       F
>>
>> 1382549008.355729       2zHHnrpDv94     128.x.y.z 49268   128.a.b.c
>> 53      udp     48578   yxhlnnrvnxwhvjb.info    1       C_INTERNET
>> 1       A       -       -       F       F       T       F       0
>> -       -       F
>>
>> 1382549009.401946       XGYKkM7TJHb     128.x.y.z 58084   128.a.b.c
>> 53      udp     21374   ypqijlryiuibvra.com     1       C_INTERNET
>> 1       A       -       -       F       F       T       F       0
>> -       -       F
>>
>> 1382549011.483780       jPbHypWQKyh     128.x.y.z 56556   128.a.b.c
>> 53      udp     38615   gfidmpcvtbjipor.biz     1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382549014.515443       ndy7OcvfED      128.x.y.z 49785   128.a.b.c
>> 53      udp     11355   -       -       -       -       -       3
>> NXDOMAIN        F       F       F       F       0       -       -       F
>>
>> 1382549015.564495       qkrQfYjmd8g     128.x.y.z 64433   128.a.b.c
>> 53      udp     45      -       -       -       -       -       0
>> NOERROR F       F       F       T       0
>> 212.71.250.4,212.71.250.4       0.000000,0.000000       F
>>
>> 1382549017.104583       bQbmeVq6PSl     128.x.y.z 60956   128.a.b.c
>> 53      udp     21595   epmydibaismctwn.info    1       C_INTERNET
>> 1       A       -       -       F       F       T       F       0
>> -       -       F
>>
>> 1382549020.276359       ZyCXQrFDUie     128.x.y.z 58936   128.a.b.c
>> 53      udp     45237   taxkcsutphxwues.biz     1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382549021.295831       DDxa09moudg     128.x.y.z 51396   128.a.b.c
>> 53      udp     14981   ooqydautbpucsxk.ru      1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382549024.077917       utOUlYH43La     128.x.y.z 61588   128.a.b.c
>> 53      udp     33615   -       -       -       -       -       0
>> NOERROR F       F       F       T       0       212.71.250.4
>> 0.000000        F
>>
>> 1382549026.376626       7NYXLG3zOJ4     128.x.y.z 52200   128.a.b.c
>> 53      udp     30833   myuutstxphxvlmn.com     1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382549028.599961       MBxVPKOcOl3     128.x.y.z 58592   128.a.b.c
>> 53      udp     49290   ohfvyihiguvwuxp.biz     1       C_INTERNET
>> 1       A       -       -       F       F       T       F       0
>> -       -       F
>>
>> 1382549031.847178       vD02D08eII4     128.x.y.z 61924   128.a.b.c
>> 53      udp     23377   shocdnhyfmdfsoj.co.uk   1       C_INTERNET
>> 1       A       -       -       F       F       T       F       0       -
>>       -       F
>>
>> 1382549034.478314       n3WCj7AlLU2     128.x.y.z 60108   128.a.b.c
>> 53      udp     33753   tmyedwcqvvykcjj.com     1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382549036.575201       caR4StggyDa     128.x.y.z 52132   128.a.b.c
>> 53      udp     4039    oxsaegepxdvieuh.biz     1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382549037.595521       OgiZzasfva3     128.x.y.z 52622   128.a.b.c
>> 53      udp     49144   cbcrkxjuurixfpe.ru      1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382549038.784184       fbHvNBwyQr6     128.x.y.z 65484   128.a.b.c
>> 53      udp     51376   pddcepyhomrngqq.org     1       C_INTERNET
>> 1       A       3       NXDOMAIN        F       F       T       F
>> 0       -       -       F
>>
>> 1382549039.995781       MdZxaa06IYh     128.x.y.z 56073   128.a.b.c
>> 53      udp     1505    novnagkvsgbfbvv.co.uk   1       C_INTERNET
>> 1       A       0       NOERROR F       F       T       T       0
>> 212.71.250.4,212.71.250.4       0.000000,0.00000
>>
>>
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Tyler
>>
>>
>>
>>
>>
>> --
>>
>> --
>>
>> Tyler Schoenke
>>
>> Network Security Program Manager
>>
>> IT Security Office
>>
>> University of Colorado at Boulder
>>
>>
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Liam Randall
Managing Partner
510-281-0760
www.Broala.com <http://www.broala.com/>
>From the creators of Bro <http://www.bro.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131106/8323fccc/attachment.html 

From christopher.p.crawford at gmail.com  Wed Nov  6 09:51:28 2013
From: christopher.p.crawford at gmail.com (Chris Crawford)
Date: Wed, 6 Nov 2013 12:51:28 -0500
Subject: [Bro] Load Single Column Table with Input Framework
Message-ID: <CAG1VgObO2F20VOCdcxp9mLekQ2zyopoxG3P1n-or1DzkGX5O1Q@mail.gmail.com>

I'm following the tutorial on the input framework:
http://www.bro.org/sphinx/input.html

Everything works great.  But, if my blacklist is only one column (i.e. I
remove the other columns so that I'm only left with the "ip" column), I run
into issues.

The docs for Input::add_table say that val is optional:
<quote>
val: any &optional

Record that defines the values used as the elements of the table If val is
undefined, destination has to be a set.
</quote>

So, I fixed my script to look like this:
<new_script>

type Idx: record {
        ip: addr;
};

global blacklist: set[addr];

event bro_init() {
        Input::add_table([$source="blacklist.file", $name="blacklist",
$idx=Idx, $destination=blacklist]);
        print(|blacklist|);
        Input::remove("blacklist");
}

</new_script>

No more val and changed the table to a set of addr.

The size for blacklist that gets printed out is 0, even though
blacklist.file looks like this:

<blacklist.file>
#fields ip
#types  addr
192.168.17.1
192.168.27.2
192.168.250.3
</blacklist.file>

I expected print(|blacklist|) to print out 3.

I know I must be missing something simple.  What am I missing?  How should
I read in a single column table?

-Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131106/dcf39351/attachment.html 

From jazoff at albany.edu  Wed Nov  6 10:05:54 2013
From: jazoff at albany.edu (Azoff, Justin)
Date: Wed, 6 Nov 2013 18:05:54 +0000
Subject: [Bro] Load Single Column Table with Input Framework
In-Reply-To: <CAG1VgObO2F20VOCdcxp9mLekQ2zyopoxG3P1n-or1DzkGX5O1Q@mail.gmail.com>
References: <CAG1VgObO2F20VOCdcxp9mLekQ2zyopoxG3P1n-or1DzkGX5O1Q@mail.gmail.com>
Message-ID: <03tyfsvwog98pwpl0p4s038g.1383761148804@email.android.com>

The table doesn't get loaded immediately.  There is an "input file loaded" event you can print from, or for testing try using bro_done

Chris Crawford <christopher.p.crawford at gmail.com> wrote:



I'm following the tutorial on the input framework:
http://www.bro.org/sphinx/input.html

Everything works great.  But, if my blacklist is only one column (i.e. I remove the other columns so that I'm only left with the "ip" column), I run into issues.

The docs for Input::add_table say that val is optional:
<quote>
val: any &optional

Record that defines the values used as the elements of the table If val is undefined, destination has to be a set.
</quote>

So, I fixed my script to look like this:
<new_script>

type Idx: record {
        ip: addr;
};

global blacklist: set[addr];

event bro_init() {
        Input::add_table([$source="blacklist.file", $name="blacklist", $idx=Idx, $destination=blacklist]);
        print(|blacklist|);
        Input::remove("blacklist");
}

</new_script>

No more val and changed the table to a set of addr.

The size for blacklist that gets printed out is 0, even though blacklist.file looks like this:

<blacklist.file>
#fields ip
#types  addr
192.168.17.1
192.168.27.2
192.168.250.3
</blacklist.file>

I expected print(|blacklist|) to print out 3.

I know I must be missing something simple.  What am I missing?  How should I read in a single column table?

-Chris

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131106/fce251f6/attachment.html 

From seth at icir.org  Wed Nov  6 10:06:49 2013
From: seth at icir.org (Seth Hall)
Date: Wed, 6 Nov 2013 13:06:49 -0500
Subject: [Bro] Load Single Column Table with Input Framework
In-Reply-To: <CAG1VgObO2F20VOCdcxp9mLekQ2zyopoxG3P1n-or1DzkGX5O1Q@mail.gmail.com>
References: <CAG1VgObO2F20VOCdcxp9mLekQ2zyopoxG3P1n-or1DzkGX5O1Q@mail.gmail.com>
Message-ID: <1FDD5690-77E1-4DDD-ACD0-4715E15B1389@icir.org>


On Nov 6, 2013, at 12:51 PM, Chris Crawford <christopher.p.crawford at gmail.com> wrote:

> I expected print(|blacklist|) to print out 3.
> 
> I know I must be missing something simple.  What am I missing?  How should I read in a single column table?

The input framework is asynchronous.  You are printing before that data has been loaded in.  You could try waiting a moment (by scheduling an event perhaps) and checking again.

event try_again()
	{
	print |blacklist|;
	}

event bro_init() 
	{
        Input::add_table([$source="blacklist.file", $name="blacklist", $idx=Idx, $destination=blacklist]);
	schedule 2secs { try_again() };
        Input::remove("blacklist");
	}

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131106/986ca599/attachment.bin 

From anthony.kasza at gmail.com  Wed Nov  6 10:30:50 2013
From: anthony.kasza at gmail.com (anthony kasza)
Date: Wed, 6 Nov 2013 10:30:50 -0800
Subject: [Bro] Load Single Column Table with Input Framework
In-Reply-To: <1FDD5690-77E1-4DDD-ACD0-4715E15B1389@icir.org>
References: <CAG1VgObO2F20VOCdcxp9mLekQ2zyopoxG3P1n-or1DzkGX5O1Q@mail.gmail.com>
	<1FDD5690-77E1-4DDD-ACD0-4715E15B1389@icir.org>
Message-ID: <CAEZw2bxv__Gi+C4kyFM3xMmemd3iJ1NQmy7hY7BYVJoEeqAWTA@mail.gmail.com>

This is a bother when running Bro on trace files. Bro will finish
processing a trace before reading in an entire table. One hack is to build
the table in a separate file and @load it.
On Nov 6, 2013 10:23 AM, "Seth Hall" <seth at icir.org> wrote:

>
> On Nov 6, 2013, at 12:51 PM, Chris Crawford <
> christopher.p.crawford at gmail.com> wrote:
>
> > I expected print(|blacklist|) to print out 3.
> >
> > I know I must be missing something simple.  What am I missing?  How
> should I read in a single column table?
>
> The input framework is asynchronous.  You are printing before that data
> has been loaded in.  You could try waiting a moment (by scheduling an event
> perhaps) and checking again.
>
> event try_again()
>         {
>         print |blacklist|;
>         }
>
> event bro_init()
>         {
>         Input::add_table([$source="blacklist.file", $name="blacklist",
> $idx=Idx, $destination=blacklist]);
>         schedule 2secs { try_again() };
>         Input::remove("blacklist");
>         }
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131106/f13c9e52/attachment.html 

From seth at icir.org  Wed Nov  6 11:00:29 2013
From: seth at icir.org (Seth Hall)
Date: Wed, 6 Nov 2013 14:00:29 -0500
Subject: [Bro] Load Single Column Table with Input Framework
In-Reply-To: <CAEZw2bxv__Gi+C4kyFM3xMmemd3iJ1NQmy7hY7BYVJoEeqAWTA@mail.gmail.com>
References: <CAG1VgObO2F20VOCdcxp9mLekQ2zyopoxG3P1n-or1DzkGX5O1Q@mail.gmail.com>
	<1FDD5690-77E1-4DDD-ACD0-4715E15B1389@icir.org>
	<CAEZw2bxv__Gi+C4kyFM3xMmemd3iJ1NQmy7hY7BYVJoEeqAWTA@mail.gmail.com>
Message-ID: <6461277B-EDE6-4747-B6C7-8A541BB26B99@icir.org>


On Nov 6, 2013, at 1:30 PM, anthony kasza <anthony.kasza at gmail.com> wrote:

> This is a bother when running Bro on trace files. Bro will finish processing a trace before reading in an entire table. One hack is to build the table in a separate file and @load it.

We also have the exit_only_after_terminate variable to prevent Bro from terminating automatically.  You will have to manually kill it if you set this.

redef exit_only_after_terminate = T;

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131106/606771e0/attachment.bin 

From gary at doit.wisc.edu  Wed Nov  6 13:36:52 2013
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Wed, 06 Nov 2013 15:36:52 -0600
Subject: [Bro] Broctl pf_ring_DNA support / Bro at 100G
In-Reply-To: <52796E3B.5050806@doit.wisc.edu>
References: <52713748.1020908@doit.wisc.edu> <5271447D.3060909@illinois.edu>
	<52796E3B.5050806@doit.wisc.edu>
Message-ID: <527AB674.2080402@doit.wisc.edu>

It looks like this behavior may be a case of not having a libzero 
license. I had licensed the DNA drivers, and hadn't realized I also 
needed the libzero piece. I'll try this again once I have the proper 
licensing. Thanks to Scott Campbell for pointing me in the right direction.

Regards,

Gary Faulkner
UW Madison
Office of Campus Information Security
608-262-8591

On 11/5/2013 4:16 PM, Gary Faulkner wrote:
> First off, I'll admit I'm new to both pf_ring and bro cluster set-up, 
> so quite possibly I've made some rookie mistakes, but I've been trying 
> to read documentation, source comments, and lists to try to fill in 
> the gaps as best I can with a full helping of trial an error. I also 
> understand that I'm attempting to test some features that are in 
> development and not necessarily ready for prime-time.
>
> I've been experimenting with the broctl with DNA support 
> (topic/dnthayer/ticket845) on a single node to start.  I have tried 
> testing this with various RSS settings (0,1 and 4) as well as 
> transparent mode 0 and 2 by tweaking the shell script 
> load_dna_driver.sh that comes with pf_ring, but I could be horribly 
> misconfiguring something somewhere. What seems to happen based on the 
> output from running diag within an interactive broctl  (and I may be 
> misinterpreting things) is that every worker process tries to listen 
> on the same cluster ID(21). pfdnacluster_master appears to run and 
> then crash and then the workers seem to start in a non-DNA mode. 
> Running capstats from within broctl usually returns an error that 
> cluster ID 21 does not exist at this point, and attempting to run the 
> stop command typically results in one or more worker process being 
> hung up and having to be killed or crashing brotctl in some way. I 
> thought I ran across a previous issue for vanilla pf_ring where there 
> was another bug ID related to needing to spawn each process with a 
> different cluster id, but can't recall. Maybe there are two different 
> branches addressing different issues related to what I'm trying to do.
>
> Here is what my node.cfg looks like (where xx.xx.xx.xx is currently 
> the same IP for manager/proxy/worker):
>
> [manager]
> type=manager
> host=xx.xx.xx.xx
>
> [proxy-1]
> type=proxy
> host=xx.xx.xx.xx
>
> [worker-1]
> type=worker
> host=xx.xx.xx.xx
> interface=dna0
> lb_procs=4
> lb_method=pf_ring_dna
>
> Typically what I end up seeing in /proc/net/pf_ring/ is something like 
> this where processid-none.xx matches each bro worker process:
>
> 30194-dna0.12  30319-none.13  30320-none.14  30321-none.16 30322-none.15
>
> and then after some time has passed:
>
> 30319-none.13  30320-none.14  30321-none.16  30322-none.15
>
> Output from each looks a such:
>
> # cat 30194-dna0.12
> Bound Device(s)    :
> Active             : 1
> Breed              : DNA
> Sampling Rate      : 1
> Capture Direction  : RX+TX
> Socket Mode        : RX only
> Appl. Name         : pfdnacluster_master-cluster-21-
> IP Defragment      : No
> BPF Filtering      : Disabled
> # Sw Filt. Rules   : 0
> # Hw Filt. Rules   : 0
> Poll Pkt Watermark : 128
> Num Poll Calls     : 0
> Channel Id         : 0
> Num RX Slots       : 8192
> Num TX Slots       : 8192
> Tot Memory         : 25952256 bytes
> Cluster: Tot Recvd : 2217888
> Cluster: Tot Sent  : 0
>
> # cat 30319-none.13
> Bound Device(s)    :
> Active             : 1
> Breed              : Non-DNA
> Sampling Rate      : 1
> Capture Direction  : RX+TX
> Socket Mode        : RX+TX
> Appl. Name         : <unknown>
> IP Defragment      : No
> BPF Filtering      : Disabled
> # Sw Filt. Rules   : 0
> # Hw Filt. Rules   : 0
> Poll Pkt Watermark : 1
> Num Poll Calls     : 600262
>
> # cat 30320-none.14
> Bound Device(s)    :
> Active             : 1
> Breed              : Non-DNA
> Sampling Rate      : 1
> Capture Direction  : RX+TX
> Socket Mode        : RX+TX
> Appl. Name         : <unknown>
> IP Defragment      : No
> BPF Filtering      : Disabled
> # Sw Filt. Rules   : 0
> # Hw Filt. Rules   : 0
> Poll Pkt Watermark : 1
> Num Poll Calls     : 706408
>
> cat 30321-none.16
> Bound Device(s)    :
> Active             : 1
> Breed              : Non-DNA
> Sampling Rate      : 1
> Capture Direction  : RX+TX
> Socket Mode        : RX+TX
> Appl. Name         : <unknown>
> IP Defragment      : No
> BPF Filtering      : Disabled
> # Sw Filt. Rules   : 0
> # Hw Filt. Rules   : 0
> Poll Pkt Watermark : 1
> Num Poll Calls     : 775591
>
> # cat 30322-none.15
> Bound Device(s)    :
> Active             : 1
> Breed              : Non-DNA
> Sampling Rate      : 1
> Capture Direction  : RX+TX
> Socket Mode        : RX+TX
> Appl. Name         : <unknown>
> IP Defragment      : No
> BPF Filtering      : Disabled
> # Sw Filt. Rules   : 0
> # Hw Filt. Rules   : 0
> Poll Pkt Watermark : 1
> Num Poll Calls     : 886131
>
> Any thoughts? Is anything I've said at all useful in seeing where I 
> may be failing or where bro might not do what it is I'm trying to get 
> it to do?
>
> Regards,
>
> Gary Faulkner
> UW Madison
> Office of Campus Information Security
> 608-262-8591
>
> On 10/30/2013 12:40 PM, Daniel Thayer wrote:
>>
>> If you want to test the PF_RING/DNA plugin, then you'll need to use
>> the BroControl in the branch "topic/dnthayer/ticket845" (in the broctl
>> git repo), but I'm not sure if anyone has successfully used it yet.
>>
>
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131106/b2595e7a/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131106/b2595e7a/attachment.bin 

From avf at eldamar.org.uk  Thu Nov  7 03:35:40 2013
From: avf at eldamar.org.uk (Alexander Frolkin)
Date: Thu, 7 Nov 2013 11:35:40 +0000
Subject: [Bro] Bro and flood protection
Message-ID: <20131107113540.GS32654@eldamar.org.uk>

Hi,

I'm currently looking around for open-source IDSes.  What we'd like
is to have an IDS machine which monitors our Internet traffic and
responds to events by blocking the traffic using Flowspec.  This is easy
to do with Bro and ExaBGP using custom event handlers and/or hooks, and
piped_exec.

I'm currently trying to understand Bro's ability to detect floods, e.g.,
SYN flood, ACK flood, or any other kind of flood, for that matter.

The feeling I have so far is that Bro wasn't really designed for this
sort of thing, and that it's designed more for L7 stuff.

I'm playing with 2.2 beta, and I can't see anything built-in to detect
floods (although maybe I haven't looked hard enough).  In older
versions, though, there was a script called synflood.bro, but it seems
to have disappeared at some point.  Does anyone know what the history of
this is, and whether there is equivalent funtionality in the latest
version?

More generally, if I want to detect network floods, is Bro the right
thing to be using, or should I be looking elsewhere?

Thanks!


Alex



From itsecderek at gmail.com  Thu Nov  7 05:01:53 2013
From: itsecderek at gmail.com (Derek Banks)
Date: Thu, 7 Nov 2013 08:01:53 -0500
Subject: [Bro] Extract files from SMTP
Message-ID: <CAF9WhNbwt5PGqv-LUSiQ1N2hgKTW=xtBOCKZPBKK0XFStU7vEA@mail.gmail.com>

Hello All,

I've been using bro now for a good few months and I still feel like a
complete noob.  I need to extract out mime types in smtp traffic - I am
looking to extract docx files from our last few weeks of pcaps to then go
check for embedded TIFF files (latest 0 day out on MS apps).  Time is not
on my side at the moment - management is bothered about this one for some
reason.

I am running from git master and cannot seem to figure out how the new file
handling works.  Has anyone done something like this recently after the
file handling change and would be willing to share?

Once I get the docx files extracted my intent was to use yara to look for
tiffs then foremost to carve any out.

Regards,
Derek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131107/2b51bae5/attachment.html 

From jlay at slave-tothe-box.net  Thu Nov  7 05:53:57 2013
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 7 Nov 2013 06:53:57 -0700
Subject: [Bro] Extract links in SMTP
Message-ID: <795292A0-8A5C-4E82-9D98-8EAC887611C9@slave-tothe-box.net>

Just saw the Extract files from SMTP, and I?d love to be able to extract links from SMTP as well.  Many times I have to track down from my http logs a bad link that was gone to?would love to be able to just look for the link in my smtp log to find out if it was clicked on via an email.  I too am still a noob at bro, so any assistance with getting something like this to go would be great?thanks all.

James
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131107/ec6af471/attachment.bin 

From liam at broala.com  Thu Nov  7 06:00:10 2013
From: liam at broala.com (Liam Randall)
Date: Thu, 7 Nov 2013 09:00:10 -0500
Subject: [Bro] Extract files from SMTP
In-Reply-To: <CAF9WhNbwt5PGqv-LUSiQ1N2hgKTW=xtBOCKZPBKK0XFStU7vEA@mail.gmail.com>
References: <CAF9WhNbwt5PGqv-LUSiQ1N2hgKTW=xtBOCKZPBKK0XFStU7vEA@mail.gmail.com>
Message-ID: <CAGGyr=4VRpeVhrP85O4aMoG1QtiBbORD-bwMxEE-kt2PhargYQ@mail.gmail.com>

Hey Derek,

Attached is a script to dump "All files" out to disk; you would want to
modify that and check to see if they are "SMTP" first.

The documentation here should have enough examples to get you started:

http://www.bro.org/sphinx-git/frameworks/file-analysis.html

Hope all is well buddy.

Thanks,

Liam Randall





On Thu, Nov 7, 2013 at 8:01 AM, Derek Banks <itsecderek at gmail.com> wrote:

> Hello All,
>
> I've been using bro now for a good few months and I still feel like a
> complete noob.  I need to extract out mime types in smtp traffic - I am
> looking to extract docx files from our last few weeks of pcaps to then go
> check for embedded TIFF files (latest 0 day out on MS apps).  Time is not
> on my side at the moment - management is bothered about this one for some
> reason.
>
> I am running from git master and cannot seem to figure out how the new
> file handling works.  Has anyone done something like this recently after
> the file handling change and would be willing to share?
>
> Once I get the docx files extracted my intent was to use yara to look for
> tiffs then foremost to carve any out.
>
> Regards,
> Derek
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Liam Randall
Managing Partner
510-281-0760
www.Broala.com <http://www.broala.com/>
>From the creators of Bro <http://www.bro.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131107/ec4e7fad/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: extract-all-files.bro
Type: application/octet-stream
Size: 82 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131107/ec4e7fad/attachment.obj 

From himself at louruppert.com  Thu Nov  7 10:10:38 2013
From: himself at louruppert.com (Lou RUPPERT)
Date: Thu, 07 Nov 2013 13:10:38 -0500
Subject: [Bro] Notice suppression in beta
Message-ID: <527BD79E.5010806@louruppert.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hey,

I'm still having trouble suppressing SSL notices in 2.2. I have this
code, which should work:

- --- snip ---
const safe_vendor_netblocks = { 192.168.0.0/16, 10.0.0.0/8, };

function suppress_ssl_notice(n: Notice::Info): bool
{
    # Vendors
    if (n$dst in safe_vendor_netblocks)
        return T;

    return F;
}

hook Notice::policy(n: Notice::Info) &priority=5
{
  if ( n$note == SSL::Invalid_Server_Cert && suppress_ssl_notice(n) )
    break;
}

- --- snip ---

But still I see notices coming through with IPs in the netblocks
listed and with a note for SSL::Invalid_Server_Cert. Shouldn't a break
issued from a hook with a greater priority than the default process
prevent the notice from being logged?

- -- 
I prefer encrypted email.  Get my key here:
http://www.louruppert.com/keys/115DCF62.asc
PGP Fingerprint: 3261 B9F9 9363 D512 56F8  12DD 127F 4D6A 115D CF62
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Icedove - http://www.enigmail.net/

iEYEARECAAYFAlJ7154ACgkQEn9NahFdz2KsdgCfWo5l8wd0TcEXOYjuPvSvcm7k
UEIAoMvRUTudBUj0mEgUGGDfRU5PSdGh
=gM4F
-----END PGP SIGNATURE-----


From jlay at slave-tothe-box.net  Thu Nov  7 10:39:23 2013
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 7 Nov 2013 11:39:23 -0700
Subject: [Bro] Extract links in SMTP
In-Reply-To: <795292A0-8A5C-4E82-9D98-8EAC887611C9@slave-tothe-box.net>
References: <795292A0-8A5C-4E82-9D98-8EAC887611C9@slave-tothe-box.net>
Message-ID: <C9669C7F-A180-434A-9858-004E1CD0A745@slave-tothe-box.net>


On Nov 7, 2013, at 6:53 AM, James Lay <jlay at slave-tothe-box.net> wrote:

> Just saw the Extract files from SMTP, and I?d love to be able to extract links from SMTP as well.  Many times I have to track down from my http logs a bad link that was gone to?would love to be able to just look for the link in my smtp log to find out if it was clicked on via an email.  I too am still a noob at bro, so any assistance with getting something like this to go would be great?thanks all.
> 
> James
> 

Any chance someone can point me in the right direction with this?  My goal is to add an http field in the smtp_entities file, so I won?t have to create a completely new log file.  I have this code (thanks to the gent from the IRC channel):

@load base/protocols/smtp
@load base/utils

event mime_entity_data(c:connection, length: count, data:string)
        { print find_all_urls(data); }

But that?s all I got so far.  I?ve spent a good portion of the morning reading the docs at:

http://www.bro.org/sphinx-beta/scripting/index.html#understanding-bro-scripts

And I?m still pretty much at the same spot I was at?completely lost :D.  My understanding is that I need to create a new .bro script, and then add a redef in my in my local.bro, but that?s the extent of my knowledge at this point.  Any help would really be appreciated.  Thank you.

James

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131107/5766d639/attachment.bin 

From hunarame at gmail.com  Thu Nov  7 10:48:02 2013
From: hunarame at gmail.com (Naveed Anwar)
Date: Thu, 7 Nov 2013 23:48:02 +0500
Subject: [Bro] Traffic Volume Calculation Using Bro's Connection Log
Message-ID: <CANg8oi8aRob-KrsUYjvx-_z=9NCF4_hZZwZ+DW=LGy-iMs4npg@mail.gmail.com>

Hi,

I'm facing a small problem when running Bro. I'm trying to calculate the
volume of traffic generated per host. I have a set of pcap files, each
containing traffic from a single host. I thought I could run Bro on each
pcap file, and then sum the orig_bytes and resp_bytes columns in conn.log
to get the total volume of traffic for one host. However when I run Bro on
a 250 MB pcap file, the sum of these two columns  is only 107 MB
approximately, and not 250 MB as I expected. Is there any alternate method
for calculating the volume of traffic generated by one host?

Here's the script I ran to get the sum:
cat conn.log | awk 'BEGIN{FS="\t"; count=0;} {count=count+$10; count+=$11}
END {print count;}'

This was the output of the script (which I expected would be 250 MB
instead):
107790112 bytes

It would be great if you could help me resolve this issue!

Thank you,
Zainab
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131107/4b0d1c37/attachment.html 

From hunarame at gmail.com  Thu Nov  7 10:58:02 2013
From: hunarame at gmail.com (Naveed Anwar)
Date: Thu, 7 Nov 2013 23:58:02 +0500
Subject: [Bro] Traffic Volume Calculation Using Bro's Connection Log
Message-ID: <CANg8oi_3arFHhpQii-m+SV21AampZzJuBvzgm4weXbM78DWnTg@mail.gmail.com>

Hi,
I'm facing a small problem when running Bro. I'm trying to calculate the
volume of traffic generated per host. I have a set of pcap files, each
containing traffic from a single host. I thought I could run Bro on each
pcap file, and then sum the orig_bytes and resp_bytes columns in conn.log
to get the total volume of traffic for one host. However when I run Bro on
a 250 MB pcap file, the sum of these two columns  is only 107 MB
approximately, and not 250 MB as I expected. Is there any alternate method
for calculating the volume of traffic generated by one host?

Here's the script I ran to get the sum:
cat conn.log | awk 'BEGIN{FS="\t"; count=0;} {count=count+$10; count+=$11}
END {print count;}'

This was the output of the script (which I expected would be 250 MB
instead):
107790112 bytes

It would be great if you could help me resolve this issue!

Thank you,

-- 
Regards,
Naveed Anwar Bhatti
Research Associate
FAST-NU Islamabad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131107/87f32258/attachment.html 

From robin at icir.org  Thu Nov  7 11:13:44 2013
From: robin at icir.org (Robin Sommer)
Date: Thu, 7 Nov 2013 11:13:44 -0800
Subject: [Bro] Bro 2.2 has arrived
Message-ID: <20131107191344.GD33505@icir.org>

See the blog posting:

    http://blog.bro.org/2013/11/bro-22.html

Robin    

-- 
Robin Sommer * Phone +1 (510) 722-6541 *     robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 * www.icir.org/robin


From seth at icir.org  Thu Nov  7 11:16:33 2013
From: seth at icir.org (Seth Hall)
Date: Thu, 7 Nov 2013 14:16:33 -0500
Subject: [Bro] Traffic Volume Calculation Using Bro's Connection Log
In-Reply-To: <CANg8oi8aRob-KrsUYjvx-_z=9NCF4_hZZwZ+DW=LGy-iMs4npg@mail.gmail.com>
References: <CANg8oi8aRob-KrsUYjvx-_z=9NCF4_hZZwZ+DW=LGy-iMs4npg@mail.gmail.com>
Message-ID: <20A3C9DE-100A-4820-A84F-6088D5810CC3@icir.org>


On Nov 7, 2013, at 1:48 PM, Naveed Anwar <hunarame at gmail.com> wrote:

>  I thought I could run Bro on each pcap file, and then sum the orig_bytes and resp_bytes columns in conn.log to get the total volume of traffic for one host. However when I run Bro on a 250 MB pcap file, the sum of these two columns  is only 107 MB approximately, and not 250 MB as I expected.

It's a matter of overhead and unmeasured data.  The orig_bytes and resp_bytes is only counting payload bytes so all of the headers (i.e. tcp, udp, icmp, ip, ethernet, etc) are not counted.  Also, if you have any packet types that we don't support those won't be counted either.  There is also some amount of overhead inherent in PCAP.

> Is there any alternate method for calculating the volume of traffic generated by one host?

You are going to need to be more specific about what you are looking for.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131107/d6fb8d4e/attachment.bin 

From michal at rsbac.org  Thu Nov  7 11:31:26 2013
From: michal at rsbac.org (Michal Purzynski)
Date: Thu, 07 Nov 2013 20:31:26 +0100
Subject: [Bro] Bro 2.2 has arrived
In-Reply-To: <20131107191344.GD33505@icir.org>
References: <20131107191344.GD33505@icir.org>
Message-ID: <527BEA8E.5000803@rsbac.org>

On 11/7/13, 8:13 PM, Robin Sommer wrote:
> See the blog posting:
>
>      http://blog.bro.org/2013/11/bro-22.html
>
> Robin
>
The world will be a better place!



From jbabio at po-box.esu.edu  Thu Nov  7 11:49:59 2013
From: jbabio at po-box.esu.edu (John Babio)
Date: Thu, 7 Nov 2013 19:49:59 +0000
Subject: [Bro] match on dns "malicious domain" and old browsers
In-Reply-To: <mailman.16854.1383850704.921.bro@bro.org>
Message-ID: <CEA15835.5670%jbabio@po-box.esu.edu>


Are there any scripts created already for matching and generating a notice
based on bad domains being queried? Also, I create a signature for
matching against a user agent string but I see this info is already pulled
in the logs. How can I create a notice for matching against something like
IE 6 being used?




From connar.rosebraugh at egov.com  Thu Nov  7 12:27:25 2013
From: connar.rosebraugh at egov.com (Rosebraugh, Connar)
Date: Thu, 7 Nov 2013 20:27:25 +0000
Subject: [Bro] Bro and Counting DNS rcodes
Message-ID: <088ED11BA811374BACE1259396F485E3DB74CD@VADC-MBX02.ad.cdc.nicusa.com>

I am trying to use Bro to count DNS rcodes, but it is returning numbers that are not correct. I am using the dns_message() event to collect the DNS messages, and I am using a pcap of 5000 packets that are all on port 53. After inspecting the packets in wireshark, I found that there were ~600 query results where rcode == 3. However, after running my script, not only did Bro only find 1 rcode == 3, but it only counted 2497 DNS messages. Is there something that I am missing?

Attached is the script that I am using to collect the rcodes. If you see some glaring logical error, please let me know.

Thanks,
Connar Rosebraugh
Intern, Security Operations
NICUSA, Inc.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131107/9fee020c/attachment.html 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test.bro.txt
Url: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131107/9fee020c/attachment.txt 

From jsiwek at illinois.edu  Thu Nov  7 12:34:26 2013
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Thu, 7 Nov 2013 20:34:26 +0000
Subject: [Bro] Notice suppression in beta
In-Reply-To: <527BD79E.5010806@louruppert.com>
References: <527BD79E.5010806@louruppert.com>
Message-ID: <B160762B-5E95-44D6-9903-2FE10624672F@illinois.edu>


On Nov 7, 2013, at 12:10 PM, Lou RUPPERT <himself at louruppert.com> wrote:

> But still I see notices coming through with IPs in the netblocks
> listed and with a note for SSL::Invalid_Server_Cert. Shouldn't a break
> issued from a hook with a greater priority than the default process
> prevent the notice from being logged?

There?s a priority 10 notice policy hook that configures some actions to take depending on the value of n$note, and by default it adds the logging action (to be performed later).

So either ?break?ing from a hook with priority greater than 10 or ?delete n$actions[Notice::ACTION_LOG]? from one with lower priority should prevent a notice from being logged.  The former would also prevent any email/alarm actions associated with the notice type.

- Jon


From liam at broala.com  Thu Nov  7 12:44:27 2013
From: liam at broala.com (Liam Randall)
Date: Thu, 7 Nov 2013 15:44:27 -0500
Subject: [Bro] Bro and Counting DNS rcodes
In-Reply-To: <088ED11BA811374BACE1259396F485E3DB74CD@VADC-MBX02.ad.cdc.nicusa.com>
References: <088ED11BA811374BACE1259396F485E3DB74CD@VADC-MBX02.ad.cdc.nicusa.com>
Message-ID: <CAGGyr=5CZZcXdveOOY6JjQuYVL=7pxij6GwGcV8F16htfk+0hQ@mail.gmail.com>

Conner are you on 2.1?  There was a bug that has been fixed in the current
code base.

You could also simply summarize the existing dns.log with something like
this:

[bro at new-host-3 dns-ad-bruteforce]$ less dns.log | bro-cut rcode rcode_name
| sort | uniq -c | sort -n
     32 - -
   1704 0 NOERROR
   2279 3 NXDOMAIN


The columns are Count / Return Code / Return Code Name.

Thanks,

Liam Randall


On Thu, Nov 7, 2013 at 3:27 PM, Rosebraugh, Connar <
connar.rosebraugh at egov.com> wrote:

>  I am trying to use Bro to count DNS rcodes, but it is returning numbers
> that are not correct. I am using the dns_message() event to collect the DNS
> messages, and I am using a pcap of 5000 packets that are all on port 53.
> After inspecting the packets in wireshark, I found that there were ~600
> query results where rcode == 3. However, after running my script, not only
> did Bro only find 1 rcode == 3, but it only counted 2497 DNS messages. Is
> there something that I am missing?
>
>
>
> Attached is the script that I am using to collect the rcodes. If you see
> some glaring logical error, please let me know.
>
>
>
> Thanks,
>
> Connar Rosebraugh
>
> Intern, Security Operations
>
> NICUSA, Inc.
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Liam Randall
Managing Partner
510-281-0760
www.Broala.com <http://www.broala.com/>
>From the creators of Bro <http://www.bro.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131107/e56d002d/attachment.html 

From hiren.panchasara at gmail.com  Thu Nov  7 22:18:10 2013
From: hiren.panchasara at gmail.com (hiren panchasara)
Date: Thu, 7 Nov 2013 22:18:10 -0800
Subject: [Bro] Bro 2.2 has arrived
In-Reply-To: <20131107191344.GD33505@icir.org>
References: <20131107191344.GD33505@icir.org>
Message-ID: <CALCpEUHwpOdOPLuKM38VJssWEDA_Tk1+j4gEMB8+z-0SE+ftbA@mail.gmail.com>

On Thu, Nov 7, 2013 at 11:13 AM, Robin Sommer <robin at icir.org> wrote:
> See the blog posting:
>
>     http://blog.bro.org/2013/11/bro-22.html

Awesome.

Waiting for FreeBSD security/bro port update!

Thanks,
Hiren


From jlay at slave-tothe-box.net  Fri Nov  8 06:25:07 2013
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 8 Nov 2013 07:25:07 -0700
Subject: [Bro] Links in SMTP round 2
Message-ID: <AEFB0140-47C8-4C71-BB32-BCAD01FED05E@slave-tothe-box.net>

So here?s where I?m at:

event bro_init()
    {
    local filter: Log::Filter = [$name="smtp-http", $path="smtp-http", $include=set("ts", "uid", "id.orig_h", "id.orig_p", "id.resp_h", "id.resp_p", "mailfrom", "rcptto", "date", "from", "to", "reply_to"
, "msg_id", "subject")];
    Log::add_filter(SMTP::LOG, filter);
    }

redef record SMTP::Info += {
    smtp_http: string &log;
};

event mime_entity_data(c:connection, length: count, data:string)

My snags are:

error in /usr/local/bro/share/bro/base/protocols/smtp/./main.bro, line 10: extension field must be &optional or have &default (SMTP::Info)
error in ./testfiles/test.bro, line 12: syntax error, at end of file

I?m hoping the first error is because I haven?t defined the new field of smtp_http yet.  As for the second, I?m not sure how to create that field.  I?ve been looking heavily at http://www.bro.org/sphinx-git/frameworks/logging.html, but so far this is all I have.  ANY help?tutorials?pointers?something would really save me some time.  Thank you.

James
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131108/ee41cf47/attachment.bin 

From JAzoff at albany.edu  Fri Nov  8 07:07:30 2013
From: JAzoff at albany.edu (Justin Azoff)
Date: Fri, 8 Nov 2013 10:07:30 -0500
Subject: [Bro] Links in SMTP round 2
In-Reply-To: <AEFB0140-47C8-4C71-BB32-BCAD01FED05E@slave-tothe-box.net>
References: <AEFB0140-47C8-4C71-BB32-BCAD01FED05E@slave-tothe-box.net>
Message-ID: <20131108150730.GP4436@datacomm.albany.edu>

On Fri, Nov 08, 2013 at 07:25:07AM -0700, James Lay wrote:
> error in /usr/local/bro/share/bro/base/protocols/smtp/./main.bro, line 10: extension field must be &optional or have &default (SMTP::Info)

Yep.. you need to mark it as &optional like it says.

> error in ./testfiles/test.bro, line 12: syntax error, at end of file

You just need to handle that event and extract the links.

> I?m hoping the first error is because I haven?t defined the new field of smtp_http yet.  As for the second, I?m not sure how to create that field.  I?ve been looking heavily at http://www.bro.org/sphinx-git/frameworks/logging.html, but so far this is all I have.  ANY help?tutorials?pointers?something would really save me some time.  Thank you.

Here is a script that adds a field to the conn log, it does all the
things you need to do:

https://github.com/JustinAzoff/bro_scripts/blob/master/conn-hostnames.bro

-- 
-- Justin Azoff
-- Network Security & Performance Analyst


From jlay at slave-tothe-box.net  Fri Nov  8 07:57:38 2013
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 8 Nov 2013 08:57:38 -0700
Subject: [Bro] Links in SMTP round 2
In-Reply-To: <20131108150730.GP4436@datacomm.albany.edu>
References: <AEFB0140-47C8-4C71-BB32-BCAD01FED05E@slave-tothe-box.net>
	<20131108150730.GP4436@datacomm.albany.edu>
Message-ID: <CE0B0EA3-2F54-4895-BF42-69BA6EBB882A@slave-tothe-box.net>


On Nov 8, 2013, at 8:07 AM, Justin Azoff <JAzoff at albany.edu> wrote:

> On Fri, Nov 08, 2013 at 07:25:07AM -0700, James Lay wrote:
>> error in /usr/local/bro/share/bro/base/protocols/smtp/./main.bro, line 10: extension field must be &optional or have &default (SMTP::Info)
> 
> Yep.. you need to mark it as &optional like it says.
> 
>> error in ./testfiles/test.bro, line 12: syntax error, at end of file
> 
> You just need to handle that event and extract the links.
> 
>> I?m hoping the first error is because I haven?t defined the new field of smtp_http yet.  As for the second, I?m not sure how to create that field.  I?ve been looking heavily at http://www.bro.org/sphinx-git/frameworks/logging.html, but so far this is all I have.  ANY help?tutorials?pointers?something would really save me some time.  Thank you.
> 
> Here is a script that adds a field to the conn log, it does all the
> things you need to do:
> 
> https://github.com/JustinAzoff/bro_scripts/blob/master/conn-hostnames.bro
> 
> -- 
> -- Justin Azoff
> -- Network Security & Performance Analyst

Thanks a BUNCH Justin?this helps.  As I?m looking at this, I think what I?m hoping for, is something like:

"if the smtp message stream contains http, then log the link to smtp_http.log, otherwise don?t log anything about the stream to smtp_http.log"

Something I?m stumbling on is?how do I specify the smtp stream, and how do I find out if it contains http ( looking at the bro cheat sheet I don?t see ?=~? ).  Again, thanks so much Justin?I think I?m getting closer.

James
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131108/afe683f8/attachment.bin 

From JAzoff at albany.edu  Fri Nov  8 08:31:30 2013
From: JAzoff at albany.edu (Justin Azoff)
Date: Fri, 8 Nov 2013 11:31:30 -0500
Subject: [Bro] Links in SMTP round 2
In-Reply-To: <CE0B0EA3-2F54-4895-BF42-69BA6EBB882A@slave-tothe-box.net>
References: <AEFB0140-47C8-4C71-BB32-BCAD01FED05E@slave-tothe-box.net>
	<20131108150730.GP4436@datacomm.albany.edu>
	<CE0B0EA3-2F54-4895-BF42-69BA6EBB882A@slave-tothe-box.net>
Message-ID: <20131108163130.GQ4436@datacomm.albany.edu>

On Fri, Nov 08, 2013 at 08:57:38AM -0700, James Lay wrote:
> Thanks a BUNCH Justin?this helps.  As I?m looking at this, I think what I?m hoping for, is something like:
> 
> "if the smtp message stream contains http, then log the link to smtp_http.log, otherwise don?t log anything about the stream to smtp_http.log"
> 
> Something I?m stumbling on is?how do I specify the smtp stream, and how do I find out if it contains http ( looking at the bro cheat sheet I don?t see ?=~? ).  Again, thanks so much Justin?I think I?m getting closer.
> 
> James

You pasted how to do this in your first message:

event mime_entity_data(c:connection, length: count, data:string)                                                                                                                                           
        { print find_all_urls(data); }                                                                                                                                                                     

The only tricky part is find_all_urls would return a vector so your log
field needs to be a 'vector of string' and not just a 'string'



-- 
-- Justin Azoff
-- Network Security & Performance Analyst


From jlay at slave-tothe-box.net  Fri Nov  8 08:34:51 2013
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 8 Nov 2013 09:34:51 -0700
Subject: [Bro] Links in SMTP round 2
In-Reply-To: <20131108163130.GQ4436@datacomm.albany.edu>
References: <AEFB0140-47C8-4C71-BB32-BCAD01FED05E@slave-tothe-box.net>
	<20131108150730.GP4436@datacomm.albany.edu>
	<CE0B0EA3-2F54-4895-BF42-69BA6EBB882A@slave-tothe-box.net>
	<20131108163130.GQ4436@datacomm.albany.edu>
Message-ID: <88763BFA-0490-49BE-A282-B3EC9AADFBE5@slave-tothe-box.net>


On Nov 8, 2013, at 9:31 AM, Justin Azoff <JAzoff at albany.edu> wrote:

> On Fri, Nov 08, 2013 at 08:57:38AM -0700, James Lay wrote:
>> Thanks a BUNCH Justin?this helps.  As I?m looking at this, I think what I?m hoping for, is something like:
>> 
>> "if the smtp message stream contains http, then log the link to smtp_http.log, otherwise don?t log anything about the stream to smtp_http.log"
>> 
>> Something I?m stumbling on is?how do I specify the smtp stream, and how do I find out if it contains http ( looking at the bro cheat sheet I don?t see ?=~? ).  Again, thanks so much Justin?I think I?m getting closer.
>> 
>> James
> 
> You pasted how to do this in your first message:
> 
> event mime_entity_data(c:connection, length: count, data:string)                                                                                                                                           
>        { print find_all_urls(data); }                                                                                                                                                                     
> 
> The only tricky part is find_all_urls would return a vector so your log
> field needs to be a 'vector of string' and not just a 'string'
> 
> 
> 
> -- 
> -- Justin Azoff
> -- Network Security & Performance Analyst


Awesome?thank you much Justin.

James
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131108/d49a27e9/attachment.bin 

From knrd at rogers.com  Mon Nov 11 06:57:16 2013
From: knrd at rogers.com (Konrad Weglowski)
Date: Mon, 11 Nov 2013 09:57:16 -0500
Subject: [Bro] BRO hourly summary connectivity reports - geoip?
Message-ID: <002a01cedeee$5092e530$f1b8af90$@com>

Hello,

Is there a way to customize/add items that are e-mailed as a part of the BRO
hourly connection summary reports? I would like to add geoip lookups for
example?

Thank You,

Konrad

 

  _____  

No virus found in this message.
Checked by AVG - www.avg.com
Version: 2014.0.4158 / Virus Database: 3629/6826 - Release Date: 11/11/13

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131111/bf2faf64/attachment.html 

From JAzoff at albany.edu  Mon Nov 11 07:20:53 2013
From: JAzoff at albany.edu (Justin Azoff)
Date: Mon, 11 Nov 2013 10:20:53 -0500
Subject: [Bro] BRO hourly summary connectivity reports - geoip?
In-Reply-To: <002a01cedeee$5092e530$f1b8af90$@com>
References: <002a01cedeee$5092e530$f1b8af90$@com>
Message-ID: <20131111152053.GB15771@datacomm.albany.edu>

On Mon, Nov 11, 2013 at 09:57:16AM -0500, Konrad Weglowski wrote:
> Hello,
> 
> Is there a way to customize/add items that are e-mailed as a part of the BRO
> hourly connection summary reports? I would like to add geoip lookups for
> example?


frameworks/notice/actions/add-geodata.bro does this.

you just need to do something like:

redef Notice::lookup_location_types += {
    HTTP::MD5,
    HTTP::Incorrect_File_Type,
    HTTP::IncorrectFileTypeBadHost,
    HTTP::Sensitive_URI,
    HTTP::ResetConnection,
};

or write a notice hook that adds ACTION_ADD_GEODATA to all notices.
which means taking this:

hook policy(n: Notice::Info) &priority=10
{
    if ( n$note in Notice::lookup_location_types )
        add n$actions[ACTION_ADD_GEODATA];
}

and adding one of your own like

hook policy(n: Notice::Info) &priority=10
{
    add n$actions[ACTION_ADD_GEODATA];
}

-- 
-- Justin Azoff
-- Network Security & Performance Analyst


From la_arshadi at yahoo.com  Mon Nov 11 22:19:08 2013
From: la_arshadi at yahoo.com (Laleh Arshadi)
Date: Mon, 11 Nov 2013 22:19:08 -0800 (PST)
Subject: [Bro] Bro and flood protection - revisited
In-Reply-To: <20131107113540.GS32654@eldamar.org.uk>
References: <20131107113540.GS32654@eldamar.org.uk> 
Message-ID: <1384237148.40495.YahooMailNeo@web140606.mail.bf1.yahoo.com>

Dear All,
?
This message was sent a while ago but I see no one has replied to it. As I have almost a similar question myself, I would be thankful if someone took another look at the email and responded.
?
Regards
Laleh


________________________________
From: Alexander Frolkin <avf at eldamar.org.uk>
To: bro at bro.org 
Sent: Thursday, November 7, 2013 3:05 PM
Subject: [Bro] Bro and flood protection


Hi,

I'm currently looking around for open-source IDSes.? What we'd like
is to have an IDS machine which monitors our Internet traffic and
responds to events by blocking the traffic using Flowspec.? This is easy
to do with Bro and ExaBGP using custom event handlers and/or hooks, and
piped_exec.

I'm currently trying to understand Bro's ability to detect floods, e.g.,
SYN flood, ACK flood, or any other kind of flood, for that matter.

The feeling I have so far is that Bro wasn't really designed for this
sort of thing, and that it's designed more for L7 stuff.

I'm playing with 2.2 beta, and I can't see anything built-in to detect
floods (although maybe I haven't looked hard enough).? In older
versions, though, there was a script called synflood.bro, but it seems
to have disappeared at some point.? Does anyone know what the history of
this is, and whether there is equivalent funtionality in the latest
version?

More generally, if I want to detect network floods, is Bro the right
thing to be using, or should I be looking elsewhere?

Thanks!


Alex

_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131111/23205983/attachment.html 

From mattchess50 at gmail.com  Wed Nov 13 07:11:49 2013
From: mattchess50 at gmail.com (Matt Stucky)
Date: Wed, 13 Nov 2013 09:11:49 -0600
Subject: [Bro] hook vs. redef
Message-ID: <CAMM5JYpfOwwo=mjB1Y7vc43kAFmc1ZPtcYgaWh9BR0v8=FDmBg@mail.gmail.com>

In an older implementation of Bro we had some lines in our site file that
would "redef" a notice policy to add criteria to the notice, i.e. if the
notice was for a SQL_Injection_Victim AND the resp_h was in a particular
subnet, then trigger the notice.  I've been testing 2.2 (the upgrade from
2.1 to 2.2 went smoothly) and trying to figure out the best way to
duplicate that functionality.  It seems it would be done with a hook, but
do I have to first add it to ignored_types and then re-raise it?  Or am I
barking up the wrong tree entirely?

In a general sense I guess I'm asking how best to modify the criteria for
an existing notice?

Thanks,
Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131113/d25116a3/attachment.html 

From jlay at slave-tothe-box.net  Wed Nov 13 07:30:41 2013
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 13 Nov 2013 08:30:41 -0700
Subject: [Bro] Additional service ports
Message-ID: <5728a51426b125d9004f8e752bbbc45e@localhost>

Hey all,

Quick question...say I have http traffic on the usual 80, and 8001?  
How does one tell Bro that there's an additional port to analyze?  Thank 
you.

James


From anthony.kasza at gmail.com  Wed Nov 13 07:59:22 2013
From: anthony.kasza at gmail.com (anthony kasza)
Date: Wed, 13 Nov 2013 07:59:22 -0800
Subject: [Bro] Additional service ports
In-Reply-To: <5728a51426b125d9004f8e752bbbc45e@localhost>
References: <5728a51426b125d9004f8e752bbbc45e@localhost>
Message-ID: <CAEZw2bzm7MRwnV55V4+g88aJqoXfZjD+NmPDTYZzikjLa5YyTw@mail.gmail.com>

The 'Determining Analyzer Activation' section in the following link might
be helpful.
http://www.bro.org/development/howtos/dpd.html
Hey all,

Quick question...say I have http traffic on the usual 80, and 8001?
How does one tell Bro that there's an additional port to analyze?  Thank
you.

James
_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131113/3a4cdff5/attachment.html 

From seth at icir.org  Wed Nov 13 09:39:14 2013
From: seth at icir.org (Seth Hall)
Date: Wed, 13 Nov 2013 12:39:14 -0500
Subject: [Bro] Additional service ports
In-Reply-To: <CAEZw2bzm7MRwnV55V4+g88aJqoXfZjD+NmPDTYZzikjLa5YyTw@mail.gmail.com>
References: <5728a51426b125d9004f8e752bbbc45e@localhost>
	<CAEZw2bzm7MRwnV55V4+g88aJqoXfZjD+NmPDTYZzikjLa5YyTw@mail.gmail.com>
Message-ID: <D9DE322E-988C-4079-A7AB-0826E0D1D83D@icir.org>


On Nov 13, 2013, at 10:59 AM, anthony kasza <anthony.kasza at gmail.com> wrote:

> The 'Determining Analyzer Activation' section in the following link might be helpful.
> http://www.bro.org/development/howtos/dpd.html

Oops!  That section should have been deprecated with 2.2, it doesn't work that way anymore. You can do the following, but I'm not sure if it overwrites the default list of ports used or not...

event bro_init()
	{
	Analyzer::register_for_ports(Analyzer::ANALYZER_HTTP, set(12345/tcp));
	}

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131113/07792a8e/attachment.bin 

From jsiwek at illinois.edu  Wed Nov 13 10:10:40 2013
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Wed, 13 Nov 2013 18:10:40 +0000
Subject: [Bro] Additional service ports
In-Reply-To: <D9DE322E-988C-4079-A7AB-0826E0D1D83D@icir.org>
References: <5728a51426b125d9004f8e752bbbc45e@localhost>
	<CAEZw2bzm7MRwnV55V4+g88aJqoXfZjD+NmPDTYZzikjLa5YyTw@mail.gmail.com>
	<D9DE322E-988C-4079-A7AB-0826E0D1D83D@icir.org>
Message-ID: <21EC6392-D10E-4E55-B356-206F6E5E4EAF@illinois.edu>


On Nov 13, 2013, at 11:39 AM, Seth Hall <seth at icir.org> wrote:

> You can do the following, but I'm not sure if it overwrites the default list of ports used or not...
> 
> event bro_init()
> 	{
> 	Analyzer::register_for_ports(Analyzer::ANALYZER_HTTP, set(12345/tcp));
> 	}

It?s additive; doesn?t overwrite.

- Jon


From jlay at slave-tothe-box.net  Wed Nov 13 10:20:01 2013
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 13 Nov 2013 11:20:01 -0700
Subject: [Bro] Additional service ports
In-Reply-To: <D9DE322E-988C-4079-A7AB-0826E0D1D83D@icir.org>
References: <5728a51426b125d9004f8e752bbbc45e@localhost>
	<CAEZw2bzm7MRwnV55V4+g88aJqoXfZjD+NmPDTYZzikjLa5YyTw@mail.gmail.com>
	<D9DE322E-988C-4079-A7AB-0826E0D1D83D@icir.org>
Message-ID: <3bf311cb4cb51478a4b204418084235a@localhost>

On 2013-11-13 10:39, Seth Hall wrote:
> On Nov 13, 2013, at 10:59 AM, anthony kasza <anthony.kasza at gmail.com> 
> wrote:
>
>> The 'Determining Analyzer Activation' section in the following link 
>> might be helpful.
>> http://www.bro.org/development/howtos/dpd.html
>
> Oops!  That section should have been deprecated with 2.2, it doesn't
> work that way anymore. You can do the following, but I'm not sure if
> it overwrites the default list of ports used or not...
>
> event bro_init()
> 	{
> 	Analyzer::register_for_ports(Analyzer::ANALYZER_HTTP, 
> set(12345/tcp));
> 	}
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/

Thanks Seth...I'll give that a go.

James


From jsiwek at illinois.edu  Wed Nov 13 10:41:50 2013
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Wed, 13 Nov 2013 18:41:50 +0000
Subject: [Bro] hook vs. redef
In-Reply-To: <CAMM5JYpfOwwo=mjB1Y7vc43kAFmc1ZPtcYgaWh9BR0v8=FDmBg@mail.gmail.com>
References: <CAMM5JYpfOwwo=mjB1Y7vc43kAFmc1ZPtcYgaWh9BR0v8=FDmBg@mail.gmail.com>
Message-ID: <4A4DAE9D-A399-4D84-A973-7AAE93CE04E0@illinois.edu>


On Nov 13, 2013, at 9:11 AM, Matt Stucky <mattchess50 at gmail.com> wrote:

> In an older implementation of Bro we had some lines in our site file that would "redef" a notice policy to add criteria to the notice, i.e. if the notice was for a SQL_Injection_Victim AND the resp_h was in a particular subnet, then trigger the notice.  I've been testing 2.2 (the upgrade from 2.1 to 2.2 went smoothly) and trying to figure out the best way to duplicate that functionality.  It seems it would be done with a hook, but do I have to first add it to ignored_types and then re-raise it?  Or am I barking up the wrong tree entirely?
> 
> In a general sense I guess I'm asking how best to modify the criteria for an existing notice?

To conditionally ignore notices, you can generally handle a Notice::policy hook at a &priority greater than 10 and ?break? from the hook if the notice meets criteria you deem uninteresting.  That will abort all the default notice handling for that particular notice.

More documentation on hooks at: http://bro.org/sphinx/scripts/builtins.html#type-hook

- Jon


From deltiongco at gmail.com  Wed Nov 13 12:29:20 2013
From: deltiongco at gmail.com (Del T)
Date: Wed, 13 Nov 2013 12:29:20 -0800
Subject: [Bro] Send emails from BRO Notices
Message-ID: <CAPMDeN21yZ_hy7cPQTz1RZarm8J4qC7MLyTVFuaS3BZw4qpjNA@mail.gmail.com>

Hi All,

I can't get the BRO Notices emails to work in my current Security Onion
12.04 set up. I have the below line in /opt/bro/share/bro/site/local.bro
UNcommented.

# redef Notice::policy += { [$action = Notice::ACTION_ALARM, $priority = 0] };



 I am getting the BRO connection summary emails regularly after I changed
the /opt/bro/etc/broctl.cfg.

Is there anything I am missing in my configuration?

Thanks in advance.

Del Tiongco
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131113/95bebc0f/attachment.html 

From knrd at rogers.com  Wed Nov 13 16:01:23 2013
From: knrd at rogers.com (Konrad Weglowski)
Date: Wed, 13 Nov 2013 19:01:23 -0500
Subject: [Bro] BRO hourly summary connectivity reports - geoip?
In-Reply-To: <20131111152053.GB15771@datacomm.albany.edu>
References: <002a01cedeee$5092e530$f1b8af90$@com>
	<20131111152053.GB15771@datacomm.albany.edu>
Message-ID: <005c01cee0cc$a92b23d0$fb816b70$@com>

Thanks Justin.

I am a newbie to BRO. Where would I put that code? Would that be a separate
script which would need to be loaded in local.bro?

-----Original Message-----
From: Justin Azoff [mailto:JAzoff at albany.edu] 
Sent: November-11-13 10:21 AM
To: Konrad Weglowski
Cc: bro at bro.org
Subject: Re: [Bro] BRO hourly summary connectivity reports - geoip?

On Mon, Nov 11, 2013 at 09:57:16AM -0500, Konrad Weglowski wrote:
> Hello,
> 
> Is there a way to customize/add items that are e-mailed as a part of 
> the BRO hourly connection summary reports? I would like to add geoip 
> lookups for example?


frameworks/notice/actions/add-geodata.bro does this.

you just need to do something like:

redef Notice::lookup_location_types += {
    HTTP::MD5,
    HTTP::Incorrect_File_Type,
    HTTP::IncorrectFileTypeBadHost,
    HTTP::Sensitive_URI,
    HTTP::ResetConnection,
};

or write a notice hook that adds ACTION_ADD_GEODATA to all notices.
which means taking this:

hook policy(n: Notice::Info) &priority=10 {
    if ( n$note in Notice::lookup_location_types )
        add n$actions[ACTION_ADD_GEODATA]; }

and adding one of your own like

hook policy(n: Notice::Info) &priority=10 {
    add n$actions[ACTION_ADD_GEODATA];
}

--
-- Justin Azoff
-- Network Security & Performance Analyst
-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2014.0.4158 / Virus Database: 3629/6832 - Release Date: 11/13/13

-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2014.0.4158 / Virus Database: 3629/6832 - Release Date: 11/13/13



From JAzoff at albany.edu  Wed Nov 13 19:49:00 2013
From: JAzoff at albany.edu (Justin Azoff)
Date: Wed, 13 Nov 2013 22:49:00 -0500
Subject: [Bro] BRO hourly summary connectivity reports - geoip?
In-Reply-To: <005c01cee0cc$a92b23d0$fb816b70$@com>
References: <002a01cedeee$5092e530$f1b8af90$@com>
	<20131111152053.GB15771@datacomm.albany.edu>
	<005c01cee0cc$a92b23d0$fb816b70$@com>
Message-ID: <20131114034900.GC15771@datacomm.albany.edu>

On Wed, Nov 13, 2013 at 07:01:23PM -0500, Konrad Weglowski wrote:
> Thanks Justin.
> 
> I am a newbie to BRO. Where would I put that code? Would that be a separate
> script which would need to be loaded in local.bro?

Yep.. You could add the lines directly to local.bro but that gets messy
fast.  I would make a 'notice-locations.bro' in the site directory what
contains the location specific tweaks, then add a simple

    @load notice-locations

in local.bro

-- 
-- Justin Azoff
-- Network Security & Performance Analyst


From mattchess50 at gmail.com  Thu Nov 14 08:53:46 2013
From: mattchess50 at gmail.com (Matt Stucky)
Date: Thu, 14 Nov 2013 10:53:46 -0600
Subject: [Bro] customize msg in a Notice hook
Message-ID: <CAMM5JYrJbas4WH98UA9Ss2ryYWk9ZU52+iR8Td0fYknFLF=FvA@mail.gmail.com>

How would one go about customizing the message for a notice when it matches
specific criteria?

Here's what I've tried:

hook Notice::policy(n: Notice::Info)
        {
        if ( n$note == <Notice> && <additional criteria> )
              add n$actions[Notice::ACTION_EMAIL];
              n$msg=<custom message>;
        }

However, that changes the message for every notice in the notice log...  is
there a way to scope that so it changes the message only for that one
notice instance?

Thanks,
Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131114/436d3cf4/attachment.html 

From jsiwek at illinois.edu  Thu Nov 14 09:54:52 2013
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Thu, 14 Nov 2013 17:54:52 +0000
Subject: [Bro] customize msg in a Notice hook
In-Reply-To: <CAMM5JYrJbas4WH98UA9Ss2ryYWk9ZU52+iR8Td0fYknFLF=FvA@mail.gmail.com>
References: <CAMM5JYrJbas4WH98UA9Ss2ryYWk9ZU52+iR8Td0fYknFLF=FvA@mail.gmail.com>
Message-ID: <284B7BDA-AEE7-40B0-89BC-76576E986D03@illinois.edu>


On Nov 14, 2013, at 10:53 AM, Matt Stucky <mattchess50 at gmail.com> wrote:

> How would one go about customizing the message for a notice when it matches specific criteria?
> 
> Here's what I've tried:
> 
> hook Notice::policy(n: Notice::Info)
>         {
>         if ( n$note == <Notice> && <additional criteria> )
>               add n$actions[Notice::ACTION_EMAIL];
>               n$msg=<custom message>;
>         }
> 
> However, that changes the message for every notice in the notice log...  is there a way to scope that so it changes the message only for that one notice instance?

You need to use curly braces to make a compound statement, otherwise that last statement is unconditional.  E.g.:

	if ( ? criteria ... )
		{
		add n$action[?];
		n$msg = ?;
		}

- Jon


From mattchess50 at gmail.com  Thu Nov 14 11:06:12 2013
From: mattchess50 at gmail.com (Matt Stucky)
Date: Thu, 14 Nov 2013 13:06:12 -0600
Subject: [Bro] customize msg in a Notice hook
In-Reply-To: <284B7BDA-AEE7-40B0-89BC-76576E986D03@illinois.edu>
References: <CAMM5JYrJbas4WH98UA9Ss2ryYWk9ZU52+iR8Td0fYknFLF=FvA@mail.gmail.com>
	<284B7BDA-AEE7-40B0-89BC-76576E986D03@illinois.edu>
Message-ID: <CAMM5JYoQioADK4hhFD2E2yMCSfjWBJdfjxXf-VakrtBq61TW+Q@mail.gmail.com>

Ahh, so simple and so right.

Thanks!
-matt


On Thu, Nov 14, 2013 at 11:54 AM, Siwek, Jonathan Luke
<jsiwek at illinois.edu>wrote:

>
> On Nov 14, 2013, at 10:53 AM, Matt Stucky <mattchess50 at gmail.com> wrote:
>
> > How would one go about customizing the message for a notice when it
> matches specific criteria?
> >
> > Here's what I've tried:
> >
> > hook Notice::policy(n: Notice::Info)
> >         {
> >         if ( n$note == <Notice> && <additional criteria> )
> >               add n$actions[Notice::ACTION_EMAIL];
> >               n$msg=<custom message>;
> >         }
> >
> > However, that changes the message for every notice in the notice log...
>  is there a way to scope that so it changes the message only for that one
> notice instance?
>
> You need to use curly braces to make a compound statement, otherwise that
> last statement is unconditional.  E.g.:
>
>         if ( ? criteria ... )
>                 {
>                 add n$action[?];
>                 n$msg = ?;
>                 }
>
> - Jon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131114/da450724/attachment.html 

From gary at doit.wisc.edu  Thu Nov 14 20:26:28 2013
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Thu, 14 Nov 2013 22:26:28 -0600
Subject: [Bro] Possible Bro Cluster communication issue?
Message-ID: <5285A274.1050404@doit.wisc.edu>

Hello,

Another Bro newbie here. Having an odd issue getting my bro 2.2 
(release) cluster working properly. I have 2 physical hosts. The first 
host is running the manager, proxy, and some workers, and the second 
host is running several workers. After running broctl install and broctl 
start the workers spin up on both hosts, however, the workers on host 2 
don't seem to be reliably reporting back to the master or connecting to 
the proxy.

I confirmed that the processes were running on both hosts and that ssh 
sessions were established between the two hosts, but a broctl status 
only showed peers for workers on the same host as the manager, fewer 
peers than expected for the proxy (about as many as were on host1), and 
broctl netstat didn't return any results for the workers on the second 
host.

At some point the proxy crashed on my first run, and upon restarting 
everything I had the same results minus the proxy crash. Interestingly 
enough broctl capstats did return results for both hosts showing a 
relatively even workload of about 3Gbps each. Also, I didn't find any 
logs other than stderr and stdout on the second host in /bro/log or 
/bro/spool. Any thoughts?

Regards,

-- 
Gary Faulkner
UW Madison
Office of Campus Information Security
608-262-8591


From dnthayer at illinois.edu  Thu Nov 14 20:36:06 2013
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Thu, 14 Nov 2013 22:36:06 -0600
Subject: [Bro] Possible Bro Cluster communication issue?
In-Reply-To: <5285A274.1050404@doit.wisc.edu>
References: <5285A274.1050404@doit.wisc.edu>
Message-ID: <5285A4B6.6070007@illinois.edu>

On 11/14/2013 10:26 PM, Gary Faulkner wrote:
> Hello,
>
> Another Bro newbie here. Having an odd issue getting my bro 2.2
> (release) cluster working properly. I have 2 physical hosts. The first
> host is running the manager, proxy, and some workers, and the second
> host is running several workers. After running broctl install and broctl
> start the workers spin up on both hosts, however, the workers on host 2
> don't seem to be reliably reporting back to the master or connecting to
> the proxy.
>
> I confirmed that the processes were running on both hosts and that ssh
> sessions were established between the two hosts, but a broctl status
> only showed peers for workers on the same host as the manager, fewer
> peers than expected for the proxy (about as many as were on host1), and
> broctl netstat didn't return any results for the workers on the second
> host.
>
> At some point the proxy crashed on my first run, and upon restarting
> everything I had the same results minus the proxy crash. Interestingly
> enough broctl capstats did return results for both hosts showing a
> relatively even workload of about 3Gbps each. Also, I didn't find any
> logs other than stderr and stdout on the second host in /bro/log or
> /bro/spool. Any thoughts?
>
> Regards,
>

Did you check if a there's a firewall running on either host?
If so, you could try turning it off temporarily to see if that resolves 
the problem.



From gary at doit.wisc.edu  Thu Nov 14 21:20:48 2013
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Thu, 14 Nov 2013 23:20:48 -0600
Subject: [Bro] Possible Bro Cluster communication issue?
In-Reply-To: <5285A4B6.6070007@illinois.edu>
References: <5285A274.1050404@doit.wisc.edu> <5285A4B6.6070007@illinois.edu>
Message-ID: <5285AF30.6030709@doit.wisc.edu>

Both hosts are running host based FWs, but disabling them doesn't appear 
to make a difference in the behavior. I can ssh between hosts just fine 
as the bro user with key-based auth and broctl seems to open an ssh 
session per worker between the two hosts that appear stay established 
throughout just fine. Does all the communication happen over those ssh 
sessions or are there other types of connections happening between 
master/proxy and worker?

On 11/14/2013 10:36 PM, Daniel Thayer wrote:
> On 11/14/2013 10:26 PM, Gary Faulkner wrote:
>> Hello,
>>
>> Another Bro newbie here. Having an odd issue getting my bro 2.2
>> (release) cluster working properly. I have 2 physical hosts. The first
>> host is running the manager, proxy, and some workers, and the second
>> host is running several workers. After running broctl install and broctl
>> start the workers spin up on both hosts, however, the workers on host 2
>> don't seem to be reliably reporting back to the master or connecting to
>> the proxy.
>>
>> I confirmed that the processes were running on both hosts and that ssh
>> sessions were established between the two hosts, but a broctl status
>> only showed peers for workers on the same host as the manager, fewer
>> peers than expected for the proxy (about as many as were on host1), and
>> broctl netstat didn't return any results for the workers on the second
>> host.
>>
>> At some point the proxy crashed on my first run, and upon restarting
>> everything I had the same results minus the proxy crash. Interestingly
>> enough broctl capstats did return results for both hosts showing a
>> relatively even workload of about 3Gbps each. Also, I didn't find any
>> logs other than stderr and stdout on the second host in /bro/log or
>> /bro/spool. Any thoughts?
>>
>> Regards,
>>
>
> Did you check if a there's a firewall running on either host?
> If so, you could try turning it off temporarily to see if that resolves
> the problem.
>

-- 
Gary Faulkner
UW Madison
Office of Campus Information Security
608-262-8591


From gary at doit.wisc.edu  Fri Nov 15 00:05:40 2013
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Fri, 15 Nov 2013 02:05:40 -0600
Subject: [Bro] Possible Bro Cluster communication issue?
In-Reply-To: <5285AF30.6030709@doit.wisc.edu>
References: <5285A274.1050404@doit.wisc.edu> <5285A4B6.6070007@illinois.edu>
	<5285AF30.6030709@doit.wisc.edu>
Message-ID: <5285D5D4.3090008@doit.wisc.edu>

Actually, it was the firewall, but I also had a secondary problem in 
that the proxy was constantly crashing due a lack of system resources so 
it didn't initially appear that disabling the firewall relieved the 
communication problem. I didn't recall seeing any FW considerations 
beyond ssh in the documentation, but I did eventually find an external 
document at https://gist.github.com/grigorescu/3776670 and a quick 
netstat allowed me to confirm the ports on my hosts. Thanks for the help!



On 11/14/2013 11:20 PM, Gary Faulkner wrote:
> Both hosts are running host based FWs, but disabling them doesn't appear
> to make a difference in the behavior. I can ssh between hosts just fine
> as the bro user with key-based auth and broctl seems to open an ssh
> session per worker between the two hosts that appear stay established
> throughout just fine. Does all the communication happen over those ssh
> sessions or are there other types of connections happening between
> master/proxy and worker?
>
> On 11/14/2013 10:36 PM, Daniel Thayer wrote:
>> On 11/14/2013 10:26 PM, Gary Faulkner wrote:
>>> Hello,
>>>
>>> Another Bro newbie here. Having an odd issue getting my bro 2.2
>>> (release) cluster working properly. I have 2 physical hosts. The first
>>> host is running the manager, proxy, and some workers, and the second
>>> host is running several workers. After running broctl install and broctl
>>> start the workers spin up on both hosts, however, the workers on host 2
>>> don't seem to be reliably reporting back to the master or connecting to
>>> the proxy.
>>>
>>> I confirmed that the processes were running on both hosts and that ssh
>>> sessions were established between the two hosts, but a broctl status
>>> only showed peers for workers on the same host as the manager, fewer
>>> peers than expected for the proxy (about as many as were on host1), and
>>> broctl netstat didn't return any results for the workers on the second
>>> host.
>>>
>>> At some point the proxy crashed on my first run, and upon restarting
>>> everything I had the same results minus the proxy crash. Interestingly
>>> enough broctl capstats did return results for both hosts showing a
>>> relatively even workload of about 3Gbps each. Also, I didn't find any
>>> logs other than stderr and stdout on the second host in /bro/log or
>>> /bro/spool. Any thoughts?
>>>
>>> Regards,
>>>
>>
>> Did you check if a there's a firewall running on either host?
>> If so, you could try turning it off temporarily to see if that resolves
>> the problem.
>>
>

-- 
Gary Faulkner
UW Madison
Office of Campus Information Security
608-262-8591


From dnthayer at illinois.edu  Fri Nov 15 06:24:14 2013
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Fri, 15 Nov 2013 08:24:14 -0600
Subject: [Bro] Possible Bro Cluster communication issue?
In-Reply-To: <5285D5D4.3090008@doit.wisc.edu>
References: <5285A274.1050404@doit.wisc.edu> <5285A4B6.6070007@illinois.edu>
	<5285AF30.6030709@doit.wisc.edu> <5285D5D4.3090008@doit.wisc.edu>
Message-ID: <52862E8E.5030908@illinois.edu>

Which Linux distro (and which version) are you using?  And were
you using the default FW settings?  Also, were you able to
determine why the proxy was crashing?  If so, how did
you resolve the problem?


On 11/15/2013 02:05 AM, Gary Faulkner wrote:
> Actually, it was the firewall, but I also had a secondary problem in
> that the proxy was constantly crashing due a lack of system resources so
> it didn't initially appear that disabling the firewall relieved the
> communication problem. I didn't recall seeing any FW considerations
> beyond ssh in the documentation, but I did eventually find an external
> document at https://gist.github.com/grigorescu/3776670 and a quick
> netstat allowed me to confirm the ports on my hosts. Thanks for the help!
>
>
>
> On 11/14/2013 11:20 PM, Gary Faulkner wrote:
>> Both hosts are running host based FWs, but disabling them doesn't appear
>> to make a difference in the behavior. I can ssh between hosts just fine
>> as the bro user with key-based auth and broctl seems to open an ssh
>> session per worker between the two hosts that appear stay established
>> throughout just fine. Does all the communication happen over those ssh
>> sessions or are there other types of connections happening between
>> master/proxy and worker?
>>



From gary at doit.wisc.edu  Fri Nov 15 10:34:03 2013
From: gary at doit.wisc.edu (Gary Faulkner)
Date: Fri, 15 Nov 2013 12:34:03 -0600
Subject: [Bro] Possible Bro Cluster communication issue?
In-Reply-To: <52862E8E.5030908@illinois.edu>
References: <5285A274.1050404@doit.wisc.edu> <5285A4B6.6070007@illinois.edu>
	<5285AF30.6030709@doit.wisc.edu> <5285D5D4.3090008@doit.wisc.edu>
	<52862E8E.5030908@illinois.edu>
Message-ID: <5286691B.4030003@doit.wisc.edu>

We're running RHEL 6.4 (2.6.32-358.6.2.el6.x86_64). We had our own 
fairly restrictive rule set on the hosts and simply didn't have the 
ports open as I didn't see them in the particular documentation I was 
referencing on the bro site. I knew from the documentation that bro 
needed to be able to SSH between hosts, but didn't know that 
(manager/proxy/worker) were also listening on specific ports or what 
they were.

As for the proxy crashing, I think the issue was simply running too many 
workers on the same host as the proxy and manager. I started my learning 
by running a single host with manager/proxy/workers and gradually 
ramping up the worker count, then added the second with just workers. So 
I suspect I just pushed it too far and needed to free up some system 
resources on that first host running the manager/proxy. Ideally I think 
I'd like to run the master and proxy on their own system (as others have 
suggested). For testing purposes I simply disabled the workers on that 
host, made sure the proxy didn't crash and then observed the behavior of 
the host firewalls to see if they were blocking anything. So mostly 
ignorance and misconfiguration on my part.

Regards,

Gary Faulkner
UW Madison
Office of Campus Information Security
608-262-8591

On 11/15/2013 8:24 AM, Daniel Thayer wrote:
> Which Linux distro (and which version) are you using?  And were
> you using the default FW settings?  Also, were you able to
> determine why the proxy was crashing?  If so, how did
> you resolve the problem?
>
>
> On 11/15/2013 02:05 AM, Gary Faulkner wrote:
>> Actually, it was the firewall, but I also had a secondary problem in
>> that the proxy was constantly crashing due a lack of system resources so
>> it didn't initially appear that disabling the firewall relieved the
>> communication problem. I didn't recall seeing any FW considerations
>> beyond ssh in the documentation, but I did eventually find an external
>> document at https://gist.github.com/grigorescu/3776670 and a quick
>> netstat allowed me to confirm the ports on my hosts. Thanks for the 
>> help!
>>
>>
>>
>> On 11/14/2013 11:20 PM, Gary Faulkner wrote:
>>> Both hosts are running host based FWs, but disabling them doesn't 
>>> appear
>>> to make a difference in the behavior. I can ssh between hosts just fine
>>> as the bro user with key-based auth and broctl seems to open an ssh
>>> session per worker between the two hosts that appear stay established
>>> throughout just fine. Does all the communication happen over those ssh
>>> sessions or are there other types of connections happening between
>>> master/proxy and worker?
>>>
>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6257 bytes
Desc: S/MIME Cryptographic Signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131115/a776b99b/attachment.bin 

From hunarame at gmail.com  Thu Nov 21 00:22:56 2013
From: hunarame at gmail.com (Naveed Anwar)
Date: Thu, 21 Nov 2013 03:22:56 -0500
Subject: [Bro] Traffic Volume Calculation Using Bro's Connection Log
In-Reply-To: <CANg8oi8aRob-KrsUYjvx-_z=9NCF4_hZZwZ+DW=LGy-iMs4npg@mail.gmail.com>
References: <CANg8oi8aRob-KrsUYjvx-_z=9NCF4_hZZwZ+DW=LGy-iMs4npg@mail.gmail.com>
Message-ID: <CANg8oi_074icmu4WJPHcPhqw+J+__cb8AV0piSjhTzFQ9=TaqA@mail.gmail.com>

Hi Seth/John,

Thank you for your responses. I have a follow up question.

Here's a quick recap of what I need to do: I want to use Bro to calculate
the total volume of traffic captured in a pcap file, including all headers
up to (and including) Ethernet headers.

Following your suggestion to sum the resp_ip_bytes and orig_ip_bytes
columns of the
conn.log generated over the trace, I now use this script to calculate the
volume:

cat conn.log | awk 'BEGIN{FS="\t"; count=0;} {count=count+$17; count+=$19}
END {print count}'

I've tried running this over conn logs generated from two different pcap
files. In both cases, I get a count that is smaller than the size of the
pcap file. Which is fine, because like you said, ethernet headers and pcap
headers are still not included.  The problem is that when I ran this script
on the conn.log generated over a 500 GB trace, the output was 376 GB. If I
calculate the total Ethernet header size (assuming 14 bytes per packet) AND
the total pcap header size (assuming 16 bytes per packet) for that trace,
it comes to around 21 GB. That means (500-376-20 =) ~ 104 GB is still
unaccounted for. I'm trying to understand why that would be.

Perhaps that is because of packets with unknown transport protocols? You
said that packet types that are not supported will also not be included in
the byte count. By unsupported packet types, are you referring to
connections for which the value of the enum "proto" is "unknown_protocol"?
If yes, I've seen that conn.log is showing ONLY TCP, UDP, and ICMP in the
"proto" field for my 500 GB trace. Does this mean that when the protocol is
unknown, the record is not included in conn.log at all? Because that's the
only explanation I can think of for the unaccounted 104 GB!

Can you please comment, and also tell me if I'm doing something incorrect,
and if so, how I should be calculating the volume instead?

Thankyou
Zainab


On Thu, Nov 7, 2013 at 1:48 PM, Naveed Anwar <hunarame at gmail.com> wrote:

> Hi,
>
> I'm facing a small problem when running Bro. I'm trying to calculate the
> volume of traffic generated per host. I have a set of pcap files, each
> containing traffic from a single host. I thought I could run Bro on each
> pcap file, and then sum the orig_bytes and resp_bytes columns in conn.log
> to get the total volume of traffic for one host. However when I run Bro on
> a 250 MB pcap file, the sum of these two columns  is only 107 MB
> approximately, and not 250 MB as I expected. Is there any alternate method
> for calculating the volume of traffic generated by one host?
>
> Here's the script I ran to get the sum:
> cat conn.log | awk 'BEGIN{FS="\t"; count=0;} {count=count+$10; count+=$11}
> END {print count;}'
>
> This was the output of the script (which I expected would be 250 MB
> instead):
> 107790112 bytes
>
> It would be great if you could help me resolve this issue!
>
> Thank you,
> Zainab
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131121/889cf4fb/attachment.html 

From omer007infosec at gmail.com  Thu Nov 21 04:23:06 2013
From: omer007infosec at gmail.com (omer security)
Date: Thu, 21 Nov 2013 14:23:06 +0200
Subject: [Bro] Customization for HTTP logs
Message-ID: <CAK_mo1MsCgsjKCkY8mMsAgPRZrFTdDbbA_R4uDq5A+bYMMT0QQ@mail.gmail.com>

Hi,

In order to be able to log more HTTP headers, I edited the file:
/bro/share/bro/base/protocols/http/main.bro (the edited file is attached to
this mail).

In addition to this file change I added log filter into
/bro/share/bro/site/local.bro file.

The log is created and most of fields logged well except the following
fields:
response_content_length
cookie
response_content_type

Can someone tell me what's wrong ?

Thanks,

Omer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131121/fafd098e/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: main.bro
Type: application/octet-stream
Size: 11436 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131121/fafd098e/attachment.obj 

From seth at icir.org  Thu Nov 21 06:00:58 2013
From: seth at icir.org (Seth Hall)
Date: Thu, 21 Nov 2013 09:00:58 -0500
Subject: [Bro] Traffic Volume Calculation Using Bro's Connection Log
In-Reply-To: <CANg8oi_074icmu4WJPHcPhqw+J+__cb8AV0piSjhTzFQ9=TaqA@mail.gmail.com>
References: <CANg8oi8aRob-KrsUYjvx-_z=9NCF4_hZZwZ+DW=LGy-iMs4npg@mail.gmail.com>
	<CANg8oi_074icmu4WJPHcPhqw+J+__cb8AV0piSjhTzFQ9=TaqA@mail.gmail.com>
Message-ID: <A2297EC0-1DD8-459A-A81C-FB150AF18BC5@icir.org>


On Nov 21, 2013, at 3:22 AM, Naveed Anwar <hunarame at gmail.com> wrote:

> Here's a quick recap of what I need to do: I want to use Bro to calculate the total volume of traffic captured in a pcap file, including all headers up to (and including) Ethernet headers. 

You can't do this right now. :)

Due to how we handle ethernet headers (and vlan and mpls) that data is just not made available.  Additionally, any non-ip traffic will be hard to include in the measurement.  What we likely need to do is keep global counters that track the size of data pulled from libpcap.  We already have a packet counter for that like this?

resource_usage()$num_packets

I'm not saying that the resource_usage built in function will stay around forever though, it's very possible that we'll reorganize that some in the future.

  .Seth



--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131121/e7bc8bd9/attachment.bin 

From jbabio at po-box.esu.edu  Thu Nov 21 06:22:15 2013
From: jbabio at po-box.esu.edu (John Babio)
Date: Thu, 21 Nov 2013 14:22:15 +0000
Subject: [Bro] using intel framework for scripts
In-Reply-To: <mailman.17948.1385042464.921.bro@bro.org>
Message-ID: <CEB38057.7E38%jbabio@po-box.esu.edu>


How does one leverage this framework to write scripts?




From liam at broala.com  Thu Nov 21 06:29:05 2013
From: liam at broala.com (Liam Randall)
Date: Thu, 21 Nov 2013 09:29:05 -0500
Subject: [Bro] using intel framework for scripts
In-Reply-To: <CEB38057.7E38%jbabio@po-box.esu.edu>
References: <mailman.17948.1385042464.921.bro@bro.org>
	<CEB38057.7E38%jbabio@po-box.esu.edu>
Message-ID: <CAGGyr=5gqBhBxDRHnOnsiK3OTZvhvy6inQYMGSc66bhDv8HSJQ@mail.gmail.com>

Hey John,

Start with the video:
Video <http://youtu.be/E7SwURCdi8g>

Then review these exercises:
http://bro.org/bro-exchange-2013/exercises/intel.html


These are both from the Bro Exchange 2013.

Thanks,

Liam Randall


On Thu, Nov 21, 2013 at 9:22 AM, John Babio <jbabio at po-box.esu.edu> wrote:

>
> How does one leverage this framework to write scripts?
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Liam Randall
Managing Partner
510-281-0760
www.Broala.com <http://www.broala.com/>
>From the creators of Bro <http://www.bro.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131121/9ec33905/attachment.html 

From seth at icir.org  Thu Nov 21 06:30:46 2013
From: seth at icir.org (Seth Hall)
Date: Thu, 21 Nov 2013 09:30:46 -0500
Subject: [Bro] using intel framework for scripts
In-Reply-To: <CEB38057.7E38%jbabio@po-box.esu.edu>
References: <CEB38057.7E38%jbabio@po-box.esu.edu>
Message-ID: <B6DDB423-F926-40DF-A62F-4386184611C0@icir.org>


On Nov 21, 2013, at 9:22 AM, John Babio <jbabio at po-box.esu.edu> wrote:

> How does one leverage this framework to write scripts?


It depends on what you want to do.  The docs we have for it show you how to use it (to get an intel.log file).  Is that all you're interested in?  Loading data and finding things that hit?

http://www.bro.org/sphinx/frameworks/intel.html

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131121/9eed0da7/attachment.bin 

From jbabio at po-box.esu.edu  Thu Nov 21 06:48:10 2013
From: jbabio at po-box.esu.edu (John Babio)
Date: Thu, 21 Nov 2013 14:48:10 +0000
Subject: [Bro] using intel framework for scripts
In-Reply-To: <CAGGyr=5gqBhBxDRHnOnsiK3OTZvhvy6inQYMGSc66bhDv8HSJQ@mail.gmail.com>
Message-ID: <CEB386F1.7E42%jbabio@po-box.esu.edu>

Thank you!

From: Liam Randall <liam at broala.com<mailto:liam at broala.com>>
Date: Thursday, November 21, 2013 at 9:29 AM
To: John Babio <jbabio at po-box.esu.edu<mailto:jbabio at po-box.esu.edu>>
Cc: "bro at bro.org<mailto:bro at bro.org>" <bro at bro.org<mailto:bro at bro.org>>
Subject: Re: [Bro] using intel framework for scripts

Hey John,

Start with the video:
Video<http://youtu.be/E7SwURCdi8g>

Then review these exercises:
http://bro.org/bro-exchange-2013/exercises/intel.html


These are both from the Bro Exchange 2013.

Thanks,

Liam Randall


On Thu, Nov 21, 2013 at 9:22 AM, John Babio <jbabio at po-box.esu.edu<mailto:jbabio at po-box.esu.edu>> wrote:

How does one leverage this framework to write scripts?


_______________________________________________
Bro mailing list
bro at bro-ids.org<mailto:bro at bro-ids.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



--
Liam Randall
Managing Partner
510-281-0760
www.Broala.com<http://www.broala.com/>
>From the creators of Bro<http://www.bro.org>



From mattchess50 at gmail.com  Thu Nov 21 08:17:06 2013
From: mattchess50 at gmail.com (Matt Stucky)
Date: Thu, 21 Nov 2013 10:17:06 -0600
Subject: [Bro] using intel framework for scripts
In-Reply-To: <B6DDB423-F926-40DF-A62F-4386184611C0@icir.org>
References: <CEB38057.7E38%jbabio@po-box.esu.edu>
	<B6DDB423-F926-40DF-A62F-4386184611C0@icir.org>
Message-ID: <CAMM5JYqCBSJ-yA86wGVFHUUS_Knc3RCkWWBaVdxF9u1f2=JsPw@mail.gmail.com>

Does the framework take care of updating the system on the fly if the input
files change, or is a restart needed?

-matt


On Thu, Nov 21, 2013 at 8:30 AM, Seth Hall <seth at icir.org> wrote:

>
> On Nov 21, 2013, at 9:22 AM, John Babio <jbabio at po-box.esu.edu> wrote:
>
> > How does one leverage this framework to write scripts?
>
>
> It depends on what you want to do.  The docs we have for it show you how
> to use it (to get an intel.log file).  Is that all you're interested in?
>  Loading data and finding things that hit?
>
> http://www.bro.org/sphinx/frameworks/intel.html
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131121/553fb725/attachment.html 

From bernhard at ICSI.Berkeley.EDU  Thu Nov 21 08:25:34 2013
From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann)
Date: Thu, 21 Nov 2013 08:25:34 -0800
Subject: [Bro] Customization for HTTP logs
In-Reply-To: <CAK_mo1MsCgsjKCkY8mMsAgPRZrFTdDbbA_R4uDq5A+bYMMT0QQ@mail.gmail.com>
References: <CAK_mo1MsCgsjKCkY8mMsAgPRZrFTdDbbA_R4uDq5A+bYMMT0QQ@mail.gmail.com>
Message-ID: <AE49B84C-53EC-46D0-A543-AD571257ECCF@icsi.berkeley.edu>

> 
> In order to be able to log more HTTP headers, I edited the file: 
> /bro/share/bro/base/protocols/http/main.bro (the edited file is attached to this mail).
> 
> In addition to this file change I added log filter into /bro/share/bro/site/local.bro file.
> 
> The log is created and most of fields logged well except the following fields:
> response_content_length
> cookie
> response_content_type
> 
> Can someone tell me what's wrong ?

Are you sure that the server sends the header lines? As far as I remember all
of them are optional. At a first glance, it looks fine besides that - and if the
other things you added are working, these should too.

But - one other thing - you really should not edit script-files in base. If you ever
re-install bro, it will be overwritten without any warning or promoting you. Also - if
you ever update to a new version you will have to re-apply your changes manually.

Instead, you should extend the HTTP::Info record in a separate, new script-file and
also set the values in that new file by catching the http_header event.

Bernhard


From seth at icir.org  Thu Nov 21 09:02:41 2013
From: seth at icir.org (Seth Hall)
Date: Thu, 21 Nov 2013 12:02:41 -0500
Subject: [Bro] using intel framework for scripts
In-Reply-To: <CAMM5JYqCBSJ-yA86wGVFHUUS_Knc3RCkWWBaVdxF9u1f2=JsPw@mail.gmail.com>
References: <CEB38057.7E38%jbabio@po-box.esu.edu>
	<B6DDB423-F926-40DF-A62F-4386184611C0@icir.org>
	<CAMM5JYqCBSJ-yA86wGVFHUUS_Knc3RCkWWBaVdxF9u1f2=JsPw@mail.gmail.com>
Message-ID: <A9350604-D3B4-4C1D-8508-6A7B308A6233@icir.org>


On Nov 21, 2013, at 11:17 AM, Matt Stucky <mattchess50 at gmail.com> wrote:

> Does the framework take care of updating the system on the fly if the input files change, or is a restart needed?

It updates on the fly.  If you are running a cluster you only need to update the data on the manager too.  It will auto distribute.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131121/a7e144eb/attachment.bin 

From amptcl at gmail.com  Thu Nov 21 09:06:33 2013
From: amptcl at gmail.com (Amir Mehmood)
Date: Thu, 21 Nov 2013 22:06:33 +0500
Subject: [Bro] Traffic Volume Calculation Using Bro's Connection Log
In-Reply-To: <A2297EC0-1DD8-459A-A81C-FB150AF18BC5@icir.org>
References: <CANg8oi8aRob-KrsUYjvx-_z=9NCF4_hZZwZ+DW=LGy-iMs4npg@mail.gmail.com>
	<CANg8oi_074icmu4WJPHcPhqw+J+__cb8AV0piSjhTzFQ9=TaqA@mail.gmail.com>
	<A2297EC0-1DD8-459A-A81C-FB150AF18BC5@icir.org>
Message-ID: <CAFLtYa5-t543vtcjemDtSe-agTg6RK3tZBVeD2Yuk3voQKPSAA@mail.gmail.com>

Try using ipsumdump ...

Amir


On Thu, Nov 21, 2013 at 7:00 PM, Seth Hall <seth at icir.org> wrote:

>
> On Nov 21, 2013, at 3:22 AM, Naveed Anwar <hunarame at gmail.com> wrote:
>
> > Here's a quick recap of what I need to do: I want to use Bro to
> calculate the total volume of traffic captured in a pcap file, including
> all headers up to (and including) Ethernet headers.
>
> You can't do this right now. :)
>
> Due to how we handle ethernet headers (and vlan and mpls) that data is
> just not made available.  Additionally, any non-ip traffic will be hard to
> include in the measurement.  What we likely need to do is keep global
> counters that track the size of data pulled from libpcap.  We already have
> a packet counter for that like this?
>
> resource_usage()$num_packets
>
> I'm not saying that the resource_usage built in function will stay around
> forever though, it's very possible that we'll reorganize that some in the
> future.
>
>   .Seth
>
>
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131121/9ab28a55/attachment.html 

From jwillie4020 at gmail.com  Thu Nov 21 20:37:03 2013
From: jwillie4020 at gmail.com (scottie)
Date: Fri, 22 Nov 2013 15:37:03 +1100
Subject: [Bro] Traffic Volume Calculation Using Bro's Connection,	Log
Message-ID: <528EDF6F.7040601@gmail.com>

Try this:
cat conn.log  | awk 'BEGIN{other=0;tcp=0;udp=0;icmp=0} {if($7 == "tcp")
tcp=tcp+1} {if($7 == "udp") udp=udp+1} {if($7 == "icmp") icmp=icmp+1}
{if($7 != "tcp" && $7 != "udp" && $7 != "icmp" ) other=other+1} END{
print "TCP:  " tcp, "\nUDP:  " udp, "\nICMP: " icmp, "\nOther: " other}'
| column -t

It will give you an overview of tcp/udp/icmp and 'other' connections.
What does it output for you? 99.9% of my traffic is one of these first 3.


From jwillie4020 at gmail.com  Fri Nov 22 00:13:19 2013
From: jwillie4020 at gmail.com (scottie)
Date: Fri, 22 Nov 2013 19:13:19 +1100
Subject: [Bro] Traffic Volume Calculation Using Bro's Connection,	Log
In-Reply-To: <528EDF6F.7040601@gmail.com>
References: <528EDF6F.7040601@gmail.com>
Message-ID: <528F121F.8070605@gmail.com>

Disregard that last email, i think bro can only see tcp, udp, and icmp,
the 'others' that i was seeing were headers and spaces.

On 22/11/13 15:37, scottie wrote:
> Try this:
> cat conn.log  | awk 'BEGIN{other=0;tcp=0;udp=0;icmp=0} {if($7 == "tcp")
> tcp=tcp+1} {if($7 == "udp") udp=udp+1} {if($7 == "icmp") icmp=icmp+1}
> {if($7 != "tcp" && $7 != "udp" && $7 != "icmp" ) other=other+1} END{
> print "TCP:  " tcp, "\nUDP:  " udp, "\nICMP: " icmp, "\nOther: " other}'
> | column -t
>
> It will give you an overview of tcp/udp/icmp and 'other' connections.
> What does it output for you? 99.9% of my traffic is one of these first 3.



From tritium.cat at gmail.com  Fri Nov 22 06:47:42 2013
From: tritium.cat at gmail.com (Tritium Cat)
Date: Fri, 22 Nov 2013 06:47:42 -0800
Subject: [Bro] Writing JSON logs
Message-ID: <CAMPgRd5QApHDkJ34WB97zwL0q-xh76cqX7qPpS1sTYW-BcTecA@mail.gmail.com>

Bro,

I made a patch for the Ascii log writer to write the logs in JSON format.
 This was thanks to the existing code from the ElasticSearch writer and
copy/paste skill.

But when I try to enable the writer at runtime there are errors.  Why ?

See patch.  ( cd bro-2.2; patch -p1 < bro--write_json.patch )

Thanks,

--TC


event bro_init()
{
        LogAscii::write_json=T;
}

results in

# bin/broctl check
manager failed.
   error in /usr/local/3rd-party/bro/share/bro/site/local.bro, line 7:
const is not a modifiable lvalue (LogAscii::write_json)
proxy-1 failed.
   error in /usr/local/3rd-party/bro/share/bro/site/local.bro, line 7:
const is not a modifiable lvalue (LogAscii::write_json)
worker-1 failed.
   error in /usr/local/3rd-party/bro/share/bro/site/local.bro, line 7:
const is not a modifiable lvalue (LogAscii::write_json)
worker-2 failed.
   error in /usr/local/3rd-party/bro/share/bro/site/local.bro, line 7:
const is not a modifiable lvalue (LogAscii::write_json)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131122/bd9b5528/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bro--write_json.patch
Type: application/octet-stream
Size: 8604 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131122/bd9b5528/attachment.obj 

From Bjorn.Samvik at netclean.com  Fri Nov 22 07:04:53 2013
From: Bjorn.Samvik at netclean.com (=?Windows-1252?Q?Bj=F6rn_Samvik?=)
Date: Fri, 22 Nov 2013 15:04:53 +0000
Subject: [Bro] Some events not received by broccoli
Message-ID: <1784413d504245c7ad9a32e9b60d22e1@AMSPR04MB147.eurprd04.prod.outlook.com>

Hello,

I'm using broccoli to receive bro (2.2-5) events and are having some problems. Consider the following.

The broccoli client is listening to 2 events.
bro_event_registry_add(m_bc, "file_new", (BroEventFunc)&Broccoli::newFile, this);
bro_event_registry_add(m_bc, "test_event", (BroEventFunc)&Broccoli::newFile, this);

The following bro script is used.

...
global test_event: event(f: fa_file);
event file_new(f: fa_file)
{
    event test_event(f);
}

The file_new event is correctly received by my broccoli client however the test_event is not received. If I change the content of the test_event to something else it works.

...
global test_event: event(f: string);
event file_new(f: fa_file)
{
    event test_event(f$mime_type);
}

So, is this expected and in that case why and what is the proposed way of solving the issue?

(Also noticed that the file_state_removed(f: fa_file) event is not received by the broccoli client.)

Thank you
/Bj?rn




Bj?rn Samvik
Software Developer

[NetClean]

NetClean Technologies Sweden AB
F?rsta L?nggatan 30 ? SE-413 27 G?teborg ? Sweden

Phone: +46 31 719 08 00 ? Fax: +46 31 13 89 50
Direct: +46 31 719 08 22 ? Mobile: +46 709 36 83 03

Bjorn.Samvik at netclean.com
<mailto:Bjorn.Samvik at netclean.com>www.netclean.com<http://www.netclean.com>

The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131122/e7c0d256/attachment.html 

From jsiwek at illinois.edu  Fri Nov 22 07:44:23 2013
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Fri, 22 Nov 2013 15:44:23 +0000
Subject: [Bro] Writing JSON logs
In-Reply-To: <CAMPgRd5QApHDkJ34WB97zwL0q-xh76cqX7qPpS1sTYW-BcTecA@mail.gmail.com>
References: <CAMPgRd5QApHDkJ34WB97zwL0q-xh76cqX7qPpS1sTYW-BcTecA@mail.gmail.com>
Message-ID: <E2DBCB23-CB94-42FC-93B4-407F0CC0400F@illinois.edu>


On Nov 22, 2013, at 8:47 AM, Tritium Cat <tritium.cat at gmail.com> wrote:

> event bro_init()
> {
>         LogAscii::write_json=T;
> }
> 
> results in
> 
> # bin/broctl check
> manager failed.
>    error in /usr/local/3rd-party/bro/share/bro/site/local.bro, line 7:
> const is not a modifiable lvalue (LogAscii::write_json)
It?s a ?const? so you can?t change the value at run-time.  Use `redef` to assign a new value at parse-time.

- Jon


From tritium.cat at gmail.com  Fri Nov 22 08:36:28 2013
From: tritium.cat at gmail.com (Tritium Cat)
Date: Fri, 22 Nov 2013 08:36:28 -0800
Subject: [Bro] Writing JSON logs
In-Reply-To: <E2DBCB23-CB94-42FC-93B4-407F0CC0400F@illinois.edu>
References: <CAMPgRd5QApHDkJ34WB97zwL0q-xh76cqX7qPpS1sTYW-BcTecA@mail.gmail.com>
	<E2DBCB23-CB94-42FC-93B4-407F0CC0400F@illinois.edu>
Message-ID: <CAMPgRd4udtJA7c3Jn5vg6ePuN185++axqksAqD2naY7srVXKvQ@mail.gmail.com>

On Fri, Nov 22, 2013 at 7:44 AM, Siwek, Jonathan Luke
<jsiwek at illinois.edu>wrote:

> It?s a ?const? so you can?t change the value at run-time.  Use `redef` to
> assign a new value at parse-time.
>


Did you read the patch ?  &redef is included.  Maybe I misunderstand you.

--TC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131122/cb36992a/attachment.html 

From jsiwek at illinois.edu  Fri Nov 22 13:36:45 2013
From: jsiwek at illinois.edu (Siwek, Jonathan Luke)
Date: Fri, 22 Nov 2013 21:36:45 +0000
Subject: [Bro] Writing JSON logs
In-Reply-To: <CAMPgRd4udtJA7c3Jn5vg6ePuN185++axqksAqD2naY7srVXKvQ@mail.gmail.com>
References: <CAMPgRd5QApHDkJ34WB97zwL0q-xh76cqX7qPpS1sTYW-BcTecA@mail.gmail.com>
	<E2DBCB23-CB94-42FC-93B4-407F0CC0400F@illinois.edu>
	<CAMPgRd4udtJA7c3Jn5vg6ePuN185++axqksAqD2naY7srVXKvQ@mail.gmail.com>
Message-ID: <8A3BAB20-4873-49E6-85E6-B42F4B0CBF18@illinois.edu>


On Nov 22, 2013, at 10:36 AM, Tritium Cat <tritium.cat at gmail.com> wrote:

> On Fri, Nov 22, 2013 at 7:44 AM, Siwek, Jonathan Luke <jsiwek at illinois.edu> wrote:
> It?s a ?const? so you can?t change the value at run-time.  Use `redef` to assign a new value at parse-time.
> 
> 
> Did you read the patch ?  &redef is included.  Maybe I misunderstand you.

The &redef attribute still doesn?t permit run-time modification of a const, which is what you did in the bro_init handler.  Instead what you need is to use the `redef` statement to assign a value at parse time (outside an event handler):

	redef LogAscii::write_json=T;

- Jon


From tritium.cat at gmail.com  Fri Nov 22 13:50:03 2013
From: tritium.cat at gmail.com (Tritium Cat)
Date: Fri, 22 Nov 2013 13:50:03 -0800
Subject: [Bro] Writing JSON logs
In-Reply-To: <8A3BAB20-4873-49E6-85E6-B42F4B0CBF18@illinois.edu>
References: <CAMPgRd5QApHDkJ34WB97zwL0q-xh76cqX7qPpS1sTYW-BcTecA@mail.gmail.com>
	<E2DBCB23-CB94-42FC-93B4-407F0CC0400F@illinois.edu>
	<CAMPgRd4udtJA7c3Jn5vg6ePuN185++axqksAqD2naY7srVXKvQ@mail.gmail.com>
	<8A3BAB20-4873-49E6-85E6-B42F4B0CBF18@illinois.edu>
Message-ID: <CAMPgRd6XUnStv6DnrKnR1oZNnbxH-fabCNCbs+MCEq+vpdAvmg@mail.gmail.com>

Thanks.

--TC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131122/433033d6/attachment.html 

From jbabio at po-box.esu.edu  Fri Nov 22 17:26:50 2013
From: jbabio at po-box.esu.edu (John Babio)
Date: Sat, 23 Nov 2013 01:26:50 +0000
Subject: [Bro] two issues with the intel framework
Message-ID: <6E7D7EC4661DB5438A60388F3ED1BBAA05B0CE@msxmb2.admin.esu.edu>

I followed the examples step by step and I cannot get bro to like the text file or dat file from the documentation. I get errors in the reporter.log about not being able to find the requested field indicator.
The other error is "headers are incorrect". Any help would be appreciated. Thanks!



From seth at icir.org  Fri Nov 22 18:05:21 2013
From: seth at icir.org (Seth Hall)
Date: Fri, 22 Nov 2013 21:05:21 -0500
Subject: [Bro] two issues with the intel framework
In-Reply-To: <6E7D7EC4661DB5438A60388F3ED1BBAA05B0CE@msxmb2.admin.esu.edu>
References: <6E7D7EC4661DB5438A60388F3ED1BBAA05B0CE@msxmb2.admin.esu.edu>
Message-ID: <C49732C9-0EE3-4806-A3B9-8879389AAD9E@icir.org>


On Nov 22, 2013, at 8:26 PM, John Babio <jbabio at po-box.esu.edu> wrote:

> The other error is "headers are incorrect". Any help would be appreciated. Thanks!


It's helpful to post exactly why errors you're seeing and exactly how you're configuring and running Bro (i.e., send the exact errors and send an example of something you can provide Bro to reproduce the error.

Thanks,
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131122/a8c06e51/attachment.bin 

From bernhard at ICSI.Berkeley.EDU  Fri Nov 22 18:14:56 2013
From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann)
Date: Fri, 22 Nov 2013 18:14:56 -0800
Subject: [Bro] two issues with the intel framework
In-Reply-To: <C49732C9-0EE3-4806-A3B9-8879389AAD9E@icir.org>
References: <6E7D7EC4661DB5438A60388F3ED1BBAA05B0CE@msxmb2.admin.esu.edu>
	<C49732C9-0EE3-4806-A3B9-8879389AAD9E@icir.org>
Message-ID: <434FDBC8-696D-4064-A616-0F97781CDF8A@icsi.berkeley.edu>

Also - check if the header fields are separated by tab characters and not by spaces.
That might be the problem.

Bernhard
On Nov 22, 2013, at 6:05 PM, Seth Hall <seth at icir.org> wrote:

> 
> On Nov 22, 2013, at 8:26 PM, John Babio <jbabio at po-box.esu.edu> wrote:
> 
>> The other error is "headers are incorrect". Any help would be appreciated. Thanks!
> 
> 
> It's helpful to post exactly why errors you're seeing and exactly how you're configuring and running Bro (i.e., send the exact errors and send an example of something you can provide Bro to reproduce the error.
> 
> Thanks,
>  .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro




From jbabio at po-box.esu.edu  Sat Nov 23 05:32:16 2013
From: jbabio at po-box.esu.edu (John Babio)
Date: Sat, 23 Nov 2013 13:32:16 +0000
Subject: [Bro] two issues with the intel framework
In-Reply-To: <434FDBC8-696D-4064-A616-0F97781CDF8A@icsi.berkeley.edu>
References: <6E7D7EC4661DB5438A60388F3ED1BBAA05B0CE@msxmb2.admin.esu.edu>
	<C49732C9-0EE3-4806-A3B9-8879389AAD9E@icir.org>,
	<434FDBC8-696D-4064-A616-0F97781CDF8A@icsi.berkeley.edu>
Message-ID: <6E7D7EC4661DB5438A60388F3ED1BBAA05C5FC@msxmb2.admin.esu.edu>

Reporter.log

#separator \x09
#set_separator  ,
#empty_field    (empty)
#unset_field    -
#path   reporter
#open   2013-11-23-08-28-18
#fields ts      level   message location
#types  time    enum    string  string
1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: Did not find requested field indicator in input data file /etc/bro/spool$
1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: Init: cannot open /etc/bro/spool/installed-scripts-do-not-touch/site/int$
1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: Init failed  (empty)
1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: terminating thread   (empty)

/etc/bro/share/bro/site/intel.txt

#fields         indicator       indicator_type          meta.source
instagram.com   Intel::DOMAIN   my_special_source

local.bro

@load intel1.bro

intel1.bro

@load frameworks/intel/seen

redef Intel::read_files += {
        fmt("%s/intel1.txt", @DIR)
};








________________________________________
From: Bernhard Amann [bernhard at ICSI.Berkeley.EDU]
Sent: Friday, November 22, 2013 9:14 PM
To: Seth Hall
Cc: John Babio; bro at bro.org
Subject: Re: [Bro] two issues with the intel framework

Also - check if the header fields are separated by tab characters and not by spaces.
That might be the problem.

Bernhard
On Nov 22, 2013, at 6:05 PM, Seth Hall <seth at icir.org> wrote:

>
> On Nov 22, 2013, at 8:26 PM, John Babio <jbabio at po-box.esu.edu> wrote:
>
>> The other error is "headers are incorrect". Any help would be appreciated. Thanks!
>
>
> It's helpful to post exactly why errors you're seeing and exactly how you're configuring and running Bro (i.e., send the exact errors and send an example of something you can provide Bro to reproduce the error.
>
> Thanks,
>  .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro




From bernhard at ICSI.Berkeley.EDU  Sat Nov 23 07:50:55 2013
From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann)
Date: Sat, 23 Nov 2013 07:50:55 -0800
Subject: [Bro] two issues with the intel framework
In-Reply-To: <6E7D7EC4661DB5438A60388F3ED1BBAA05C5FC@msxmb2.admin.esu.edu>
References: <6E7D7EC4661DB5438A60388F3ED1BBAA05B0CE@msxmb2.admin.esu.edu>
	<C49732C9-0EE3-4806-A3B9-8879389AAD9E@icir.org>,
	<434FDBC8-696D-4064-A616-0F97781CDF8A@icsi.berkeley.edu>
	<6E7D7EC4661DB5438A60388F3ED1BBAA05C5FC@msxmb2.admin.esu.edu>
Message-ID: <083D8263-F929-4C79-B4E5-F0BD6B9C5E3F@icsi.berkeley.edu>

Just to check - are you a hundred percent sure that the first line of your
intel.txt file looks like?
#fields[tab]indicator[tab]indicator_type[tab]mata.source

Without any other characters in between, especially not using spaces
instead of tab?

>From the paste in your mail we are unable to tell if that is the case, but
the error message really sounds like there is some kind of problem with
that line in the input file.

Bernhard

On Nov 23, 2013, at 5:32 AM, John Babio <jbabio at po-box.esu.edu> wrote:

> Reporter.log
> 
> #separator \x09
> #set_separator  ,
> #empty_field    (empty)
> #unset_field    -
> #path   reporter
> #open   2013-11-23-08-28-18
> #fields ts      level   message location
> #types  time    enum    string  string
> 1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: Did not find requested field indicator in input data file /etc/bro/spool$
> 1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: Init: cannot open /etc/bro/spool/installed-scripts-do-not-touch/site/int$
> 1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: Init failed  (empty)
> 1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: terminating thread   (empty)
> 
> /etc/bro/share/bro/site/intel.txt
> 
> #fields         indicator       indicator_type          meta.source
> instagram.com   Intel::DOMAIN   my_special_source
> 
> local.bro
> 
> @load intel1.bro
> 
> intel1.bro
> 
> @load frameworks/intel/seen
> 
> redef Intel::read_files += {
>       fmt("%s/intel1.txt", @DIR)
> };
> 
> 
> 
> 
> 
> 
> 
> 
> ________________________________________
> From: Bernhard Amann [bernhard at ICSI.Berkeley.EDU]
> Sent: Friday, November 22, 2013 9:14 PM
> To: Seth Hall
> Cc: John Babio; bro at bro.org
> Subject: Re: [Bro] two issues with the intel framework
> 
> Also - check if the header fields are separated by tab characters and not by spaces.
> That might be the problem.
> 
> Bernhard
> On Nov 22, 2013, at 6:05 PM, Seth Hall <seth at icir.org> wrote:
> 
>> 
>> On Nov 22, 2013, at 8:26 PM, John Babio <jbabio at po-box.esu.edu> wrote:
>> 
>>> The other error is "headers are incorrect". Any help would be appreciated. Thanks!
>> 
>> 
>> It's helpful to post exactly why errors you're seeing and exactly how you're configuring and running Bro (i.e., send the exact errors and send an example of something you can provide Bro to reproduce the error.
>> 
>> Thanks,
>> .Seth
>> 
>> --
>> Seth Hall
>> International Computer Science Institute
>> (Bro) because everyone has a network
>> http://www.bro.org/
>> 
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 




From jbabio at po-box.esu.edu  Sat Nov 23 11:01:05 2013
From: jbabio at po-box.esu.edu (John Babio)
Date: Sat, 23 Nov 2013 19:01:05 +0000
Subject: [Bro] two issues with the intel framework
In-Reply-To: <083D8263-F929-4C79-B4E5-F0BD6B9C5E3F@icsi.berkeley.edu>
References: <6E7D7EC4661DB5438A60388F3ED1BBAA05B0CE@msxmb2.admin.esu.edu>
	<C49732C9-0EE3-4806-A3B9-8879389AAD9E@icir.org>,
	<434FDBC8-696D-4064-A616-0F97781CDF8A@icsi.berkeley.edu>
	<6E7D7EC4661DB5438A60388F3ED1BBAA05C5FC@msxmb2.admin.esu.edu>,
	<083D8263-F929-4C79-B4E5-F0BD6B9C5E3F@icsi.berkeley.edu>
Message-ID: <6E7D7EC4661DB5438A60388F3ED1BBAA05C6D2@msxmb2.admin.esu.edu>

You guys had it. I went back in and redid the entire line with just tabs. I believe I had it this way but just to be certain I redid the entire file. Something must have through it off or was lingering in the original file. You guys are the best!
________________________________________
From: Bernhard Amann [bernhard at ICSI.Berkeley.EDU]
Sent: Saturday, November 23, 2013 10:50 AM
To: John Babio
Cc: Seth Hall; bro at bro.org
Subject: Re: [Bro] two issues with the intel framework

Just to check - are you a hundred percent sure that the first line of your
intel.txt file looks like?
#fields[tab]indicator[tab]indicator_type[tab]mata.source

Without any other characters in between, especially not using spaces
instead of tab?

>From the paste in your mail we are unable to tell if that is the case, but
the error message really sounds like there is some kind of problem with
that line in the input file.

Bernhard

On Nov 23, 2013, at 5:32 AM, John Babio <jbabio at po-box.esu.edu> wrote:

> Reporter.log
>
> #separator \x09
> #set_separator  ,
> #empty_field    (empty)
> #unset_field    -
> #path   reporter
> #open   2013-11-23-08-28-18
> #fields ts      level   message location
> #types  time    enum    string  string
> 1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: Did not find requested field indicator in input data file /etc/bro/spool$
> 1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: Init: cannot open /etc/bro/spool/installed-scripts-do-not-touch/site/int$
> 1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: Init failed  (empty)
> 1385213298.936827       Reporter::ERROR /etc/bro/spool/installed-scripts-do-not-touch/site/intel1.txt/Input::READER_ASCII: terminating thread   (empty)
>
> /etc/bro/share/bro/site/intel.txt
>
> #fields         indicator       indicator_type          meta.source
> instagram.com   Intel::DOMAIN   my_special_source
>
> local.bro
>
> @load intel1.bro
>
> intel1.bro
>
> @load frameworks/intel/seen
>
> redef Intel::read_files += {
>       fmt("%s/intel1.txt", @DIR)
> };
>
>
>
>
>
>
>
>
> ________________________________________
> From: Bernhard Amann [bernhard at ICSI.Berkeley.EDU]
> Sent: Friday, November 22, 2013 9:14 PM
> To: Seth Hall
> Cc: John Babio; bro at bro.org
> Subject: Re: [Bro] two issues with the intel framework
>
> Also - check if the header fields are separated by tab characters and not by spaces.
> That might be the problem.
>
> Bernhard
> On Nov 22, 2013, at 6:05 PM, Seth Hall <seth at icir.org> wrote:
>
>>
>> On Nov 22, 2013, at 8:26 PM, John Babio <jbabio at po-box.esu.edu> wrote:
>>
>>> The other error is "headers are incorrect". Any help would be appreciated. Thanks!
>>
>>
>> It's helpful to post exactly why errors you're seeing and exactly how you're configuring and running Bro (i.e., send the exact errors and send an example of something you can provide Bro to reproduce the error.
>>
>> Thanks,
>> .Seth
>>
>> --
>> Seth Hall
>> International Computer Science Institute
>> (Bro) because everyone has a network
>> http://www.bro.org/
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>




From jbabio at po-box.esu.edu  Sun Nov 24 17:45:56 2013
From: jbabio at po-box.esu.edu (John Babio)
Date: Mon, 25 Nov 2013 01:45:56 +0000
Subject: [Bro] sending log of choice to barnyard2
Message-ID: <6E7D7EC4661DB5438A60388F3ED1BBAA05C99F@msxmb2.admin.esu.edu>

I seen there is a way to send logs to bro with barnyard. Is there a way to have bro send the notice.log or any log for that matter to barnyard2 then have barnyard2 send them to a remote database server running something like snorby? Thanks as always.



From paul.halliday at gmail.com  Sun Nov 24 18:13:23 2013
From: paul.halliday at gmail.com (Paul Halliday)
Date: Sun, 24 Nov 2013 22:13:23 -0400
Subject: [Bro] sending log of choice to barnyard2
In-Reply-To: <6E7D7EC4661DB5438A60388F3ED1BBAA05C99F@msxmb2.admin.esu.edu>
References: <6E7D7EC4661DB5438A60388F3ED1BBAA05C99F@msxmb2.admin.esu.edu>
Message-ID: <CAJfn-RFhC=BjT1RTQmv1DgNaQTvn2Cf+HJFXTCTrBW3cwOhAjw@mail.gmail.com>

If you use sguil there is an agent that can help you:
https://github.com/int13h/bro_agent



On Sun, Nov 24, 2013 at 9:45 PM, John Babio <jbabio at po-box.esu.edu> wrote:
> I seen there is a way to send logs to bro with barnyard. Is there a way to have bro send the notice.log or any log for that matter to barnyard2 then have barnyard2 send them to a remote database server running something like snorby? Thanks as always.
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



-- 
Paul Halliday
http://www.pintumbler.org/


From jbabio at po-box.esu.edu  Mon Nov 25 05:50:43 2013
From: jbabio at po-box.esu.edu (John Babio)
Date: Mon, 25 Nov 2013 13:50:43 +0000
Subject: [Bro] sending log of choice to barnyard2
In-Reply-To: <CAJfn-RFhC=BjT1RTQmv1DgNaQTvn2Cf+HJFXTCTrBW3cwOhAjw@mail.gmail.com>
Message-ID: <CEB8BFEC.7FB2%jbabio@po-box.esu.edu>

Thanks Paul.

On 11/24/13, 9:13 PM, "Paul Halliday" <paul.halliday at gmail.com> wrote:

>If you use sguil there is an agent that can help you:
>https://github.com/int13h/bro_agent
>
>
>
>On Sun, Nov 24, 2013 at 9:45 PM, John Babio <jbabio at po-box.esu.edu> wrote:
>> I seen there is a way to send logs to bro with barnyard. Is there a way
>>to have bro send the notice.log or any log for that matter to barnyard2
>>then have barnyard2 send them to a remote database server running
>>something like snorby? Thanks as always.
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
>
>-- 
>Paul Halliday
>http://www.pintumbler.org/




From r.fulton at auckland.ac.nz  Mon Nov 25 19:21:43 2013
From: r.fulton at auckland.ac.nz (Russell Fulton)
Date: Tue, 26 Nov 2013 03:21:43 +0000
Subject: [Bro] warning: removing stale lock
Message-ID: <F1815D56-500C-4A58-8283-810E5D9B2C25@auckland.ac.nz>

Hi,

I regularly get this message from /opt/bro/bin/broctl cron

Any idea what might cause locks not to be removed?  it is happening about half a dozen times a day.

Russell





From dnthayer at illinois.edu  Mon Nov 25 20:51:25 2013
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Mon, 25 Nov 2013 22:51:25 -0600
Subject: [Bro] warning: removing stale lock
In-Reply-To: <F1815D56-500C-4A58-8283-810E5D9B2C25@auckland.ac.nz>
References: <F1815D56-500C-4A58-8283-810E5D9B2C25@auckland.ac.nz>
Message-ID: <529428CD.3@illinois.edu>

If an instance of broctl crashed somehow (perhaps
due to a bug in the code), then next time you run broctl,
you'd get that message.

Which version of BroControl are you using?
Also, are you running "broctl cron" from a cron job?  If so,
then you'll want to check that the cron job runs
as the same user you normally use to run broctl.


On 11/25/2013 09:21 PM, Russell Fulton wrote:
> Hi,
>
> I regularly get this message from /opt/bro/bin/broctl cron
>
> Any idea what might cause locks not to be removed?  it is happening about half a dozen times a day.
>
> Russell



From esalvati at gmail.com  Fri Nov 29 12:03:29 2013
From: esalvati at gmail.com (Dino)
Date: Fri, 29 Nov 2013 18:03:29 -0200
Subject: [Bro] Layer7 / DPI filter with Bro
Message-ID: <CAMSVOFkBkViVMmdXeuvUfvf9FJ+QpCONGMAziPhcyNejdLaKUQ@mail.gmail.com>

Hi.
I'm trying do integrate some DPI capabilities with iptables.
I've tried nDPI but it doesn't recognize the protocols most of the time.
I think that Bro could do that, but I found too few scrips to identify
applications like skype or tor.
Am I on the right direction?
Could someone point me to some repository with more scripts or, if you
think that Bro isn't the right tool to do that, give me some advice on
other tools?

Thanks

Edson Dino Salvati
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20131129/d7997d44/attachment.html