From capn.freako at gmail.com  Sat Aug  1 16:59:21 2015
From: capn.freako at gmail.com (David Banas)
Date: Sat, 1 Aug 2015 16:59:21 -0700
Subject: [Bro] Error trying to 'make doc' in current GitHub head.
Message-ID: <6A755C35-5228-494C-99E1-50F944BE2094@gmail.com>

Hi all,

I?m getting this error, when attempting a ?make doc? from within the root directory of the current GitHub head:

Davids-MacBook-Air-2:bro dbanas$ git show
commit 30bb17ea8d1faa7bb1c2ff41aa1579523fcbe273
Author: Robin Sommer <robin at icir.org>
Date:   Fri Jul 24 15:06:07 2015 -0700

[100%] [Sphinx] Generate HTML documentation in /Users/dbanas/Documents/Projects/bro/build/html
error in /Users/dbanas/Documents/Projects/bro/scripts/policy/protocols/ftp/detect-bruteforcing.bro, line 4: Failed to open file /Users/dbanas/Documents/Projects/bro/scripts/policy/protocols/ftp/detect-bruteforcing.bro: Too many open files
fatal error in /Users/dbanas/Documents/Projects/bro/scripts/policy/protocols/ftp/detect-bruteforcing.bro, line 4: can't open /Users/dbanas/Documents/Projects/bro/scripts/base/protocols/ftp/__load__.bro

Anyone know what?s going on?

Thanks,
-db

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150801/54a3950d/attachment.html 

From johanna at icir.org  Sat Aug  1 17:19:53 2015
From: johanna at icir.org (Johanna Amann)
Date: Sat, 1 Aug 2015 20:19:53 -0400
Subject: [Bro] Error trying to 'make doc' in current GitHub head.
In-Reply-To: <6A755C35-5228-494C-99E1-50F944BE2094@gmail.com>
References: <6A755C35-5228-494C-99E1-50F944BE2094@gmail.com>
Message-ID: <20150802001953.GA10901@Beezling.home>

Try to increase the file ulimit (ulimit -n). I assume you are on OS X --
it is a bit low there with only 256 simultaneous open files.

Johanna

On Sat, Aug 01, 2015 at 04:59:21PM -0700, David Banas wrote:
> Hi all,
> 
> I?m getting this error, when attempting a ?make doc? from within the root directory of the current GitHub head:
> 
> Davids-MacBook-Air-2:bro dbanas$ git show
> commit 30bb17ea8d1faa7bb1c2ff41aa1579523fcbe273
> Author: Robin Sommer <robin at icir.org>
> Date:   Fri Jul 24 15:06:07 2015 -0700
> 
> [100%] [Sphinx] Generate HTML documentation in /Users/dbanas/Documents/Projects/bro/build/html
> error in /Users/dbanas/Documents/Projects/bro/scripts/policy/protocols/ftp/detect-bruteforcing.bro, line 4: Failed to open file /Users/dbanas/Documents/Projects/bro/scripts/policy/protocols/ftp/detect-bruteforcing.bro: Too many open files
> fatal error in /Users/dbanas/Documents/Projects/bro/scripts/policy/protocols/ftp/detect-bruteforcing.bro, line 4: can't open /Users/dbanas/Documents/Projects/bro/scripts/base/protocols/ftp/__load__.bro
> 
> Anyone know what?s going on?
> 
> Thanks,
> -db
> 

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From billcyz at gmail.com  Mon Aug  3 06:14:36 2015
From: billcyz at gmail.com (=?UTF-8?B?6ZmI5pix56u5?=)
Date: Tue, 4 Aug 2015 01:14:36 +1200
Subject: [Bro] Store PCAP logs
Message-ID: <CAGokiGNtBH-v7SKt=2yVTZfpRUNsD9EG8vVua-z72oha-dMxSg@mail.gmail.com>

Hello,

I've installed Bro IDS on my computer, and I want to know is it possible to
make Bro generate pcap logs? Because I want to use Wireshark to analyze Bro
logs.
Another question, does anyone tried Splunk to analyze Bro logs? Can anyone
give me some advice?

Any help would be great. Thank You.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150804/c9616098/attachment.html 

From slagell at illinois.edu  Mon Aug  3 06:30:38 2015
From: slagell at illinois.edu (Slagell, Adam J)
Date: Mon, 3 Aug 2015 13:30:38 +0000
Subject: [Bro] Store PCAP logs
In-Reply-To: <CAGokiGNtBH-v7SKt=2yVTZfpRUNsD9EG8vVua-z72oha-dMxSg@mail.gmail.com>
References: <CAGokiGNtBH-v7SKt=2yVTZfpRUNsD9EG8vVua-z72oha-dMxSg@mail.gmail.com>
Message-ID: <06C60A31-2469-4560-9B63-0AF975C889B7@illinois.edu>

Bro can analyze pcaps, but it doesn't generate them.

Wire shark isn't really a log analyzer, but a raw traffic analyzer/GUI. 

There are Bro plugins for Splunk. It works well. 



> On Aug 3, 2015, at 8:19 AM, ??? <billcyz at gmail.com> wrote:
> 
> Hello,
> 
> I've installed Bro IDS on my computer, and I want to know is it possible to make Bro generate pcap logs? Because I want to use Wireshark to analyze Bro logs.
> Another question, does anyone tried Splunk to analyze Bro logs? Can anyone give me some advice?
> 
> Any help would be great. Thank You.
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From vlad at grigorescu.org  Mon Aug  3 08:44:56 2015
From: vlad at grigorescu.org (Vlad Grigorescu)
Date: Mon, 3 Aug 2015 10:44:56 -0500
Subject: [Bro] Problem identifying originator in Kerberos connections
In-Reply-To: <CACPag=0EqqHiCqi5doQsjgLGQRwEgfV95mdC4hS7Egdqmc9_4w@mail.gmail.com>
References: <CACPag=0EqqHiCqi5doQsjgLGQRwEgfV95mdC4hS7Egdqmc9_4w@mail.gmail.com>
Message-ID: <CANUJ_Fn26EHZLZD-ZJ-iTK_F7H1BXWh5N9XvGZqfpacY1tn9gg@mail.gmail.com>

Hi Peter,

This is not a known issue, so I'd like to figure out what you're seeing and
fix any problems. If you could share a few log lines exhibiting this
behavior, that'd be very helpful (any IP addresses, usernames, etc. can be
redacted or modified as long as the issue is still clear).

There are actually two Kerberos analyzers - one for TCP and one for UDP.
TCP should be a bit more reliable, but for UDP who the originator is and
who the responder is is simply an educated guess. The guess is mainly based
off of the port numbers - if a packet is going to 88/udp, it's assumed to
be from the originator to the responder.

Both the request and response packets will be written out as a single log
line, with the same originator and responder. This is consistent with other
Bro logs - the originator and responder don't refer to the packet, but to
the transaction as a whole. Loosely speaking, the originator can be thought
of as "the host that sent the request," while the responder is "the host
that replied to the request."

  --Vlad

On Wed, Jul 29, 2015 at 3:38 PM, Peter Hansen <pch66 at cornell.edu> wrote:

> Hello all,
>
> I have been working with Kerberos in bro for a bit, and a problem I am
> consistently having is that for some reason with Kerberos packets, Bro
> cannot correctly identify the correct originator IP address in
> kerberos.log. It appears that the response packets are having their orig_h
> and resp_h values (and corresponding ports) swapped, so all connections
> made in the transfer are incorrectly identified as having the same
> originating IP address.
>
> Is this a known issue? Am I doing something wrong? Looking at the packets
> in wireshark correctly identifies them.
>
> Thanks,
> Peter
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150803/b456d326/attachment.html 

From dnthayer at illinois.edu  Mon Aug  3 10:06:28 2015
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Mon, 3 Aug 2015 12:06:28 -0500
Subject: [Bro] Store PCAP logs
In-Reply-To: <CAGokiGNtBH-v7SKt=2yVTZfpRUNsD9EG8vVua-z72oha-dMxSg@mail.gmail.com>
References: <CAGokiGNtBH-v7SKt=2yVTZfpRUNsD9EG8vVua-z72oha-dMxSg@mail.gmail.com>
Message-ID: <55BF9F94.9040901@illinois.edu>

Bro can generate pcap files with the "-w" command-line option.
Example:
bro -i eth0 -w output.pcap


On 08/03/2015 08:14 AM, ??? wrote:
> Hello,
>
> I've installed Bro IDS on my computer, and I want to know is it possible
> to make Bro generate pcap logs? Because I want to use Wireshark to
> analyze Bro logs.
> Another question, does anyone tried Splunk to analyze Bro logs? Can
> anyone give me some advice?
>
> Any help would be great. Thank You.
>

From slagell at illinois.edu  Mon Aug  3 10:18:08 2015
From: slagell at illinois.edu (Slagell, Adam J)
Date: Mon, 3 Aug 2015 17:18:08 +0000
Subject: [Bro] Store PCAP logs
In-Reply-To: <55BF9F94.9040901@illinois.edu>
References: <CAGokiGNtBH-v7SKt=2yVTZfpRUNsD9EG8vVua-z72oha-dMxSg@mail.gmail.com>,
	<55BF9F94.9040901@illinois.edu>
Message-ID: <D650DE3D-160B-4E6E-B7CD-4C42A2DD0332@illinois.edu>

Keep in mind that you aren't analyzing Bro logs in this way, though. If all you want are pcaps, tcpdump should suffice. If you want both, this is a good solution. 



> On Aug 3, 2015, at 12:15 PM, Daniel Thayer <dnthayer at illinois.edu> wrote:
> 
> Bro can generate pcap files with the "-w" command-line option.
> Example:
> bro -i eth0 -w output.pcap
> 
> 
>> On 08/03/2015 08:14 AM, ??? wrote:
>> Hello,
>> 
>> I've installed Bro IDS on my computer, and I want to know is it possible
>> to make Bro generate pcap logs? Because I want to use Wireshark to
>> analyze Bro logs.
>> Another question, does anyone tried Splunk to analyze Bro logs? Can
>> anyone give me some advice?
>> 
>> Any help would be great. Thank You.
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From kevin at branchnetconsulting.com  Tue Aug  4 07:18:52 2015
From: kevin at branchnetconsulting.com (Kevin Branch)
Date: Tue, 4 Aug 2015 10:18:52 -0400
Subject: [Bro] Adjusting Bro snaplen caused multiple Security Onion systems
 to sporadically kernel panic
Message-ID: <CA+dGL9HvssoTpXcfTvDxNsYQ1xT3gQdU97mGJKEtkYHkVuaFiQ@mail.gmail.com>

Hi Seth,

Since getting involved with Security Onion over a year ago, I have really
grown to appreciate the value that Bro brings to the NSM scene.  Thanks to
you and the rest of the Bro community for your work on this!

Not long ago I noticed that my Bro instances were eating up way more memory
than necessary for my non-jumbo-frame environment because of the default
snaplen of 8192, so based on what I saw in your comment here a couple of
years ago:

https://groups.google.com/d/msg/security-onion/qDU23hx6Q5g/xFeRDJbi9LsJ

... I added "redef snaplen = 1514;" to my local.bro file on 6 different
Security Onion systems and reclaimed a heap of memory.  At first it seemed
to work great but after a short while three completely separate systems
start throwing kernel panics.  I finally traced it down to the snaplen
change in Bro and upon raising the snaplen from 1514 to 1600, all kernel
panics completely stopped.

I am looking for your recommendation as to how low should be considered
"safe" to change the Bro snaplen to for non-jumbo-frame environments, as
well as to confirm that the old redef method I used to do this is still
what should be used with modern Bro.

I brought up my kernel panic issues on the Security Onion support list and
am hoping to report back to them how changing Bro's snaplen can be safely
used to save memory on systems where numerous Bro instances and/or large
PF_RINGs are in use.  Here is that thread for your reference:

https://groups.google.com/forum/#!searchin/security-onion-testing/kernel$20panic/security-onion-testing/H_21-0DGM6E/qnSt0BPA7lMJ


Thanks again for a great tool!
Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150804/2b387c61/attachment.html 

From earl.eiland at root9b.com  Tue Aug  4 12:34:38 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Tue, 4 Aug 2015 19:34:38 +0000
Subject: [Bro] error declaring a record
Message-ID: <BY1PR0401MB1382D683ECD41E5C36E12E76D7760@BY1PR0401MB1382.namprd04.prod.outlook.com>

I?m declaring a record, and bro is reporting a syntax error ?at or near ?type??.

I?ve compared my code to functional scripts, and can find examples where variables are declared as type ?addr?.  However, the code below does not work ?.  I can replace the ?addr? type with ?string? and the declaration is accepted.

export
    {
    redef enum Log::ID += {LOG};
    }

type Service_observed_key: record
    {
    conversation:  addr;
    };

Please advise!

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150804/172b53fe/attachment.html 

From nellieyun at gmail.com  Wed Aug  5 06:12:01 2015
From: nellieyun at gmail.com (Nuyun Zhang)
Date: Wed, 5 Aug 2015 09:12:01 -0400
Subject: [Bro] Does Bro generate only one event for one network connection?
Message-ID: <CAK8DPZriFB8B7Dsp8meRxJaMUfcRVuyiXjh2o6Q-dhrOJRLb9g@mail.gmail.com>

Dear Bro team,

   I have a question about Bro. Does Bro generate only one event for one
packet/connection? Or Bro will generate multiple events for one
packet/connection?
    I have read the paper "Bro: A system for Decting Network Intruder in
Real-time." The example showed Bro did generate a "Finger" event when the
connection meet more conditions instead of a TCP_connection event. Is this
always true?

    Thanks!
--
Nuyun Zhang (Nellie) Ph.D.
Research Associate
CCIT of Clemson University
http://people.clemson.edu/~nuyun/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150805/aa2a288e/attachment.html 

From earl.eiland at root9b.com  Wed Aug  5 06:47:45 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Wed, 5 Aug 2015 13:47:45 +0000
Subject: [Bro] error declaring a record
In-Reply-To: <BY1PR0401MB1382D683ECD41E5C36E12E76D7760@BY1PR0401MB1382.namprd04.prod.outlook.com>
References: <BY1PR0401MB1382D683ECD41E5C36E12E76D7760@BY1PR0401MB1382.namprd04.prod.outlook.com>
Message-ID: <BY1PR0401MB138232572F4B7DA3DE5620BFD7750@BY1PR0401MB1382.namprd04.prod.outlook.com>

For any others that may run into a similar situation, ?conversation? is a reserved word In bro.  The problem disappeared when I renamed the variable.

Earl

From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Earl Eiland
Sent: Tuesday, August 4, 2015 2:35 PM
To: bro at bro.org
Subject: [Bro] error declaring a record

I?m declaring a record, and bro is reporting a syntax error ?at or near ?type??.

I?ve compared my code to functional scripts, and can find examples where variables are declared as type ?addr?.  However, the code below does not work ?.  I can replace the ?addr? type with ?string? and the declaration is accepted.

export
    {
    redef enum Log::ID += {LOG};
    }

type Service_observed_key: record
    {
    conversation:  addr;
    };

Please advise!

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150805/75431090/attachment.html 

From dnthayer at illinois.edu  Wed Aug  5 09:36:58 2015
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Wed, 5 Aug 2015 11:36:58 -0500
Subject: [Bro] Does Bro generate only one event for one network
	connection?
In-Reply-To: <CAK8DPZriFB8B7Dsp8meRxJaMUfcRVuyiXjh2o6Q-dhrOJRLb9g@mail.gmail.com>
References: <CAK8DPZriFB8B7Dsp8meRxJaMUfcRVuyiXjh2o6Q-dhrOJRLb9g@mail.gmail.com>
Message-ID: <55C23BAA.9030600@illinois.edu>

It is possible for Bro to generate more than one event.
For example, it is possible for one UDP packet to generate
both "udp_reply" and "udp_contents" events.
Similarly, an HTTP request will cause Bro to generate an
"http_request" event and a "tcp_packet" event.

All of the Bro events are described in the documentation:
https://www.bro.org/sphinx/script-reference/proto-analyzers.html
https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html


On 08/05/2015 08:12 AM, Nuyun Zhang wrote:
> Dear Bro team,
>
>     I have a question about Bro. Does Bro generate only one event for
> one packet/connection? Or Bro will generate multiple events for one
> packet/connection?
>      I have read the paper "Bro: A system for Decting Network Intruder
> in Real-time." The example showed Bro did generate a "Finger" event when
> the connection meet more conditions instead of a TCP_connection event.
> Is this always true?
>      Thanks!
> --
> Nuyun Zhang (Nellie) Ph.D.
> Research Associate
> CCIT of Clemson University
> http://people.clemson.edu/~nuyun/

From mdblack98 at gmail.com  Wed Aug  5 10:08:16 2015
From: mdblack98 at gmail.com (Michael Black)
Date: Wed, 5 Aug 2015 12:08:16 -0500
Subject: [Bro] No info record
In-Reply-To: <009101d0cb99$232b9d00$6982d700$@gmail.com>
References: <009101d0cb99$232b9d00$6982d700$@gmail.com>
Message-ID: <CAMYKG0YNdmC2c=7VXYM_Y-h=KBaxhBRhK+e2EDS=2zb4H8-bGg@mail.gmail.com>

Fixed the problem by using connection_state_remove event instead.
It appears many protocols don't add the conn fields until after the
connection_finished event.

Mike

On Fri, Jul 31, 2015 at 8:59 AM, Michael Black <mdblack98 at gmail.com> wrote:

> Using 2.4
>
>
>
> I'm having a problem in a connection_finished event.  I've extended the
> connection record with an extra field.
>
>
>
> But?.processing  a 512MB capture file I have I get a number of connection
> events that don't have a c$conn record in them.
>
> I get the same behavior using connection_EOF.
>
>
>
> This script demonstrates the problem.  I've attached a sample of the
> conn.log records that show a mix of good/bad where you can see the TEST1
> and N/A default on the non-conn records.
>
>
>
> 1426100429.761609 expression error in ./test.bro, line 11: field value
> missing [c$conn]
>
> It seems that if there is no "string" value or if it's an ssl, dns, for
> example, then there is no $conn field.
>
> Is there an extendable record in a connection record that is ALWAYS there?
>
>
>
> @load base/utils/site
>
> @load base/protocols/conn
>
>
>
> redef record Conn::Info += {
>
>         testfield: string &default="N/A" &log;
>
> };
>
>
>
> event connection_finished(c: connection)
>
> {
>
>         if (!c?$conn) {
>
>                 c$conn$testfield = "TEST2";
>
>         }
>
>         else {
>
>                 print("TEST1");
>
>                 c$conn$testfield = "TEST1";
>
>         }
>
> }
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150805/54c1d913/attachment.html 

From billcyz at gmail.com  Wed Aug  5 18:46:26 2015
From: billcyz at gmail.com (=?UTF-8?B?6ZmI5pix56u5?=)
Date: Thu, 6 Aug 2015 13:46:26 +1200
Subject: [Bro] Bro scripts to detect network attacks
Message-ID: <CAGokiGOCw6xbPXLZpzbuH--zzO1qY2NSG1=XZYO+LhLks9DQWQ@mail.gmail.com>

Hello

I'm running Bro in my network, and I want to perform attacks to test its
capabilities and create alert information.
I choose to use SYN Flood Attack and ARP Spoofing Attack, can anyone tell
me where to find these scripts?

Any help would be great. Thank You.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150806/54199a0b/attachment.html 

From maxfeldman14 at gmail.com  Thu Aug  6 11:15:08 2015
From: maxfeldman14 at gmail.com (Max Feldman)
Date: Thu, 6 Aug 2015 11:15:08 -0700
Subject: [Bro] Bro scripts to detect network attacks
In-Reply-To: <CAGokiGOCw6xbPXLZpzbuH--zzO1qY2NSG1=XZYO+LhLks9DQWQ@mail.gmail.com>
References: <CAGokiGOCw6xbPXLZpzbuH--zzO1qY2NSG1=XZYO+LhLks9DQWQ@mail.gmail.com>
Message-ID: <CAE9iDQY659qcnYRXEFGVd0+iKy_HUSS0nuxZpoHmK-qB4YDn=g@mail.gmail.com>

It looks like there's a syn flood detection script here:
http://www.gnu-darwin.org/www001/src/ports/security/bro/work/bro-1.2.1/policy/synflood.bro
and an arp spoofing detection script here:
https://github.com/maxfeldman14/brospects/blob/master/arpspoof.bro

But I'm not sure if there is anything more up-to-date.

On Wed, Aug 5, 2015 at 6:46 PM, ??? <billcyz at gmail.com> wrote:

> Hello
>
> I'm running Bro in my network, and I want to perform attacks to test its
> capabilities and create alert information.
> I choose to use SYN Flood Attack and ARP Spoofing Attack, can anyone tell
> me where to find these scripts?
>
> Any help would be great. Thank You.
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Max Feldman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150806/4dab928a/attachment.html 

From capn.freako at gmail.com  Fri Aug  7 09:07:10 2015
From: capn.freako at gmail.com (David Banas)
Date: Fri, 7 Aug 2015 09:07:10 -0700
Subject: [Bro] Error trying to 'make doc' in current GitHub head.
In-Reply-To: <20150802001953.GA10901@Beezling.home>
References: <6A755C35-5228-494C-99E1-50F944BE2094@gmail.com>
	<20150802001953.GA10901@Beezling.home>
Message-ID: <EAE0E3A5-9749-43EA-9B1E-0D514DD9E529@gmail.com>

Thanks! That worked.
-db

On Aug 1, 2015, at 5:19 PM, Johanna Amann <johanna at icir.org> wrote:

> Try to increase the file ulimit (ulimit -n). I assume you are on OS X --
> it is a bit low there with only 256 simultaneous open files.
> 
> Johanna
> 
> On Sat, Aug 01, 2015 at 04:59:21PM -0700, David Banas wrote:
>> Hi all,
>> 
>> I?m getting this error, when attempting a ?make doc? from within the root directory of the current GitHub head:
>> 
>> Davids-MacBook-Air-2:bro dbanas$ git show
>> commit 30bb17ea8d1faa7bb1c2ff41aa1579523fcbe273
>> Author: Robin Sommer <robin at icir.org>
>> Date:   Fri Jul 24 15:06:07 2015 -0700
>> 
>> [100%] [Sphinx] Generate HTML documentation in /Users/dbanas/Documents/Projects/bro/build/html
>> error in /Users/dbanas/Documents/Projects/bro/scripts/policy/protocols/ftp/detect-bruteforcing.bro, line 4: Failed to open file /Users/dbanas/Documents/Projects/bro/scripts/policy/protocols/ftp/detect-bruteforcing.bro: Too many open files
>> fatal error in /Users/dbanas/Documents/Projects/bro/scripts/policy/protocols/ftp/detect-bruteforcing.bro, line 4: can't open /Users/dbanas/Documents/Projects/bro/scripts/base/protocols/ftp/__load__.bro
>> 
>> Anyone know what?s going on?
>> 
>> Thanks,
>> -db
>> 
> 
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 



From earl.eiland at root9b.com  Fri Aug  7 11:01:13 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Fri, 7 Aug 2015 18:01:13 +0000
Subject: [Bro] Conflicting errors when creating a table
Message-ID: <BY1PR0401MB1382FABD5893D1EA0E61F699D7730@BY1PR0401MB1382.namprd04.prod.outlook.com>

Hello.

I'm trying to read in a table where the key is a set.  The Input::add_table operation seems fine with this, but the table declaration fails.  The line "global whitelist: table[set[addr]] of Service_whitelist_data = table(); " generates the error "bad index type (set[addr])"

I can avoid this error by declaring "global whitelist: table[addr] of Service_whitelist_data = table(); ", but then receive the error "Input stream whitelist: Table type does not match index type. Need type 'table':set[addr], got 'addr':addr".

I'm assuming that my type declaration "table[set[addr]]" is incorrect, and that the error message is suggesting a solution.  I've tried a number of type declaration variations: all have failed.  I've also looked through the documentation and input framework examples, but came away empty handed.  How do I satisfy the 'table':set[addr] requirement?

Best Regards,

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150807/d92f1610/attachment.html 

From damonrouse at gmail.com  Fri Aug  7 11:41:06 2015
From: damonrouse at gmail.com (Damon Rouse)
Date: Fri, 7 Aug 2015 11:41:06 -0700
Subject: [Bro] Bro Crashing on Reboot (Cluster-Layout)
Message-ID: <CAHoWat4Y-7nZix8PxbWXvS8tR8tz8fWHe82uyicJbtUHZtpJYw@mail.gmail.com>

I posted this to the SO list, but haven't received any responses, so wanted
to post here too.

Anytime I need to reboot my SO box, Bro tries to start normally but then
terminates and I get crash alerts.  They seem to point to something with
the cluster layout, but I'm not sure how to fix it or why it started.

I usually need to a couple rounds of broctl deploy, kill bro, then do the
same and it eventually starts fine.

This Bro 2.4 under Security Onion

See the crash log below--any ideas or help is much appreciated

Crash Report from Manager:
Bro 2.4
Linux 3.2.0-88-generic


==== No reporter.log

==== stderr.log
fatal error in /opt/bro/share/bro/base/frameworks/cluster/__load__.bro,
line 16: can't find cluster-layout

==== stdout.log
max memory size         (kbytes, -m) unlimited
data seg size           (kbytes, -d) unlimited
virtual memory          (kbytes, -v) unlimited
core file size          (blocks, -c) unlimited

==== .cmdline
-U .status -p broctl -p broctl-live -p local -p manager local.bro broctl
base/frameworks/cluster local-manager.bro broctl/auto

==== .env_vars
PATH=/opt/bro/bin:/opt/bro/share/broctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
BROPATH=/nsm/bro/spool/installed-scripts-do-not-touch/site::/nsm/bro/spool/installed-scripts-do-not-touch/auto:/opt/bro/share/bro:/opt/bro/share/bro/policy:/opt/bro/share/bro/site
CLUSTER_NODE=manager

==== .status
TERMINATED [atexit]

==== No prof.log

==== No packet_filter.log

==== No loaded_scripts.log
--
[Automatically generated.]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150807/47cf1525/attachment.html 

From dnthayer at illinois.edu  Fri Aug  7 12:39:50 2015
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Fri, 7 Aug 2015 14:39:50 -0500
Subject: [Bro] Conflicting errors when creating a table
In-Reply-To: <BY1PR0401MB1382FABD5893D1EA0E61F699D7730@BY1PR0401MB1382.namprd04.prod.outlook.com>
References: <BY1PR0401MB1382FABD5893D1EA0E61F699D7730@BY1PR0401MB1382.namprd04.prod.outlook.com>
Message-ID: <55C50986.1090509@illinois.edu>

The error message "bad index type" means that a table's index type
cannot be a set.

To solve this problem, you will probably need to add a new column
to your input file.  The new column will be the index to your table.


On 08/07/2015 01:01 PM, Earl Eiland wrote:
> Hello.
>
> I?m trying to read in a table where the key is a set.  The
> Input::add_table operation seems fine with this, but the table
> declaration fails.  The line ?global whitelist: table[set[addr]] of
> Service_whitelist_data = table(); ? generates the error ?bad index type
> (set[addr])?
>
> I can avoid this error by declaring ?global whitelist: table[addr] of
> Service_whitelist_data = table(); ?, but then receive the error ?Input
> stream whitelist: Table type does not match index type. Need type
> 'table':set[addr], got 'addr':addr?.
>
> I?m assuming that my type declaration ?table[set[addr]]? is incorrect,
> and that the error message is suggesting a solution.  I?ve tried a
> number of type declaration variations: all have failed.  I?ve also
> looked through the documentation and input framework examples, but came
> away empty handed.  How do I satisfy the 'table':set[addr] requirement?
>
> Best Regards,
>
> Earl Eiland,
>
> Sr. Cyber Security Engineer,
>
> Emerging Technologies, root9B,
>
> San Antonio, Texas
>

From capn.freako at gmail.com  Fri Aug  7 17:35:25 2015
From: capn.freako at gmail.com (David Banas)
Date: Fri, 7 Aug 2015 17:35:25 -0700
Subject: [Bro] Anything unusual about the broker_data_as_vector() function?
Message-ID: <498F8005-799E-44FF-8658-3364CDFCEC0C@gmail.com>

Hi all,

I?m trying to use the broker_data_as_vector() function and am getting a null pointer returned to me.
I?ve checked the validity of the broker_data* value I?m sending in.
Is there anything unusual about this function? Why might it return a null pointer?

Thanks,
-db

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150807/2cbd0eed/attachment.html 

From capn.freako at gmail.com  Sat Aug  8 07:46:42 2015
From: capn.freako at gmail.com (David Banas)
Date: Sat, 8 Aug 2015 07:46:42 -0700
Subject: [Bro] Where is broker::get<T> defined?
Message-ID: <3A74C84C-8779-4555-859D-D0D75EF2E6B2@gmail.com>

Hi all,

Does anyone know where broker::get<T> is defined?

Thanks,
-db


From gfaulkner.nsm at gmail.com  Sun Aug  9 16:45:01 2015
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Sun, 9 Aug 2015 18:45:01 -0500
Subject: [Bro] Passing options to Bro plugins from a file
Message-ID: <55C7E5FD.703@gmail.com>

I'd like to read options into a plugin such as IP addresses, hostnames,
ports, etc from a configuration file. Looking at the redis plugin for
inspiration, it looks like I could set types in the bif, set initial
values in the plugin's init.bro and then redef in local.bro? Is this
correct? Am I missing anything?

For example I'd like to have a bro script that can call a function from
scriptland to shove data into external DB. A use case may be something
such as writing time series data directly to InfluxDB. Instead of hard
coding the IP, port, etc into the plugin I'd like to be able to specify
the location in a file. I might also use the file to define a tag
structure for InfluxDB.

 

From capn.freako at gmail.com  Mon Aug 10 06:22:21 2015
From: capn.freako at gmail.com (David Banas)
Date: Mon, 10 Aug 2015 06:22:21 -0700
Subject: [Bro] Trouble passing a message between two local endpoints.
Message-ID: <36E1532B-83FB-4939-8AED-4FC748C80AE6@gmail.com>

Hi all,

I?m trying to pass a message between two local endpoints, using only the C-interface in broker.h:

#include <stdio.h>
#include "broker.h"

int main (int argc, char* argv[]) {
    broker_init(0);
    broker_endpoint* ep1 = broker_endpoint_create_with_flags("ep1", 3);
    broker_endpoint* ep2 = broker_endpoint_create_with_flags("ep2", 3);
    broker_peering*    p = broker_endpoint_peer_locally(ep2, ep1);
    
    broker_string*       bs = broker_string_create("");
    broker_message_queue* q = broker_message_queue_create(bs, ep2);

    broker_string*  msg_str = broker_string_create("Hello, World!\n");
    broker_string*    topic = broker_string_create("test");
    broker_data*        msg = broker_data_from_string(msg_str);
    
    broker_vector* vec = broker_vector_create();
    int            res = broker_vector_insert(vec, msg, 0L);
                   res = broker_endpoint_send(ep1, topic, vec);
    
    broker_deque_of_message* msg_list = broker_message_queue_want_pop(q);
    size_t                   num_msgs = broker_deque_of_message_size(msg_list);
    
    printf("There are %ld messages.\n", num_msgs);
}

And I?m failing:

davids-air-2:broker-haskell dbanas$ ./a.out 
There are 0 messages.

Does anyone see what I?m doing wrong?

Thanks!
-db

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150810/1f4f3b78/attachment.html 

From earl.eiland at root9b.com  Mon Aug 10 07:13:03 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Mon, 10 Aug 2015 14:13:03 +0000
Subject: [Bro] error inputting a table with sets in '$val'
Message-ID: <BY1PR0401MB13822268204FC8C9EF525AFFD7700@BY1PR0401MB1382.namprd04.prod.outlook.com>

Hello.

I?m reading a table into a script.  The table includes two sets in the values fields.  When executing the script, I?m getting the error message ?Did not find requested field service in input data file model2.log?

Following the example in bro/testing/btest/scripts/base/frameworks/input/setseparator.bro, I?ve redefined the set separator as ?|? (redef InputAscii::set_separator = "|";).

The

The table key consists of two addresses, node_A and node_B.
My value inputs consist of two sets, which can consist of just a single value; all fields are separated by tabs.  The first two lines  of my input file are:
#fields  node_A                                node_B                                layer_3_4            service
xxx.yyy.zzz.30   xxx.yyy.255.255                udp        dns

xxx.yyy.zzz are valid IP address values.
It appears that the strings ?udp? and ?dns? are both being read as part of the layer_3_4 set.  Since they are separated by a tab instead of ?|?, they should be interpreted as separate fields.  How do I correct this read error?

Best Regards,

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150810/6f49753b/attachment.html 

From jsiwek at illinois.edu  Mon Aug 10 07:34:55 2015
From: jsiwek at illinois.edu (Siwek, Jon)
Date: Mon, 10 Aug 2015 14:34:55 +0000
Subject: [Bro] Anything unusual about the broker_data_as_vector()
 function?
In-Reply-To: <498F8005-799E-44FF-8658-3364CDFCEC0C@gmail.com>
References: <498F8005-799E-44FF-8658-3364CDFCEC0C@gmail.com>
Message-ID: <DDEF987A-5D7F-48DF-97C5-859D6B7ACE32@illinois.edu>


> On Aug 7, 2015, at 7:35 PM, David Banas <capn.freako at gmail.com> wrote:
> 
> I?m trying to use the broker_data_as_vector() function and am getting a null pointer returned to me.
> I?ve checked the validity of the broker_data* value I?m sending in.
> Is there anything unusual about this function? Why might it return a null pointer?

I think the only reason would be if the broker_data* argument isn?t actually a vector type.  You could use broker_data_which(), for example, to inspect the actual type as a sanity check.

- Jon

From jsiwek at illinois.edu  Mon Aug 10 07:36:00 2015
From: jsiwek at illinois.edu (Siwek, Jon)
Date: Mon, 10 Aug 2015 14:36:00 +0000
Subject: [Bro] Where is broker::get<T> defined?
In-Reply-To: <3A74C84C-8779-4555-859D-D0D75EF2E6B2@gmail.com>
References: <3A74C84C-8779-4555-859D-D0D75EF2E6B2@gmail.com>
Message-ID: <4633F225-1A04-450E-8476-BCEF7CEAEF3E@illinois.edu>


> On Aug 8, 2015, at 9:46 AM, David Banas <capn.freako at gmail.com> wrote:
> 
> Does anyone know where broker::get<T> is defined?

broker/util/variant.hh

- Jon


From robin at icir.org  Mon Aug 10 08:00:59 2015
From: robin at icir.org (Robin Sommer)
Date: Mon, 10 Aug 2015 08:00:59 -0700
Subject: [Bro] Passing options to Bro plugins from a file
In-Reply-To: <55C7E5FD.703@gmail.com>
References: <55C7E5FD.703@gmail.com>
Message-ID: <20150810150059.GE83844@icir.org>



On Sun, Aug 09, 2015 at 18:45 -0500, you wrote:

> I'd like to read options into a plugin such as IP addresses, hostnames,
> ports, etc from a configuration file. Looking at the redis plugin for
> inspiration, it looks like I could set types in the bif, set initial
> values in the plugin's init.bro and then redef in local.bro? Is this
> correct? Am I missing anything?

Yes, correct. It's kind of the "normal" Bro way: you define a set of
global options with defaults through the bifs and people can redef
them as they need to. A few of the plugins do it this way, another
example is dataseries/src/dataseries.bif.

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From jsiwek at illinois.edu  Mon Aug 10 08:10:15 2015
From: jsiwek at illinois.edu (Siwek, Jon)
Date: Mon, 10 Aug 2015 15:10:15 +0000
Subject: [Bro] Trouble passing a message between two local endpoints.
In-Reply-To: <36E1532B-83FB-4939-8AED-4FC748C80AE6@gmail.com>
References: <36E1532B-83FB-4939-8AED-4FC748C80AE6@gmail.com>
Message-ID: <230469BF-DEB0-4FFC-9A1B-4690AD15B9ED@illinois.edu>


> On Aug 10, 2015, at 8:22 AM, David Banas <capn.freako at gmail.com> wrote:
> 
> Does anyone see what I?m doing wrong?

There?s a few race conditions to be aware of:

1) For the sample code you made, you want to establish/create the message queues attached to an endpoint before initiating peerings/connections with other endpoints.  The point here is to allow topic advertisements/subscriptions to be established before actually sending any messages.

2) If you request a message to be sent before the connection w/ a peer is actually established, it may just get dropped because it?s seen that no one is interested in that message.  In your example, you can check the outgoing/incoming connection status queues to wait for the connection to establish.  Using the blocking ?need? version of the function works fine as a convenience in this case, but I?d think it more typical in real code to use the non-blocking ?want? version and to have integrated queues into event loop (e.g. select(), poll(), etc.)

3) Using the non-blocking ?want? version of popping the message queue doesn?t give any time for the message to actually be sent and arrive at the peer endpoint.  Either integrate into an event loop or just use the blocking ?need? version to wait for the message to arrive.

Here?s an example of revising your code w/ those 3 suggestions:

#include <stdio.h>
#include "broker.h"

int main (int argc, char* argv[]) {
    broker_init(0);
    broker_endpoint* ep1 = broker_endpoint_create_with_flags("ep1", 3);
    broker_endpoint* ep2 = broker_endpoint_create_with_flags("ep2", 3);
    broker_string*       bs = broker_string_create("");
    broker_message_queue* q = broker_message_queue_create(bs, ep2);
    
    broker_peering*    p = broker_endpoint_peer_locally(ep2, ep1);

    const broker_outgoing_connection_status_queue* ocsq =
            broker_endpoint_outgoing_connection_status(ep2);
    broker_deque_of_outgoing_connection_status_delete(
        broker_outgoing_connection_status_queue_need_pop(ocsq));

    broker_string*  msg_str = broker_string_create("Hello, World!\n");
    broker_string*    topic = broker_string_create("test");
    broker_data*        msg = broker_data_from_string(msg_str);
    
    broker_vector* vec = broker_vector_create();
    int            res = broker_vector_insert(vec, msg, 0L);
                   res = broker_endpoint_send(ep1, topic, vec);
    
    broker_deque_of_message* msg_list = broker_message_queue_need_pop(q);
    size_t                   num_msgs = broker_deque_of_message_size(msg_list);
    
    printf("There are %ld messages.\n", num_msgs);
}

Hope that helps.

- Jon

From vstoffer at lbl.gov  Mon Aug 10 08:53:29 2015
From: vstoffer at lbl.gov (Vincent Stoffer)
Date: Mon, 10 Aug 2015 08:53:29 -0700
Subject: [Bro] 100G Bro monitoring technical paper
Message-ID: <CAER+47JsoDEYPf+_U78H5nxNfCA7J6qN39-PEtK4oetz+V3BrA@mail.gmail.com>

Hello,

As announced at Brocon, we have completed the technical document which
describes the architecture of our 100G Bro monitoring system.  As part of
our project, we created this comprehensive document meant to be shared
widely within the security community:

http://go.lbl.gov/100g

The document begins with the background and design decisions and then
describes the build process including specific part numbers and
configurations.  We also include a review of  performance and a description
of our shunting mechanism, which increases performance by removing large
and long-running flows from analysis.

Please feel free to share this link and the document with anyone and direct
any questions or comments to security at lbl.gov.  A huge thanks to the many
folks in our community who helped influence the design of the system and
this document.

Thank you,

Vince

-- 
Vincent Stoffer, Cyber Security Engineer
Cyber Security, Information Technology Division
Lawrence Berkeley National Laboratory
(510) 486-4531
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150810/7ad75827/attachment.html 

From jxbatchelor at gmail.com  Mon Aug 10 11:17:11 2015
From: jxbatchelor at gmail.com (Jason Batchelor)
Date: Mon, 10 Aug 2015 13:17:11 -0500
Subject: [Bro] Specifying File Extraction Limit
Message-ID: <CANWz5X9riG6=j_fn7m+u-qomUri51Q+qmrD8rs1fq2WCa++hKA@mail.gmail.com>

Hello all:

With the 2.4 release is it still best practice so specify file extraction
size limit as follows...

Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname,
$extract_limit=<FILE LIMIT>]);

I ask because I seem to be getting files extracted greater than my imposed
limit on occasion and was wondering if something had changed?

Thanks,
Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150810/281ef721/attachment.html 

From damonrouse at gmail.com  Mon Aug 10 11:53:45 2015
From: damonrouse at gmail.com (Damon Rouse)
Date: Mon, 10 Aug 2015 11:53:45 -0700
Subject: [Bro] Specifying File Extraction Limit
In-Reply-To: <CANWz5X9riG6=j_fn7m+u-qomUri51Q+qmrD8rs1fq2WCa++hKA@mail.gmail.com>
References: <CANWz5X9riG6=j_fn7m+u-qomUri51Q+qmrD8rs1fq2WCa++hKA@mail.gmail.com>
Message-ID: <CAHoWat49i=S1+fvN3GUXDMTE98g_FE5woarUAg-_Ete742nn_w@mail.gmail.com>

I seem to have having a similar issue with the way I was limiting the size
of my extracted files too.  Under 2.3.2, popping the following redef in my
local.bro worked perfectly:  redef FileExtract::default_limit = 25000000;

Under 2.4, I have larger files being extracted like Jason.

Thanks
Damon

On Mon, Aug 10, 2015 at 11:17 AM, Jason Batchelor <jxbatchelor at gmail.com>
wrote:

> Hello all:
>
> With the 2.4 release is it still best practice so specify file extraction
> size limit as follows...
>
> Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname,
> $extract_limit=<FILE LIMIT>]);
>
> I ask because I seem to be getting files extracted greater than my imposed
> limit on occasion and was wondering if something had changed?
>
> Thanks,
> Jason
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150810/3433f81e/attachment.html 

From capn.freako at gmail.com  Mon Aug 10 12:05:22 2015
From: capn.freako at gmail.com (David Banas)
Date: Mon, 10 Aug 2015 12:05:22 -0700
Subject: [Bro] Trouble passing a message between two local endpoints.
In-Reply-To: <230469BF-DEB0-4FFC-9A1B-4690AD15B9ED@illinois.edu>
References: <36E1532B-83FB-4939-8AED-4FC748C80AE6@gmail.com>
	<230469BF-DEB0-4FFC-9A1B-4690AD15B9ED@illinois.edu>
Message-ID: <71F7EEFF-A244-4775-8BF2-F3F790C35C87@gmail.com>

Hi Jon,

Thanks for your reply!

I implemented your recommended changes, recompiled, and re-ran, but am getting the same result:

davids-air-2:broker-haskell dbanas$ ./a.out 
There are 0 messages.

Any thoughts?

Thanks!
-db

On Aug 10, 2015, at 8:10 AM, Siwek, Jon <jsiwek at illinois.edu> wrote:

> 
>> On Aug 10, 2015, at 8:22 AM, David Banas <capn.freako at gmail.com> wrote:
>> 
>> Does anyone see what I?m doing wrong?
> 
> There?s a few race conditions to be aware of:
> 
> 1) For the sample code you made, you want to establish/create the message queues attached to an endpoint before initiating peerings/connections with other endpoints.  The point here is to allow topic advertisements/subscriptions to be established before actually sending any messages.
> 
> 2) If you request a message to be sent before the connection w/ a peer is actually established, it may just get dropped because it?s seen that no one is interested in that message.  In your example, you can check the outgoing/incoming connection status queues to wait for the connection to establish.  Using the blocking ?need? version of the function works fine as a convenience in this case, but I?d think it more typical in real code to use the non-blocking ?want? version and to have integrated queues into event loop (e.g. select(), poll(), etc.)
> 
> 3) Using the non-blocking ?want? version of popping the message queue doesn?t give any time for the message to actually be sent and arrive at the peer endpoint.  Either integrate into an event loop or just use the blocking ?need? version to wait for the message to arrive.
> 
> Here?s an example of revising your code w/ those 3 suggestions:
> 
> #include <stdio.h>
> #include "broker.h"
> 
> int main (int argc, char* argv[]) {
>    broker_init(0);
>    broker_endpoint* ep1 = broker_endpoint_create_with_flags("ep1", 3);
>    broker_endpoint* ep2 = broker_endpoint_create_with_flags("ep2", 3);
>    broker_string*       bs = broker_string_create("");
>    broker_message_queue* q = broker_message_queue_create(bs, ep2);
> 
>    broker_peering*    p = broker_endpoint_peer_locally(ep2, ep1);
> 
>    const broker_outgoing_connection_status_queue* ocsq =
>            broker_endpoint_outgoing_connection_status(ep2);
>    broker_deque_of_outgoing_connection_status_delete(
>        broker_outgoing_connection_status_queue_need_pop(ocsq));
> 
>    broker_string*  msg_str = broker_string_create("Hello, World!\n");
>    broker_string*    topic = broker_string_create("test");
>    broker_data*        msg = broker_data_from_string(msg_str);
> 
>    broker_vector* vec = broker_vector_create();
>    int            res = broker_vector_insert(vec, msg, 0L);
>                   res = broker_endpoint_send(ep1, topic, vec);
> 
>    broker_deque_of_message* msg_list = broker_message_queue_need_pop(q);
>    size_t                   num_msgs = broker_deque_of_message_size(msg_list);
> 
>    printf("There are %ld messages.\n", num_msgs);
> }
> 
> Hope that helps.
> 
> - Jon

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150810/4c18d477/attachment.html 

From seth at icir.org  Mon Aug 10 12:47:02 2015
From: seth at icir.org (Seth Hall)
Date: Mon, 10 Aug 2015 15:47:02 -0400
Subject: [Bro] Specifying File Extraction Limit
In-Reply-To: <CAHoWat49i=S1+fvN3GUXDMTE98g_FE5woarUAg-_Ete742nn_w@mail.gmail.com>
References: <CANWz5X9riG6=j_fn7m+u-qomUri51Q+qmrD8rs1fq2WCa++hKA@mail.gmail.com>
	<CAHoWat49i=S1+fvN3GUXDMTE98g_FE5woarUAg-_Ete742nn_w@mail.gmail.com>
Message-ID: <1E99DD3A-AB04-4C63-ABC7-FE8727479A54@icir.org>


> On Aug 10, 2015, at 2:53 PM, Damon Rouse <damonrouse at gmail.com> wrote:
> 
> I seem to have having a similar issue with the way I was limiting the size of my extracted files too.  Under 2.3.2, popping the following redef in my local.bro worked perfectly:  redef FileExtract::default_limit = 25000000;
> 
> Under 2.4, I have larger files being extracted like Jason.

Oh, interesting.  I don?t think we have a test case which covers that.  I suspect that it?s the file reassembly that was added into 2.4.  I filed a ticket to make sure we track this.
	https://bro-tracker.atlassian.net/browse/BIT-1451

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From blackhole.em at gmail.com  Mon Aug 10 16:00:47 2015
From: blackhole.em at gmail.com (Joe Blow)
Date: Mon, 10 Aug 2015 19:00:47 -0400
Subject: [Bro] 100G Bro monitoring technical paper
In-Reply-To: <CAER+47JsoDEYPf+_U78H5nxNfCA7J6qN39-PEtK4oetz+V3BrA@mail.gmail.com>
References: <CAER+47JsoDEYPf+_U78H5nxNfCA7J6qN39-PEtK4oetz+V3BrA@mail.gmail.com>
Message-ID: <CAMCSCoW2VD486Hk-HdL67v5aY9E6ajQAiMWEEOOXpL8Rr2nSkw@mail.gmail.com>

This is an amazing document.  It has pretty much everything you'd need to
get off the ground.  Arista configs, Bro configs, Bro hardware specs...
They did pretty much everything except build it for you.

Add a logging cluster and you've got an amazing analytics platform on top
of all of your packets.

Fantastic work fellas.

Cheers,

JB

On Mon, Aug 10, 2015 at 11:53 AM, Vincent Stoffer <vstoffer at lbl.gov> wrote:

> Hello,
>
> As announced at Brocon, we have completed the technical document which
> describes the architecture of our 100G Bro monitoring system.  As part of
> our project, we created this comprehensive document meant to be shared
> widely within the security community:
>
> http://go.lbl.gov/100g
>
> The document begins with the background and design decisions and then
> describes the build process including specific part numbers and
> configurations.  We also include a review of  performance and a description
> of our shunting mechanism, which increases performance by removing large
> and long-running flows from analysis.
>
> Please feel free to share this link and the document with anyone and
> direct any questions or comments to security at lbl.gov.  A huge thanks to
> the many folks in our community who helped influence the design of the
> system and this document.
>
> Thank you,
>
> Vince
>
> --
> Vincent Stoffer, Cyber Security Engineer
> Cyber Security, Information Technology Division
> Lawrence Berkeley National Laboratory
> (510) 486-4531
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150810/306b3b64/attachment.html 

From billcyz at gmail.com  Mon Aug 10 20:23:45 2015
From: billcyz at gmail.com (=?UTF-8?B?6ZmI5pix56u5?=)
Date: Tue, 11 Aug 2015 15:23:45 +1200
Subject: [Bro] Bro performance information
Message-ID: <CAGokiGOY4dQGWZ4M7=+O0_CoSMmtBVg=xpBhnTtEgr1CphW+VA@mail.gmail.com>

Hello everyone

I want to know how can to get performance information of Bro? Such as
number of packets received, analyzed, dropped during detection, and also
how many packets analyzed per minute? Are there any way to show these
information?

Any help would be great. Thank You.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150811/ff075293/attachment.html 

From jsiwek at illinois.edu  Tue Aug 11 07:05:41 2015
From: jsiwek at illinois.edu (Siwek, Jon)
Date: Tue, 11 Aug 2015 14:05:41 +0000
Subject: [Bro] Trouble passing a message between two local endpoints.
In-Reply-To: <71F7EEFF-A244-4775-8BF2-F3F790C35C87@gmail.com>
References: <36E1532B-83FB-4939-8AED-4FC748C80AE6@gmail.com>
	<230469BF-DEB0-4FFC-9A1B-4690AD15B9ED@illinois.edu>
	<71F7EEFF-A244-4775-8BF2-F3F790C35C87@gmail.com>
Message-ID: <13152031-5A91-4343-9080-DF7F03D2FA62@illinois.edu>


> On Aug 10, 2015, at 2:05 PM, David Banas <capn.freako at gmail.com> wrote:
> 
> I implemented your recommended changes, recompiled, and re-ran, but am getting the same result:
> 
> davids-air-2:broker-haskell dbanas$ ./a.out 
> There are 0 messages.
> 
> Any thoughts?

It shouldn?t be possible for broker_message_queue_need_pop() to return zero messages (it?s a bug if it does).  Double check your revised code against what I posted and that you compiled/ran that version.  Else post the exact code you ran and other details you may think would help reproduce those results (OS, compiler version, Broker version, CAF version, etc.).

- Jon

From capn.freako at gmail.com  Tue Aug 11 11:32:25 2015
From: capn.freako at gmail.com (David Banas)
Date: Tue, 11 Aug 2015 11:32:25 -0700
Subject: [Bro] Trouble passing a message between two local endpoints.
In-Reply-To: <230469BF-DEB0-4FFC-9A1B-4690AD15B9ED@illinois.edu>
References: <36E1532B-83FB-4939-8AED-4FC748C80AE6@gmail.com>
	<230469BF-DEB0-4FFC-9A1B-4690AD15B9ED@illinois.edu>
Message-ID: <CADz=fO2NFE7w8bsaQ5ubU2nJWqgn6GzW5OQXWuAuSDFcptTvsA@mail.gmail.com>

Hi Jon,

Thanks for your reply!

I implemented your recommended changes, recompiled, and re-ran, but am
getting the same result:

davids-air-2:broker-haskell dbanas$ ./a.out
There are 0 messages.

Any thoughts?

Thanks!
-db

On Aug 10, 2015, at 8:10 AM, Siwek, Jon <jsiwek at illinois.edu> wrote:


On Aug 10, 2015, at 8:22 AM, David Banas <capn.freako at gmail.com> wrote:

Does anyone see what I?m doing wrong?


There?s a few race conditions to be aware of:

1) For the sample code you made, you want to establish/create the message
queues attached to an endpoint before initiating peerings/connections with
other endpoints.  The point here is to allow topic
advertisements/subscriptions to be established before actually sending any
messages.

2) If you request a message to be sent before the connection w/ a peer is
actually established, it may just get dropped because it?s seen that no one
is interested in that message.  In your example, you can check the
outgoing/incoming connection status queues to wait for the connection to
establish.  Using the blocking ?need? version of the function works fine as
a convenience in this case, but I?d think it more typical in real code to
use the non-blocking ?want? version and to have integrated queues into
event loop (e.g. select(), poll(), etc.)

3) Using the non-blocking ?want? version of popping the message queue
doesn?t give any time for the message to actually be sent and arrive at the
peer endpoint.  Either integrate into an event loop or just use the
blocking ?need? version to wait for the message to arrive.

Here?s an example of revising your code w/ those 3 suggestions:

#include <stdio.h>
#include "broker.h"

int main (int argc, char* argv[]) {
   broker_init(0);
   broker_endpoint* ep1 = broker_endpoint_create_with_flags("ep1", 3);
   broker_endpoint* ep2 = broker_endpoint_create_with_flags("ep2", 3);
   broker_string*       bs = broker_string_create("");
   broker_message_queue* q = broker_message_queue_create(bs, ep2);

   broker_peering*    p = broker_endpoint_peer_locally(ep2, ep1);

   const broker_outgoing_connection_status_queue* ocsq =
           broker_endpoint_outgoing_connection_status(ep2);
   broker_deque_of_outgoing_connection_status_delete(
       broker_outgoing_connection_status_queue_need_pop(ocsq));

   broker_string*  msg_str = broker_string_create("Hello, World!\n");
   broker_string*    topic = broker_string_create("test");
   broker_data*        msg = broker_data_from_string(msg_str);

   broker_vector* vec = broker_vector_create();
   int            res = broker_vector_insert(vec, msg, 0L);
                  res = broker_endpoint_send(ep1, topic, vec);

   broker_deque_of_message* msg_list = broker_message_queue_need_pop(q);
   size_t                   num_msgs =
broker_deque_of_message_size(msg_list);

   printf("There are %ld messages.\n", num_msgs);
}

Hope that helps.

- Jon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150811/365ee4c6/attachment-0001.html 

From gfaulkner.nsm at gmail.com  Tue Aug 11 15:39:07 2015
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Tue, 11 Aug 2015 17:39:07 -0500
Subject: [Bro] Modifying the Fox-IT Meterpreter script to raise a notice
Message-ID: <55CA798B.5010500@gmail.com>

Fox-IT shared a script after Bro Con that looks for evidence of 
meterpreter payloads being downloaded, but it prints the results, which 
should work fine with pcaps, but doesn't seem useful for running on live 
traffic. To run this against live traffic it seems like it would be 
preferable to raise a notice instead. What I was thinking was something 
such as below, but I'm not sure if I'm missing any pieces, or if I'm 
even thinking this through correctly. Will this work? Is it likely to be 
cluster safe?

Modified code is below:

module Meterpreter;

export {
     #Add new notice type for Meterpreter
     redef enum Notice::Type += {
         Meterpreter_Seen,
     };
     redef record connection += {
         meterpreter_payload_size: count &optional;
     };
}

event tcp_packet(c: connection, is_orig: bool, flags: string,
                  seq: count, ack: count, len: count, payload: string)
{
     if(|payload| == 4 && seq == 1)
         {
         c$meterpreter_payload_size = bytestring_to_count(payload, T);
         }
     else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP" 
&& ack > 5)
         {
         if (c$meterpreter_payload_size == ack-5)
             {
             #Raise a notice if we think we've seen a payload
             NOTICE([$note=Meterpreter_Seen,
             $msg=fmt("%DT: Possible Meterpreter Payload transfered! 
%s:%s -> %s:%s",
             c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h, 
c$id$orig_p)]);
             }
         }
}


The original code is here:

https://github.com/fox-it/bro-scripts/blob/master/meterpreter.bro

## meterpreter.bro
##
## Bro-IDS policy to detect Metasploit's meterpreter payload transfer
## Note that it does not detect payload transfers over SSL
##
## Fox-IT
## Security Research Team
##
## https://github.com/fox-it/bro-scripts

export {
     redef record connection += {
         meterpreter_payload_size: count &optional;
     };
}

event tcp_packet(c: connection, is_orig: bool, flags: string,
                  seq: count, ack: count, len: count, payload: string)
{
     if(|payload| == 4 && seq == 1)
         {
         c$meterpreter_payload_size = bytestring_to_count(payload, T);
         }
     else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP" 
&& ack > 5)
         {
         if (c$meterpreter_payload_size == ack-5)
             {
             print( fmt("%DT: Possible Meterpreter Payload transfered! 
%s:%s -> %s:%s",
                c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h, 
c$id$orig_p));
             }
         }
}



From dopheide at gmail.com  Tue Aug 11 15:44:11 2015
From: dopheide at gmail.com (Mike Dopheide)
Date: Tue, 11 Aug 2015 17:44:11 -0500
Subject: [Bro] Modifying the Fox-IT Meterpreter script to raise a notice
In-Reply-To: <55CA798B.5010500@gmail.com>
References: <55CA798B.5010500@gmail.com>
Message-ID: <CAPy2kFb7KaNffp=5GFW9JBOi9RJgtS6cYmPL0sZAoQQot_WpLA@mail.gmail.com>

I did something similar, but my Notice code looks like this to help
populate the other fields.  I haven't gotten around to doing a production
test yet.


  NOTICE([$note=FoxIT::Meterpreter,
                       $msg=fmt("%DT: Possible Meterpreter Payload
transfered! %s:%s -> %s:%s",
               c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
c$id$orig_p),
                                $conn=c,
                                $src=c$id$orig_h,
                                $dst=c$id$resp_h,
                                $identifier=cat(c$id$resp_h,c$id$orig_h)]);

On Tue, Aug 11, 2015 at 5:39 PM, Gary Faulkner <gfaulkner.nsm at gmail.com>
wrote:

> Fox-IT shared a script after Bro Con that looks for evidence of
> meterpreter payloads being downloaded, but it prints the results, which
> should work fine with pcaps, but doesn't seem useful for running on live
> traffic. To run this against live traffic it seems like it would be
> preferable to raise a notice instead. What I was thinking was something
> such as below, but I'm not sure if I'm missing any pieces, or if I'm
> even thinking this through correctly. Will this work? Is it likely to be
> cluster safe?
>
> Modified code is below:
>
> module Meterpreter;
>
> export {
>      #Add new notice type for Meterpreter
>      redef enum Notice::Type += {
>          Meterpreter_Seen,
>      };
>      redef record connection += {
>          meterpreter_payload_size: count &optional;
>      };
> }
>
> event tcp_packet(c: connection, is_orig: bool, flags: string,
>                   seq: count, ack: count, len: count, payload: string)
> {
>      if(|payload| == 4 && seq == 1)
>          {
>          c$meterpreter_payload_size = bytestring_to_count(payload, T);
>          }
>      else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP"
> && ack > 5)
>          {
>          if (c$meterpreter_payload_size == ack-5)
>              {
>              #Raise a notice if we think we've seen a payload
>              NOTICE([$note=Meterpreter_Seen,
>              $msg=fmt("%DT: Possible Meterpreter Payload transfered!
> %s:%s -> %s:%s",
>              c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
> c$id$orig_p)]);
>              }
>          }
> }
>
>
> The original code is here:
>
> https://github.com/fox-it/bro-scripts/blob/master/meterpreter.bro
>
> ## meterpreter.bro
> ##
> ## Bro-IDS policy to detect Metasploit's meterpreter payload transfer
> ## Note that it does not detect payload transfers over SSL
> ##
> ## Fox-IT
> ## Security Research Team
> ##
> ## https://github.com/fox-it/bro-scripts
>
> export {
>      redef record connection += {
>          meterpreter_payload_size: count &optional;
>      };
> }
>
> event tcp_packet(c: connection, is_orig: bool, flags: string,
>                   seq: count, ack: count, len: count, payload: string)
> {
>      if(|payload| == 4 && seq == 1)
>          {
>          c$meterpreter_payload_size = bytestring_to_count(payload, T);
>          }
>      else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP"
> && ack > 5)
>          {
>          if (c$meterpreter_payload_size == ack-5)
>              {
>              print( fmt("%DT: Possible Meterpreter Payload transfered!
> %s:%s -> %s:%s",
>                 c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
> c$id$orig_p));
>              }
>          }
> }
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150811/b1ea2c58/attachment.html 

From gfaulkner.nsm at gmail.com  Tue Aug 11 16:26:08 2015
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Tue, 11 Aug 2015 18:26:08 -0500
Subject: [Bro] Modifying the Fox-IT Meterpreter script to raise a notice
In-Reply-To: <CAPy2kFb7KaNffp=5GFW9JBOi9RJgtS6cYmPL0sZAoQQot_WpLA@mail.gmail.com>
References: <55CA798B.5010500@gmail.com>
	<CAPy2kFb7KaNffp=5GFW9JBOi9RJgtS6cYmPL0sZAoQQot_WpLA@mail.gmail.com>
Message-ID: <55CA8490.4040601@gmail.com>

Good pointer. I had gotten a couple hits already, and noticed the notice
line had a few empty fields, but hadn't looked into it further yet. I'll
give that a shot.

On 8/11/2015 5:44 PM, Mike Dopheide wrote:
> I did something similar, but my Notice code looks like this to help
> populate the other fields.  I haven't gotten around to doing a production
> test yet.
>
>
>   NOTICE([$note=FoxIT::Meterpreter,
>                        $msg=fmt("%DT: Possible Meterpreter Payload
> transfered! %s:%s -> %s:%s",
>                c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
> c$id$orig_p),
>                                 $conn=c,
>                                 $src=c$id$orig_h,
>                                 $dst=c$id$resp_h,
>                                 $identifier=cat(c$id$resp_h,c$id$orig_h)]);
>
> On Tue, Aug 11, 2015 at 5:39 PM, Gary Faulkner <gfaulkner.nsm at gmail.com>
> wrote:
>
>> Fox-IT shared a script after Bro Con that looks for evidence of
>> meterpreter payloads being downloaded, but it prints the results, which
>> should work fine with pcaps, but doesn't seem useful for running on live
>> traffic. To run this against live traffic it seems like it would be
>> preferable to raise a notice instead. What I was thinking was something
>> such as below, but I'm not sure if I'm missing any pieces, or if I'm
>> even thinking this through correctly. Will this work? Is it likely to be
>> cluster safe?
>>
>> Modified code is below:
>>
>> module Meterpreter;
>>
>> export {
>>      #Add new notice type for Meterpreter
>>      redef enum Notice::Type += {
>>          Meterpreter_Seen,
>>      };
>>      redef record connection += {
>>          meterpreter_payload_size: count &optional;
>>      };
>> }
>>
>> event tcp_packet(c: connection, is_orig: bool, flags: string,
>>                   seq: count, ack: count, len: count, payload: string)
>> {
>>      if(|payload| == 4 && seq == 1)
>>          {
>>          c$meterpreter_payload_size = bytestring_to_count(payload, T);
>>          }
>>      else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP"
>> && ack > 5)
>>          {
>>          if (c$meterpreter_payload_size == ack-5)
>>              {
>>              #Raise a notice if we think we've seen a payload
>>              NOTICE([$note=Meterpreter_Seen,
>>              $msg=fmt("%DT: Possible Meterpreter Payload transfered!
>> %s:%s -> %s:%s",
>>              c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
>> c$id$orig_p)]);
>>              }
>>          }
>> }
>>
>>
>> The original code is here:
>>
>> https://github.com/fox-it/bro-scripts/blob/master/meterpreter.bro
>>
>> ## meterpreter.bro
>> ##
>> ## Bro-IDS policy to detect Metasploit's meterpreter payload transfer
>> ## Note that it does not detect payload transfers over SSL
>> ##
>> ## Fox-IT
>> ## Security Research Team
>> ##
>> ## https://github.com/fox-it/bro-scripts
>>
>> export {
>>      redef record connection += {
>>          meterpreter_payload_size: count &optional;
>>      };
>> }
>>
>> event tcp_packet(c: connection, is_orig: bool, flags: string,
>>                   seq: count, ack: count, len: count, payload: string)
>> {
>>      if(|payload| == 4 && seq == 1)
>>          {
>>          c$meterpreter_payload_size = bytestring_to_count(payload, T);
>>          }
>>      else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP"
>> && ack > 5)
>>          {
>>          if (c$meterpreter_payload_size == ack-5)
>>              {
>>              print( fmt("%DT: Possible Meterpreter Payload transfered!
>> %s:%s -> %s:%s",
>>                 c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
>> c$id$orig_p));
>>              }
>>          }
>> }
>>
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>


From gfaulkner.nsm at gmail.com  Tue Aug 11 16:59:45 2015
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Tue, 11 Aug 2015 18:59:45 -0500
Subject: [Bro] Modifying the Fox-IT Meterpreter script to raise a notice
In-Reply-To: <55CA8490.4040601@gmail.com>
References: <55CA798B.5010500@gmail.com>
	<CAPy2kFb7KaNffp=5GFW9JBOi9RJgtS6cYmPL0sZAoQQot_WpLA@mail.gmail.com>
	<55CA8490.4040601@gmail.com>
Message-ID: <55CA8C71.8040500@gmail.com>

The identifier is used for notice suppression correct? If I'm reading
this correctly won't this suppress any further notices of this type that
have the same combination of src ip and dst ip for the default
suppression interval? Wouldn't this potentially result in missing
additional payloads? 

On 8/11/2015 6:26 PM, Gary Faulkner wrote:
> Good pointer. I had gotten a couple hits already, and noticed the notice
> line had a few empty fields, but hadn't looked into it further yet. I'll
> give that a shot.
>
> On 8/11/2015 5:44 PM, Mike Dopheide wrote:
>> I did something similar, but my Notice code looks like this to help
>> populate the other fields.  I haven't gotten around to doing a production
>> test yet.
>>
>>
>>   NOTICE([$note=FoxIT::Meterpreter,
>>                        $msg=fmt("%DT: Possible Meterpreter Payload
>> transfered! %s:%s -> %s:%s",
>>                c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
>> c$id$orig_p),
>>                                 $conn=c,
>>                                 $src=c$id$orig_h,
>>                                 $dst=c$id$resp_h,
>>                                 $identifier=cat(c$id$resp_h,c$id$orig_h)]);
>>
>> On Tue, Aug 11, 2015 at 5:39 PM, Gary Faulkner <gfaulkner.nsm at gmail.com>
>> wrote:
>>
>>> Fox-IT shared a script after Bro Con that looks for evidence of
>>> meterpreter payloads being downloaded, but it prints the results, which
>>> should work fine with pcaps, but doesn't seem useful for running on live
>>> traffic. To run this against live traffic it seems like it would be
>>> preferable to raise a notice instead. What I was thinking was something
>>> such as below, but I'm not sure if I'm missing any pieces, or if I'm
>>> even thinking this through correctly. Will this work? Is it likely to be
>>> cluster safe?
>>>
>>> Modified code is below:
>>>
>>> module Meterpreter;
>>>
>>> export {
>>>      #Add new notice type for Meterpreter
>>>      redef enum Notice::Type += {
>>>          Meterpreter_Seen,
>>>      };
>>>      redef record connection += {
>>>          meterpreter_payload_size: count &optional;
>>>      };
>>> }
>>>
>>> event tcp_packet(c: connection, is_orig: bool, flags: string,
>>>                   seq: count, ack: count, len: count, payload: string)
>>> {
>>>      if(|payload| == 4 && seq == 1)
>>>          {
>>>          c$meterpreter_payload_size = bytestring_to_count(payload, T);
>>>          }
>>>      else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP"
>>> && ack > 5)
>>>          {
>>>          if (c$meterpreter_payload_size == ack-5)
>>>              {
>>>              #Raise a notice if we think we've seen a payload
>>>              NOTICE([$note=Meterpreter_Seen,
>>>              $msg=fmt("%DT: Possible Meterpreter Payload transfered!
>>> %s:%s -> %s:%s",
>>>              c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
>>> c$id$orig_p)]);
>>>              }
>>>          }
>>> }
>>>
>>>
>>> The original code is here:
>>>
>>> https://github.com/fox-it/bro-scripts/blob/master/meterpreter.bro
>>>
>>> ## meterpreter.bro
>>> ##
>>> ## Bro-IDS policy to detect Metasploit's meterpreter payload transfer
>>> ## Note that it does not detect payload transfers over SSL
>>> ##
>>> ## Fox-IT
>>> ## Security Research Team
>>> ##
>>> ## https://github.com/fox-it/bro-scripts
>>>
>>> export {
>>>      redef record connection += {
>>>          meterpreter_payload_size: count &optional;
>>>      };
>>> }
>>>
>>> event tcp_packet(c: connection, is_orig: bool, flags: string,
>>>                   seq: count, ack: count, len: count, payload: string)
>>> {
>>>      if(|payload| == 4 && seq == 1)
>>>          {
>>>          c$meterpreter_payload_size = bytestring_to_count(payload, T);
>>>          }
>>>      else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP"
>>> && ack > 5)
>>>          {
>>>          if (c$meterpreter_payload_size == ack-5)
>>>              {
>>>              print( fmt("%DT: Possible Meterpreter Payload transfered!
>>> %s:%s -> %s:%s",
>>>                 c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
>>> c$id$orig_p));
>>>              }
>>>          }
>>> }
>>>
>>>
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>


From dopheide at gmail.com  Tue Aug 11 17:08:05 2015
From: dopheide at gmail.com (Mike Dopheide)
Date: Tue, 11 Aug 2015 19:08:05 -0500
Subject: [Bro] Modifying the Fox-IT Meterpreter script to raise a notice
In-Reply-To: <55CA8C71.8040500@gmail.com>
References: <55CA798B.5010500@gmail.com>
	<CAPy2kFb7KaNffp=5GFW9JBOi9RJgtS6cYmPL0sZAoQQot_WpLA@mail.gmail.com>
	<55CA8490.4040601@gmail.com> <55CA8C71.8040500@gmail.com>
Message-ID: <CAPy2kFa73v4J-LUs00AULv0PdRFZ9XAeP_eCcyWqQw80so1Qqw@mail.gmail.com>

Yes, it's a trade off.  I generally prefer the one notice/alert to start an
investigation into everything the attacking IP was doing.

On Tuesday, August 11, 2015, Gary Faulkner <gfaulkner.nsm at gmail.com> wrote:

> The identifier is used for notice suppression correct? If I'm reading
> this correctly won't this suppress any further notices of this type that
> have the same combination of src ip and dst ip for the default
> suppression interval? Wouldn't this potentially result in missing
> additional payloads?
>
> On 8/11/2015 6:26 PM, Gary Faulkner wrote:
> > Good pointer. I had gotten a couple hits already, and noticed the notice
> > line had a few empty fields, but hadn't looked into it further yet. I'll
> > give that a shot.
> >
> > On 8/11/2015 5:44 PM, Mike Dopheide wrote:
> >> I did something similar, but my Notice code looks like this to help
> >> populate the other fields.  I haven't gotten around to doing a
> production
> >> test yet.
> >>
> >>
> >>   NOTICE([$note=FoxIT::Meterpreter,
> >>                        $msg=fmt("%DT: Possible Meterpreter Payload
> >> transfered! %s:%s -> %s:%s",
> >>                c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
> >> c$id$orig_p),
> >>                                 $conn=c,
> >>                                 $src=c$id$orig_h,
> >>                                 $dst=c$id$resp_h,
> >>
>  $identifier=cat(c$id$resp_h,c$id$orig_h)]);
> >>
> >> On Tue, Aug 11, 2015 at 5:39 PM, Gary Faulkner <gfaulkner.nsm at gmail.com
> <javascript:;>>
> >> wrote:
> >>
> >>> Fox-IT shared a script after Bro Con that looks for evidence of
> >>> meterpreter payloads being downloaded, but it prints the results, which
> >>> should work fine with pcaps, but doesn't seem useful for running on
> live
> >>> traffic. To run this against live traffic it seems like it would be
> >>> preferable to raise a notice instead. What I was thinking was something
> >>> such as below, but I'm not sure if I'm missing any pieces, or if I'm
> >>> even thinking this through correctly. Will this work? Is it likely to
> be
> >>> cluster safe?
> >>>
> >>> Modified code is below:
> >>>
> >>> module Meterpreter;
> >>>
> >>> export {
> >>>      #Add new notice type for Meterpreter
> >>>      redef enum Notice::Type += {
> >>>          Meterpreter_Seen,
> >>>      };
> >>>      redef record connection += {
> >>>          meterpreter_payload_size: count &optional;
> >>>      };
> >>> }
> >>>
> >>> event tcp_packet(c: connection, is_orig: bool, flags: string,
> >>>                   seq: count, ack: count, len: count, payload: string)
> >>> {
> >>>      if(|payload| == 4 && seq == 1)
> >>>          {
> >>>          c$meterpreter_payload_size = bytestring_to_count(payload, T);
> >>>          }
> >>>      else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP"
> >>> && ack > 5)
> >>>          {
> >>>          if (c$meterpreter_payload_size == ack-5)
> >>>              {
> >>>              #Raise a notice if we think we've seen a payload
> >>>              NOTICE([$note=Meterpreter_Seen,
> >>>              $msg=fmt("%DT: Possible Meterpreter Payload transfered!
> >>> %s:%s -> %s:%s",
> >>>              c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
> >>> c$id$orig_p)]);
> >>>              }
> >>>          }
> >>> }
> >>>
> >>>
> >>> The original code is here:
> >>>
> >>> https://github.com/fox-it/bro-scripts/blob/master/meterpreter.bro
> >>>
> >>> ## meterpreter.bro
> >>> ##
> >>> ## Bro-IDS policy to detect Metasploit's meterpreter payload transfer
> >>> ## Note that it does not detect payload transfers over SSL
> >>> ##
> >>> ## Fox-IT
> >>> ## Security Research Team
> >>> ##
> >>> ## https://github.com/fox-it/bro-scripts
> >>>
> >>> export {
> >>>      redef record connection += {
> >>>          meterpreter_payload_size: count &optional;
> >>>      };
> >>> }
> >>>
> >>> event tcp_packet(c: connection, is_orig: bool, flags: string,
> >>>                   seq: count, ack: count, len: count, payload: string)
> >>> {
> >>>      if(|payload| == 4 && seq == 1)
> >>>          {
> >>>          c$meterpreter_payload_size = bytestring_to_count(payload, T);
> >>>          }
> >>>      else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP"
> >>> && ack > 5)
> >>>          {
> >>>          if (c$meterpreter_payload_size == ack-5)
> >>>              {
> >>>              print( fmt("%DT: Possible Meterpreter Payload transfered!
> >>> %s:%s -> %s:%s",
> >>>                 c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
> >>> c$id$orig_p));
> >>>              }
> >>>          }
> >>> }
> >>>
> >>>
> >>> _______________________________________________
> >>> Bro mailing list
> >>> bro at bro-ids.org <javascript:;>
> >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150811/c50ae184/attachment.html 

From kevin at branchnetconsulting.com  Tue Aug 11 17:08:37 2015
From: kevin at branchnetconsulting.com (Kevin Branch)
Date: Tue, 11 Aug 2015 20:08:37 -0400
Subject: [Bro] Adjusting Bro snaplen caused multiple Security Onion
 systems to sporadically kernel panic
In-Reply-To: <CA+dGL9HvssoTpXcfTvDxNsYQ1xT3gQdU97mGJKEtkYHkVuaFiQ@mail.gmail.com>
References: <CA+dGL9HvssoTpXcfTvDxNsYQ1xT3gQdU97mGJKEtkYHkVuaFiQ@mail.gmail.com>
Message-ID: <CA+dGL9EnfJ7O6CyNFA3i+zbVFu=7SXv8gfN1i=XWi4bLvpzjAA@mail.gmail.com>

Seth,

Is it still OK to use "redef snaplen = *N*;" in my local.bro file if I want
to drop Bro's default snaplen to save PF_RING memory?  If so, how low would
you say "*N*" can be set to safely when jumbo frames are not involved?  I
got burned with sporadic kernel panics when I set N to *1514*.  I'd sure
appreciate your input.

Kevin

On Tue, Aug 4, 2015 at 10:18 AM, Kevin Branch <kevin at branchnetconsulting.com
> wrote:

> Hi Seth,
>
> Since getting involved with Security Onion over a year ago, I have really
> grown to appreciate the value that Bro brings to the NSM scene.  Thanks to
> you and the rest of the Bro community for your work on this!
>
> Not long ago I noticed that my Bro instances were eating up way more
> memory than necessary for my non-jumbo-frame environment because of the
> default snaplen of 8192, so based on what I saw in your comment here a
> couple of years ago:
>
> https://groups.google.com/d/msg/security-onion/qDU23hx6Q5g/xFeRDJbi9LsJ
>
> ... I added "redef snaplen = 1514;" to my local.bro file on 6 different
> Security Onion systems and reclaimed a heap of memory.  At first it seemed
> to work great but after a short while three completely separate systems
> start throwing kernel panics.  I finally traced it down to the snaplen
> change in Bro and upon raising the snaplen from 1514 to 1600, all kernel
> panics completely stopped.
>
> I am looking for your recommendation as to how low should be considered
> "safe" to change the Bro snaplen to for non-jumbo-frame environments, as
> well as to confirm that the old redef method I used to do this is still
> what should be used with modern Bro.
>
> I brought up my kernel panic issues on the Security Onion support list and
> am hoping to report back to them how changing Bro's snaplen can be safely
> used to save memory on systems where numerous Bro instances and/or large
> PF_RINGs are in use.  Here is that thread for your reference:
>
>
> https://groups.google.com/forum/#!searchin/security-onion-testing/kernel$20panic/security-onion-testing/H_21-0DGM6E/qnSt0BPA7lMJ
>
>
> Thanks again for a great tool!
> Kevin
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150811/320245a3/attachment.html 

From seth at icir.org  Tue Aug 11 17:43:49 2015
From: seth at icir.org (Seth Hall)
Date: Tue, 11 Aug 2015 20:43:49 -0400
Subject: [Bro] Adjusting Bro snaplen caused multiple Security Onion
	systems to sporadically kernel panic
In-Reply-To: <CA+dGL9EnfJ7O6CyNFA3i+zbVFu=7SXv8gfN1i=XWi4bLvpzjAA@mail.gmail.com>
References: <CA+dGL9HvssoTpXcfTvDxNsYQ1xT3gQdU97mGJKEtkYHkVuaFiQ@mail.gmail.com>
	<CA+dGL9EnfJ7O6CyNFA3i+zbVFu=7SXv8gfN1i=XWi4bLvpzjAA@mail.gmail.com>
Message-ID: <A311025F-1F07-47B4-85AC-2DC94267EFC5@icir.org>


> On Aug 11, 2015, at 8:08 PM, Kevin Branch <kevin at branchnetconsulting.com> wrote:
> 
> Is it still OK to use "redef snaplen = N;" in my local.bro file if I want to drop Bro's default snaplen to save PF_RING memory?  If so, how low would you say "N" can be set to safely when jumbo frames are not involved?  I got burned with sporadic kernel panics when I set N to 1514.  I'd sure appreciate your input.

Yes, it should be fine to do that but I really don?t know how that change in snap length affects pf_ring.  It may not actually do anything (except apparently cause crashes!?).

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From anthony.kasza at gmail.com  Tue Aug 11 21:13:19 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Tue, 11 Aug 2015 21:13:19 -0700
Subject: [Bro] Broker - File Extraction
Message-ID: <CAEZw2by98g708qkOtVptCDOSw-v3suXV4AnH968QDfe+Nbo-PQ@mail.gmail.com>

Hi All,

I was doing some reading on broker and came across the remote logging
section of the documentation. This seems very useful.
Is there a mechanism for remote file extraction? I think it would be useful
to be able to extract files to a remote system instead of a local
directory. Is this possible with broker?

-AK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150811/9943acf6/attachment-0001.html 

From giedrius.ramas at gmail.com  Wed Aug 12 05:37:44 2015
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Wed, 12 Aug 2015 15:37:44 +0300
Subject: [Bro] bro v2.4 Bro Intel Framework Extensions do not work
Message-ID: <CAHr6V5qiDNeQS2fSyqhoijC6dKDtf8v5+b7u1zp1b=Uxut-DMQ@mail.gmail.com>

Hi, I had working BRO Intel extensions
https://github.com/sethhall/intel-ext fine
until bro v2.4 version.
Now I get errors :

checking configurations ...
manager scripts failed.
error in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/extend.bro, line
19: no such field in record (Intel::s$f?$mime_type)
error in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/extend.bro, line
20: no such field in record (Intel::s$f$mime_type)

Have anyone a clue where should I fix the issue ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150812/699baa04/attachment.html 

From jlay at slave-tothe-box.net  Wed Aug 12 05:58:39 2015
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 12 Aug 2015 06:58:39 -0600
Subject: [Bro] bro v2.4 Bro Intel Framework Extensions do not work
In-Reply-To: <CAHr6V5qiDNeQS2fSyqhoijC6dKDtf8v5+b7u1zp1b=Uxut-DMQ@mail.gmail.com>
References: <CAHr6V5qiDNeQS2fSyqhoijC6dKDtf8v5+b7u1zp1b=Uxut-DMQ@mail.gmail.com>
Message-ID: <1439384319.3798.1.camel@JamesiMac>

On Wed, 2015-08-12 at 15:37 +0300, Giedrius Ramas wrote:
> Hi, I had working BRO Intel extensions
> https://github.com/sethhall/intel-ext fine until bro v2.4 version.  
> Now I get errors :
>  
> checking configurations ...
> manager scripts failed.
> error in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/extend.bro,
> line 19: no such field in record (Intel::s$f?$mime_type)
> error in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/extend.bro,
> line 20: no such field in record (Intel::s$f$mime_type)
> 
>  
> Have anyone a clue where should I fix the issue ?
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


Answers are in this thread:

http://mailman.icsi.berkeley.edu/pipermail/bro/2015-July/008712.html

the tl;dr is f?$mime_type is now meta$mime_type

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150812/1fc8bdc2/attachment.html 

From jsiwek at illinois.edu  Wed Aug 12 06:18:15 2015
From: jsiwek at illinois.edu (Siwek, Jon)
Date: Wed, 12 Aug 2015 13:18:15 +0000
Subject: [Bro] Broker - File Extraction
In-Reply-To: <CAEZw2by98g708qkOtVptCDOSw-v3suXV4AnH968QDfe+Nbo-PQ@mail.gmail.com>
References: <CAEZw2by98g708qkOtVptCDOSw-v3suXV4AnH968QDfe+Nbo-PQ@mail.gmail.com>
Message-ID: <BE9A8411-F8E9-40D1-90D0-9403A0D6F57E@illinois.edu>


> On Aug 11, 2015, at 11:13 PM, anthony kasza <anthony.kasza at gmail.com> wrote:
> 
> I was doing some reading on broker and came across the remote logging section of the documentation. This seems very useful.
> Is there a mechanism for remote file extraction?

There?s not a direct/built-in mechanism for that like there is w/ remote logging.

> I think it would be useful to be able to extract files to a remote system instead of a local directory. Is this possible with broker?

Yes, it should be possible, in a couple different ways.  Using the Broker library directly and implementing it in Bro core (similar to remote logging) would be an option.  Or using Bro?s scripting interface to the Broker library in combination w/ the scripting interface for file analysis should also work ? e.g. ask for access to the contents of a file via events then send it to a remote peer via Broker.

- Jon


From giedrius.ramas at gmail.com  Wed Aug 12 06:44:50 2015
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Wed, 12 Aug 2015 16:44:50 +0300
Subject: [Bro] bro v2.4 Bro Intel Framework Extensions do not work
In-Reply-To: <1439384319.3798.1.camel@JamesiMac>
References: <CAHr6V5qiDNeQS2fSyqhoijC6dKDtf8v5+b7u1zp1b=Uxut-DMQ@mail.gmail.com>
	<1439384319.3798.1.camel@JamesiMac>
Message-ID: <CAHr6V5pMh3gxKj54p0u=fhhtD_mCDpa=8uE-1-SKY6a=s8CeWw@mail.gmail.com>

Thank you for instant reply.

However I am not good at scripting . As far as I understand I should just
change line f?$mime_type to meta$mime_type right ?
And what about f$mime_type  ?

On Wed, Aug 12, 2015 at 3:58 PM, James Lay <jlay at slave-tothe-box.net> wrote:

> On Wed, 2015-08-12 at 15:37 +0300, Giedrius Ramas wrote:
>
> Hi, I had working BRO Intel extensions
> https://github.com/sethhall/intel-ext fine until bro v2.4 version.
>
> Now I get errors :
>
>
>
> checking configurations ...
> manager scripts failed.
> error in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/extend.bro,
> line 19: no such field in record (Intel::s$f?$mime_type)
> error in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/extend.bro,
> line 20: no such field in record (Intel::s$f$mime_type)
>
>
>
> Have anyone a clue where should I fix the issue ?
>
> _______________________________________________
> Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> Answers are in this thread:
>
> http://mailman.icsi.berkeley.edu/pipermail/bro/2015-July/008712.html
>
> the tl;dr is f?$mime_type is now meta$mime_type
>
> James
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150812/486d682e/attachment.html 

From seth at icir.org  Wed Aug 12 06:56:31 2015
From: seth at icir.org (Seth Hall)
Date: Wed, 12 Aug 2015 09:56:31 -0400
Subject: [Bro] bro v2.4 Bro Intel Framework Extensions do not work
In-Reply-To: <CAHr6V5qiDNeQS2fSyqhoijC6dKDtf8v5+b7u1zp1b=Uxut-DMQ@mail.gmail.com>
References: <CAHr6V5qiDNeQS2fSyqhoijC6dKDtf8v5+b7u1zp1b=Uxut-DMQ@mail.gmail.com>
Message-ID: <C99AB9F2-4DE6-43D6-BA00-BC33AC8AEEFF@icir.org>


> On Aug 12, 2015, at 8:37 AM, Giedrius Ramas <giedrius.ramas at gmail.com> wrote:
> 
> error in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/extend.bro, line 19: no such field in record (Intel::s$f?$mime_type)
> error in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/extend.bro, line 20: no such field in record (Intel::s$f$mime_type)

Sorry about that.  I just pushed a fix to the repository (and now it only works with Bro 2.4).

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Wed Aug 12 07:00:29 2015
From: seth at icir.org (Seth Hall)
Date: Wed, 12 Aug 2015 10:00:29 -0400
Subject: [Bro] Broker - File Extraction
In-Reply-To: <CAEZw2by98g708qkOtVptCDOSw-v3suXV4AnH968QDfe+Nbo-PQ@mail.gmail.com>
References: <CAEZw2by98g708qkOtVptCDOSw-v3suXV4AnH968QDfe+Nbo-PQ@mail.gmail.com>
Message-ID: <4B85C444-BC6E-4EE3-878B-6ECADB56834E@icir.org>


> On Aug 12, 2015, at 12:13 AM, anthony kasza <anthony.kasza at gmail.com> wrote:
> 
> I was doing some reading on broker and came across the remote logging section of the documentation. This seems very useful.
> Is there a mechanism for remote file extraction? I think it would be useful to be able to extract files to a remote system instead of a local directory. Is this possible with broker?

Adding to what Jon said, this was something he and I discussed a lot while the files framework was being developed.  I suspect that at some point it will be added as a supported feature in Bro but there are so many edge cases to how this needs to be handled that it wasn?t quite an immediately obvious feature to implement so we skipped it initially.

My aim for it is to be able to extract BitTorrent transfers on clusters.  That?s super complicated and will take some time unfortunately but we have continued laying the groundwork for it.  For instance, full file reassembly went into Bro 2.4 which was a requirement for actually doing this correctly.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From gfaulkner.nsm at gmail.com  Wed Aug 12 07:40:09 2015
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Wed, 12 Aug 2015 09:40:09 -0500
Subject: [Bro] Modifying the Fox-IT Meterpreter script to raise a notice
In-Reply-To: <CAPy2kFa73v4J-LUs00AULv0PdRFZ9XAeP_eCcyWqQw80so1Qqw@mail.gmail.com>
References: <55CA798B.5010500@gmail.com>
	<CAPy2kFb7KaNffp=5GFW9JBOi9RJgtS6cYmPL0sZAoQQot_WpLA@mail.gmail.com>
	<55CA8490.4040601@gmail.com> <55CA8C71.8040500@gmail.com>
	<CAPy2kFa73v4J-LUs00AULv0PdRFZ9XAeP_eCcyWqQw80so1Qqw@mail.gmail.com>
Message-ID: <55CB5AC9.8060207@gmail.com>

If any one is curious, the modified script I ended up with after 
incorporating Mike's suggestions is below. In the 14 hours or so I've 
been running it against production traffic I've already seen several 
hits. At least one of those hosts has multiple hits every few hours and 
appears to need further investigation, while another looks like it may 
be a false positive triggering on what appears to be a stock 
checking/trading app that connects to HDS3.ninjatrader.com on port 31654.

## meterpreter.bro
##
## Bro-IDS policy to detect Metasploit's meterpreter payload transfer
## Note that it does not detect payload transfers over SSL
##
## Original Script by Fox-IT
## Security Research Team
##
## https://github.com/fox-it/bro-scripts
##
## This version has been modified to raise a NOTICE

module Meterpreter;

export {
     #Add new notice type for Meterpreter
     redef enum Notice::Type += {
         Meterpreter_Seen,
     };
     redef record connection += {
         meterpreter_payload_size: count &optional;
     };
}

event tcp_packet(c: connection, is_orig: bool, flags: string,
                  seq: count, ack: count, len: count, payload: string)
{
     if(|payload| == 4 && seq == 1)
         {
         c$meterpreter_payload_size = bytestring_to_count(payload, T);
         }
     else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP" 
&& ack > 5)
         {
         if (c$meterpreter_payload_size == ack-5)
             {
             #Raise a notice if we think we've seen a payload
             NOTICE([$note=Meterpreter_Seen,
             $msg=fmt("%DT: Possible Meterpreter Payload transferred! 
%s:%s -> %s:%s",
             c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h, 
c$id$orig_p),
             $conn=c,
             $src=c$id$orig_h,
             $dst=c$id$resp_h,
             $identifier=cat(c$id$resp_h,c$id$orig_h)]);
         }
         }
}


On 8/11/15 7:08 PM, Mike Dopheide wrote:
> Yes, it's a trade off.  I generally prefer the one notice/alert to start an
> investigation into everything the attacking IP was doing.
>
> On Tuesday, August 11, 2015, Gary Faulkner <gfaulkner.nsm at gmail.com> wrote:
>
>> The identifier is used for notice suppression correct? If I'm reading
>> this correctly won't this suppress any further notices of this type that
>> have the same combination of src ip and dst ip for the default
>> suppression interval? Wouldn't this potentially result in missing
>> additional payloads?
>>
>> On 8/11/2015 6:26 PM, Gary Faulkner wrote:
>>> Good pointer. I had gotten a couple hits already, and noticed the notice
>>> line had a few empty fields, but hadn't looked into it further yet. I'll
>>> give that a shot.
>>>
>>> On 8/11/2015 5:44 PM, Mike Dopheide wrote:
>>>> I did something similar, but my Notice code looks like this to help
>>>> populate the other fields.  I haven't gotten around to doing a
>> production
>>>> test yet.
>>>>
>>>>
>>>>    NOTICE([$note=FoxIT::Meterpreter,
>>>>                         $msg=fmt("%DT: Possible Meterpreter Payload
>>>> transfered! %s:%s -> %s:%s",
>>>>                 c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
>>>> c$id$orig_p),
>>>>                                  $conn=c,
>>>>                                  $src=c$id$orig_h,
>>>>                                  $dst=c$id$resp_h,
>>>>
>>   $identifier=cat(c$id$resp_h,c$id$orig_h)]);
>>>> On Tue, Aug 11, 2015 at 5:39 PM, Gary Faulkner <gfaulkner.nsm at gmail.com
>> <javascript:;>>
>>>> wrote:
>>>>
>>>>> Fox-IT shared a script after Bro Con that looks for evidence of
>>>>> meterpreter payloads being downloaded, but it prints the results, which
>>>>> should work fine with pcaps, but doesn't seem useful for running on
>> live
>>>>> traffic. To run this against live traffic it seems like it would be
>>>>> preferable to raise a notice instead. What I was thinking was something
>>>>> such as below, but I'm not sure if I'm missing any pieces, or if I'm
>>>>> even thinking this through correctly. Will this work? Is it likely to
>> be
>>>>> cluster safe?
>>>>>
>>>>> Modified code is below:
>>>>>
>>>>> module Meterpreter;
>>>>>
>>>>> export {
>>>>>       #Add new notice type for Meterpreter
>>>>>       redef enum Notice::Type += {
>>>>>           Meterpreter_Seen,
>>>>>       };
>>>>>       redef record connection += {
>>>>>           meterpreter_payload_size: count &optional;
>>>>>       };
>>>>> }
>>>>>
>>>>> event tcp_packet(c: connection, is_orig: bool, flags: string,
>>>>>                    seq: count, ack: count, len: count, payload: string)
>>>>> {
>>>>>       if(|payload| == 4 && seq == 1)
>>>>>           {
>>>>>           c$meterpreter_payload_size = bytestring_to_count(payload, T);
>>>>>           }
>>>>>       else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP"
>>>>> && ack > 5)
>>>>>           {
>>>>>           if (c$meterpreter_payload_size == ack-5)
>>>>>               {
>>>>>               #Raise a notice if we think we've seen a payload
>>>>>               NOTICE([$note=Meterpreter_Seen,
>>>>>               $msg=fmt("%DT: Possible Meterpreter Payload transfered!
>>>>> %s:%s -> %s:%s",
>>>>>               c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
>>>>> c$id$orig_p)]);
>>>>>               }
>>>>>           }
>>>>> }
>>>>>
>>>>>
>>>>> The original code is here:
>>>>>
>>>>> https://github.com/fox-it/bro-scripts/blob/master/meterpreter.bro
>>>>>
>>>>> ## meterpreter.bro
>>>>> ##
>>>>> ## Bro-IDS policy to detect Metasploit's meterpreter payload transfer
>>>>> ## Note that it does not detect payload transfers over SSL
>>>>> ##
>>>>> ## Fox-IT
>>>>> ## Security Research Team
>>>>> ##
>>>>> ## https://github.com/fox-it/bro-scripts
>>>>>
>>>>> export {
>>>>>       redef record connection += {
>>>>>           meterpreter_payload_size: count &optional;
>>>>>       };
>>>>> }
>>>>>
>>>>> event tcp_packet(c: connection, is_orig: bool, flags: string,
>>>>>                    seq: count, ack: count, len: count, payload: string)
>>>>> {
>>>>>       if(|payload| == 4 && seq == 1)
>>>>>           {
>>>>>           c$meterpreter_payload_size = bytestring_to_count(payload, T);
>>>>>           }
>>>>>       else if (c?$meterpreter_payload_size && seq == 1 && flags == "AP"
>>>>> && ack > 5)
>>>>>           {
>>>>>           if (c$meterpreter_payload_size == ack-5)
>>>>>               {
>>>>>               print( fmt("%DT: Possible Meterpreter Payload transfered!
>>>>> %s:%s -> %s:%s",
>>>>>                  c$start_time, c$id$resp_h, c$id$resp_p, c$id$orig_h,
>>>>> c$id$orig_p));
>>>>>               }
>>>>>           }
>>>>> }
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Bro mailing list
>>>>> bro at bro-ids.org <javascript:;>
>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>>>
>>


From robin at icir.org  Wed Aug 12 08:40:33 2015
From: robin at icir.org (Robin Sommer)
Date: Wed, 12 Aug 2015 08:40:33 -0700
Subject: [Bro] Broker - File Extraction
In-Reply-To: <4B85C444-BC6E-4EE3-878B-6ECADB56834E@icir.org>
References: <CAEZw2by98g708qkOtVptCDOSw-v3suXV4AnH968QDfe+Nbo-PQ@mail.gmail.com>
	<4B85C444-BC6E-4EE3-878B-6ECADB56834E@icir.org>
Message-ID: <20150812154033.GA52761@icir.org>



On Wed, Aug 12, 2015 at 10:00 -0400, Seth Hall wrote:

> full file reassembly went into Bro 2.4 which was a requirement for
> actually doing this correctly.

Yeah, I think we're actually in good shape now for tackling this. I've
added it to the list of project ideas on bro.org so that we keep it on
the radar.

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From earl.eiland at root9b.com  Wed Aug 12 09:19:34 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Wed, 12 Aug 2015 16:19:34 +0000
Subject: [Bro] running setseparator.bro fails
Message-ID: <BY1PR0401MB1382FA21F91DCD0DD7DBBAFCD77E0@BY1PR0401MB1382.namprd04.prod.outlook.com>

Hello.

I've copied https://github.com/bro/bro/blob/master/testing/btest/scripts/base/frameworks/input/setseparator.bro

Into my pwd and trying to run it.  I'm getting the error unrecognized character - @ when it gets to line 5.

What am I doing wrong?

Best Regards,

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150812/d1a8b9c0/attachment.html 

From dnthayer at illinois.edu  Wed Aug 12 09:39:04 2015
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Wed, 12 Aug 2015 11:39:04 -0500
Subject: [Bro] running setseparator.bro fails
In-Reply-To: <BY1PR0401MB1382FA21F91DCD0DD7DBBAFCD77E0@BY1PR0401MB1382.namprd04.prod.outlook.com>
References: <BY1PR0401MB1382FA21F91DCD0DD7DBBAFCD77E0@BY1PR0401MB1382.namprd04.prod.outlook.com>
Message-ID: <55CB76A8.10902@illinois.edu>

On 08/12/2015 11:19 AM, Earl Eiland wrote:
> Hello.
>
> I?ve copied
> https://github.com/bro/bro/blob/master/testing/btest/scripts/base/frameworks/input/setseparator.bro
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_bro_bro_blob_master_testing_btest_scripts_base_frameworks_input_setseparator.bro&d=AwMFAg&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Bi5qPBnY0NmYPqnRTPj_AfXQKpfQTZUpCzpfFBcawv0&m=sTsVQUDlNukOtreU7YJtt4sempYjNJ7sgeqa9hYuSts&s=GuyvPMy77QPhuknn5iQ4_2XI1StCMXGgOkl0ZWSC4OU&e=>
>
> Into my pwd and trying to run it.  I?m getting the error unrecognized
> character - @ when it gets to line 5.
>
> What am I doing wrong?
>
> Best Regards,
>
> Earl Eiland,

That file isn't meant to be run directly as a Bro script, if you want
to do that then you'll need to first extract the part between
"@TEST-START-FILE" and "@TEST-END-FILE" and save it as file "input.log".
Then delete those lines (including lines starting with @TEST) in the 
setseparator.bro file.


From giedrius.ramas at gmail.com  Thu Aug 13 01:22:36 2015
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Thu, 13 Aug 2015 11:22:36 +0300
Subject: [Bro] bro v2.4 Bro Intel Framework Extensions do not work
In-Reply-To: <C99AB9F2-4DE6-43D6-BA00-BC33AC8AEEFF@icir.org>
References: <CAHr6V5qiDNeQS2fSyqhoijC6dKDtf8v5+b7u1zp1b=Uxut-DMQ@mail.gmail.com>
	<C99AB9F2-4DE6-43D6-BA00-BC33AC8AEEFF@icir.org>
Message-ID: <CAHr6V5q14ABuNFgJQf38eyXG8h4DBs7RmdvmOCqtGpN9j1NSfg@mail.gmail.com>

thank you for the fix. It is working fine now.

On Wed, Aug 12, 2015 at 4:56 PM, Seth Hall <seth at icir.org> wrote:

>
> > On Aug 12, 2015, at 8:37 AM, Giedrius Ramas <giedrius.ramas at gmail.com>
> wrote:
> >
> > error in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/extend.bro,
> line 19: no such field in record (Intel::s$f?$mime_type)
> > error in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/extend.bro,
> line 20: no such field in record (Intel::s$f$mime_type)
>
> Sorry about that.  I just pushed a fix to the repository (and now it only
> works with Bro 2.4).
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150813/dae0f7cf/attachment.html 

From giedrius.ramas at gmail.com  Thu Aug 13 03:51:29 2015
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Thu, 13 Aug 2015 13:51:29 +0300
Subject: [Bro] BRO 2.3.2 Intel email indicator do not work
In-Reply-To: <CAPy2kFbxneNuzrRB3xoHNQZOte2vFo7UKt6TPiaKfm0YcAndeQ@mail.gmail.com>
References: <CAHr6V5pNOmSW55+MHruMtgFDe44pZNpMGxa8EtjgQj8-HSNM3A@mail.gmail.com>
	<CAHr6V5qDUCuH9aEGp3RzurEesEexFgC5w0+6zKZTWv+9MEV9Hw@mail.gmail.com>
	<CAPy2kFbxneNuzrRB3xoHNQZOte2vFo7UKt6TPiaKfm0YcAndeQ@mail.gmail.com>
Message-ID: <CAHr6V5rv=xOma4_mxDmSwUqPMwoLFMOUa9ZGFrH52NrB8YQStw@mail.gmail.com>

I have just tested email indicator on BRO 2.4 version and no luck
. Intel::EMAIL does not work with BRO 2.4.
BRO 2.2 version works fine with Intel::EMAIL, double tested .

On Wed, Jun 10, 2015 at 12:53 PM, Mike Dopheide <dopheide at gmail.com> wrote:

> If nobody gets back to you sooner, I'll have time to test later this week
> if you hit me up then.  In the meantime, I'd suggest testing with 2.4 that
> was just released.
>
> Dop
>
>
> On Tuesday, June 9, 2015, Giedrius Ramas <giedrius.ramas at gmail.com> wrote:
>
>> anyone faced the same issue ?
>>
>> On Tue, Jun 2, 2015 at 9:09 AM, Giedrius Ramas <giedrius.ramas at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I found that BRO 2.3.4 Intel do not work with email's indicators. I have
>>> played on my infrastructure to get BRO intel work and found that email
>>> indicator won't work.
>>>
>>> I also tested it on  try.bro.org/ the same results . However BRO 2.2
>>> version works well with Intel email's indicators .
>>>
>>> Please let me know if more details needed to troubleshoot
>>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150813/2d55f53e/attachment.html 

From earl.eiland at root9b.com  Thu Aug 13 08:54:13 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Thu, 13 Aug 2015 15:54:13 +0000
Subject: [Bro] [JIRA] (BIT-1453) Input::add_table is not properly
 reading in sets
In-Reply-To: <JIRA.19308.1439411395000.40.1439477641799@Atlassian.JIRA>
References: <JIRA.19308.1439411395000@Atlassian.JIRA>
	<JIRA.19308.1439411395760@bro-tracker.atlassian.net>
	<JIRA.19308.1439411395000.40.1439477641799@Atlassian.JIRA>
Message-ID: <BY1PR0401MB13822F454235586564E85BF7D77D0@BY1PR0401MB1382.namprd04.prod.outlook.com>

If you're writing bro input files with python and using csv, then csv.writer must have the correct parameters.  For example,
write_model = csv.writer(model_file, delimiter='\x09', lineterminator = '\n')

These settings place tabs between the columns and terminate lines with the newline character.

Earl

-----Original Message-----
From: Daniel Thayer (JIRA) [mailto:jira at bro-tracker.atlassian.net] 
Sent: Thursday, August 13, 2015 9:54 AM
To: Earl Eiland <earl.eiland at root9b.com>
Subject: [JIRA] (BIT-1453) Input::add_table is not properly reading in sets


    [ https://bro-tracker.atlassian.net/browse/BIT-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21607#comment-21607 ] 

Daniel Thayer commented on BIT-1453:
------------------------------------

I can reproduce those errors.  The problem is that the file "model2.log.txt"
contains newline characters that do not match those used in Linux (perhaps you created the file on another OS, and then copied it over to your Linux machine?).  If you look at the file using the "vi" editor in Linux, you can see "^M" characters at the end of each line.  If you remove those, then the errors disappear.


> Input::add_table is not properly reading in sets
> ------------------------------------------------
>
>                 Key: BIT-1453
>                 URL: https://bro-tracker.atlassian.net/browse/BIT-1453
>             Project: Bro Issue Tracker
>          Issue Type: Problem
>          Components: Bro
>    Affects Versions: 2.4
>         Environment: ArchLinux on VMware
>            Reporter: earl eiland
>            Assignee: Johanna Amann
>              Labels: Input::add_table
>         Attachments: input.bro, input.log, model2.log.txt
>
>
> I?m reading a table into a script.  The table includes two sets in the values fields.  When executing the script, I?m getting the error message ?Did not find requested field service in input data file model2.log?
> Following the example in bro/testing/btest/scripts/base/frameworks/input/setseparator.bro, I?ve redefined the set separator as ?|? (redef InputAscii::set_separator = "|";).
> The
> The table key consists of two addresses, node_A and node_B.
> My value inputs consist of two sets, which can consist of just a single value; all fields are separated by tabs.  The first two lines  of my input file are:
> #fields  node_A                                node_B                                layer_3_4            service
> xxx.yyy.zzz.30   xxx.yyy.255.255                udp        dns
> xxx.yyy.zzz are valid IP address values.
> It appears that the strings ?udp? and ?dns? are both being read as part of the layer_3_4 set.  Since they are separated by a tab instead of ?|?, they should be interpreted as separate fields.  



--
This message was sent by Atlassian JIRA
(v6.5-OD-08-001#65007)


From capn.freako at gmail.com  Thu Aug 13 18:52:23 2015
From: capn.freako at gmail.com (David Banas)
Date: Thu, 13 Aug 2015 18:52:23 -0700
Subject: [Bro] Trouble passing a message between two local endpoints.
In-Reply-To: <13152031-5A91-4343-9080-DF7F03D2FA62@illinois.edu>
References: <36E1532B-83FB-4939-8AED-4FC748C80AE6@gmail.com>
	<230469BF-DEB0-4FFC-9A1B-4690AD15B9ED@illinois.edu>
	<71F7EEFF-A244-4775-8BF2-F3F790C35C87@gmail.com>
	<13152031-5A91-4343-9080-DF7F03D2FA62@illinois.edu>
Message-ID: <1E802A35-FE78-481D-A88F-DE37F769CF26@gmail.com>

Thanks, Jon.

Commands:

davids-air-2:broker-haskell dbanas$ touch test.c
davids-air-2:broker-haskell dbanas$ gcc test.c -lbroker
davids-air-2:broker-haskell dbanas$ ./a.out
There are 0 messages.

Code:

#include <stdio.h>
#include <stdlib.h>
#include "../bro/aux/broker/broker/broker.h"

void my_exit (char* msg) {
    printf("%s", msg);
    printf("\n");
    exit(-1);
}

int main (int argc, char* argv[]) {
    int res = broker_init(0);
    if(res) my_exit("broker_init() failed!");

    broker_endpoint* ep1 = broker_endpoint_create_with_flags("ep1", 3);
    if(!ep1) my_exit("Failed to create first endpoint!");

    broker_endpoint* ep2 = broker_endpoint_create_with_flags("ep2", 3);
    if(!ep2) my_exit("Failed to create second endpoint!");

    broker_string*    bs = broker_string_create("");
    if(!bs) my_exit("Failed to create topic string!");

    broker_message_queue* q = broker_message_queue_create(bs, ep2);
    if(!q) my_exit("Failed to create message queue!");

    broker_peering*    p = broker_endpoint_peer_locally(ep2, ep1);
    if(!p) my_exit("Failed to create peering!");

    const broker_outgoing_connection_status_queue* ocsq =
        broker_endpoint_outgoing_connection_status(ep2);
    if(!ocsq) my_exit("Failed to create status queue!");

    broker_deque_of_outgoing_connection_status_delete(
       broker_outgoing_connection_status_queue_need_pop(ocsq));
    
    broker_string*  msg_str = broker_string_create("Hello, World!\n");
    if(!msg_str) my_exit("Failed to create message string!");

    broker_string*    topic = broker_string_create("test");
    if(!topic) my_exit("Failed to create topic string!");

    broker_data*        msg = broker_data_from_string(msg_str);
    if(!msg) my_exit("Failed to create message data!");

    broker_vector*      vec = broker_vector_create();
    if(!vec) my_exit("Failed to create message vector!");

    res = broker_vector_insert(vec, msg, 0L);
    if(!res) my_exit("Failed to insert into vector!");

    res = broker_endpoint_send(ep1, topic, vec);
    if(!res) my_exit("Failed to send message!");

    broker_deque_of_message* msg_list = broker_message_queue_want_pop(q);
    if(!msg_list) my_exit("Failed to pop queue!");

    size_t num_msgs = broker_deque_of_message_size(msg_list);
    printf("There are %ld messages.\n", num_msgs);
}

-db

On Aug 11, 2015, at 7:05 AM, Siwek, Jon <jsiwek at illinois.edu> wrote:

> 
>> On Aug 10, 2015, at 2:05 PM, David Banas <capn.freako at gmail.com> wrote:
>> 
>> I implemented your recommended changes, recompiled, and re-ran, but am getting the same result:
>> 
>> davids-air-2:broker-haskell dbanas$ ./a.out 
>> There are 0 messages.
>> 
>> Any thoughts?
> 
> It shouldn?t be possible for broker_message_queue_need_pop() to return zero messages (it?s a bug if it does).  Double check your revised code against what I posted and that you compiled/ran that version.  Else post the exact code you ran and other details you may think would help reproduce those results (OS, compiler version, Broker version, CAF version, etc.).
> 
> - Jon

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150813/5cd0ea71/attachment.html 

From jsiwek at illinois.edu  Fri Aug 14 07:57:01 2015
From: jsiwek at illinois.edu (Siwek, Jon)
Date: Fri, 14 Aug 2015 14:57:01 +0000
Subject: [Bro] Trouble passing a message between two local endpoints.
In-Reply-To: <1E802A35-FE78-481D-A88F-DE37F769CF26@gmail.com>
References: <36E1532B-83FB-4939-8AED-4FC748C80AE6@gmail.com>
	<230469BF-DEB0-4FFC-9A1B-4690AD15B9ED@illinois.edu>
	<71F7EEFF-A244-4775-8BF2-F3F790C35C87@gmail.com>
	<13152031-5A91-4343-9080-DF7F03D2FA62@illinois.edu>
	<1E802A35-FE78-481D-A88F-DE37F769CF26@gmail.com>
Message-ID: <02DA0473-17AC-4E29-AB56-B8FAC4A075B1@illinois.edu>


> On Aug 13, 2015, at 8:52 PM, David Banas <capn.freako at gmail.com> wrote:
> 
>     broker_deque_of_message* msg_list = broker_message_queue_want_pop(q);

This ends up checking for messages while the one you just sent is still in-flight.  Try changing it to use ?broker_message_queue_need_pop(q)?.  That version will block until at least one message can be retrieved from the queue.

Alternatively, what you may want in a real application is integrate the ?want_pop? version into a poll()/select() loop so you get signaled when something is actually available to retrieve.

- Jon

From earl.eiland at root9b.com  Fri Aug 14 09:50:34 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Fri, 14 Aug 2015 16:50:34 +0000
Subject: [Bro] error when defining an enumerated data type
Message-ID: <BY1PR0401MB13827878E110E17DCAE82702D77C0@BY1PR0401MB1382.namprd04.prod.outlook.com>

Bro has an enumerated type for layer 3 protocols:  enum { tcp, udp, icmp, unknown }.  I want to create a variable with this type, but haven?t found the label in the documentation.  Where do I find such information?  I tried defining my own enumerated data type, but that generates the error ?identifier or enumerator value in enumerated type definition already exists?.

Best Regards,

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity named.  If you are not the named addressee you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.  Please notify the sender immediately by email if you received this email in error and delete this email from your system. Any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of root9B LLC.?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150814/039c5116/attachment.html 

From dnthayer at illinois.edu  Fri Aug 14 09:58:29 2015
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Fri, 14 Aug 2015 11:58:29 -0500
Subject: [Bro] error when defining an enumerated data type
In-Reply-To: <BY1PR0401MB13827878E110E17DCAE82702D77C0@BY1PR0401MB1382.namprd04.prod.outlook.com>
References: <BY1PR0401MB13827878E110E17DCAE82702D77C0@BY1PR0401MB1382.namprd04.prod.outlook.com>
Message-ID: <55CE1E35.1060702@illinois.edu>

There is some reference documentation here:
https://www.bro.org/sphinx/script-reference/index.html


On 08/14/2015 11:50 AM, Earl Eiland wrote:
> Bro has an enumerated type for layer 3 protocols:  enum { tcp, udp,
> icmp, unknown }.  I want to create a variable with this type, but
> haven?t found the label in the documentation.  Where do I find such
> information?  I tried defining my own enumerated data type, but that
> generates the error ?identifier or enumerator value in enumerated type
> definition already exists?.
>
> Best Regards,
>
> Earl Eiland,
>
> Sr. Cyber Security Engineer,
>
> Emerging Technologies, root9B,
>
> San Antonio, Texas
>
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity named.  If you
> are not the named addressee you are notified that disclosing, copying,
> distributing or taking any action in reliance on the contents of this
> information is strictly prohibited.  Please notify the sender
> immediately by email if you received this email in error and delete this
> email from your system. Any views or opinions presented in this e-mail
> are solely those of the author and do not necessarily represent those of
> root9B LLC.?
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>

From earl.eiland at root9b.com  Fri Aug 14 10:06:21 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Fri, 14 Aug 2015 17:06:21 +0000
Subject: [Bro] error when defining an enumerated data type
In-Reply-To: <55CE1E35.1060702@illinois.edu>
References: <BY1PR0401MB13827878E110E17DCAE82702D77C0@BY1PR0401MB1382.namprd04.prod.outlook.com>
	<55CE1E35.1060702@illinois.edu>
Message-ID: <BY1PR0401MB1382D343B832D4D370A6DD5CD77C0@BY1PR0401MB1382.namprd04.prod.outlook.com>

I looked at that.  It covers enumerated data types in general, and the port type.  The port type includes both the port # and protocol.   It also covers how to extract the protocol from the port type with the function get_port_transport_proto.  However, there's nothing on the protocol type's label.

-----Original Message-----
From: Daniel Thayer [mailto:dnthayer at illinois.edu] 
Sent: Friday, August 14, 2015 11:58 AM
To: Earl Eiland <earl.eiland at root9b.com>; bro at bro.org
Subject: Re: [Bro] error when defining an enumerated data type

There is some reference documentation here:
https://www.bro.org/sphinx/script-reference/index.html


On 08/14/2015 11:50 AM, Earl Eiland wrote:
> Bro has an enumerated type for layer 3 protocols:  enum { tcp, udp, 
> icmp, unknown }.  I want to create a variable with this type, but 
> haven?t found the label in the documentation.  Where do I find such 
> information?  I tried defining my own enumerated data type, but that 
> generates the error ?identifier or enumerator value in enumerated type 
> definition already exists?.
>
> Best Regards,
>
> Earl Eiland,
>
> Sr. Cyber Security Engineer,
>
> Emerging Technologies, root9B,
>
> San Antonio, Texas
>
> This email and any files transmitted with it are confidential and 
> intended solely for the use of the individual or entity named.  If you 
> are not the named addressee you are notified that disclosing, copying, 
> distributing or taking any action in reliance on the contents of this 
> information is strictly prohibited.  Please notify the sender 
> immediately by email if you received this email in error and delete 
> this email from your system. Any views or opinions presented in this 
> e-mail are solely those of the author and do not necessarily represent 
> those of root9B LLC.?
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>


From dnthayer at illinois.edu  Fri Aug 14 10:18:10 2015
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Fri, 14 Aug 2015 12:18:10 -0500
Subject: [Bro] error when defining an enumerated data type
In-Reply-To: <BY1PR0401MB1382D343B832D4D370A6DD5CD77C0@BY1PR0401MB1382.namprd04.prod.outlook.com>
References: <BY1PR0401MB13827878E110E17DCAE82702D77C0@BY1PR0401MB1382.namprd04.prod.outlook.com>
	<55CE1E35.1060702@illinois.edu>
	<BY1PR0401MB1382D343B832D4D370A6DD5CD77C0@BY1PR0401MB1382.namprd04.prod.outlook.com>
Message-ID: <55CE22D2.3000804@illinois.edu>

In the reference documentation for type "port", it mentions you can
use function "get_port_transport_proto" to obtain the protocol type of
a port.  If you look at the reference documentation for that function,
you can see it returns something of type "transport_proto".
If you look at the reference documentation for type "transport_proto",
you can see it's the enum type that you mentioned.

Now that you know the type, you can declare a local variable like this:

event bro_init()
{
   local x: transport_proto = tcp;
   print x;
}


On 08/14/2015 12:06 PM, Earl Eiland wrote:
> I looked at that.  It covers enumerated data types in general, and the port type.  The port type includes both the port # and protocol.   It also covers how to extract the protocol from the port type with the function get_port_transport_proto.  However, there's nothing on the protocol type's label.
>
> -----Original Message-----
> From: Daniel Thayer [mailto:dnthayer at illinois.edu]
> Sent: Friday, August 14, 2015 11:58 AM
> To: Earl Eiland <earl.eiland at root9b.com>; bro at bro.org
> Subject: Re: [Bro] error when defining an enumerated data type
>
> There is some reference documentation here:
> https://www.bro.org/sphinx/script-reference/index.html
>
>
> On 08/14/2015 11:50 AM, Earl Eiland wrote:
>> Bro has an enumerated type for layer 3 protocols:  enum { tcp, udp,
>> icmp, unknown }.  I want to create a variable with this type, but
>> haven?t found the label in the documentation.  Where do I find such
>> information?  I tried defining my own enumerated data type, but that
>> generates the error ?identifier or enumerator value in enumerated type
>> definition already exists?.
>>
>> Best Regards,
>>
>> Earl Eiland,
>>
>> Sr. Cyber Security Engineer,
>>
>> Emerging Technologies, root9B,
>>
>> San Antonio, Texas
>>
>> This email and any files transmitted with it are confidential and
>> intended solely for the use of the individual or entity named.  If you
>> are not the named addressee you are notified that disclosing, copying,
>> distributing or taking any action in reliance on the contents of this
>> information is strictly prohibited.  Please notify the sender
>> immediately by email if you received this email in error and delete
>> this email from your system. Any views or opinions presented in this
>> e-mail are solely those of the author and do not necessarily represent
>> those of root9B LLC.?
>>
>>
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>

From capn.freako at gmail.com  Fri Aug 14 11:16:29 2015
From: capn.freako at gmail.com (David Banas)
Date: Fri, 14 Aug 2015 11:16:29 -0700
Subject: [Bro] Trouble passing a message between two local endpoints.
In-Reply-To: <02DA0473-17AC-4E29-AB56-B8FAC4A075B1@illinois.edu>
References: <36E1532B-83FB-4939-8AED-4FC748C80AE6@gmail.com>
	<230469BF-DEB0-4FFC-9A1B-4690AD15B9ED@illinois.edu>
	<71F7EEFF-A244-4775-8BF2-F3F790C35C87@gmail.com>
	<13152031-5A91-4343-9080-DF7F03D2FA62@illinois.edu>
	<1E802A35-FE78-481D-A88F-DE37F769CF26@gmail.com>
	<02DA0473-17AC-4E29-AB56-B8FAC4A075B1@illinois.edu>
Message-ID: <8E2333B0-0A69-44E3-A072-0C17B3B41188@gmail.com>

That worked!:

Davids-MacBook-Air-2:broker-haskell dbanas$ ./a.out
There are 1 messages.

Thanks, so much, for hand holding me through this, Jon!

Have a great weekend,
-db

On Aug 14, 2015, at 7:57 AM, Siwek, Jon <jsiwek at illinois.edu> wrote:

> 
>> On Aug 13, 2015, at 8:52 PM, David Banas <capn.freako at gmail.com> wrote:
>> 
>>    broker_deque_of_message* msg_list = broker_message_queue_want_pop(q);
> 
> This ends up checking for messages while the one you just sent is still in-flight.  Try changing it to use ?broker_message_queue_need_pop(q)?.  That version will block until at least one message can be retrieved from the queue.
> 
> Alternatively, what you may want in a real application is integrate the ?want_pop? version into a poll()/select() loop so you get signaled when something is actually available to retrieve.
> 
> - Jon

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150814/0ad3304c/attachment.html 

From lists at g-clef.net  Fri Aug 14 11:17:09 2015
From: lists at g-clef.net (Aaron Gee-Clough)
Date: Fri, 14 Aug 2015 14:17:09 -0400
Subject: [Bro] Bro Kafka logging plugin
Message-ID: <55CE30A5.40100@g-clef.net>


All,

I have a bro plugin that I've just finished writing that I'm hoping some
folks will help test. It's a logging plugin that will send
JSON-formatted bro logs to a Kafka message broker.

The code is at: https://github.com/g-clef/KafkaLogger .

Rather than writing a simple log forwarder, I modified things a bit:
	* You can specify which logs to send to Kafka in the bro config.
	* It will add a "type" field to the JSON message to clarify which log
the message came from ("http" vs "conn" vs "ssl", for example).
	* It will add a "sensor" field to the JSON message to allow you to tag
logs from particular sensors in your network (if you have multiple bro
sensors, you may want to be able to distinguish between logs from
different sensors).
	* It will rename the "ts", "id.orig_h", "id.orig_p", "id.resp_h", and
"id.resp_p" fields to more commonly-used names (for example, "ts"
becomes "@timestamp" to interoperate with logstash-style logs).

The plugin is pretty young, so I would consider this beta testing at the
moment. It is working and seems to be stable in my testing, but I'd love
to have some other folks than me testing it.

Feel free to send me questions or pull requests.

aaron

From earl.eiland at root9b.com  Mon Aug 17 08:03:48 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Mon, 17 Aug 2015 15:03:48 +0000
Subject: [Bro] confusing error on if statement
Message-ID: <BY1PR0401MB13823FCD8B3384C10C978EB5D7790@BY1PR0401MB1382.namprd04.prod.outlook.com>

I?m testing to see if c$service is a subset of a set[string] field in a table, and getting the error message

?error in string and ./bro_scripts/Screen_protocols.bro, line 79: type clash (string and Screen_protocols::c$service)
error in ./bro_scripts/Screen_protocols.bro, line 79 and string: type mismatch (Screen_protocols::c$service and string)
error in ./bro_scripts/Screen_protocols.bro, line 79: not an index type (Screen_protocols::c$service in Screen_protocols::whitelist[Screen_protocols::c$id$orig_h, Screen_protocols::c$id$resp_h]$service)?

The command is: ?if (c$service !in whitelist[c$id$orig_h, c$id$resp_h]$service)?
Where the table is defined as ?global whitelist: table[addr, addr] of Service_whitelist_data = table();? and the table field service is defined as ?service: set[string];?

Please advise!

Best Regards,

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150817/4d0a7cf8/attachment.html 

From marcus at randomhack.org  Mon Aug 17 12:30:44 2015
From: marcus at randomhack.org (Marcus LaFerrera)
Date: Mon, 17 Aug 2015 15:30:44 -0400
Subject: [Bro] Bro Kafka logging plugin
In-Reply-To: <55CE30A5.40100@g-clef.net>
References: <55CE30A5.40100@g-clef.net>
Message-ID: <CANsWCU=Cr74CvaFGBXzXp0Ny9bBzW8_dZROiV656Pa912DF=Xg@mail.gmail.com>

Thanks for sharing Aaron. This is terrific.

On Fri, Aug 14, 2015 at 2:17 PM, Aaron Gee-Clough <lists at g-clef.net> wrote:

>
> All,
>
> I have a bro plugin that I've just finished writing that I'm hoping some
> folks will help test. It's a logging plugin that will send
> JSON-formatted bro logs to a Kafka message broker.
>
> The code is at: https://github.com/g-clef/KafkaLogger .
>
> Rather than writing a simple log forwarder, I modified things a bit:
>         * You can specify which logs to send to Kafka in the bro config.
>         * It will add a "type" field to the JSON message to clarify which
> log
> the message came from ("http" vs "conn" vs "ssl", for example).
>         * It will add a "sensor" field to the JSON message to allow you to
> tag
> logs from particular sensors in your network (if you have multiple bro
> sensors, you may want to be able to distinguish between logs from
> different sensors).
>         * It will rename the "ts", "id.orig_h", "id.orig_p", "id.resp_h",
> and
> "id.resp_p" fields to more commonly-used names (for example, "ts"
> becomes "@timestamp" to interoperate with logstash-style logs).
>
> The plugin is pretty young, so I would consider this beta testing at the
> moment. It is working and seems to be stable in my testing, but I'd love
> to have some other folks than me testing it.
>
> Feel free to send me questions or pull requests.
>
> aaron
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Cheers,
Marcus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150817/9eebda40/attachment.html 

From anthony.kasza at gmail.com  Mon Aug 17 12:33:34 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Mon, 17 Aug 2015 12:33:34 -0700
Subject: [Bro] confusing error on if statement
In-Reply-To: <BY1PR0401MB13823FCD8B3384C10C978EB5D7790@BY1PR0401MB1382.namprd04.prod.outlook.com>
References: <BY1PR0401MB13823FCD8B3384C10C978EB5D7790@BY1PR0401MB1382.namprd04.prod.outlook.com>
Message-ID: <CAEZw2bxjkvZuucMqfj++9U5BjgTc5VkgPvN908sqe4jL1TiaUg@mail.gmail.com>

Hi Earl,

Are you able to include the script here?

-AK
On Aug 17, 2015 8:16 AM, "Earl Eiland" <earl.eiland at root9b.com> wrote:

> I?m testing to see if c$service is a subset of a set[string] field in a
> table, and getting the error message
>
>
>
> ?error in string and ./bro_scripts/Screen_protocols.bro, line 79: type
> clash (string and Screen_protocols::c$service)
>
> error in ./bro_scripts/Screen_protocols.bro, line 79 and string: type
> mismatch (Screen_protocols::c$service and string)
>
> error in ./bro_scripts/Screen_protocols.bro, line 79: not an index type
> (Screen_protocols::c$service in
> Screen_protocols::whitelist[Screen_protocols::c$id$orig_h,
> Screen_protocols::c$id$resp_h]$service)?
>
>
>
> The command is: ?if (c$service !in whitelist[c$id$orig_h,
> c$id$resp_h]$service)?
>
> Where the table is defined as ?global whitelist: table[addr, addr] of
> Service_whitelist_data = table();? and the table field service is defined
> as ?service: set[string];?
>
>
>
> Please advise!
>
>
>
> Best Regards,
>
>
>
> Earl Eiland,
>
> Sr. Cyber Security Engineer,
>
> Emerging Technologies, root9B,
>
> San Antonio, Texas
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150817/71a1be82/attachment.html 

From earl.eiland at root9b.com  Tue Aug 18 11:49:36 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Tue, 18 Aug 2015 18:49:36 +0000
Subject: [Bro] testing against uninitialized fields
Message-ID: <BY1PR0401MB1382BC531D1907BDD8C71F40D7780@BY1PR0401MB1382.namprd04.prod.outlook.com>

I?m writing a script that works with the connection$service field.  Sometimes, this field is unintialized, which is causing my script to fail.  How can I catch an unintialized field?

Best Regards,

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity named.  If you are not the named addressee you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.  Please notify the sender immediately by email if you received this email in error and delete this email from your system. Any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of root9B LLC.?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150818/a5b20012/attachment.html 

From lists at g-clef.net  Tue Aug 18 12:56:20 2015
From: lists at g-clef.net (Aaron Gee-Clough)
Date: Tue, 18 Aug 2015 15:56:20 -0400
Subject: [Bro] testing against uninitialized fields
References: <BY1PR0401MB1382BC531D1907BDD8C71F40D7780@BY1PR0401MB1382.namprd04.prod.outlook.com>
Message-ID: <55D38DE4.4070500@g-clef.net>


Try:

   if (connection?$service){
      <do stuff that relies on service field>
   }

or if you'd prefer to explicitly bail out if it's not there:

   if (!connection?$service){return;}



aaron

On 08/18/2015 02:49 PM, Earl Eiland wrote:
> I?m writing a script that works with the connection$service field. 
> Sometimes, this field is unintialized, which is causing my script to
> fail.  How can I catch an unintialized field?
> 
>  
> 
> Best Regards,
> 
>  
> 
> Earl Eiland,
> 
> Sr. Cyber Security Engineer,
> 
> Emerging Technologies, root9B,
> 
> San Antonio, Texas
> 
>  
> 
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity named.  If you
> are not the named addressee you are notified that disclosing, copying,
> distributing or taking any action in reliance on the contents of this
> information is strictly prohibited.  Please notify the sender
> immediately by email if you received this email in error and delete this
> email from your system. Any views or opinions presented in this e-mail
> are solely those of the author and do not necessarily represent those of
> root9B LLC.?
> 
>  
> 
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 

From anthony.kasza at gmail.com  Tue Aug 18 12:58:13 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Tue, 18 Aug 2015 12:58:13 -0700
Subject: [Bro] testing against uninitialized fields
In-Reply-To: <BY1PR0401MB1382BC531D1907BDD8C71F40D7780@BY1PR0401MB1382.namprd04.prod.outlook.com>
References: <BY1PR0401MB1382BC531D1907BDD8C71F40D7780@BY1PR0401MB1382.namprd04.prod.outlook.com>
Message-ID: <CAEZw2bwLQ0m2riraektEzu5ndUmuLqpHF2-yr8t_LbR_DfmDSQ@mail.gmail.com>

The ?$ operator is what you want.

-AK
On Aug 18, 2015 11:53 AM, "Earl Eiland" <earl.eiland at root9b.com> wrote:

> I?m writing a script that works with the connection$service field.
> Sometimes, this field is unintialized, which is causing my script to fail.
> How can I catch an unintialized field?
>
>
>
> Best Regards,
>
>
>
> Earl Eiland,
>
> Sr. Cyber Security Engineer,
>
> Emerging Technologies, root9B,
>
> San Antonio, Texas
>
>
>
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity named.  If you are not the
> named addressee you are notified that disclosing, copying, distributing or
> taking any action in reliance on the contents of this information is
> strictly prohibited.  Please notify the sender immediately by email if you
> received this email in error and delete this email from your system. Any
> views or opinions presented in this e-mail are solely those of the author
> and do not necessarily represent those of root9B LLC.?
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150818/51f6eb01/attachment.html 

From easetheworld at gmail.com  Tue Aug 18 14:48:32 2015
From: easetheworld at gmail.com (Hyun Yoo)
Date: Wed, 19 Aug 2015 06:48:32 +0900
Subject: [Bro] spam mail message collector
Message-ID: <CAJzQz-GhCOU1eYvM4Oi1dAzxAQZ9szVoP+RtfQ8VeNMSkTR2wg@mail.gmail.com>

Hello Bro. I am new to bro.
I think my task is more suitable to Bro than other NIDS.

There is a list of spammer email addresses and
I want to save the email subject and whole message of them.
(reassembled payload of tcp segments)
I tried a few events like log_smtp, tcp_contents but couldn't save the
whole stream.

Can anybody guide me to the right way, please?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150819/b146e440/attachment.html 

From gfaulkner.nsm at gmail.com  Tue Aug 18 20:00:11 2015
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Tue, 18 Aug 2015 22:00:11 -0500
Subject: [Bro] [security-onion] Bro IDS: binapc exception in dpd.log
In-Reply-To: <CAK8kjrDi1QzOK1tiBaYKAFFW=ah9nPC_9sKVnYifJfKJsF_NLg@mail.gmail.com>
References: <c532f5e4-bbbe-4dba-9987-61ec04b9821b@googlegroups.com>
	<CAK8kjrDi1QzOK1tiBaYKAFFW=ah9nPC_9sKVnYifJfKJsF_NLg@mail.gmail.com>
Message-ID: <55D3F13B.1020806@gmail.com>

Cross-posting over to bro list... I took a look on my own Bro cluster
built from git master 2.4-10 on RHEL 6.6, and I am seeing similar binpac
errors in dpd.log. Probably worthy of an issue report to the Bro team.

Also, it seems odd to see binpac error messages in dpd.log. This seems
more like something that would be in reporter.log, so I wonder if that
is intended? I also see some binpac errors for rdp, and SSL IN dpd.log.

Here are some more samples:

1439952507.945287       C0Zth33h2gy9HEGM4k      10.10.250.141  5070   
10.10.146.171  5060    udp     SIP     Binpac exception: binpac
exception: string mismatch at
/nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70: \x0aexpected
pattern: ":"\x0aactual data: " 1702356679 1793741124 IN IP4
10.10.250.141\x0d\x0as=sipcli\x0d\x0ac=IN IP4 10.10.250.141\x0d\x0at=0
0\x0d\x0am=audio 5072 RTP/AVP 18 0 8 101\x0d\x0aa=fmtp:101
0-15\x0d\x0aa=rtpmap:18 G729/8000\x0d\x0aa=rtpmap:0
PCMU/8000\x0d\x0aa=rtpmap:8 PCMA/8000\x0d\x0aa=rtpmap:101
telephone-event/8000\x0d\x0aa=ptime:20\x0d\x0aa=sendrecv\x0d\x0a"

1439952508.235601       CfnJdC2wJa7QObDdK7      10.10.250.141  5110   
10.10.146.171  5060    udp     SIP     Binpac exception: binpac
exception: string mismatch at
/nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70: \x0aexpected
pattern: ":"\x0aactual data: " 2046637637 2105833686 IN IP4
10.10.250.141\x0d\x0as=sipcli\x0d\x0ac=IN IP4 10.10.250.141\x0d\x0at=0
0\x0d\x0am=audio 5111 RTP/AVP 18 0 8 101\x0d\x0aa=fmtp:101
0-15\x0d\x0aa=rtpmap:18 G729/8000\x0d\x0aa=rtpmap:0
PCMU/8000\x0d\x0aa=rtpmap:8 PCMA/8000\x0d\x0aa=rtpmap:101
telephone-event/8000\x0d\x0aa=ptime:20\x0d\x0aa=sendrecv\x0d\x0a"

1439952508.245335       CfnJdC2wJa7QObDdK7      10.10.250.141  5110   
10.10.146.171  5060    udp     SIP     Binpac exception: binpac
exception: string mismatch at
/nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70: \x0aexpected
pattern: ":"\x0aactual data: " 2046637637 2105833686 IN IP4
10.10.250.141\x0d\x0as=sipcli\x0d\x0ac=IN IP4 10.10.250.141\x0d\x0at=0
0\x0d\x0am=audio 5111 RTP/AVP 18 0 8 101\x0d\x0aa=fmtp:101
0-15\x0d\x0aa=rtpmap:18 G729/8000\x0d\x0aa=rtpmap:0
PCMU/8000\x0d\x0aa=rtpmap:8 PCMA/8000\x0d\x0aa=rtpmap:101
telephone-event/8000\x0d\x0aa=ptime:20\x0d\x0aa=sendrecv\x0d\x0a"

1439952508.597857       C2vuSQ3duZlPtt6Njl      10.10.44.245  5060   
10.10.7.100    5060    udp     SIP     Binpac exception: binpac
exception: string mismatch at
/nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70: \x0aexpected
pattern: ":"\x0aactual data: " version='1.0'
encoding='UTF-8'?><!--PUA--><presence
xmlns='urn:ietf:params:xml:ns:pidf'
xmlns:dm='urn:ietf:params:xml:ns:pidf:data-model'
xmlns:rpid='urn:ietf:params:xml:ns:pidf:rpid'
xmlns:c='urn:ietf:params:xml:ns:pidf:cipid'
entity='sip:CIO-EX90 at EXAMPLE.COM    '><tuple
id='f71ad0ae-dc51-4be2-977d-39c9ccc2d29b'><status><basic>open</basic></status></tuple></presence>"

On 8/18/2015 6:26 PM, Doug Burks wrote:
> Hi Tommy,
>
> My guess is that this isn't strictly related to Security Onion, as we
> have a fairly standard build of Bro.  The reason for the
> "/build/securityonion-bro-C1BIlk/securityonion-bro-2.4/src/analyzer/protocol/sip/sip-protocol.pac"
> is that that's the build directory where the Ubuntu Launchpad build
> server builds our binaries.
>
> I would take a look at the actual traffic and see if it's valid SIP or
> perhaps just a scan or some other kind of traffic.
>
> On Tue, Aug 18, 2015 at 5:59 PM,  <tommydew at gmail.com> wrote:
>> While looking through the 'dpd.log' in '/nsm/bro/logs/current/', I found several log entries that reported 'Binapc exception'. Here's a sample with redacted IPs:
>>
>> 1439934408.353389       CMUcGx4TXPPDGCIb65      xxx.xxx.xxx.xxx 40046   xxx.xxx.xxx.xxx 5060    udp     SIP     Binpac exception: binpac exception: string mismatch at /build/securityonion-bro-C1BIlk/securityonion-bro-2.4/src/analyzer/protocol/sip/sip-protocol.pac:34: \x0aexpected pattern: "[[:alnum:]@[:punct:]]+"\x0aactual data: ""
>>
>> It appears that the issue may be related to Security Onion, but I can always move this to the Bro IDS mailing list if it's specific to Bro. I'll try to see what could be causing the exception, but I was curious if anyone else had any ideas.
>>
>> Thanks.
>>
>> --
>> Tommy
>>
>> --
>> You received this message because you are subscribed to the Google Groups "security-onion" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe at googlegroups.com.
>> To post to this group, send email to security-onion at googlegroups.com.
>> Visit this group at http://groups.google.com/group/security-onion.
>> For more options, visit https://groups.google.com/d/optout.
>
>


From michalpurzynski1 at gmail.com  Tue Aug 18 21:23:33 2015
From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=)
Date: Tue, 18 Aug 2015 21:23:33 -0700
Subject: [Bro] [security-onion] Bro IDS: binapc exception in dpd.log
In-Reply-To: <55D3F13B.1020806@gmail.com>
References: <c532f5e4-bbbe-4dba-9987-61ec04b9821b@googlegroups.com>
	<CAK8kjrDi1QzOK1tiBaYKAFFW=ah9nPC_9sKVnYifJfKJsF_NLg@mail.gmail.com>
	<55D3F13B.1020806@gmail.com>
Message-ID: <CAJ6bFK0+guBW5689g+X0Q+eFb55zhrk0YqXymO2F9+OuP-iJ+g@mail.gmail.com>

Can you tell us what kind of error code you have in sip.log for this
connection id?

I have similar errors, with user agent sipcli/v1.8 and result 401
Unauthorized so that's a scan of some kind.

I've filed a Bro bug

https://bro-tracker.atlassian.net/browse/BIT-1458

We might consider moving discussion to the Bro mailing list and/or
BIT-1458, as the problem is not SO specific.


On Tue, Aug 18, 2015 at 8:00 PM, Gary Faulkner <gfaulkner.nsm at gmail.com>
wrote:
> Cross-posting over to bro list... I took a look on my own Bro cluster
> built from git master 2.4-10 on RHEL 6.6, and I am seeing similar binpac
> errors in dpd.log. Probably worthy of an issue report to the Bro team.
>
> Also, it seems odd to see binpac error messages in dpd.log. This seems
> more like something that would be in reporter.log, so I wonder if that
> is intended? I also see some binpac errors for rdp, and SSL IN dpd.log.
>
> Here are some more samples:
>
> 1439952507.945287       C0Zth33h2gy9HEGM4k      10.10.250.141  5070
> 10.10.146.171  5060    udp     SIP     Binpac exception: binpac
> exception: string mismatch at
> /nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70:
\x0aexpected
> pattern: ":"\x0aactual data: " 1702356679 1793741124 IN IP4
> 10.10.250.141\x0d\x0as=sipcli\x0d\x0ac=IN IP4 10.10.250.141\x0d\x0at=0
> 0\x0d\x0am=audio 5072 RTP/AVP 18 0 8 101\x0d\x0aa=fmtp:101
> 0-15\x0d\x0aa=rtpmap:18 G729/8000\x0d\x0aa=rtpmap:0
> PCMU/8000\x0d\x0aa=rtpmap:8 PCMA/8000\x0d\x0aa=rtpmap:101
> telephone-event/8000\x0d\x0aa=ptime:20\x0d\x0aa=sendrecv\x0d\x0a"
>
> 1439952508.235601       CfnJdC2wJa7QObDdK7      10.10.250.141  5110
> 10.10.146.171  5060    udp     SIP     Binpac exception: binpac
> exception: string mismatch at
> /nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70:
\x0aexpected
> pattern: ":"\x0aactual data: " 2046637637 2105833686 IN IP4
> 10.10.250.141\x0d\x0as=sipcli\x0d\x0ac=IN IP4 10.10.250.141\x0d\x0at=0
> 0\x0d\x0am=audio 5111 RTP/AVP 18 0 8 101\x0d\x0aa=fmtp:101
> 0-15\x0d\x0aa=rtpmap:18 G729/8000\x0d\x0aa=rtpmap:0
> PCMU/8000\x0d\x0aa=rtpmap:8 PCMA/8000\x0d\x0aa=rtpmap:101
> telephone-event/8000\x0d\x0aa=ptime:20\x0d\x0aa=sendrecv\x0d\x0a"
>
> 1439952508.245335       CfnJdC2wJa7QObDdK7      10.10.250.141  5110
> 10.10.146.171  5060    udp     SIP     Binpac exception: binpac
> exception: string mismatch at
> /nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70:
\x0aexpected
> pattern: ":"\x0aactual data: " 2046637637 2105833686 IN IP4
> 10.10.250.141\x0d\x0as=sipcli\x0d\x0ac=IN IP4 10.10.250.141\x0d\x0at=0
> 0\x0d\x0am=audio 5111 RTP/AVP 18 0 8 101\x0d\x0aa=fmtp:101
> 0-15\x0d\x0aa=rtpmap:18 G729/8000\x0d\x0aa=rtpmap:0
> PCMU/8000\x0d\x0aa=rtpmap:8 PCMA/8000\x0d\x0aa=rtpmap:101
> telephone-event/8000\x0d\x0aa=ptime:20\x0d\x0aa=sendrecv\x0d\x0a"
>
> 1439952508.597857       C2vuSQ3duZlPtt6Njl      10.10.44.245  5060
> 10.10.7.100    5060    udp     SIP     Binpac exception: binpac
> exception: string mismatch at
> /nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70:
\x0aexpected
> pattern: ":"\x0aactual data: " version='1.0'
> encoding='UTF-8'?><!--PUA--><presence
> xmlns='urn:ietf:params:xml:ns:pidf'
> xmlns:dm='urn:ietf:params:xml:ns:pidf:data-model'
> xmlns:rpid='urn:ietf:params:xml:ns:pidf:rpid'
> xmlns:c='urn:ietf:params:xml:ns:pidf:cipid'
> entity='sip:CIO-EX90 at EXAMPLE.COM    '><tuple
>
id='f71ad0ae-dc51-4be2-977d-39c9ccc2d29b'><status><basic>open</basic></status></tuple></presence>"
>
> On 8/18/2015 6:26 PM, Doug Burks wrote:
>> Hi Tommy,
>>
>> My guess is that this isn't strictly related to Security Onion, as we
>> have a fairly standard build of Bro.  The reason for the
>>
"/build/securityonion-bro-C1BIlk/securityonion-bro-2.4/src/analyzer/protocol/sip/sip-protocol.pac"
>> is that that's the build directory where the Ubuntu Launchpad build
>> server builds our binaries.
>>
>> I would take a look at the actual traffic and see if it's valid SIP or
>> perhaps just a scan or some other kind of traffic.
>>
>> On Tue, Aug 18, 2015 at 5:59 PM,  <tommydew at gmail.com> wrote:
>>> While looking through the 'dpd.log' in '/nsm/bro/logs/current/', I
found several log entries that reported 'Binapc exception'. Here's a sample
with redacted IPs:
>>>
>>> 1439934408.353389       CMUcGx4TXPPDGCIb65      xxx.xxx.xxx.xxx 40046
xxx.xxx.xxx.xxx 5060    udp     SIP     Binpac exception: binpac exception:
string mismatch at
/build/securityonion-bro-C1BIlk/securityonion-bro-2.4/src/analyzer/protocol/sip/sip-protocol.pac:34:
\x0aexpected pattern: "[[:alnum:]@[:punct:]]+"\x0aactual data: ""
>>>
>>> It appears that the issue may be related to Security Onion, but I can
always move this to the Bro IDS mailing list if it's specific to Bro. I'll
try to see what could be causing the exception, but I was curious if anyone
else had any ideas.
>>>
>>> Thanks.
>>>
>>> --
>>> Tommy
>>>
>>> --
>>> You received this message because you are subscribed to the Google
Groups "security-onion" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
an email to security-onion+unsubscribe at googlegroups.com.
>>> To post to this group, send email to security-onion at googlegroups.com.
>>> Visit this group at http://groups.google.com/group/security-onion.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
"security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email to security-onion+unsubscribe at googlegroups.com.
> To post to this group, send email to security-onion at googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150818/9711d27e/attachment-0001.html 

From gfaulkner.nsm at gmail.com  Tue Aug 18 21:39:31 2015
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Tue, 18 Aug 2015 23:39:31 -0500
Subject: [Bro] [security-onion] Bro IDS: binapc exception in dpd.log
In-Reply-To: <CAJ6bFK0+guBW5689g+X0Q+eFb55zhrk0YqXymO2F9+OuP-iJ+g@mail.gmail.com>
References: <c532f5e4-bbbe-4dba-9987-61ec04b9821b@googlegroups.com>
	<CAK8kjrDi1QzOK1tiBaYKAFFW=ah9nPC_9sKVnYifJfKJsF_NLg@mail.gmail.com>
	<55D3F13B.1020806@gmail.com>
	<CAJ6bFK0+guBW5689g+X0Q+eFb55zhrk0YqXymO2F9+OuP-iJ+g@mail.gmail.com>
Message-ID: <55D40883.8000705@gmail.com>

Yes, the corresponding entries in sip.log are for sipcli/v1.8, but with
result 404 Not Found. I am seeing a lot of repeating source addresses,
so could very likely be a scanner.

On 8/18/2015 11:23 PM, Micha? Purzy?ski wrote:
> Can you tell us what kind of error code you have in sip.log for this
> connection id?
>
> I have similar errors, with user agent sipcli/v1.8 and result 401
> Unauthorized so that's a scan of some kind.
>
> I've filed a Bro bug
>
> https://bro-tracker.atlassian.net/browse/BIT-1458
>
> We might consider moving discussion to the Bro mailing list and/or
> BIT-1458, as the problem is not SO specific.
>
>
> On Tue, Aug 18, 2015 at 8:00 PM, Gary Faulkner <gfaulkner.nsm at gmail.com>
> wrote:
>> Cross-posting over to bro list... I took a look on my own Bro cluster
>> built from git master 2.4-10 on RHEL 6.6, and I am seeing similar binpac
>> errors in dpd.log. Probably worthy of an issue report to the Bro team.
>>
>> Also, it seems odd to see binpac error messages in dpd.log. This seems
>> more like something that would be in reporter.log, so I wonder if that
>> is intended? I also see some binpac errors for rdp, and SSL IN dpd.log.
>>
>> Here are some more samples:
>>
>> 1439952507.945287       C0Zth33h2gy9HEGM4k      10.10.250.141  5070
>> 10.10.146.171  5060    udp     SIP     Binpac exception: binpac
>> exception: string mismatch at
>> /nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70:
> \x0aexpected
>> pattern: ":"\x0aactual data: " 1702356679 1793741124 IN IP4
>> 10.10.250.141\x0d\x0as=sipcli\x0d\x0ac=IN IP4 10.10.250.141\x0d\x0at=0
>> 0\x0d\x0am=audio 5072 RTP/AVP 18 0 8 101\x0d\x0aa=fmtp:101
>> 0-15\x0d\x0aa=rtpmap:18 G729/8000\x0d\x0aa=rtpmap:0
>> PCMU/8000\x0d\x0aa=rtpmap:8 PCMA/8000\x0d\x0aa=rtpmap:101
>> telephone-event/8000\x0d\x0aa=ptime:20\x0d\x0aa=sendrecv\x0d\x0a"
>>
>> 1439952508.235601       CfnJdC2wJa7QObDdK7      10.10.250.141  5110
>> 10.10.146.171  5060    udp     SIP     Binpac exception: binpac
>> exception: string mismatch at
>> /nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70:
> \x0aexpected
>> pattern: ":"\x0aactual data: " 2046637637 2105833686 IN IP4
>> 10.10.250.141\x0d\x0as=sipcli\x0d\x0ac=IN IP4 10.10.250.141\x0d\x0at=0
>> 0\x0d\x0am=audio 5111 RTP/AVP 18 0 8 101\x0d\x0aa=fmtp:101
>> 0-15\x0d\x0aa=rtpmap:18 G729/8000\x0d\x0aa=rtpmap:0
>> PCMU/8000\x0d\x0aa=rtpmap:8 PCMA/8000\x0d\x0aa=rtpmap:101
>> telephone-event/8000\x0d\x0aa=ptime:20\x0d\x0aa=sendrecv\x0d\x0a"
>>
>> 1439952508.245335       CfnJdC2wJa7QObDdK7      10.10.250.141  5110
>> 10.10.146.171  5060    udp     SIP     Binpac exception: binpac
>> exception: string mismatch at
>> /nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70:
> \x0aexpected
>> pattern: ":"\x0aactual data: " 2046637637 2105833686 IN IP4
>> 10.10.250.141\x0d\x0as=sipcli\x0d\x0ac=IN IP4 10.10.250.141\x0d\x0at=0
>> 0\x0d\x0am=audio 5111 RTP/AVP 18 0 8 101\x0d\x0aa=fmtp:101
>> 0-15\x0d\x0aa=rtpmap:18 G729/8000\x0d\x0aa=rtpmap:0
>> PCMU/8000\x0d\x0aa=rtpmap:8 PCMA/8000\x0d\x0aa=rtpmap:101
>> telephone-event/8000\x0d\x0aa=ptime:20\x0d\x0aa=sendrecv\x0d\x0a"
>>
>> 1439952508.597857       C2vuSQ3duZlPtt6Njl      10.10.44.245  5060
>> 10.10.7.100    5060    udp     SIP     Binpac exception: binpac
>> exception: string mismatch at
>> /nsm/bro/git/bro2.4-10/bro/src/analyzer/protocol/sip/sip-protocol.pac:70:
> \x0aexpected
>> pattern: ":"\x0aactual data: " version='1.0'
>> encoding='UTF-8'?><!--PUA--><presence
>> xmlns='urn:ietf:params:xml:ns:pidf'
>> xmlns:dm='urn:ietf:params:xml:ns:pidf:data-model'
>> xmlns:rpid='urn:ietf:params:xml:ns:pidf:rpid'
>> xmlns:c='urn:ietf:params:xml:ns:pidf:cipid'
>> entity='sip:CIO-EX90 at EXAMPLE.COM    '><tuple
>>
> id='f71ad0ae-dc51-4be2-977d-39c9ccc2d29b'><status><basic>open</basic></status></tuple></presence>"
>> On 8/18/2015 6:26 PM, Doug Burks wrote:
>>> Hi Tommy,
>>>
>>> My guess is that this isn't strictly related to Security Onion, as we
>>> have a fairly standard build of Bro.  The reason for the
>>>
> "/build/securityonion-bro-C1BIlk/securityonion-bro-2.4/src/analyzer/protocol/sip/sip-protocol.pac"
>>> is that that's the build directory where the Ubuntu Launchpad build
>>> server builds our binaries.
>>>
>>> I would take a look at the actual traffic and see if it's valid SIP or
>>> perhaps just a scan or some other kind of traffic.
>>>
>>> On Tue, Aug 18, 2015 at 5:59 PM,  <tommydew at gmail.com> wrote:
>>>> While looking through the 'dpd.log' in '/nsm/bro/logs/current/', I
> found several log entries that reported 'Binapc exception'. Here's a sample
> with redacted IPs:
>>>> 1439934408.353389       CMUcGx4TXPPDGCIb65      xxx.xxx.xxx.xxx 40046
> xxx.xxx.xxx.xxx 5060    udp     SIP     Binpac exception: binpac exception:
> string mismatch at
> /build/securityonion-bro-C1BIlk/securityonion-bro-2.4/src/analyzer/protocol/sip/sip-protocol.pac:34:
> \x0aexpected pattern: "[[:alnum:]@[:punct:]]+"\x0aactual data: ""
>>>> It appears that the issue may be related to Security Onion, but I can
> always move this to the Bro IDS mailing list if it's specific to Bro. I'll
> try to see what could be causing the exception, but I was curious if anyone
> else had any ideas.
>>>> Thanks.
>>>>
>>>> --
>>>> Tommy
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
> Groups "security-onion" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
> an email to security-onion+unsubscribe at googlegroups.com.
>>>> To post to this group, send email to security-onion at googlegroups.com.
>>>> Visit this group at http://groups.google.com/group/security-onion.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
> "security-onion" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
> email to security-onion+unsubscribe at googlegroups.com.
>> To post to this group, send email to security-onion at googlegroups.com.
>> Visit this group at http://groups.google.com/group/security-onion.
>> For more options, visit https://groups.google.com/d/optout.


From hosom at battelle.org  Wed Aug 19 05:17:56 2015
From: hosom at battelle.org (Hosom, Stephen M)
Date: Wed, 19 Aug 2015 12:17:56 +0000
Subject: [Bro] spam mail message collector
In-Reply-To: <CAJzQz-GhCOU1eYvM4Oi1dAzxAQZ9szVoP+RtfQ8VeNMSkTR2wg@mail.gmail.com>
References: <CAJzQz-GhCOU1eYvM4Oi1dAzxAQZ9szVoP+RtfQ8VeNMSkTR2wg@mail.gmail.com>
Message-ID: <E5A2071C6FFFF640B714C224139C9B33123859C1@WP-MBX2B.milky-way.battelle.org>

You could just use file extraction. This will extract many files for multipart messages.

Try: https://github.com/hosom/bro-file-extraction

Add a file and load it that does the following hook:

hook FileExtraction::extract(f: fa_file, meta: fa_metadata) &priority=10
                {
                if ( f$source == ?SMTP? )
                                break;
                }

From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Hyun Yoo
Sent: Tuesday, August 18, 2015 5:49 PM
To: bro at bro.org
Subject: [Bro] spam mail message collector

Hello Bro. I am new to bro.
I think my task is more suitable to Bro than other NIDS.
There is a list of spammer email addresses and
I want to save the email subject and whole message of them.
(reassembled payload of tcp segments)
I tried a few events like log_smtp, tcp_contents but couldn't save the whole stream.

Can anybody guide me to the right way, please?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150819/e81365da/attachment.html 

From hckim at narusec.com  Wed Aug 19 17:21:13 2015
From: hckim at narusec.com (=?UTF-8?B?6rmA7Z2s7LKg?=)
Date: Thu, 20 Aug 2015 09:21:13 +0900
Subject: [Bro] conn.log history has letter 'Q'?
Message-ID: <CAHs13Ajr1n9fSsAqYSs=B+SoAUTEGB1f5_rkDUZT_Maehz_n2A@mail.gmail.com>

Hi Bro community
I am using bro version 2.3-316

In side a Conn.log history I have letter 'Q' in it.
I can not find any info about 'Q'
am I missing something?

1439941988.068044 C3FNvf40Sa0n7jtNTf 10.122.100.26 63394 10.122.110.8 22
tcp - 1.796387 0 0 SH T Qah 1 60 4 224 (empty) (empty) (empty)

1439942990.248722 CqADp939XKyVf7j03i 10.122.100.26 63119 10.122.103.10 22
tcp - 3.000317 0 0 S2 T Qh 1 60 4 240 (empty) (empty) (empty)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150820/bf79d416/attachment.html 

From seth at icir.org  Wed Aug 19 18:30:37 2015
From: seth at icir.org (Seth Hall)
Date: Wed, 19 Aug 2015 21:30:37 -0400
Subject: [Bro] conn.log history has letter 'Q'?
In-Reply-To: <CAHs13Ajr1n9fSsAqYSs=B+SoAUTEGB1f5_rkDUZT_Maehz_n2A@mail.gmail.com>
References: <CAHs13Ajr1n9fSsAqYSs=B+SoAUTEGB1f5_rkDUZT_Maehz_n2A@mail.gmail.com>
Message-ID: <6F23BEA2-C4AF-4885-8D56-9864DDDF02E4@icir.org>


> On Aug 19, 2015, at 8:21 PM, ??? <hckim at narusec.com> wrote:
> 
> In side a Conn.log history I have letter 'Q' in it.
> I can not find any info about 'Q' 
> am I missing something?
> 
> 1439941988.068044	C3FNvf40Sa0n7jtNTf	10.122.100.26	63394	10.122.110.8	22	tcp	-	1.796387	0	0	SH	T	Qah	1	60	4	224	(empty)	(empty)	(empty)

?Q? indicates a multi flag packet.  It should be either a syn/fin or syn/rst packet.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From jlay at slave-tothe-box.net  Wed Aug 19 19:59:43 2015
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 19 Aug 2015 20:59:43 -0600
Subject: [Bro] conn.log history has letter 'Q'?
In-Reply-To: <6F23BEA2-C4AF-4885-8D56-9864DDDF02E4@icir.org>
References: <CAHs13Ajr1n9fSsAqYSs=B+SoAUTEGB1f5_rkDUZT_Maehz_n2A@mail.gmail.com>
	<6F23BEA2-C4AF-4885-8D56-9864DDDF02E4@icir.org>
Message-ID: <1440039583.4942.1.camel@JamesiMac>

On Wed, 2015-08-19 at 21:30 -0400, Seth Hall wrote:

> > On Aug 19, 2015, at 8:21 PM, ??? <hckim at narusec.com> wrote:
> > 
> > In side a Conn.log history I have letter 'Q' in it.
> > I can not find any info about 'Q' 
> > am I missing something?
> > 
> > 1439941988.068044	C3FNvf40Sa0n7jtNTf	10.122.100.26	63394	10.122.110.8	22	tcp	-	1.796387	0	0	SH	T	Qah	1	60	4	224	(empty)	(empty)	(empty)
> 
> ?Q? indicates a multi flag packet.  It should be either a syn/fin or syn/rst packet.
> 
>   .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


That's interesting..I don't have Q at all....and I would agree that
maybe that should be documented somewhere, but I couldn't find it here:

https://www.bro.org/sphinx/scripts/base/protocols/conn/main.bro.html#type-Conn::Info

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150819/653367bb/attachment.html 

From hckim at narusec.com  Wed Aug 19 23:16:17 2015
From: hckim at narusec.com (=?UTF-8?B?6rmA7Z2s7LKg?=)
Date: Thu, 20 Aug 2015 15:16:17 +0900
Subject: [Bro] conn.log history has letter 'Q'?
In-Reply-To: <6F23BEA2-C4AF-4885-8D56-9864DDDF02E4@icir.org>
References: <CAHs13Ajr1n9fSsAqYSs=B+SoAUTEGB1f5_rkDUZT_Maehz_n2A@mail.gmail.com>
	<6F23BEA2-C4AF-4885-8D56-9864DDDF02E4@icir.org>
Message-ID: <CAHs13AjHmi2FeNKcBGKJc4ferEQWkajkSnbqeOoHh85gOYnCrA@mail.gmail.com>

Hi Seth
Thank you for fast replay



On Thu, Aug 20, 2015 at 10:30 AM, Seth Hall <seth at icir.org> wrote:

>
> > On Aug 19, 2015, at 8:21 PM, ??? <hckim at narusec.com> wrote:
> >
> > In side a Conn.log history I have letter 'Q' in it.
> > I can not find any info about 'Q'
> > am I missing something?
> >
> > 1439941988.068044     C3FNvf40Sa0n7jtNTf      10.122.100.26   63394
>  10.122.110.8    22      tcp     -       1.796387        0       0
>  SH      T       Qah     1       60      4       224     (empty) (empty)
> (empty)
>
> ?Q? indicates a multi flag packet.  It should be either a syn/fin or
> syn/rst packet.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150820/746cb8e2/attachment.html 

From earl.eiland at root9b.com  Thu Aug 20 07:40:42 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Thu, 20 Aug 2015 14:40:42 +0000
Subject: [Bro] problem setting default variable value
Message-ID: <BY1PR0401MB13823744EEB95AADAE29A937D7660@BY1PR0401MB1382.namprd04.prod.outlook.com>

I?m trying to define a record variable and getting the error ?&default value has inconsistent type?.  The line is:
whitelist_l4_protocol: set[transport_proto] &default=unknown_transport      &log;
unknown_transport is one of the values for enum transport_proto.  I?ve tried enclosing the value in quotes.  I get the same error.

I?ve also tried
whitelist_l4_protocol: set[transport_proto] &default={unknown_transport}      &log;
This throws the error ?syntax error, at or near ?{?.

Please advise!

Best Regards,

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150820/65d0cec6/attachment.html 

From seth at icir.org  Thu Aug 20 09:44:02 2015
From: seth at icir.org (Seth Hall)
Date: Thu, 20 Aug 2015 12:44:02 -0400
Subject: [Bro] conn.log history has letter 'Q'?
In-Reply-To: <1440039583.4942.1.camel@JamesiMac>
References: <CAHs13Ajr1n9fSsAqYSs=B+SoAUTEGB1f5_rkDUZT_Maehz_n2A@mail.gmail.com>
	<6F23BEA2-C4AF-4885-8D56-9864DDDF02E4@icir.org>
	<1440039583.4942.1.camel@JamesiMac>
Message-ID: <F588D765-6C4C-426B-BAB2-DA483BC17787@icir.org>


> On Aug 19, 2015, at 10:59 PM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> That's interesting..I don't have Q at all....and I would agree that maybe that should be documented somewhere, but I couldn't find it here:

To make it worse, there is also ?I? which indicates fin/rst (and possibly other flags).  James, would you mind filing a ticket about adding Q/I to the docs? (he who brings up docs files the ticket!)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From dnthayer at illinois.edu  Thu Aug 20 11:08:09 2015
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Thu, 20 Aug 2015 13:08:09 -0500
Subject: [Bro] conn.log history has letter 'Q'?
In-Reply-To: <F588D765-6C4C-426B-BAB2-DA483BC17787@icir.org>
References: <CAHs13Ajr1n9fSsAqYSs=B+SoAUTEGB1f5_rkDUZT_Maehz_n2A@mail.gmail.com>	<6F23BEA2-C4AF-4885-8D56-9864DDDF02E4@icir.org>	<1440039583.4942.1.camel@JamesiMac>
	<F588D765-6C4C-426B-BAB2-DA483BC17787@icir.org>
Message-ID: <55D61789.7000804@illinois.edu>

I already fixed this (I've had a branch for a while now
where I've been collecting small documentation fixes like this).



On 08/20/2015 11:44 AM, Seth Hall wrote:
> To make it worse, there is also ?I? which indicates fin/rst (and possibly other flags).  James, would you mind filing a ticket about adding Q/I to the docs? (he who brings up docs files the ticket!)
>
>    .Seth
>

From jlay at slave-tothe-box.net  Thu Aug 20 11:25:34 2015
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 20 Aug 2015 12:25:34 -0600
Subject: [Bro] conn.log history has letter 'Q'?
In-Reply-To: <F588D765-6C4C-426B-BAB2-DA483BC17787@icir.org>
References: <CAHs13Ajr1n9fSsAqYSs=B+SoAUTEGB1f5_rkDUZT_Maehz_n2A@mail.gmail.com>
	<6F23BEA2-C4AF-4885-8D56-9864DDDF02E4@icir.org>
	<1440039583.4942.1.camel@JamesiMac>
	<F588D765-6C4C-426B-BAB2-DA483BC17787@icir.org>
Message-ID: <7BA0F2A1-64A3-48B0-BE13-F7236511C4BA@slave-tothe-box.net>

LoL...I sure will Seth thanks.

Sent from my iPhone

> On Aug 20, 2015, at 10:44, Seth Hall <seth at icir.org> wrote:
> 
> 
>> On Aug 19, 2015, at 10:59 PM, James Lay <jlay at slave-tothe-box.net> wrote:
>> 
>> That's interesting..I don't have Q at all....and I would agree that maybe that should be documented somewhere, but I couldn't find it here:
> 
> To make it worse, there is also ?I? which indicates fin/rst (and possibly other flags).  James, would you mind filing a ticket about adding Q/I to the docs? (he who brings up docs files the ticket!)
> 
>  .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/


From jdopheid at illinois.edu  Thu Aug 20 12:00:47 2015
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Thu, 20 Aug 2015 19:00:47 +0000
Subject: [Bro] BroCon '15: Slides are posted
Message-ID: <D1FB8E0E.192AE%jdopheid@illinois.edu>

Bro Community,

The slides from this year's BroCon are linked off the Agenda. We just
received the videos from MIT's AV team and so we need a couple days to get
them posted to YouTube.

https://www.bro.org/community/brocon2015.html#agenda


Thanks for your support and enthusiasm,
Jeannette Dopheide



------

Jeannette Dopheide
Bro Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign






From seth at icir.org  Thu Aug 20 20:01:11 2015
From: seth at icir.org (Seth Hall)
Date: Thu, 20 Aug 2015 23:01:11 -0400
Subject: [Bro] conn.log history has letter 'Q'?
In-Reply-To: <55D61789.7000804@illinois.edu>
References: <CAHs13Ajr1n9fSsAqYSs=B+SoAUTEGB1f5_rkDUZT_Maehz_n2A@mail.gmail.com>
	<6F23BEA2-C4AF-4885-8D56-9864DDDF02E4@icir.org>
	<1440039583.4942.1.camel@JamesiMac>
	<F588D765-6C4C-426B-BAB2-DA483BC17787@icir.org>
	<55D61789.7000804@illinois.edu>
Message-ID: <390B47D0-E3C4-415A-B36A-123A5ECEF198@icir.org>


> On Aug 20, 2015, at 2:08 PM, Daniel Thayer <dnthayer at illinois.edu> wrote:
> 
> I already fixed this (I've had a branch for a while now
> where I've been collecting small documentation fixes like this).

Great!

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/


From hckim at narusec.com  Thu Aug 20 23:20:09 2015
From: hckim at narusec.com (=?UTF-8?B?6rmA7Z2s7LKg?=)
Date: Fri, 21 Aug 2015 15:20:09 +0900
Subject: [Bro] conn.log history has letter 'Q'?
Message-ID: <CAHs13AjFutkthq=cKRuPpYKQZ_axi9Qy1QEAnMrE3DCg9rJFTg@mail.gmail.com>

so for the history

S a SYN w/o the ACK bit set H a SYN+ACK (?handshake?) A a pure ACK D packet
with payload (?data?) F packet with FIN bit set R packet with RST bit
set C packet
with a bad checksum I inconsistent packet (e.g. SYN+RST bits both set) Q a
syn/fin or syn/rst L a  fin/rst


On Fri, Aug 21, 2015 at 4:00 AM, <bro-request at bro.org> wrote:

> Send Bro mailing list submissions to
>         bro at bro.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> or, via email, send a message with subject or body 'help' to
>         bro-request at bro.org
>
> You can reach the person managing the list at
>         bro-owner at bro.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Bro digest..."
>
>
> Today's Topics:
>
>    1. Re: conn.log history has letter 'Q'? (Daniel Thayer)
>    2. Re: conn.log history has letter 'Q'? (James Lay)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 20 Aug 2015 13:08:09 -0500
> From: Daniel Thayer <dnthayer at illinois.edu>
> Subject: Re: [Bro] conn.log history has letter 'Q'?
> To: Seth Hall <seth at icir.org>
> Cc: bro at bro.org
> Message-ID: <55D61789.7000804 at illinois.edu>
> Content-Type: text/plain; charset="utf-8"; format=flowed
>
> I already fixed this (I've had a branch for a while now
> where I've been collecting small documentation fixes like this).
>
>
>
> On 08/20/2015 11:44 AM, Seth Hall wrote:
> > To make it worse, there is also ?I? which indicates fin/rst (and
> possibly other flags).  James, would you mind filing a ticket about adding
> Q/I to the docs? (he who brings up docs files the ticket!)
> >
> >    .Seth
> >
>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 20 Aug 2015 12:25:34 -0600
> From: James Lay <jlay at slave-tothe-box.net>
> Subject: Re: [Bro] conn.log history has letter 'Q'?
> To: Seth Hall <seth at icir.org>
> Cc: "bro at bro.org" <bro at bro.org>
> Message-ID: <7BA0F2A1-64A3-48B0-BE13-F7236511C4BA at slave-tothe-box.net>
> Content-Type: text/plain;       charset=utf-8
>
> LoL...I sure will Seth thanks.
>
> Sent from my iPhone
>
> > On Aug 20, 2015, at 10:44, Seth Hall <seth at icir.org> wrote:
> >
> >
> >> On Aug 19, 2015, at 10:59 PM, James Lay <jlay at slave-tothe-box.net>
> wrote:
> >>
> >> That's interesting..I don't have Q at all....and I would agree that
> maybe that should be documented somewhere, but I couldn't find it here:
> >
> > To make it worse, there is also ?I? which indicates fin/rst (and
> possibly other flags).  James, would you mind filing a ticket about adding
> Q/I to the docs? (he who brings up docs files the ticket!)
> >
> >  .Seth
> >
> > --
> > Seth Hall
> > International Computer Science Institute
> > (Bro) because everyone has a network
> > http://www.bro.org/
>
>
>
> ------------------------------
>
> _______________________________________________
> Bro mailing list
> Bro at bro.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> End of Bro Digest, Vol 112, Issue 25
> ************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150821/27e4ce9a/attachment.html 

From seth at icir.org  Fri Aug 21 06:49:11 2015
From: seth at icir.org (Seth Hall)
Date: Fri, 21 Aug 2015 09:49:11 -0400
Subject: [Bro] conn.log history has letter 'Q'?
In-Reply-To: <CAHs13AjFutkthq=cKRuPpYKQZ_axi9Qy1QEAnMrE3DCg9rJFTg@mail.gmail.com>
References: <CAHs13AjFutkthq=cKRuPpYKQZ_axi9Qy1QEAnMrE3DCg9rJFTg@mail.gmail.com>
Message-ID: <9CDE01E0-058D-40F6-BE82-19B1F1FBBF19@icir.org>


> On Aug 21, 2015, at 2:20 AM, ??? <hckim at narusec.com> wrote:
> 
> I	inconsistent packet (e.g. SYN+RST bits both set)

I don?t actually know what ?I? stands for, but it?s for fin/rst packets, not syn/rst (although that would also be viable as long as fin is also set)

> L	a  fin/rst 

I don?t believe that ?L' is a valid flag for the history field.  Where did you find this?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From Emmanuel.TORQUATO at monext.net  Fri Aug 21 08:53:59 2015
From: Emmanuel.TORQUATO at monext.net (Emmanuel TORQUATO)
Date: Fri, 21 Aug 2015 15:53:59 +0000
Subject: [Bro] access right changed on log directory
Message-ID: <93a0f86962b042b08b5b0bc9313ab4bb@EXCB1P1.monext.net>

Hello,

I have upgraded from 2.3.2 to 2.4 and find that spool/manager directory has not the same access right than before:

In 2.4:
drwxr-x---  3 root root   4096 Aug 21 17:36 manager

in 2.3.2:
drwxr-xr-x  2 root root 4096 Aug 21 17:13 manager


It's a problem for me because I would like to view logs without being root. Is there a way to change the access right to log directory when starting bro?

Thanks
Regards,

Emmanuel.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150821/51825925/attachment-0001.html 

From dnthayer at illinois.edu  Fri Aug 21 09:24:41 2015
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Fri, 21 Aug 2015 11:24:41 -0500
Subject: [Bro] access right changed on log directory
In-Reply-To: <93a0f86962b042b08b5b0bc9313ab4bb@EXCB1P1.monext.net>
References: <93a0f86962b042b08b5b0bc9313ab4bb@EXCB1P1.monext.net>
Message-ID: <55D750C9.6070109@illinois.edu>

Broctl creates that directory, but it doesn't explicitly
set or change its permissions.  You should check the umask of the root
user on your system, and the umask setting for sudo (if you're using
sudo to run broctl).  You want a umask of 0022.


On 08/21/2015 10:53 AM, Emmanuel TORQUATO wrote:
> Hello,
>
> I have upgraded from 2.3.2 to 2.4 and find that spool/manager directory
> has not the same access right than before:
>
> In 2.4:
>
> drwxr-x---  3 root root   4096 Aug 21 17:36 manager
>
> in 2.3.2:
>
> drwxr-xr-x  2 root root 4096 Aug 21 17:13 manager
>
> It?s a problem for me because I would like to view logs without being
> root. Is there a way to change the access right to log directory when
> starting bro?
>
> Thanks
>
> Regards,
>
> Emmanuel.

From bmixonb1 at cs.unm.edu  Fri Aug 21 10:36:30 2015
From: bmixonb1 at cs.unm.edu (nhtvl)
Date: Fri, 21 Aug 2015 11:36:30 -0600
Subject: [Bro] Detecting Encryption
Message-ID: <55D7619E.1090505@cs.unm.edu>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

I am relatively new to Bro and was wondering if Bro has any way of
detecting encryption and/or plain text in the dpd module or anywhere els
e.

I have several use cases.

1. I wish to determine whether a program that has an auto-update feature
is sending the updates using encryption.

2. I wish to determine if a chat application is sending data encrypted.

I had a suggestion from my advisor that I should compress the data
being sent over the wire to see if it is compressible or not and use
that in determining whether a stream is using encryption or not.

Any suggestions or advice on this problem would be greatly appreciated.

Regards,
Ben Mixon-Baca
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJV12GeAAoJEO09Oz0uXqnKUX4IAMfFBsaEvcOMSMn/7kg4J5AH
xOvTlpmzUYXXHWHj/J+5rGf4VkHGej7I4vmIaQ1dxmCxGy/34is5m9y767f4AAuH
jazvC2ZLNOixYBq/H4sVKX7Vl5zUY8wU7ptKdbo2HxnaX4MHkbJg/bnD2c4mIhPN
3EuOIZgzdYGJIQWsIhCaZmuaiaO2JE+Kp6JlleYcbg+J7lUQd/34YU3Sv6snysGM
ON5hmbPISukkFXUAVCsIuRWYXkiAhdDPR1XHtp4pClu2EHOITcIChM9/6qsmqgr/
RXWHU5UOthJ/IgjLaNkTQ/YlBmFkTVJ9QnKCKNOQv8Uhc4+e1c4vVF7F8jrefVE=
=TgbT
-----END PGP SIGNATURE-----

From jlay at slave-tothe-box.net  Fri Aug 21 10:48:04 2015
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 21 Aug 2015 11:48:04 -0600
Subject: [Bro] Detecting Encryption
In-Reply-To: <55D7619E.1090505@cs.unm.edu>
References: <55D7619E.1090505@cs.unm.edu>
Message-ID: <c126e4f1bbe17a07b404867a43c61fd9@localhost>

On 2015-08-21 11:36 AM, nhtvl wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi all,
> 
> I am relatively new to Bro and was wondering if Bro has any way of
> detecting encryption and/or plain text in the dpd module or anywhere 
> els
> e.
> 
> I have several use cases.
> 
> 1. I wish to determine whether a program that has an auto-update 
> feature
> is sending the updates using encryption.
> 
> 2. I wish to determine if a chat application is sending data encrypted.
> 
> I had a suggestion from my advisor that I should compress the data
> being sent over the wire to see if it is compressible or not and use
> that in determining whether a stream is using encryption or not.
> 
> Any suggestions or advice on this problem would be greatly appreciated.
> 
> Regards,
> Ben Mixon-Baca
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
> 
> iQEcBAEBAgAGBQJV12GeAAoJEO09Oz0uXqnKUX4IAMfFBsaEvcOMSMn/7kg4J5AH
> xOvTlpmzUYXXHWHj/J+5rGf4VkHGej7I4vmIaQ1dxmCxGy/34is5m9y767f4AAuH
> jazvC2ZLNOixYBq/H4sVKX7Vl5zUY8wU7ptKdbo2HxnaX4MHkbJg/bnD2c4mIhPN
> 3EuOIZgzdYGJIQWsIhCaZmuaiaO2JE+Kp6JlleYcbg+J7lUQd/34YU3Sv6snysGM
> ON5hmbPISukkFXUAVCsIuRWYXkiAhdDPR1XHtp4pClu2EHOITcIChM9/6qsmqgr/
> RXWHU5UOthJ/IgjLaNkTQ/YlBmFkTVJ9QnKCKNOQv8Uhc4+e1c4vVF7F8jrefVE=
> =TgbT
> -----END PGP SIGNATURE-----
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

Check out ssl.log and x509.log.

James

From Brett.Hite at parsons.com  Fri Aug 21 11:52:24 2015
From: Brett.Hite at parsons.com (Hite, Brett)
Date: Fri, 21 Aug 2015 18:52:24 +0000
Subject: [Bro] When statement not executing?
Message-ID: <D09036F1F9F5BD488CE108D34A13438203966B0C@HSV-MB001.huntsville.ads.sparta.com>

Hi everyone,

I'm new to Bro and made it through some example problems I found but am stumped with a current problem. I'm trying to do either an HTTP request using ActiveHTTP::Request or an external command using Exec::Command. I found some example code using both and am first trying to run those to see if I can get expected output.

Example code I'm following: https://gist.github.com/hillar/825c36269c2f684a45b3

To summarize where I'm having difficulty:

function do_mhr_lookup(hash: string, fi: Notice::FileInfo)
    {
        local data = fmt("resource=%s", hash);
        local key = fmt("-d apikey=%s", vt_apikey);
        local req: ActiveHTTP::Request = ActiveHTTP::Request($url...);

        when (local res = ActiveHTTP::request(req))
            {
                print res;
            }
    }
>From what I've read, the ActiveHTTP::request can only be run in a when statement. The same is true for the Exec::run command. In the above sample code, the req variable is exactly what I want it to be (confirmed with a print), but nothing is happening after that. I've come across similar questions to mine that recommended using "redef exit_only_after_terminate = T;" or "@load frameworks/communication/listen". Both of these are not affecting the behavior of the script.

How can I ensure that the ActiveHTTP::request is run?

Thanks,

Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150821/f536c107/attachment.html 

From anthony.kasza at gmail.com  Fri Aug 21 12:26:18 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Fri, 21 Aug 2015 12:26:18 -0700
Subject: [Bro] When statement not executing?
In-Reply-To: <D09036F1F9F5BD488CE108D34A13438203966B0C@HSV-MB001.huntsville.ads.sparta.com>
References: <D09036F1F9F5BD488CE108D34A13438203966B0C@HSV-MB001.huntsville.ads.sparta.com>
Message-ID: <CAEZw2bzufrTu=E_13BtjQ37RbRb9LZaCRgWBfK+wKZ=0m28JOw@mail.gmail.com>

If I recall correctly there is a timeout attribute you can apply to when
statements. Try adding one and see if it ever returns.

-AK
On Aug 21, 2015 11:54 AM, "Hite, Brett" <Brett.Hite at parsons.com> wrote:

> Hi everyone,
>
> I'm new to Bro and made it through some example problems I found but am
> stumped with a current problem. I'm trying to do either an HTTP request
> using ActiveHTTP::Request or an external command using Exec::Command. I
> found some example code using both and am first trying to run those to see
> if I can get expected output.
>
> Example code I'm following:
> https://gist.github.com/hillar/825c36269c2f684a45b3
>
> To summarize where I'm having difficulty:
>
> function do_mhr_lookup(hash: string, fi: Notice::FileInfo)
>     {
>         local data = fmt("resource=%s", hash);
>         local key = fmt("-d apikey=%s", vt_apikey);
>         local req: ActiveHTTP::Request = ActiveHTTP::Request($url...);
>
>         when (local res = ActiveHTTP::request(req))
>             {
>                 print res;
>             }
>     }
>
> From what I've read, the ActiveHTTP::request can only be run in a when
> statement. The same is true for the Exec::run command. In the above sample
> code, the req variable is exactly what I want it to be (confirmed with a
> print), but nothing is happening after that. I've come across similar
> questions to mine that recommended using "redef exit_only_after_terminate =
> T;" or "@load frameworks/communication/listen". Both of these are not
> affecting the behavior of the script.
>
> How can I ensure that the ActiveHTTP::request is run?
>
> Thanks,
>
> Brett
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150821/9dfd2e3d/attachment.html 

From basvermeulen80 at yahoo.com  Fri Aug 21 12:31:00 2015
From: basvermeulen80 at yahoo.com (Bas Vermeulen)
Date: Fri, 21 Aug 2015 19:31:00 +0000 (UTC)
Subject: [Bro] Plugin doesn't seem te get instantiated
Message-ID: <1560293342.8768798.1440185460561.JavaMail.yahoo@mail.yahoo.com>

Hi all, 

I want to create my own bro plugin but I'm stuck in the playing-around phase. Below is my current code and information about my system. I know packet counts are available in the normal logs, this is just my hello world for bro. The problem is that while bro seems to recognize that there is a plugin, it doesn't seem to instantiate the analyzer when is is processing a pcap. I've tried to activate it using the environment variables, the Available function and the EnableHook. I need to process all connections so I can't use port numbers or signatures. 

The only output the plugin creates is 'hello world!' from the plugin.cc If the Analyzer gets instantiated, I would expect more output.

Could someone please help me?

Bas

Plugin.cc:
----------
#include "plugin/Plugin.h"
#include "plugin/Manager.h"

#include "PluginAnalyzer.h"
namespace plugin {
namespace mynamespace_myplugin {

class Plugin : public plugin::Plugin {
public:
?? ?plugin::Configuration Configure()
?? ??? ?{
?? ??? ??? ?AddComponent(new ::analyzer::Component("PluginAnalyzer", ::analyzer::mynamespace_myplugin::PluginAnalyzer::Instantiate));

?? ??? ??? ?plugin::Configuration config;
?? ??? ??? ?config.name = "mynamespace::myplugin";
?? ??? ??? ?config.description = "Test_plugin";
?? ??? ??? ?config.version.major = 0;
?? ??? ??? ?config.version.minor = 2;
?? ??? ??? ?cout << "hello world!\n";

//?? ??? ??? ?Attempt to enable the plugin, this doesn't seem to
//?? ??? ??? ?do anything
?? ??? ??? ?EnableHook(HOOK_SETUP_ANALYZER_TREE, 1);

?? ??? ??? ?return config;?? ?;
?? ??? ?}
} plugin;

}
}

PluginAnalyzer.h
-----------------
#ifndef PLUGINPROTOCOL_H
#define PLUGINPROTOCOL_H

//#include "analyzer/Analyzer.h"
#include "analyzer/protocol/tcp/TCP.h"

namespace analyzer { namespace mynamespace_myplugin {

//class PluginAnalyzer : public analyzer::Analyzer {
class PluginAnalyzer :? public tcp::TCP_ApplicationAnalyzer {
public:
?? ?PluginAnalyzer(Connection* c);
?? ?virtual ~PluginAnalyzer();

?? ?virtual void Init();
?? ?virtual void Done();

?? ?// from Analyzer.h
?? ?virtual void UpdateConnVal(RecordVal *conn_val);
?? ?virtual void FlipRoles();
?? ?static bool Available()
?? ?{
?? ??? ?cout << "availability checked\n";
?? ??? ?return true;
?? ?}

?? ?static analyzer::Analyzer* Instantiate(Connection* conn)
?? ??? ?{ cout << "instantiate\n"; return new PluginAnalyzer(conn); }

?? ?virtual void DeliverStream(int len, const u_char* data, bool orig);
?? ?protected:
?? ?uint64_t total_packets;
};

} } // namespace analyzer::* 

#endif

PluginAnalyer.cc
----------------
#include "PluginAnalyzer.h"
#include "analyzer/protocol/tcp/TCP.h"

using namespace analyzer::mynamespace_myplugin;

PluginAnalyzer::PluginAnalyzer(Connection* c)
: tcp::TCP_ApplicationAnalyzer("MyPluginAnalyzer", c)
?? ?{
?? ??? ?cout << "pluginanalyzer constructor\n " ;
?? ?}


PluginAnalyzer::~PluginAnalyzer()
?? ?{
?? ?}

void PluginAnalyzer::Init()
?? ?{
?? ?cout << "init \n";
?? ?Analyzer::Init();

?? ?total_packets = 0;
?? ?}

void PluginAnalyzer::Done()
?? ?{
?? ?Analyzer::Done();
?? ?}

void PluginAnalyzer::DeliverStream(int length, const u_char* data, bool orig)
?? ?{
?? ?tcp::TCP_ApplicationAnalyzer::DeliverStream(length, data, orig);

?? ?cout << "deliverStream \n";
?? ?total_packets++;
}


void PluginAnalyzer::UpdateConnVal(RecordVal *conn_val)
?? ?{
?? ?cout << "UpdateConnVal begin\n";
?? ?int totalidx = conn_val->Type()->AsRecordType()->FieldOffset("total_packets");
?? ?if ( totalidx < 0 ) 
?? ??? ?reporter->InternalError("missing total packets field");

?? ?conn_val->Assign(totalidx, new Val(total_packets, TYPE_COUNT));

?? ?Analyzer::UpdateConnVal(conn_val);
?? ?cout << "UpdateConnVal end\n";

?? ?}


void PluginAnalyzer::FlipRoles()
?? ?{
?? ?}

This is what I have done...

$ make
< no error messages >
$ sudo make install 
< no error messages >

$ export BRO_PLUGIN_PATH=~/plugin
$ export BRO_PLUGIN_ACTIVATE=mynamespace::myplugin

$ bro -N
hello world!
mynamespace::myplugin - Test_plugin (dynamic, version 0.2)
Bro::ARP - ARP Parsing (built-in)
Bro::AsciiReader - ASCII input reader (built-in)
.....


$ rm *.log
$ bro -C -r test.pcap 
hello world!
$ ls *.log
conn.log? packet_filter.log? ssh.log

This is info about my system and installation...

$ bro -v
bro version 2.4-84

$ uname -srvpio
Linux 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 GNU/Linux

When I installed from source I used:
./configure --disable-broker
make
sudo make install

The plugin was originally create with the init-plugin tool
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150821/bb750776/attachment.html 

From robin at icir.org  Fri Aug 21 12:50:15 2015
From: robin at icir.org (Robin Sommer)
Date: Fri, 21 Aug 2015 12:50:15 -0700
Subject: [Bro] Detecting Encryption
In-Reply-To: <55D7619E.1090505@cs.unm.edu>
References: <55D7619E.1090505@cs.unm.edu>
Message-ID: <20150821195015.GM48017@icir.org>



On Fri, Aug 21, 2015 at 11:36 -0600, nhtvl wrote:

> I had a suggestion from my advisor that I should compress the data
> being sent over the wire to see if it is compressible or not and use
> that in determining whether a stream is using encryption or not.

Bro has functions to measure entropy, see
https://www.bro.org/sphinx-git/scripts/base/bif/bro.bif.bro.html#id-find_entropy.

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From bmixonb1 at cs.unm.edu  Fri Aug 21 12:52:11 2015
From: bmixonb1 at cs.unm.edu (nhtvl)
Date: Fri, 21 Aug 2015 13:52:11 -0600
Subject: [Bro] Detecting Encryption
In-Reply-To: <20150821195015.GM48017@icir.org>
References: <55D7619E.1090505@cs.unm.edu> <20150821195015.GM48017@icir.org>
Message-ID: <55D7816B.6000506@cs.unm.edu>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

That is amazing! Would I need to make additions in a script of mine in
order to differentiate between encryption and compression?

On 08/21/2015 01:50 PM, Robin Sommer wrote:
> 
> 
> On Fri, Aug 21, 2015 at 11:36 -0600, nhtvl wrote:
> 
>> I had a suggestion from my advisor that I should compress the
>> data being sent over the wire to see if it is compressible or not
>> and use that in determining whether a stream is using encryption
>> or not.
> 
> Bro has functions to measure entropy, see 
> https://www.bro.org/sphinx-git/scripts/base/bif/bro.bif.bro.html#id-fi
nd_entropy.
>
>  Robin
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJV14FrAAoJEO09Oz0uXqnKo8EH/04UIxvj/hiyKJjbdxj1VJX1
crc8fhQ5WT9jSdIUzxRNRsN4JvwXPLPqAUBJP+kW3dCl58HX/2vk6EP1HTOB2LYS
2jNU3FVj8vgol/tBUKOdAsCYJrlh2c8kJlEWYVDU8F3wRWlaPAMPhVOY3FEuaWIZ
kYo6Gvdugywdj29hEOumRPNFDgPQqcDjmxG0lAoDKMO7mkO99F0aGmojgkXzjWJh
H12fNkImywS7/ZHIDB8zLD34tM1gWk8CB0M1fXQQIMcxOdsfahCt1hMMlBFr5x6g
4qsNtxlTTtO139a+oJMD7H+0Q7YgRv1q3LmEXU+iRiPjZb35PwfmZb5MYnbtsOo=
=u9Ii
-----END PGP SIGNATURE-----

From basvermeulen80 at yahoo.com  Sun Aug 23 08:40:01 2015
From: basvermeulen80 at yahoo.com (Bas Vermeulen)
Date: Sun, 23 Aug 2015 15:40:01 +0000 (UTC)
Subject: [Bro] Plugin doesn't seem te get instantiated
In-Reply-To: <1560293342.8768798.1440185460561.JavaMail.yahoo@mail.yahoo.com>
References: <1560293342.8768798.1440185460561.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <1369111374.216164.1440344401991.JavaMail.yahoo@mail.yahoo.com>

Hi,
In order to find the reason that my plugin isn't doing anything, I have recompiled with --enable-debug and I run with bro -B plugins. The debug.log now contains
???????? 0.000000/1440343663.376984 [plugins] Found plugin mynamespace::myplugin in /usr/local/bro/lib/bro/plugins/mynamespace_myplugin
???????? 0.000000/1440343663.383816 [plugins] Activating plugin mynamespace::myplugin
???????? 0.000000/1440343663.383855 [plugins]?? Adding /usr/local/bro/lib/bro/plugins/mynamespace_myplugin/scripts to BROPATH
???????? 0.000000/1440343663.383892 [plugins]?? Loading /usr/local/bro/lib/bro/plugins/mynamespace_myplugin/scripts/__preload__.bro
???????? 0.000000/1440343663.383908 [plugins]?? Loading /usr/local/bro/lib/bro/plugins/mynamespace_myplugin/lib/bif/__load__.bro
???????? 0.000000/1440343663.383921 [plugins]?? Loading /usr/local/bro/lib/bro/plugins/mynamespace_myplugin/scripts/__load__.bro
???????? 0.000000/1440343663.383932 [plugins]?? Searching for shared libraries /usr/local/bro/lib/bro/plugins/mynamespace_myplugin//lib/*.linux-x86_64.so
???????? 0.000000/1440343663.384400 [plugins] Registering component PluginAnalyzer (tag 68/0)
???????? 0.000000/1440343663.384527 [plugins]?? Loaded /usr/local/bro/lib/bro/plugins/mynamespace_myplugin//lib/mynamespace-myplugin.linux-x86_64.so
I also added the Available() function to? Plugin.h.

Is this all that is required? The plugin still doesn't do anything... Any hints? Or does anyone have an example non-built-in plugin that work on all connections regardless of ports and signatures?
Best regards,Bas

 


     On Friday, August 21, 2015 9:31 PM, Bas Vermeulen <basvermeulen80 at yahoo.com> wrote:
   

 Hi all, 

I want to create my own bro plugin but I'm stuck in the playing-around phase. Below is my current code and information about my system. I know packet counts are available in the normal logs, this is just my hello world for bro. The problem is that while bro seems to recognize that there is a plugin, it doesn't seem to instantiate the analyzer when is is processing a pcap. I've tried to activate it using the environment variables, the Available function and the EnableHook. I need to process all connections so I can't use port numbers or signatures. 

The only output the plugin creates is 'hello world!' from the plugin.cc If the Analyzer gets instantiated, I would expect more output.

Could someone please help me?

Bas

Plugin.cc:
----------
#include "plugin/Plugin.h"
#include "plugin/Manager.h"

#include "PluginAnalyzer.h"
namespace plugin {
namespace mynamespace_myplugin {

class Plugin : public plugin::Plugin {
public:
?? ?plugin::Configuration Configure()
?? ??? ?{
?? ??? ??? ?AddComponent(new ::analyzer::Component("PluginAnalyzer", ::analyzer::mynamespace_myplugin::PluginAnalyzer::Instantiate));

?? ??? ??? ?plugin::Configuration config;
?? ??? ??? ?config.name = "mynamespace::myplugin";
?? ??? ??? ?config.description = "Test_plugin";
?? ??? ??? ?config.version.major = 0;
?? ??? ??? ?config.version.minor = 2;
?? ??? ??? ?cout << "hello world!\n";

//?? ??? ??? ?Attempt to enable the plugin, this doesn't seem to
//?? ??? ??? ?do anything
?? ??? ??? ?EnableHook(HOOK_SETUP_ANALYZER_TREE, 1);

?? ??? ??? ?return config;?? ?;
?? ??? ?}
} plugin;

}
}

PluginAnalyzer.h
-----------------
#ifndef PLUGINPROTOCOL_H
#define PLUGINPROTOCOL_H

//#include "analyzer/Analyzer.h"
#include "analyzer/protocol/tcp/TCP.h"

namespace analyzer { namespace mynamespace_myplugin {

//class PluginAnalyzer : public analyzer::Analyzer {
class PluginAnalyzer :? public tcp::TCP_ApplicationAnalyzer {
public:
?? ?PluginAnalyzer(Connection* c);
?? ?virtual ~PluginAnalyzer();

?? ?virtual void Init();
?? ?virtual void Done();

?? ?// from Analyzer.h
?? ?virtual void UpdateConnVal(RecordVal *conn_val);
?? ?virtual void FlipRoles();
?? ?static bool Available()
?? ?{
?? ??? ?cout << "availability checked\n";
?? ??? ?return true;
?? ?}

?? ?static analyzer::Analyzer* Instantiate(Connection* conn)
?? ??? ?{ cout << "instantiate\n"; return new PluginAnalyzer(conn); }

?? ?virtual void DeliverStream(int len, const u_char* data, bool orig);
?? ?protected:
?? ?uint64_t total_packets;
};

} } // namespace analyzer::* 

#endif

PluginAnalyer.cc
----------------
#include "PluginAnalyzer.h"
#include "analyzer/protocol/tcp/TCP.h"

using namespace analyzer::mynamespace_myplugin;

PluginAnalyzer::PluginAnalyzer(Connection* c)
: tcp::TCP_ApplicationAnalyzer("MyPluginAnalyzer", c)
?? ?{
?? ??? ?cout << "pluginanalyzer constructor\n " ;
?? ?}


PluginAnalyzer::~PluginAnalyzer()
?? ?{
?? ?}

void PluginAnalyzer::Init()
?? ?{
?? ?cout << "init \n";
?? ?Analyzer::Init();

?? ?total_packets = 0;
?? ?}

void PluginAnalyzer::Done()
?? ?{
?? ?Analyzer::Done();
?? ?}

void PluginAnalyzer::DeliverStream(int length, const u_char* data, bool orig)
?? ?{
?? ?tcp::TCP_ApplicationAnalyzer::DeliverStream(length, data, orig);

?? ?cout << "deliverStream \n";
?? ?total_packets++;
}


void PluginAnalyzer::UpdateConnVal(RecordVal *conn_val)
?? ?{
?? ?cout << "UpdateConnVal begin\n";
?? ?int totalidx = conn_val->Type()->AsRecordType()->FieldOffset("total_packets");
?? ?if ( totalidx < 0 ) 
?? ??? ?reporter->InternalError("missing total packets field");

?? ?conn_val->Assign(totalidx, new Val(total_packets, TYPE_COUNT));

?? ?Analyzer::UpdateConnVal(conn_val);
?? ?cout << "UpdateConnVal end\n";

?? ?}


void PluginAnalyzer::FlipRoles()
?? ?{
?? ?}

This is what I have done...

$ make
< no error messages >
$ sudo make install 
< no error messages >

$ export BRO_PLUGIN_PATH=~/plugin
$ export BRO_PLUGIN_ACTIVATE=mynamespace::myplugin

$ bro -N
hello world!
mynamespace::myplugin - Test_plugin (dynamic, version 0.2)
Bro::ARP - ARP Parsing (built-in)
Bro::AsciiReader - ASCII input reader (built-in)
.....


$ rm *.log
$ bro -C -r test.pcap 
hello world!
$ ls *.log
conn.log? packet_filter.log? ssh.log

This is info about my system and installation...

$ bro -v
bro version 2.4-84

$ uname -srvpio
Linux 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 GNU/Linux

When I installed from source I used:
./configure --disable-broker
make
sudo make install

The plugin was originally create with the init-plugin tool


  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150823/b791d22f/attachment.html 

From cosmotraumatika at gmail.com  Sun Aug 23 12:46:07 2015
From: cosmotraumatika at gmail.com (Jamie Saker)
Date: Sun, 23 Aug 2015 14:46:07 -0500
Subject: [Bro] Worker in cluster with different processor architecture
Message-ID: <A465EAF7-79A9-447D-BE73-C3535C412D18@gmail.com>

I?ve got a new cluster being run with a manager on Intel hardware but the worker  on Raspberry Pi 2.  The Raspberry Pi was previously working in standalone mode successfully with Bro 2.4 compiled and installed. When I moved it into the manager/worker configuration, all went well with the installation except for the inevitable error due to the manager pushing an Intel executable over to the worker which it can?t recognize/run:

/usr/local/bro/bin/bro: 3: /usr/local/bro/bin/bro: Syntax error: Unterminated quoted string
/usr/local/bro/bin/bro: 1: /usr/local/bro/bin/bro: ELF: not found

I?ve scanned the online documentation and googled but haven?t yet found a workaround; I?d imagine there might be a way to have the master not overwrite the binaries and allow the native Raspberry Pi compiled binaries run? Or another approach I?m missing? Processor info follows the .sig. Thanks much.  

Jamie Saker
cosmotraumatika at gmail.com
BroCon 2015 Attendee (Yea!)


[Manager] (VMWare)
 
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 6
model name	: QEMU Virtual CPU version 2.0.0

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 6
model name	: QEMU Virtual CPU version 2.0.0

[Worker-1] 

processor	: 0
model name	: ARMv7 Processor rev 5 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtst
rm 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xc07
CPU revision	: 5

processor	: 1
model name	: ARMv7 Processor rev 5 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtst
rm 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xc07
CPU revision	: 5

processor	: 2
model name	: ARMv7 Processor rev 5 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtst
rm 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xc07
CPU revision	: 5

processor	: 3
model name	: ARMv7 Processor rev 5 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtst
rm 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xc07
CPU revision	: 5

Hardware	: BCM2709
Revision	: a21041
Serial		: 000000002c3a22ee

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150823/5fb6e8e5/attachment.html 

From cgaylord at vt.edu  Mon Aug 24 05:48:56 2015
From: cgaylord at vt.edu (Clark Gaylord)
Date: Mon, 24 Aug 2015 08:48:56 -0400
Subject: [Bro] Standalone vs cluster
Message-ID: <CADzU5g4OY+Jh=gyLAc=k3S5FC-cawY3seGfwpCPSm7Q7WQagqg@mail.gmail.com>

This appears to have been discussed in 2009, so I thought I might re-ask to
see if anything has changed, and to add a follow on question/clarification.
I don't see any further discussion from searching the archives.

If using a single box to run bro, is there any advantage to running cluster
mode (all localhost) rather than standalone?

The previous answer was: no reason to do so, with additional clarification
that a) if you're thinking of eventually migrating to cluster mode, getting
the configuration correct will be the least of your trouble and b) unless
you want to take advantage of multiple cores.

The latter point is why I am posing the question again: on a 12-core box,
for example, how does one (and should one) take advantage of these cores.
The last I have seen is a) bro is single threaded and b) the rule of thumb
is 80Mbps/core. If this is so, then am I at risk of dropping data on the
floor if I don't specifically have more workers?

Say I can expect to see 500 Mbps peak, with occasional sustained load of
say 300 Mbps.

To accommodate this traffic load, should six workers be defined all on
localhost? Or does a single localhost worker (the default in standalone,
right?) already utilize the cores to achieve the desired performance?

Thanks for your suggestions
Clark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150824/ec1453e0/attachment.html 

From mike.patterson at uwaterloo.ca  Mon Aug 24 06:29:21 2015
From: mike.patterson at uwaterloo.ca (Mike Patterson)
Date: Mon, 24 Aug 2015 13:29:21 +0000
Subject: [Bro] Standalone vs cluster
In-Reply-To: <CADzU5g4OY+Jh=gyLAc=k3S5FC-cawY3seGfwpCPSm7Q7WQagqg@mail.gmail.com>
References: <CADzU5g4OY+Jh=gyLAc=k3S5FC-cawY3seGfwpCPSm7Q7WQagqg@mail.gmail.com>
Message-ID: <98617C86-3265-4CA6-B84B-E9B192D2D0B5@uwaterloo.ca>

You're going to want to run it as a cluster, even if it's all on one box.
80Mbps/core seems low nowadays, although it depends on your CPUs. We're easily handling loads[0] in the 3-4Gbps range on 16 workers, 4 proxies, and a manager (all on the same 20 core box). My CPUs are  E5-2687W v3 @ 3.10GHz. Pin your processes and you should be ok. But yes, if the load is too much, then you'll drop traffic. Enable the capture loss script and graph its output to get an idea.

[0] asterisk: two workers drop more traffic than the other 14 due to CPUs at 100%, load follows the workers, gave up trying to figure that one out for now, those drop 5-10% - I'm assuming it's some prolonged traffic and/or some weird hashing on my network card, an Endace DAG 9.2X2.

Mike

-- 
My grandfather on why he has no computer in his house: "it's just a
passing fad." I'm feeling less and less of an urge to beg to differ
with him.  - Omri Schwarz

> On Aug 24, 2015, at 8:48 AM, Clark Gaylord <cgaylord at vt.edu> wrote:
> 
> This appears to have been discussed in 2009, so I thought I might re-ask to see if anything has changed, and to add a follow on question/clarification. I don't see any further discussion from searching the archives.
> 
> If using a single box to run bro, is there any advantage to running cluster mode (all localhost) rather than standalone?
> 
> The previous answer was: no reason to do so, with additional clarification that a) if you're thinking of eventually migrating to cluster mode, getting the configuration correct will be the least of your trouble and b) unless you want to take advantage of multiple cores.
> 
> The latter point is why I am posing the question again: on a 12-core box, for example, how does one (and should one) take advantage of these cores. The last I have seen is a) bro is single threaded and b) the rule of thumb is 80Mbps/core. If this is so, then am I at risk of dropping data on the floor if I don't specifically have more workers?
> 
> Say I can expect to see 500 Mbps peak, with occasional sustained load of say 300 Mbps.
> 
> To accommodate this traffic load, should six workers be defined all on localhost? Or does a single localhost worker (the default in standalone, right?) already utilize the cores to achieve the desired performance?
> 
> Thanks for your suggestions
> Clark
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From luke at geekempire.com  Mon Aug 24 06:34:58 2015
From: luke at geekempire.com (Mike Reeves)
Date: Mon, 24 Aug 2015 09:34:58 -0400
Subject: [Bro] Standalone vs cluster
In-Reply-To: <CADzU5g4OY+Jh=gyLAc=k3S5FC-cawY3seGfwpCPSm7Q7WQagqg@mail.gmail.com>
References: <CADzU5g4OY+Jh=gyLAc=k3S5FC-cawY3seGfwpCPSm7Q7WQagqg@mail.gmail.com>
Message-ID: <CADC84wRQOLBBNxj+O8Cr4OFR1MRds_JAYx_N5VW0jr2FGBs1qA@mail.gmail.com>

I run all of my single boxes as clusters. This is how you get it to scale
locally. That way you can take full advantage of all the cores on the box.
The amount of workers really depends on the traffic and types of traffic.
start with 6 and see how it does.

Thanks

Mike

On Mon, Aug 24, 2015 at 8:48 AM, Clark Gaylord <cgaylord at vt.edu> wrote:

> This appears to have been discussed in 2009, so I thought I might re-ask
> to see if anything has changed, and to add a follow on
> question/clarification. I don't see any further discussion from searching
> the archives.
>
> If using a single box to run bro, is there any advantage to running
> cluster mode (all localhost) rather than standalone?
>
> The previous answer was: no reason to do so, with additional clarification
> that a) if you're thinking of eventually migrating to cluster mode, getting
> the configuration correct will be the least of your trouble and b) unless
> you want to take advantage of multiple cores.
>
> The latter point is why I am posing the question again: on a 12-core box,
> for example, how does one (and should one) take advantage of these cores.
> The last I have seen is a) bro is single threaded and b) the rule of thumb
> is 80Mbps/core. If this is so, then am I at risk of dropping data on the
> floor if I don't specifically have more workers?
>
> Say I can expect to see 500 Mbps peak, with occasional sustained load of
> say 300 Mbps.
>
> To accommodate this traffic load, should six workers be defined all on
> localhost? Or does a single localhost worker (the default in standalone,
> right?) already utilize the cores to achieve the desired performance?
>
> Thanks for your suggestions
> Clark
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150824/67436279/attachment-0001.html 

From seth at icir.org  Mon Aug 24 06:49:06 2015
From: seth at icir.org (Seth Hall)
Date: Mon, 24 Aug 2015 09:49:06 -0400
Subject: [Bro] Worker in cluster with different processor architecture
In-Reply-To: <A465EAF7-79A9-447D-BE73-C3535C412D18@gmail.com>
References: <A465EAF7-79A9-447D-BE73-C3535C412D18@gmail.com>
Message-ID: <8680AB4E-DE2B-4670-A6BE-FF15EB125CEA@icir.org>


> On Aug 23, 2015, at 3:46 PM, Jamie Saker <cosmotraumatika at gmail.com> wrote:
> 
> I?ve got a new cluster being run with a manager on Intel hardware but the worker  on Raspberry Pi 2.  The Raspberry Pi was previously working in standalone mode successfully with Bro 2.4 compiled and installed. When I moved it into the manager/worker configuration, all went well with the installation except for the inevitable error due to the manager pushing an Intel executable over to the worker which it can?t recognize/run:

Unfortunately there isn?t a work around right now that wouldn?t be really hacky (perhaps someone else can provide that extreme hackiness?).  This is a problem we?re aware of though and the way that BroControl works is in the middle of being redesigned right now and this scenario will be better accommodated under the new model.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Mon Aug 24 06:56:59 2015
From: seth at icir.org (Seth Hall)
Date: Mon, 24 Aug 2015 09:56:59 -0400
Subject: [Bro] Standalone vs cluster
In-Reply-To: <CADzU5g4OY+Jh=gyLAc=k3S5FC-cawY3seGfwpCPSm7Q7WQagqg@mail.gmail.com>
References: <CADzU5g4OY+Jh=gyLAc=k3S5FC-cawY3seGfwpCPSm7Q7WQagqg@mail.gmail.com>
Message-ID: <C96DF60F-FB07-4BC1-B774-6E7AADDAC827@icir.org>


> On Aug 24, 2015, at 8:48 AM, Clark Gaylord <cgaylord at vt.edu> wrote:
> 
> If using a single box to run bro, is there any advantage to running cluster mode (all localhost) rather than standalone?

Simple answer here, you almost never want to run standalone.

> The previous answer was: no reason to do so, with additional clarification that a) if you're thinking of eventually migrating to cluster mode, getting the configuration correct will be the least of your trouble and b) unless you want to take advantage of multiple cores.
> 
> The latter point is why I am posing the question again: on a 12-core box, for example, how does one (and should one) take advantage of these cores. The last I have seen is a) bro is single threaded and b) the rule of thumb is 80Mbps/core. If this is so, then am I at risk of dropping data on the floor if I don't specifically have more workers?

That rule of thumb was actually created for this box: 
	http://www.amazon.com/Dell-Computer-Professional-Extremely-Operation/dp/B002Q6ZTZM

I don?t recommend using those anymore (or ever), but the first production Bro cluster was running on a big stack of those because I got them for free. :)   That documentation needs to be updated at some point, but generally these days with modern hardware people will see ~200-250Mbps per core although it?s possible to make it run faster.

> To accommodate this traffic load, should six workers be defined all on localhost? Or does a single localhost worker (the default in standalone, right?) already utilize the cores to achieve the desired performance?

Did you read the load balancing documentation?
	https://www.bro.org/documentation/load-balancing.html

It?s a bit out of date, and unfortunately only includes directions for load balancing with pf_ring, but it should give you a first direction.  I?ll see if I can update that with a second mechanism soon too.  We?re working on adding another mechanism to the on-host load balancing options as well which we think should be really flexible and nice.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Mon Aug 24 06:59:25 2015
From: seth at icir.org (Seth Hall)
Date: Mon, 24 Aug 2015 09:59:25 -0400
Subject: [Bro] Detecting Encryption
In-Reply-To: <20150821195015.GM48017@icir.org>
References: <55D7619E.1090505@cs.unm.edu> <20150821195015.GM48017@icir.org>
Message-ID: <8A178547-9A25-4260-96EE-69405BFA2A08@icir.org>


> On Aug 21, 2015, at 3:50 PM, Robin Sommer <robin at icir.org> wrote:
> 
> Bro has functions to measure entropy, see
> https://www.bro.org/sphinx-git/scripts/base/bif/bro.bif.bro.html#id-find_entropy.

Unfortunately we still haven?t added file and connection entropy analyzers yet.  I have a file entropy analyzer floating around somewhere, but generally both of those are extremely easy to write.  I think that Ben would need those to do what he?s trying to do.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From jan.grashofer at cern.ch  Mon Aug 24 07:23:26 2015
From: jan.grashofer at cern.ch (Jan Grashofer)
Date: Mon, 24 Aug 2015 14:23:26 +0000
Subject: [Bro] NTP
Message-ID: <E3142ED5D6744B4C8E17323B862FEDE5DFBDB5@CERNXCHG62.cern.ch>

Hi,

I would like to use the NTP analyzer and documentation says:

"Bro?s current default configuration does not activate the protocol analyzer that generates this event; the corresponding script has not yet been ported to Bro 2.x. To still enable this event, one needs to register a port for it or add a DPD payload signature." (https://www.bro.org/sphinx/script-reference/proto-analyzers.html#bro-ntp)

I think a DPD signature would be preferable compared to registering a port. Before I start digging into that I thought I might ask here, whether someone has already done this.

Best regards,
Jan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150824/edd32143/attachment.html 

From bmixonb1 at cs.unm.edu  Mon Aug 24 07:27:56 2015
From: bmixonb1 at cs.unm.edu (nhtvl)
Date: Mon, 24 Aug 2015 08:27:56 -0600
Subject: [Bro] Detecting Encryption
In-Reply-To: <8A178547-9A25-4260-96EE-69405BFA2A08@icir.org>
References: <55D7619E.1090505@cs.unm.edu> <20150821195015.GM48017@icir.org>
	<8A178547-9A25-4260-96EE-69405BFA2A08@icir.org>
Message-ID: <55DB29EC.6060901@cs.unm.edu>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Seth, was the link provided not a file entropy analyzer?

On 08/24/2015 07:59 AM, Seth Hall wrote:
> 
>> On Aug 21, 2015, at 3:50 PM, Robin Sommer <robin at icir.org> 
>> wrote:
>> 
>> Bro has functions to measure entropy, see 
>> https://www.bro.org/sphinx-git/scripts/base/bif/bro.bif.bro.html#id-f
ind_entropy.
>
>>
>>
>> 
> Unfortunately we still haven?t added file and connection entropy 
> analyzers yet.  I have a file entropy analyzer floating around 
> somewhere, but generally both of those are extremely easy to write.
> I think that Ben would need those to do what he?s trying to do.
> 
> .Seth
> 
> -- Seth Hall International Computer Science Institute (Bro) because
> everyone has a network http://www.bro.org/
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJV2ynsAAoJEO09Oz0uXqnK1K8H/RmbTLvghesVPVoPzjdmk5Ou
kGz1jGRA0rePMngbtAxMxSpnHi/S5FcMvpz8hyhvT5le23655hqpjtU98facdc6r
r+cNIp9nf9K3wWU5ToxaAXE+CCl+jc9+APwAwGDUgsF2f0QnVCejvXYxN77mBaS5
jAlWQLxV2hRdYzusb2WkaSOE2NtHnHzrdP4xzQrLiQDialhaUQBOvhH2537RUkSF
m1RLzuYY7Dd7ufpm6ERxTzoCuYoe6AxgSzYkOCaOl3kXf9nezfImXrSoaYyxfrlO
X9eRhsKDQRbjR8by25U5NRF3h9wZbFt7KIvCxhTJnG1MugncQyFomOWvtH3LA34=
=FaYJ
-----END PGP SIGNATURE-----

From seth at icir.org  Mon Aug 24 07:50:12 2015
From: seth at icir.org (Seth Hall)
Date: Mon, 24 Aug 2015 10:50:12 -0400
Subject: [Bro] Detecting Encryption
In-Reply-To: <55DB29EC.6060901@cs.unm.edu>
References: <55D7619E.1090505@cs.unm.edu> <20150821195015.GM48017@icir.org>
	<8A178547-9A25-4260-96EE-69405BFA2A08@icir.org>
	<55DB29EC.6060901@cs.unm.edu>
Message-ID: <9ABCE9A5-C7BC-4A12-9DEF-7931996C3F5E@icir.org>


> On Aug 24, 2015, at 10:27 AM, nhtvl <bmixonb1 at cs.unm.edu> wrote:
> 
> Seth, was the link provided not a file entropy analyzer?

No, those are built-in-functions (BiFs) to calculate entropy on strings that are available in Bro?s scripting language.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150824/7eab66b8/attachment.bin 

From bmixonb1 at cs.unm.edu  Mon Aug 24 07:51:15 2015
From: bmixonb1 at cs.unm.edu (nhtvl)
Date: Mon, 24 Aug 2015 08:51:15 -0600
Subject: [Bro] Detecting Encryption
In-Reply-To: <9ABCE9A5-C7BC-4A12-9DEF-7931996C3F5E@icir.org>
References: <55D7619E.1090505@cs.unm.edu> <20150821195015.GM48017@icir.org>
	<8A178547-9A25-4260-96EE-69405BFA2A08@icir.org>
	<55DB29EC.6060901@cs.unm.edu>
	<9ABCE9A5-C7BC-4A12-9DEF-7931996C3F5E@icir.org>
Message-ID: <55DB2F63.1030002@cs.unm.edu>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

OK thanks. So I would write my own broscripts to do connection and
file entropy analysis then right?

On 08/24/2015 08:50 AM, Seth Hall wrote:
> 
>> On Aug 24, 2015, at 10:27 AM, nhtvl <bmixonb1 at cs.unm.edu> wrote:
>> 
>> Seth, was the link provided not a file entropy analyzer?
> 
> No, those are built-in-functions (BiFs) to calculate entropy on
> strings that are available in Bro?s scripting language.
> 
> .Seth
> 
> -- Seth Hall International Computer Science Institute (Bro) because
> everyone has a network http://www.bro.org/
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJV2y9jAAoJEO09Oz0uXqnKvzUIAKrP0Ow3F1BLjuZUyxBsubf2
MGLz0PcHoRtOK0RFZhYW57n5tvX9IbVQyGcRq63bXejiIRwKB0giZWb95kNZY8HP
sU9nhRfhnEiS+3ahXHYOLmbQLW9k0oh8aAMIy5ocoxjm8p5y1Zl+560bQjv4k4T3
CzIzz9VjrWJyqjNW+tI+m9XBoMKzjVMHjHJf1H2UMVBo+mAFlcP3SHpg2YsNfpaD
9agWx53DY29UIWyhuEVblVjv6UMYa/fGO0TCArYGlzvi9YdcLyYnCAFuCQJARDum
6jwd/2coOqPO2oxlOxJkeXNw+Jszq8xuSae0KwKMmNoe4tu1q0q6CH4jpKZXXZU=
=TXfm
-----END PGP SIGNATURE-----

From seth at icir.org  Mon Aug 24 07:56:19 2015
From: seth at icir.org (Seth Hall)
Date: Mon, 24 Aug 2015 10:56:19 -0400
Subject: [Bro] Detecting Encryption
In-Reply-To: <55DB2F63.1030002@cs.unm.edu>
References: <55D7619E.1090505@cs.unm.edu> <20150821195015.GM48017@icir.org>
	<8A178547-9A25-4260-96EE-69405BFA2A08@icir.org>
	<55DB29EC.6060901@cs.unm.edu>
	<9ABCE9A5-C7BC-4A12-9DEF-7931996C3F5E@icir.org>
	<55DB2F63.1030002@cs.unm.edu>
Message-ID: <A547501F-44E1-485C-9A3A-5E2AC765283C@icir.org>


> On Aug 24, 2015, at 10:51 AM, nhtvl <bmixonb1 at cs.unm.edu> wrote:
> 
> OK thanks. So I would write my own broscripts to do connection and
> file entropy analysis then right?

These wouldn?t be written as scripts.  Connection and file analyzers needs to be written as plugins or in the core.  They are typically implemented in C++ or BinPAC.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From dnthayer at illinois.edu  Mon Aug 24 07:55:16 2015
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Mon, 24 Aug 2015 09:55:16 -0500
Subject: [Bro] Worker in cluster with different processor architecture
In-Reply-To: <A465EAF7-79A9-447D-BE73-C3535C412D18@gmail.com>
References: <A465EAF7-79A9-447D-BE73-C3535C412D18@gmail.com>
Message-ID: <55DB3054.1020500@illinois.edu>

If you apply the attached patch (untested, but I think it might work),
and then do a fresh install of Bro on all machines in your cluster (it
must be installed on the same path on all machines), then when you do
a "broctl install" or "broctl deploy", it shouldn't copy your binaries
from the manager to the workers.  If you later want to install a Bro
or broctl plugin, you must do so separately on all machines in your
cluster.




On 08/23/2015 02:46 PM, Jamie Saker wrote:
> I?ve got a new cluster being run with a manager on Intel hardware but
> the worker  on Raspberry Pi 2.  The Raspberry Pi was previously working
> in standalone mode successfully with Bro 2.4 compiled and installed.
> When I moved it into the manager/worker configuration, all went well
> with the installation except for the inevitable error due to the manager
> pushing an Intel executable over to the worker which it can?t recognize/run:
>
> /usr/local/bro/bin/bro: 3: /usr/local/bro/bin/bro: Syntax error:
> Unterminated quoted string
> /usr/local/bro/bin/bro: 1: /usr/local/bro/bin/bro: ELF: not found
>
> I?ve scanned the online documentation and googled but haven?t yet found
> a workaround; I?d imagine there might be a way to have the master not
> overwrite the binaries and allow the native Raspberry Pi compiled
> binaries run? Or another approach I?m missing? Processor info follows
> the .sig. Thanks much.
>
> Jamie Saker
> cosmotraumatika at gmail.com <mailto:cosmotraumatika at gmail.com>
> BroCon 2015 Attendee (Yea!)
>
>
> _[Manager] (VMWare)_
> processor: 0
> vendor_id: GenuineIntel
> cpu family: 6
> model: 6
> model name: QEMU Virtual CPU version 2.0.0
>
> processor: 1
> vendor_id: GenuineIntel
> cpu family: 6
> model: 6
> model name: QEMU Virtual CPU version 2.0.0
>
> _[Worker-1] _
>
> processor: 0
> model name: ARMv7 Processor rev 5 (v7l)
> BogoMIPS: 38.40
> Features: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
> vfpd32 lpae evtst
> rm
> CPU implementer: 0x41
> CPU architecture: 7
> CPU variant: 0x0
> CPU part: 0xc07
> CPU revision: 5
>
> processor: 1
> model name: ARMv7 Processor rev 5 (v7l)
> BogoMIPS: 38.40
> Features: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
> vfpd32 lpae evtst
> rm
> CPU implementer: 0x41
> CPU architecture: 7
> CPU variant: 0x0
> CPU part: 0xc07
> CPU revision: 5
>
> processor: 2
> model name: ARMv7 Processor rev 5 (v7l)
> BogoMIPS: 38.40
> Features: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
> vfpd32 lpae evtst
> rm
> CPU implementer: 0x41
> CPU architecture: 7
> CPU variant: 0x0
> CPU part: 0xc07
> CPU revision: 5
>
> processor: 3
> model name: ARMv7 Processor rev 5 (v7l)
> BogoMIPS: 38.40
> Features: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
> vfpd32 lpae evtst
> rm
> CPU implementer: 0x41
> CPU architecture: 7
> CPU variant: 0x0
> CPU part: 0xc07
> CPU revision: 5
>
> Hardware: BCM2709
> Revision: a21041
> Serial: 000000002c3a22ee
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: broctl.diff
Type: text/x-patch
Size: 602 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150824/2f18c52a/attachment-0001.bin 

From Emmanuel.TORQUATO at monext.net  Mon Aug 24 08:02:59 2015
From: Emmanuel.TORQUATO at monext.net (Emmanuel TORQUATO)
Date: Mon, 24 Aug 2015 15:02:59 +0000
Subject: [Bro] access right changed on log directory
In-Reply-To: <55D750C9.6070109@illinois.edu>
References: <93a0f86962b042b08b5b0bc9313ab4bb@EXCB1P1.monext.net>
	<55D750C9.6070109@illinois.edu>
Message-ID: <2919e335a6c14a1eba7440b4e5db948f@EXCB1P1.monext.net>

Thanks Daniel,

I have used setgid to set read on the low privilege's group on the spool folder. Any files or folders created under spool by bro are now accessible by this group. 

-----Message d'origine-----
De?: Daniel Thayer [mailto:dnthayer at illinois.edu] 
Envoy??: vendredi 21 ao?t 2015 18:25
??: Emmanuel TORQUATO; bro at bro.org
Objet?: Re: [Bro] access right changed on log directory

Broctl creates that directory, but it doesn't explicitly set or change its permissions.  You should check the umask of the root user on your system, and the umask setting for sudo (if you're using sudo to run broctl).  You want a umask of 0022.


On 08/21/2015 10:53 AM, Emmanuel TORQUATO wrote:
> Hello,
>
> I have upgraded from 2.3.2 to 2.4 and find that spool/manager 
> directory has not the same access right than before:
>
> In 2.4:
>
> drwxr-x---  3 root root   4096 Aug 21 17:36 manager
>
> in 2.3.2:
>
> drwxr-xr-x  2 root root 4096 Aug 21 17:13 manager
>
> It's a problem for me because I would like to view logs without being 
> root. Is there a way to change the access right to log directory when 
> starting bro?
>
> Thanks
>
> Regards,
>
> Emmanuel.


From basvermeulen80 at yahoo.com  Mon Aug 24 08:31:16 2015
From: basvermeulen80 at yahoo.com (Bas Vermeulen)
Date: Mon, 24 Aug 2015 08:31:16 -0700
Subject: [Bro] Detecting Encryption
Message-ID: <1440430276.88932.BPMail_high_carrier@web181104.mail.ne1.yahoo.com>


Hi,

That touches my problem... Is it possible to have such a c++ analyzer plugin that looks at all connections? Or is a signature or port required for dynamic plugins?

Bas



------------------------------
On Mon, Aug 24, 2015 7:56 AM PDT Seth Hall wrote:

>
> On Aug 24, 2015, at 10:51 AM, nhtvl <bmixonb1 at cs.unm.edu> wrote:
> 
> OK thanks. So I would write my own broscripts to do connection and
> file entropy analysis then right?
>
>These wouldn?t be written as scripts.  Connection and file analyzers needs to be written as plugins or in the core.  They are typically implemented in C++ or BinPAC.
>
>  .Seth
>
>--
>Seth Hall
>International Computer Science Institute
>(Bro) because everyone has a network
>http://www.bro.org/
>
>
>_______________________________________________
>Bro mailing list
>bro at bro-ids.org
>http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From robin at icir.org  Mon Aug 24 08:57:54 2015
From: robin at icir.org (Robin Sommer)
Date: Mon, 24 Aug 2015 08:57:54 -0700
Subject: [Bro] Plugin doesn't seem te get instantiated
In-Reply-To: <1369111374.216164.1440344401991.JavaMail.yahoo@mail.yahoo.com>
References: <1560293342.8768798.1440185460561.JavaMail.yahoo@mail.yahoo.com>
	<1369111374.216164.1440344401991.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <20150824155754.GF71711@icir.org>



On Sun, Aug 23, 2015 at 15:40 +0000, you wrote:

> Is this all that is required? The plugin still doesn't do anything...
> Any hints? Or does anyone have an example non-built-in plugin that
> work on all connections regardless of ports and signatures?

Sounds like you're suspecting the right thing already: the analyzer
needs to be explicitly activated for all connections it's supposed to
look at, it's not going to receive everything automatically. There
there 4 ways to activate an analyzer: (1) by port, (2) by signature,
(3) in script-land for a future connection, and (4) with recent git
master you can write a C++ hook function that gets called once at the
beginning of each connection when the default analyzer setup has been
determined; that C++ function can then add the customer analyzer at
that point as well.

The interface for (1)-(3) is documented here
https://www.bro.org/sphinx/scripts/base/frameworks/analyzer/main.bro.html.

For (4), this is the merge commit (not further documented yet).

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From aaronhaycraft1 at gmail.com  Mon Aug 24 09:44:10 2015
From: aaronhaycraft1 at gmail.com (Aaron Haycraft)
Date: Mon, 24 Aug 2015 12:44:10 -0400
Subject: [Bro] Log file issues
Message-ID: <CAHi0i3Henbhoq=qW520ue1O+3i7dPNCch6=9=wsOfaAGEAq90g@mail.gmail.com>

Good afternoon everyone,

My name is Aaron Haycraft. I am attempting to work with Bro for a project
and I am having some
issues. I am running Bro from the command line on a Fedora machine, and I
want to run a lot of PCAP files through it. For example, the lines of
code I run are "bro -r test.pcap" and "bro -r test2.pcap", and so on.
However, when I do so, the logs seem to overwrite after awhile and I lose a
lot of data. I know that when Bro runs, it stores the logs in the local
directory, but after the log gets full, it goes somewhere that I don't
know. Is there anyway to get around this, such that everything goes into
one big log file?

Thank you for your time,

Aaron H.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150824/ff918ff4/attachment.html 

From robin at icir.org  Mon Aug 24 10:44:21 2015
From: robin at icir.org (Robin Sommer)
Date: Mon, 24 Aug 2015 10:44:21 -0700
Subject: [Bro] Log file issues
In-Reply-To: <CAHi0i3Henbhoq=qW520ue1O+3i7dPNCch6=9=wsOfaAGEAq90g@mail.gmail.com>
References: <CAHi0i3Henbhoq=qW520ue1O+3i7dPNCch6=9=wsOfaAGEAq90g@mail.gmail.com>
Message-ID: <20150824174421.GA3571@icir.org>



On Mon, Aug 24, 2015 at 12:44 -0400, Aaron Haycraft wrote:

> want to run a lot of PCAP files through it. For example, the lines of
> code I run are "bro -r test.pcap" and "bro -r test2.pcap", and so on.
> However, when I do so, the logs seem to overwrite after awhile and I lose a
> lot of data.

When you run Bro from the command line, everything in the trace you
give to an invocation will end up in a single set of logs files inside
the current directory. But if you then restart Bro with a different
trace, these logs will be overwritten with new ones (i.e., Bro won't
append the new data). You'll either need move them away before you
start the new Bro, or you could concanate all your traces into one
pcap stream on stdin and have Bro read from "-" (tcpslice and mergecap
can both do that).

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From basvermeulen80 at yahoo.com  Mon Aug 24 13:15:50 2015
From: basvermeulen80 at yahoo.com (Bas Vermeulen)
Date: Mon, 24 Aug 2015 20:15:50 +0000 (UTC)
Subject: [Bro] Plugin doesn't seem te get instantiated
In-Reply-To: <20150824155754.GF71711@icir.org>
References: <20150824155754.GF71711@icir.org>
Message-ID: <930421014.9347.1440447350778.JavaMail.yahoo@mail.yahoo.com>

Thanks! I still have a lot of work to do, but now at least my UpdateConnVal is being called, so there is progress :)
You made my day!
For other people that want to create a dynamic plugin that is invoked for all connections (option 4), this is the progress so far:
In my Plugin.cc I added:??? void HookSetupAnalyzerTree(Connection *conn)
??? {
??? ???? ??? ::analyzer::mynamespace_myplugin::PluginAnalyzer::Instantiate(conn);
??? }
And in my PluginAnalyzer.cc I now have:PluginAnalyzer::PluginAnalyzer(Connection* c)
: tcp::TCP_ApplicationAnalyzer("TEST", c)
??? {
??? ??? TCP()->AddChildAnalyzer(this); // this line took me quite a while ;)
??? }


 


     On Monday, August 24, 2015 5:57 PM, Robin Sommer <robin at icir.org> wrote:
   

 

On Sun, Aug 23, 2015 at 15:40 +0000, you wrote:

> Is this all that is required? The plugin still doesn't do anything...
> Any hints? Or does anyone have an example non-built-in plugin that
> work on all connections regardless of ports and signatures?

Sounds like you're suspecting the right thing already: the analyzer
needs to be explicitly activated for all connections it's supposed to
look at, it's not going to receive everything automatically. There
there 4 ways to activate an analyzer: (1) by port, (2) by signature,
(3) in script-land for a future connection, and (4) with recent git
master you can write a C++ hook function that gets called once at the
beginning of each connection when the default analyzer setup has been
determined; that C++ function can then add the customer analyzer at
that point as well.

The interface for (1)-(3) is documented here
https://www.bro.org/sphinx/scripts/base/frameworks/analyzer/main.bro.html.

For (4), this is the merge commit (not further documented yet).

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin


  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150824/eefdbec0/attachment.html 

From laaziz.lahlou at etu.parisdescartes.fr  Mon Aug 24 19:05:08 2015
From: laaziz.lahlou at etu.parisdescartes.fr (Laaziz Lahlou)
Date: Tue, 25 Aug 2015 02:05:08 +0000
Subject: [Bro] Deploying Bro Cluster using Docker container technology
Message-ID: <DBXPR01MB079AFE5BC253E002ACA3852AB610@DBXPR01MB079.eurprd01.prod.exchangelabs.com>

Hi guys,


I'm trying to deploy Bro Cluster using Docker container technology for my master's research project on Network Function Virtualization.


The objective is to use pf_send and replay a pcap file obtained from http://download.netresec.com/pcap/smia-2011/SMIA_2011-10-12_07%253A41%253A40_CEST_606532000_file2.pcap.<http://download.netresec.com/pcap/smia-2011/SMIA_2011-10-12_07%253A41%253A40_CEST_606532000_file2.pcap>

I configured PF_RING and created 5 containers as workers.


I guess here I'm violating what is cited in : https://www.bro.org/sphinx/cluster/index.html

The PF_RING software for Linux has a "clustering" feature which will do flow-based load balancing across a number of processes that are sniffing the same interface". What I mean here is each container has it's own interface and the workers are not listening on the same interface,so am I right or should I deploy the whole Bro Cluster just on one container ? I will appreciate any comment and guidance.


Best regards.

Aziz

MSc S?curit?, R?seaux et e-Sant?
Universit? Paris Descartes


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150825/ae4b9ded/attachment.html 

From michalpurzynski1 at gmail.com  Mon Aug 24 23:35:49 2015
From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=)
Date: Mon, 24 Aug 2015 23:35:49 -0700
Subject: [Bro] Bro and scan detection - the new script. And performance.
Message-ID: <CAJ6bFK27bik4sCzHO9PN4N1jtXvtdbbiP329SEsv4tX7wVOZ-w@mail.gmail.com>

If memory serves me right, there was an old scanning detection script
and now there is a new one in ./scripts/policy/misc/scan.bro

The old one was discouraged on large clusters, is the new one better?

TL;DR to my surprise I have like 60Gbit of traffic here (OK, spikes
;-), millions of connections, insane amount of logs and I'm wondering
if I could enable it. It's not like I want to sacrifice lots of
performance, though. At best it will tell me that people are scanning
us 24/7, something that's quite obvious, but it would be a nice thing
to correlate and trace the attacker what he's doing, what other
services was he looking for before he started hammering some innocent
HTTP site and so on.

What do you think?

From hckim at narusec.com  Tue Aug 25 02:57:47 2015
From: hckim at narusec.com (=?UTF-8?B?6rmA7Z2s7LKg?=)
Date: Tue, 25 Aug 2015 18:57:47 +0900
Subject: [Bro] conn.log history has letter 'Q'?
Message-ID: <CAHs13AjrAMOZgDxeJhBitV_h=hDqnqTrp5SGxjes8gpP3vJ3Gg@mail.gmail.com>

>> I     inconsistent packet (e.g. SYN+RST bits both set)

>I don?t actually know what ?I? stands for, but it?s for fin/rst packets,
not syn/rst (although that would also be viable as long as fin is also set)

I got 'I' from bro document
https://www.bro.org/sphinx/scripts/base/protocols/conn/main.bro.html



>> L     a  fin/rst

>I don?t believe that ?L' is a valid flag for the history field.  Where did
you find this?

Sorry I got mix up with capital 'I' and lower case 'L'


On Fri, Aug 21, 2015 at 10:49 PM, Seth Hall <seth at icir.org> wrote:

>
> > On Aug 21, 2015, at 2:20 AM, ??? <hckim at narusec.com> wrote:
> >
> > I     inconsistent packet (e.g. SYN+RST bits both set)
>
> I don?t actually know what ?I? stands for, but it?s for fin/rst packets,
> not syn/rst (although that would also be viable as long as fin is also set)
>
> > L     a  fin/rst
>
> I don?t believe that ?L' is a valid flag for the history field.  Where did
> you find this?
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150825/ebd76ff1/attachment.html 

From seth at icir.org  Tue Aug 25 07:01:27 2015
From: seth at icir.org (Seth Hall)
Date: Tue, 25 Aug 2015 10:01:27 -0400
Subject: [Bro] Bro and scan detection - the new script. And performance.
In-Reply-To: <CAJ6bFK27bik4sCzHO9PN4N1jtXvtdbbiP329SEsv4tX7wVOZ-w@mail.gmail.com>
References: <CAJ6bFK27bik4sCzHO9PN4N1jtXvtdbbiP329SEsv4tX7wVOZ-w@mail.gmail.com>
Message-ID: <2C77E6E7-2967-4E24-A414-901743771358@icir.org>


> On Aug 25, 2015, at 2:35 AM, Micha? Purzy?ski <michalpurzynski1 at gmail.com> wrote:
> 
> The old one was discouraged on large clusters, is the new one better?

Yes, vastly.

> TL;DR to my surprise I have like 60Gbit of traffic here (OK, spikes
> ;-), millions of connections, insane amount of logs and I'm wondering
> if I could enable it.

It should work just fine.  We spent several years figuring out how to do it and extended SumStats a lot with that aim, but I haven?t yet heard of a network where it doesn?t work (although I?m sure I will now!).

A bit more information about why it works...
It was built on top of SumStats and was even one of the driving motivations for SumStats which gives us cluster transparency.  There are a couple of reasons that SumStats works in general, even in crazy cases like scan detection.  It uses lazy synchronization to wait until the end of an epoch (i.e. some time interval) to collect data from all of the nodes seeing traffic (workers).  This means that the cluster is not synchronizing everything all the time.  It?s only synchronizing results and defined intervals which greatly reduces the communication overhead.  Additionally, all of the measurements that it does are composable (we can merge results from many different systems) which enables us to cope with traffic being split across processes and even machines like all of you are doing on clusters but then bring that data back and creates the final composed result which can then be checked against thresholds or you can have your own code run on the result.

Give it a try.  I think it?ll surprise you. :)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From vitologrillo at gmail.com  Tue Aug 25 08:40:49 2015
From: vitologrillo at gmail.com (Vito Logrillo)
Date: Tue, 25 Aug 2015 17:40:49 +0200
Subject: [Bro] Broadcast detection
Message-ID: <CANdYiFGo6U4mDvM7K2kXoQLMkowTCa0e+5wO1CPz11oAf7ijEg@mail.gmail.com>

Hi all,
i want to flag if a given ip is an ip broadcast/multicast or not:
there are some built-in functions able to recognize an ip broadcast in
Bro?
Thanks,
Vito

From anthony.kasza at gmail.com  Tue Aug 25 10:17:19 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Tue, 25 Aug 2015 10:17:19 -0700
Subject: [Bro] Broadcast detection
In-Reply-To: <CANdYiFGo6U4mDvM7K2kXoQLMkowTCa0e+5wO1CPz11oAf7ijEg@mail.gmail.com>
References: <CANdYiFGo6U4mDvM7K2kXoQLMkowTCa0e+5wO1CPz11oAf7ijEg@mail.gmail.com>
Message-ID: <CAEZw2bwCwAQRQWiQQ57h6j7NtWRur0ZHGmt0EJjupqsnsqusGQ@mail.gmail.com>

This sounds like something the Hosts module in scriptland does. If not, you
could define subnets of multicast/broadcast addresses in a script and check
on new_connection if id.orig_h or id.resp_h is in those subnets.

-AK
On Aug 25, 2015 8:51 AM, "Vito Logrillo" <vitologrillo at gmail.com> wrote:

> Hi all,
> i want to flag if a given ip is an ip broadcast/multicast or not:
> there are some built-in functions able to recognize an ip broadcast in
> Bro?
> Thanks,
> Vito
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150825/57b99f97/attachment.html 

From jswan at sugf.com  Tue Aug 25 10:39:58 2015
From: jswan at sugf.com (Swan, Jay)
Date: Tue, 25 Aug 2015 17:39:58 +0000
Subject: [Bro] Broadcast detection
In-Reply-To: <CANdYiFGo6U4mDvM7K2kXoQLMkowTCa0e+5wO1CPz11oAf7ijEg@mail.gmail.com>
References: <CANdYiFGo6U4mDvM7K2kXoQLMkowTCa0e+5wO1CPz11oAf7ijEg@mail.gmail.com>
Message-ID: <20150825174016.C5AE22C4063@rock.ICSI.Berkeley.EDU>

One way would be to check the packet destination against the IP multicast range:

global mcast = 224.0.0.0/4;
global bcast = 255.255.255.255;
event new_packet(c:connection,p:pkt_hdr) {
    if (c$id$resp_h in mcast || c$id$resp_h == bcast)
        print "mutlicast or broadcast found";
}

You wouldn't want to use the new_packet event of course.

-----Original Message-----
From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Vito Logrillo
Sent: Tuesday, August 25, 2015 9:41 AM
To: bro at bro.org
Subject: [Bro] Broadcast detection

Hi all,
i want to flag if a given ip is an ip broadcast/multicast or not:
there are some built-in functions able to recognize an ip broadcast in Bro?
Thanks,
Vito
_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From easetheworld at gmail.com  Tue Aug 25 14:59:03 2015
From: easetheworld at gmail.com (Hyun Yoo)
Date: Wed, 26 Aug 2015 06:59:03 +0900
Subject: [Bro] spam mail message collector
Message-ID: <CAJzQz-Frx_MqfD_e6vaN=6saEvjoE8aroknGjvHhfpuoFKf+Eg@mail.gmail.com>

(I added mailing list addresss to recipient.)
I found 'set_contents_file() in connection_established event' does what I
want.

But it doesn't work for uni-direction packets. Any option for this?

And if I use -b option (bare mode) for performance, connection_established
is called much less. I thought only difference in bare mode was using less
protocol parser..
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150826/23c62ae8/attachment.html 

From hhoffman at ip-solutions.net  Tue Aug 25 19:16:40 2015
From: hhoffman at ip-solutions.net (Harry Hoffman)
Date: Tue, 25 Aug 2015 22:16:40 -0400
Subject: [Bro] Bro and scan detection - the new script. And performance.
In-Reply-To: <2C77E6E7-2967-4E24-A414-901743771358@icir.org>
References: <CAJ6bFK27bik4sCzHO9PN4N1jtXvtdbbiP329SEsv4tX7wVOZ-w@mail.gmail.com>
	<2C77E6E7-2967-4E24-A414-901743771358@icir.org>
Message-ID: <55DD2188.8060402@ip-solutions.net>

Are you looking for examples where SumStats doesn't work from the latest
pull of Bro or a specific version? It's been my experience that it
depends heavily on the amount of networks configured in local nets.

Cheers,
Harry


On 8/25/15 10:01 AM, Seth Hall wrote:
>> On Aug 25, 2015, at 2:35 AM, Micha? Purzy?ski <michalpurzynski1 at gmail.com> wrote:
>>
>> The old one was discouraged on large clusters, is the new one better?
> Yes, vastly.
>
>> TL;DR to my surprise I have like 60Gbit of traffic here (OK, spikes
>> ;-), millions of connections, insane amount of logs and I'm wondering
>> if I could enable it.
> It should work just fine.  We spent several years figuring out how to do it and extended SumStats a lot with that aim, but I haven?t yet heard of a network where it doesn?t work (although I?m sure I will now!).
>
> A bit more information about why it works...
> It was built on top of SumStats and was even one of the driving motivations for SumStats which gives us cluster transparency.  There are a couple of reasons that SumStats works in general, even in crazy cases like scan detection.  It uses lazy synchronization to wait until the end of an epoch (i.e. some time interval) to collect data from all of the nodes seeing traffic (workers).  This means that the cluster is not synchronizing everything all the time.  It?s only synchronizing results and defined intervals which greatly reduces the communication overhead.  Additionally, all of the measurements that it does are composable (we can merge results from many different systems) which enables us to cope with traffic being split across processes and even machines like all of you are doing on clusters but then bring that data back and creates the final composed result which can then be checked against thresholds or you can have your own code run on the result.
>
> Give it a try.  I think it?ll surprise you. :)
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro




From seth at icir.org  Tue Aug 25 19:52:49 2015
From: seth at icir.org (Seth Hall)
Date: Tue, 25 Aug 2015 22:52:49 -0400
Subject: [Bro] Bro and scan detection - the new script. And performance.
In-Reply-To: <55DD2188.8060402@ip-solutions.net>
References: <CAJ6bFK27bik4sCzHO9PN4N1jtXvtdbbiP329SEsv4tX7wVOZ-w@mail.gmail.com>
	<2C77E6E7-2967-4E24-A414-901743771358@icir.org>
	<55DD2188.8060402@ip-solutions.net>
Message-ID: <AD07962F-BF3A-4DE4-B98D-D286B35B8FDD@icir.org>


> On Aug 25, 2015, at 10:16 PM, Harry Hoffman <hhoffman at ip-solutions.net> wrote:
> 
> Are you looking for examples where SumStats doesn't work from the latest
> pull of Bro or a specific version? It's been my experience that it
> depends heavily on the amount of networks configured in local nets.

Ah!  I knew it.  I?d be curious to learn conditions when and why (if we can figure it out) SumStats isn?t working.  SumStats also hasn?t seen massive changes in a while so particular versions shouldn?t make much of a difference.

I?m really curious to figure out why more local nets would cause the issue.  I?m mentally running through all of the cases where sumstats is used to think of places where more data is collected for local hosts.  I?m at a loss right now. :/

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From vitologrillo at gmail.com  Wed Aug 26 06:02:36 2015
From: vitologrillo at gmail.com (Vito Logrillo)
Date: Wed, 26 Aug 2015 15:02:36 +0200
Subject: [Bro] Broadcast detection
In-Reply-To: <20150825174016.C5AE22C4063@rock.ICSI.Berkeley.EDU>
References: <CANdYiFGo6U4mDvM7K2kXoQLMkowTCa0e+5wO1CPz11oAf7ijEg@mail.gmail.com>
	<20150825174016.C5AE22C4063@rock.ICSI.Berkeley.EDU>
Message-ID: <CANdYiFF29NN+m_LQ1TSWLG+2A_74ZkP9Dp9E=v6Es1zHQzi_mg@mail.gmail.com>

Hi,
thanks for your reply.
What i'm trying to do is to create a flag if an ip broadcast is found.
For example, in networks.cfg i've written this subnet
172.20.1.0/24
It's broadcast address is 172.20.1.255.
I can read all subnets written in networks.cfg with the variable
Site::local_nets_table: to calculate the ip broadcast i can use this
method
https://en.wikipedia.org/wiki/Broadcast_address
What i'm not able to do is to transform a subnet variable (in this
case 172.20.1.0/24) into an ip variable (172.20.1.0) plus a count
(24).
Any suggestion?
Thanks
Vito

2015-08-25 19:39 GMT+02:00 Swan, Jay <jswan at sugf.com>:
> One way would be to check the packet destination against the IP multicast range:
>
> global mcast = 224.0.0.0/4;
> global bcast = 255.255.255.255;
> event new_packet(c:connection,p:pkt_hdr) {
>     if (c$id$resp_h in mcast || c$id$resp_h == bcast)
>         print "mutlicast or broadcast found";
> }
>
> You wouldn't want to use the new_packet event of course.
>
> -----Original Message-----
> From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Vito Logrillo
> Sent: Tuesday, August 25, 2015 9:41 AM
> To: bro at bro.org
> Subject: [Bro] Broadcast detection
>
> Hi all,
> i want to flag if a given ip is an ip broadcast/multicast or not:
> there are some built-in functions able to recognize an ip broadcast in Bro?
> Thanks,
> Vito
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

From brett.hite at parsons.com  Wed Aug 26 06:42:01 2015
From: brett.hite at parsons.com (Brett Hite)
Date: Wed, 26 Aug 2015 09:42:01 -0400
Subject: [Bro]  When statement not executing?
In-Reply-To: D09036F1F9F5BD488CE108D34A13438203966B0C@HSV-MB001.huntsville.ads.sparta.com
Message-ID: <1440596521.30122.2.camel@Barn>

The timeout attribute wasn't working for me, but thank you. I'll keep
that in mind for future projects. I was able to get it working by doing
a few things:

* Use the "redef exit_only_after_terminate = T;"
* create an output log to display what I was expecting


From anthony.kasza at gmail.com  Wed Aug 26 07:17:56 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Wed, 26 Aug 2015 07:17:56 -0700
Subject: [Bro] Broadcast detection
In-Reply-To: <CANdYiFF29NN+m_LQ1TSWLG+2A_74ZkP9Dp9E=v6Es1zHQzi_mg@mail.gmail.com>
References: <CANdYiFGo6U4mDvM7K2kXoQLMkowTCa0e+5wO1CPz11oAf7ijEg@mail.gmail.com>
	<20150825174016.C5AE22C4063@rock.ICSI.Berkeley.EDU>
	<CANdYiFF29NN+m_LQ1TSWLG+2A_74ZkP9Dp9E=v6Es1zHQzi_mg@mail.gmail.com>
Message-ID: <CAEZw2bwfOHuR8f1a5HnbBZNGxwNebwFFczBeRof8asHTTStz8w@mail.gmail.com>

Oh I see what you're saying. What you'd like is a function that takes a
subnet as input and returns the broadcast address, correct?

-AK
On Aug 26, 2015 6:11 AM, "Vito Logrillo" <vitologrillo at gmail.com> wrote:

> Hi,
> thanks for your reply.
> What i'm trying to do is to create a flag if an ip broadcast is found.
> For example, in networks.cfg i've written this subnet
> 172.20.1.0/24
> It's broadcast address is 172.20.1.255.
> I can read all subnets written in networks.cfg with the variable
> Site::local_nets_table: to calculate the ip broadcast i can use this
> method
> https://en.wikipedia.org/wiki/Broadcast_address
> What i'm not able to do is to transform a subnet variable (in this
> case 172.20.1.0/24) into an ip variable (172.20.1.0) plus a count
> (24).
> Any suggestion?
> Thanks
> Vito
>
> 2015-08-25 19:39 GMT+02:00 Swan, Jay <jswan at sugf.com>:
> > One way would be to check the packet destination against the IP
> multicast range:
> >
> > global mcast = 224.0.0.0/4;
> > global bcast = 255.255.255.255;
> > event new_packet(c:connection,p:pkt_hdr) {
> >     if (c$id$resp_h in mcast || c$id$resp_h == bcast)
> >         print "mutlicast or broadcast found";
> > }
> >
> > You wouldn't want to use the new_packet event of course.
> >
> > -----Original Message-----
> > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of
> Vito Logrillo
> > Sent: Tuesday, August 25, 2015 9:41 AM
> > To: bro at bro.org
> > Subject: [Bro] Broadcast detection
> >
> > Hi all,
> > i want to flag if a given ip is an ip broadcast/multicast or not:
> > there are some built-in functions able to recognize an ip broadcast in
> Bro?
> > Thanks,
> > Vito
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150826/b0baf3c9/attachment.html 

From vitologrillo at gmail.com  Wed Aug 26 10:48:03 2015
From: vitologrillo at gmail.com (Vito Logrillo)
Date: Wed, 26 Aug 2015 19:48:03 +0200
Subject: [Bro] Broadcast detection
In-Reply-To: <CAEZw2bwfOHuR8f1a5HnbBZNGxwNebwFFczBeRof8asHTTStz8w@mail.gmail.com>
References: <CANdYiFGo6U4mDvM7K2kXoQLMkowTCa0e+5wO1CPz11oAf7ijEg@mail.gmail.com>
	<20150825174016.C5AE22C4063@rock.ICSI.Berkeley.EDU>
	<CANdYiFF29NN+m_LQ1TSWLG+2A_74ZkP9Dp9E=v6Es1zHQzi_mg@mail.gmail.com>
	<CAEZw2bwfOHuR8f1a5HnbBZNGxwNebwFFczBeRof8asHTTStz8w@mail.gmail.com>
Message-ID: <CANdYiFGjaBJKkA2x+9y+-nFGfsqg6oiwtAV5GzS5Mp_7YzmS6A@mail.gmail.com>

Correct!
Any suggestion?

2015-08-26 16:17 GMT+02:00 anthony kasza <anthony.kasza at gmail.com>:
> Oh I see what you're saying. What you'd like is a function that takes a
> subnet as input and returns the broadcast address, correct?
>
> -AK
>
> On Aug 26, 2015 6:11 AM, "Vito Logrillo" <vitologrillo at gmail.com> wrote:
>>
>> Hi,
>> thanks for your reply.
>> What i'm trying to do is to create a flag if an ip broadcast is found.
>> For example, in networks.cfg i've written this subnet
>> 172.20.1.0/24
>> It's broadcast address is 172.20.1.255.
>> I can read all subnets written in networks.cfg with the variable
>> Site::local_nets_table: to calculate the ip broadcast i can use this
>> method
>> https://en.wikipedia.org/wiki/Broadcast_address
>> What i'm not able to do is to transform a subnet variable (in this
>> case 172.20.1.0/24) into an ip variable (172.20.1.0) plus a count
>> (24).
>> Any suggestion?
>> Thanks
>> Vito
>>
>> 2015-08-25 19:39 GMT+02:00 Swan, Jay <jswan at sugf.com>:
>> > One way would be to check the packet destination against the IP
>> > multicast range:
>> >
>> > global mcast = 224.0.0.0/4;
>> > global bcast = 255.255.255.255;
>> > event new_packet(c:connection,p:pkt_hdr) {
>> >     if (c$id$resp_h in mcast || c$id$resp_h == bcast)
>> >         print "mutlicast or broadcast found";
>> > }
>> >
>> > You wouldn't want to use the new_packet event of course.
>> >
>> > -----Original Message-----
>> > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Vito
>> > Logrillo
>> > Sent: Tuesday, August 25, 2015 9:41 AM
>> > To: bro at bro.org
>> > Subject: [Bro] Broadcast detection
>> >
>> > Hi all,
>> > i want to flag if a given ip is an ip broadcast/multicast or not:
>> > there are some built-in functions able to recognize an ip broadcast in
>> > Bro?
>> > Thanks,
>> > Vito
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> >
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

From gfaulkner.nsm at gmail.com  Wed Aug 26 11:38:27 2015
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Wed, 26 Aug 2015 13:38:27 -0500
Subject: [Bro] Getting info records when log events happen,
 but where the logging script has no specific log event type
Message-ID: <55DE07A3.6040803@gmail.com>

In my quest to graph event statistics tied to bro logs I've run across a 
few scripts that seem to break the idiom of logging being a separate 
event from the rest of the events in a script. A couple examples are 
capture-loss and tunnels. Both scripts call the LOG function within some 
other event that doesn't expose the underlying info record to other 
scripts as far as I can tell. A lot of my meta-data collection acts on 
the log events and the data contained within the info records at the 
time those events are logged. I'm wondering if there is another way to 
grab that data without modifying the base scripts or if these scripts 
can be easily made to have a logging event?

Here are a couple examples of things I'd like to be able to do.

  * Increment a counter whenever a new log line is written (useful for
    troubleshooting upstream log aggregator inputs)
  * Send raw data such as percent_lost per peer to an external time
    series database (could be useful for seeing loss over time, or
    identifying problems with load-balancing and filtering of flows).
  * Track number of tunnels seen by tunnel type (knowing how often and
    when traffic is being tunneled could be interesting)

I also find tracking event counters can be useful for identifying things 
that are outside the norm, especially in cases where seeing similar 
trends in a log management system or SEM involves a very expensive 
query. For example a sudden spike in TCP connection attempts / SYNs that 
could indicate DOS participation, spikes in the number of DNS ANY 
queries (probably an open resolver being abused) etc.

Here are a few simplistic examples of some counters I'm already 
collecting that show how the log event and info record are used (These 
rely on JA's statsd plugin and some stats may be borrowed/derived):

# DNS Events

event DNS::log_dns(rec: DNS::Info)
{
     statsd_increment("bro.log.dns", 1); #Track DNS log volume

     if(rec?$rcode && rec$rcode == 3) 
{statsd_increment("bro.dns.error.nxdomain", 1);}

     if(rec?$qtype_name && /query/ !in rec$qtype_name)
     {
         local s = fmt("bro.dns.query.type.%s", rec$qtype_name);
         statsd_increment(s, 1);
     }
}

# Notice Events

event Notice::log_notice(rec: Notice::Info)
{
     statsd_increment("bro.log.notice", 1); #Track Notice log volume

     if(rec?$note)
     {
         local s = fmt("bro.notice.type.%s", rec$note);
         local s2 = sub(s, /::/, "_"); #influxdb doesn't like :: so 
replace it with _
         statsd_increment(s2, 1);
     }
}


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150826/6f1f2ed9/attachment.html 

From anthony.kasza at gmail.com  Wed Aug 26 12:41:54 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Wed, 26 Aug 2015 12:41:54 -0700
Subject: [Bro] Broadcast detection
In-Reply-To: <CANdYiFGjaBJKkA2x+9y+-nFGfsqg6oiwtAV5GzS5Mp_7YzmS6A@mail.gmail.com>
References: <CANdYiFGo6U4mDvM7K2kXoQLMkowTCa0e+5wO1CPz11oAf7ijEg@mail.gmail.com>
	<20150825174016.C5AE22C4063@rock.ICSI.Berkeley.EDU>
	<CANdYiFF29NN+m_LQ1TSWLG+2A_74ZkP9Dp9E=v6Es1zHQzi_mg@mail.gmail.com>
	<CAEZw2bwfOHuR8f1a5HnbBZNGxwNebwFFczBeRof8asHTTStz8w@mail.gmail.com>
	<CANdYiFGjaBJKkA2x+9y+-nFGfsqg6oiwtAV5GzS5Mp_7YzmS6A@mail.gmail.com>
Message-ID: <CAEZw2bz9q10wKFS1b3d5hmQRO1rWwBCv0=qXG46LO5HPG2x=Yg@mail.gmail.com>

The following code feels like it should work, but the last print statement
breaks it.

local my_subnet: subnet = 1.1.1.1/8;
print fmt("%s", my_subnet);
print fmt("%s", |my_subnet|);
print fmt("%s", my_subnet[ |my_subnet| ]);

I don't believe there is currently a built in way to do what you want. Is
there a way to convert a subnet to a vector of addr?

-AK
On Aug 26, 2015 10:50 AM, "Vito Logrillo" <vitologrillo at gmail.com> wrote:

> Correct!
> Any suggestion?
>
> 2015-08-26 16:17 GMT+02:00 anthony kasza <anthony.kasza at gmail.com>:
> > Oh I see what you're saying. What you'd like is a function that takes a
> > subnet as input and returns the broadcast address, correct?
> >
> > -AK
> >
> > On Aug 26, 2015 6:11 AM, "Vito Logrillo" <vitologrillo at gmail.com> wrote:
> >>
> >> Hi,
> >> thanks for your reply.
> >> What i'm trying to do is to create a flag if an ip broadcast is found.
> >> For example, in networks.cfg i've written this subnet
> >> 172.20.1.0/24
> >> It's broadcast address is 172.20.1.255.
> >> I can read all subnets written in networks.cfg with the variable
> >> Site::local_nets_table: to calculate the ip broadcast i can use this
> >> method
> >> https://en.wikipedia.org/wiki/Broadcast_address
> >> What i'm not able to do is to transform a subnet variable (in this
> >> case 172.20.1.0/24) into an ip variable (172.20.1.0) plus a count
> >> (24).
> >> Any suggestion?
> >> Thanks
> >> Vito
> >>
> >> 2015-08-25 19:39 GMT+02:00 Swan, Jay <jswan at sugf.com>:
> >> > One way would be to check the packet destination against the IP
> >> > multicast range:
> >> >
> >> > global mcast = 224.0.0.0/4;
> >> > global bcast = 255.255.255.255;
> >> > event new_packet(c:connection,p:pkt_hdr) {
> >> >     if (c$id$resp_h in mcast || c$id$resp_h == bcast)
> >> >         print "mutlicast or broadcast found";
> >> > }
> >> >
> >> > You wouldn't want to use the new_packet event of course.
> >> >
> >> > -----Original Message-----
> >> > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of
> Vito
> >> > Logrillo
> >> > Sent: Tuesday, August 25, 2015 9:41 AM
> >> > To: bro at bro.org
> >> > Subject: [Bro] Broadcast detection
> >> >
> >> > Hi all,
> >> > i want to flag if a given ip is an ip broadcast/multicast or not:
> >> > there are some built-in functions able to recognize an ip broadcast in
> >> > Bro?
> >> > Thanks,
> >> > Vito
> >> > _______________________________________________
> >> > Bro mailing list
> >> > bro at bro-ids.org
> >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >> >
> >> > _______________________________________________
> >> > Bro mailing list
> >> > bro at bro-ids.org
> >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >> _______________________________________________
> >> Bro mailing list
> >> bro at bro-ids.org
> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150826/5d1f01a1/attachment.html 

From anthony.kasza at gmail.com  Wed Aug 26 18:12:18 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Wed, 26 Aug 2015 18:12:18 -0700
Subject: [Bro] Broadcast detection
In-Reply-To: <CAEZw2bz9q10wKFS1b3d5hmQRO1rWwBCv0=qXG46LO5HPG2x=Yg@mail.gmail.com>
References: <CANdYiFGo6U4mDvM7K2kXoQLMkowTCa0e+5wO1CPz11oAf7ijEg@mail.gmail.com>
	<20150825174016.C5AE22C4063@rock.ICSI.Berkeley.EDU>
	<CANdYiFF29NN+m_LQ1TSWLG+2A_74ZkP9Dp9E=v6Es1zHQzi_mg@mail.gmail.com>
	<CAEZw2bwfOHuR8f1a5HnbBZNGxwNebwFFczBeRof8asHTTStz8w@mail.gmail.com>
	<CANdYiFGjaBJKkA2x+9y+-nFGfsqg6oiwtAV5GzS5Mp_7YzmS6A@mail.gmail.com>
	<CAEZw2bz9q10wKFS1b3d5hmQRO1rWwBCv0=qXG46LO5HPG2x=Yg@mail.gmail.com>
Message-ID: <CAEZw2bxyYcSf-S5DLEe7Hbmwrc0p64Unb-skuORWx4heuJT-Ww@mail.gmail.com>

I'm looking to write a bif which does this. How can I access a subnet's
prefix as an int? Here's what I have so far.

function get_broadcast%(snet: subnet%): addr
  %{
  return new AddrVal( snet->Prefix() + (snet->Width() - 1) );
  %}

-AK
On Aug 26, 2015 12:41 PM, "anthony kasza" <anthony.kasza at gmail.com> wrote:

> The following code feels like it should work, but the last print statement
> breaks it.
>
> local my_subnet: subnet = 1.1.1.1/8;
> print fmt("%s", my_subnet);
> print fmt("%s", |my_subnet|);
> print fmt("%s", my_subnet[ |my_subnet| ]);
>
> I don't believe there is currently a built in way to do what you want. Is
> there a way to convert a subnet to a vector of addr?
>
> -AK
> On Aug 26, 2015 10:50 AM, "Vito Logrillo" <vitologrillo at gmail.com> wrote:
>
>> Correct!
>> Any suggestion?
>>
>> 2015-08-26 16:17 GMT+02:00 anthony kasza <anthony.kasza at gmail.com>:
>> > Oh I see what you're saying. What you'd like is a function that takes a
>> > subnet as input and returns the broadcast address, correct?
>> >
>> > -AK
>> >
>> > On Aug 26, 2015 6:11 AM, "Vito Logrillo" <vitologrillo at gmail.com>
>> wrote:
>> >>
>> >> Hi,
>> >> thanks for your reply.
>> >> What i'm trying to do is to create a flag if an ip broadcast is found.
>> >> For example, in networks.cfg i've written this subnet
>> >> 172.20.1.0/24
>> >> It's broadcast address is 172.20.1.255.
>> >> I can read all subnets written in networks.cfg with the variable
>> >> Site::local_nets_table: to calculate the ip broadcast i can use this
>> >> method
>> >> https://en.wikipedia.org/wiki/Broadcast_address
>> >> What i'm not able to do is to transform a subnet variable (in this
>> >> case 172.20.1.0/24) into an ip variable (172.20.1.0) plus a count
>> >> (24).
>> >> Any suggestion?
>> >> Thanks
>> >> Vito
>> >>
>> >> 2015-08-25 19:39 GMT+02:00 Swan, Jay <jswan at sugf.com>:
>> >> > One way would be to check the packet destination against the IP
>> >> > multicast range:
>> >> >
>> >> > global mcast = 224.0.0.0/4;
>> >> > global bcast = 255.255.255.255;
>> >> > event new_packet(c:connection,p:pkt_hdr) {
>> >> >     if (c$id$resp_h in mcast || c$id$resp_h == bcast)
>> >> >         print "mutlicast or broadcast found";
>> >> > }
>> >> >
>> >> > You wouldn't want to use the new_packet event of course.
>> >> >
>> >> > -----Original Message-----
>> >> > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of
>> Vito
>> >> > Logrillo
>> >> > Sent: Tuesday, August 25, 2015 9:41 AM
>> >> > To: bro at bro.org
>> >> > Subject: [Bro] Broadcast detection
>> >> >
>> >> > Hi all,
>> >> > i want to flag if a given ip is an ip broadcast/multicast or not:
>> >> > there are some built-in functions able to recognize an ip broadcast
>> in
>> >> > Bro?
>> >> > Thanks,
>> >> > Vito
>> >> > _______________________________________________
>> >> > Bro mailing list
>> >> > bro at bro-ids.org
>> >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> >> >
>> >> > _______________________________________________
>> >> > Bro mailing list
>> >> > bro at bro-ids.org
>> >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> >> _______________________________________________
>> >> Bro mailing list
>> >> bro at bro-ids.org
>> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150826/6d2f6724/attachment.html 

From drakearonhalt at gmail.com  Thu Aug 27 07:57:52 2015
From: drakearonhalt at gmail.com (Drake Aronhalt)
Date: Thu, 27 Aug 2015 10:57:52 -0400
Subject: [Bro] Bro on CentOS 7
Message-ID: <CAMfxL3Eci6sbd=9-VoqM7AR6Rni2OOPnFDKJdUkxT6GtTMkjWA@mail.gmail.com>

Has anyone had any issues running bro on CentOS 7.1? It crashes the entire
system every time I run 'broctl start'. I've configured with and without
pfring as well as with and without c++11 support. The kernel I'm using is
3.10.0-229.11.1.el7.x86_64 (I've also tried to the previous kernel). The
NIC is an intel I350 NIC with the stock driver, although I tried updating
the driver to the current intel driver with the same results.

I think this is a kernel issue (getting ready to file a bug there) I just
wanted to make sure someone in the community didn't have a solution for
this already.

Thanks.
Drake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150827/5d1749a4/attachment.html 

From robin at icir.org  Thu Aug 27 08:03:21 2015
From: robin at icir.org (Robin Sommer)
Date: Thu, 27 Aug 2015 08:03:21 -0700
Subject: [Bro] Broadcast detection
In-Reply-To: <CAEZw2bxyYcSf-S5DLEe7Hbmwrc0p64Unb-skuORWx4heuJT-Ww@mail.gmail.com>
References: <CANdYiFGo6U4mDvM7K2kXoQLMkowTCa0e+5wO1CPz11oAf7ijEg@mail.gmail.com>
	<20150825174016.C5AE22C4063@rock.ICSI.Berkeley.EDU>
	<CANdYiFF29NN+m_LQ1TSWLG+2A_74ZkP9Dp9E=v6Es1zHQzi_mg@mail.gmail.com>
	<CAEZw2bwfOHuR8f1a5HnbBZNGxwNebwFFczBeRof8asHTTStz8w@mail.gmail.com>
	<CANdYiFGjaBJKkA2x+9y+-nFGfsqg6oiwtAV5GzS5Mp_7YzmS6A@mail.gmail.com>
	<CAEZw2bz9q10wKFS1b3d5hmQRO1rWwBCv0=qXG46LO5HPG2x=Yg@mail.gmail.com>
	<CAEZw2bxyYcSf-S5DLEe7Hbmwrc0p64Unb-skuORWx4heuJT-Ww@mail.gmail.com>
Message-ID: <20150827150321.GA2780@icir.org>



On Wed, Aug 26, 2015 at 18:12 -0700, anthony kasza wrote:

> I'm looking to write a bif which does this. How can I access a subnet's
> prefix as an int?

snet->Prefix() yields an IPAddr. You don't easily get that as an int,
but it has a method for getting it as a sequence of bytes:

    int GetBytes(const uint32_t** bytes)

That works for both IPv4 and v6.

That said, I think you can solve this more easily by combining some
other methods that IPAddr offers as well:

     /**
         * Masks out lower bits of the address.
         [...]
         */
        void Mask(int top_bits_to_keep);

     /**
         * Masks out top bits of the address.
        [...]
         */
        void ReverseMask(int top_bits_to_chop);

      /**
         * Bitwise OR operator returns the IP address resulting from the bitwise
         * OR operation on the raw bytes of this address with another.
         */
       IPAddr operator|(const IPAddr& other)

You'd mask out the lower bits of the prefix, mask out the upper bits
of 255.255.255.255 (for IPv4), and then "or" the two together.

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From johanna at icir.org  Sat Aug 29 13:15:26 2015
From: johanna at icir.org (Johanna Amann)
Date: Sat, 29 Aug 2015 13:15:26 -0700
Subject: [Bro] [Bro-Dev] Aide broDev
In-Reply-To: <CAK64S6zZtySvDJRCHQxkbux+0sdHa==-Rh1ZDxpapXwLXbh9fA@mail.gmail.com>
References: <CAK64S6wL+CwegD7RvuYqNbAXSFSmOw-CgeEtnMErzX-QZLAZdQ@mail.gmail.com>
	<CAK64S6zZtySvDJRCHQxkbux+0sdHa==-Rh1ZDxpapXwLXbh9fA@mail.gmail.com>
Message-ID: <20150829201526.GA30996@Beezling.local>

Hello,

since my French is a bit bad - English reply first. Most, if not all of
the tasks in your list seem to require an active in-path system like an
IPS system, not a network monitoring system like Bro.

Tasks like requiring a password for users that want to use the Internet or
only allowing certain servers to access DNS cannot be easily accomplished
with Bro.

That being said -- we recently started working on the NetControl framework
of Bro that allows you to install rules in certain switches/firewalls to
e.g. block traffic that is recognized as being malicious - you could
examine if that might fulfil your requirements; installation instructions
are available at https://github.com/bro/bro-netcontrol

--

Bonjour,

beaucoup, sinon tous les taches dans le fichier necessite une machine dans
le chemin de reseau. Bro est un logiciel qu'ecoute le resau d'ordinateur
passivement; donc Bro ne peut pas influencer le trafic dans le reseau.

Taches comme demander un mot de passe pour permettre des connexions sont
tres difficile d'accompiler avec Bro.

Mais -- recemment nous avons commonce le travail sur un nouveau systeme
dans Bro appelle 'NetControl'. Netcontrol permets, par example, de ne pas
permettre certain connexions. NetControl pousse des regles aux, par
example, pare-feux ou switchs. Les instructions pour l'installations sont
disponsibles a https://github.com/bro/bro-netcontrol

Johanna


On Sat, Aug 29, 2015 at 08:05:09PM +0100, Edgar D. AYENA wrote:
> Bonjour chers amis d?veloppeurs Bro,
> Je suis Edgar, et je suis d?butant sur Bro. Mon m?moire de fin de
> formation m'a amen? ? mettre en place des politiques de s?curit? avec
> l'outil Bro. J'ai ?num?r? quelques taches ? ex?cuter dans un fichier
> que j'ai joint ? ce mail. Je ne sais pas si tout est faisable avec bro
> mais je voudrais s?rieusement de l'aide car la date de ma soutenance
> se rapproche et je voudrais pouvoir appliquer tout au moins certaines
> lors de ma pr?sentation.
> Merci de m'aider SVP.
> 
> -- 
> Cordialement,
> ------
> Edgar D. AYENA,
> T?l: (00229) 96 055 506 - 95 805 326
> 03 BP 3172 Cotonou, R. B?nin
> ayenadedgar at yahoo.fr
> ayenadedg at gmail.com


> _______________________________________________
> bro-dev mailing list
> bro-dev at bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev


From timbr.data at gmail.com  Sun Aug 30 08:05:02 2015
From: timbr.data at gmail.com (Tim Brennan)
Date: Sun, 30 Aug 2015 11:05:02 -0400
Subject: [Bro] DHCP Analyzer question/issue
Message-ID: <CAPt3nMKZBsPPiKsHq_+xk_p+xc=-cps4V0LYwvn+gHikkxKNfA@mail.gmail.com>

I am running Bro 2.4 as part of a recent security-onion installation.  I am
seeing very few entries in the dhcp.log files. In weird.log, I see tons of
entries similar to the below:

binpac exception: out_of_bound: DHCP_Message:file: 236 > 187
binpac exception: out_of_bound: DHCP_Message:giaddr: 28 > 14
binpac exception: out_of_bound: DHCP_Message:chaddr: 44 > 32
binpac exception: out_of_bound: DHCP_Message:sname: 108 > 73

Do I have an configuration issue?  Any ideas on what is going on or how to
troubleshoot?

Thanks,
Tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150830/45e646f2/attachment.html 

From j2om1350 at unibw.de  Mon Aug 31 00:20:44 2015
From: j2om1350 at unibw.de (j2om1350 at unibw.de)
Date: Mon, 31 Aug 2015 09:20:44 +0200
Subject: [Bro] Implementing new layer 2 Protocol
Message-ID: <a223fb54b02d87d76b9ac8364a2838d7.squirrel@mail.unibw.de>

Hi all,

My goal is to integrate a new protocol analyzer in Bro. This protocol
(PROFINET dyscovery and Basic Configuration Protocol) is working on layer
2. My question is, are there special considerations to get at the data of
the layer 2? My colleague has tried creating an analyzer by following your
instructions for coding an analyzer by binpac. Before he went on vacation,
he told me, he could access data with binpac of layer 3 but not of layer
2? Is that correct? If so does it work with the new binpac ++? Any pieces
of advice or suggestions how to get started would be greatly appreciated.

Kind regards

Marcel Odenwald



From robin at icir.org  Mon Aug 31 08:31:20 2015
From: robin at icir.org (Robin Sommer)
Date: Mon, 31 Aug 2015 08:31:20 -0700
Subject: [Bro] DHCP Analyzer question/issue
In-Reply-To: <CAPt3nMKZBsPPiKsHq_+xk_p+xc=-cps4V0LYwvn+gHikkxKNfA@mail.gmail.com>
References: <CAPt3nMKZBsPPiKsHq_+xk_p+xc=-cps4V0LYwvn+gHikkxKNfA@mail.gmail.com>
Message-ID: <20150831153120.GM39752@icir.org>



On Sun, Aug 30, 2015 at 11:05 -0400, you wrote:

> Do I have an configuration issue?  Any ideas on what is going on or how to
> troubleshoot?

If you could capture a small trace reproducing the messages, that
would be helpful.

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From earl.eiland at root9b.com  Mon Aug 31 12:17:11 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Mon, 31 Aug 2015 19:17:11 +0000
Subject: [Bro] using bro for file extraction
Message-ID: <BY1PR0401MB13821FAA0905AD2E45DDDAC4D76B0@BY1PR0401MB1382.namprd04.prod.outlook.com>

I want to use bro to extract files for external analysis.  Bro::FileDataEvent appears to be the proper approach.  However, I?m not finding the event to write a script for, nor do I know how to write to anything other than a log file.

Please advise!

Best Regards,

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150831/50f7dec6/attachment.html 

From easetheworld at gmail.com  Mon Aug 31 14:33:45 2015
From: easetheworld at gmail.com (Hyun Yoo)
Date: Tue, 1 Sep 2015 06:33:45 +0900
Subject: [Bro] using bro for file extraction
In-Reply-To: <BY1PR0401MB13821FAA0905AD2E45DDDAC4D76B0@BY1PR0401MB1382.namprd04.prod.outlook.com>
References: <BY1PR0401MB13821FAA0905AD2E45DDDAC4D76B0@BY1PR0401MB1382.namprd04.prod.outlook.com>
Message-ID: <CAJzQz-G7TRq3g1faiX0zgWdZSbbL8pP6U0cvnv=LUDvoQOeAkQ@mail.gmail.com>

Look at 'scripts/base/protocols/conn/contents.bro' for example. That
extract sessions and save to disks.
2015. 9. 1. ?? 4:22? "Earl Eiland" <earl.eiland at root9b.com>?? ??:

> I want to use bro to extract files for external analysis.
> Bro::FileDataEvent appears to be the proper approach.  However, I?m not
> finding the event to write a script for, nor do I know how to write to
> anything other than a log file.
>
>
>
> Please advise!
>
>
>
> Best Regards,
>
>
>
> Earl Eiland,
>
> Sr. Cyber Security Engineer,
>
> Emerging Technologies, root9B,
>
> San Antonio, Texas
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150901/7c9ad4d3/attachment.html 

From scotty.b.brown at gmail.com  Mon Aug 31 16:45:10 2015
From: scotty.b.brown at gmail.com (Scotty Brown)
Date: Tue, 1 Sep 2015 09:45:10 +1000
Subject: [Bro] Email Notice Suppression
Message-ID: <55E4E706.7090403@gmail.com>

Hi All,

I'm running bro 2.4 and have just added a bunch of critical stack intel
feeds.  All is working well.

One of the feeds I have is a list of TOR ips, and once I set notices to
true for the critical stack intel I start getting emails (I've set up
email alerting for notices).

What I would like to do is suppress email alerts for a particular notice
from a particular src host.

ie (intel.log):

1441063489.889373       CEyDP6zbg6ngOFFa        10.10.10.10    45969  
213.163.70.234  443     -       -       -       213.163.70.234 
Intel::ADDR     Conn::IN_RESP   sensor-eth1-1 from
https://www.dan.me.uk/torlist/ via intel.criticalstack.com

So any notice that fires from src 10.10.10.10 for the torlist intel -
I'd still like to see the notice in the intel file - but not get the
email alert.

Any pointers?

Cheers,

Scotty



From michalpurzynski1 at gmail.com  Mon Aug 31 19:01:41 2015
From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=)
Date: Mon, 31 Aug 2015 19:01:41 -0700
Subject: [Bro] Bro manager dies in a large cluster
Message-ID: <CAJ6bFK3MGcPLUvhwGP0_8v4hzQm5byo_3y+7cJBWwYWSxtiaSQ@mail.gmail.com>

Hello :-)

I've finished the long process of merging all sensors in a large
cluster. To my surprise, every time I enable all of them and run
"broctl deploy" all workers start, so do proxies, but manager dies
right away.

This cluster has almost 200 workers, 9 servers, between 8 and 16
proxies (tried 8 and 16, didn't change anything).

I have lots of traffic, lots of connections, lots of everything ;-) My
guess is that manager can't keep up with the amount of logs it is
expected to generate and it gives up.

manager and proxies run on a server dedicated just for them, 64GB RAM,
16 physical cores, dedicated network for the cluster traffic.

Now, when I divide the cluster more or less in half (4 nodes enabled,
5 disabled) everything is stable.

The amount of logs with 4 sensors enabled (almost exactly an hour, I'm
like 2 minutes from rotation). Hm. Maybe I should do something about
the Mysql traffic ;-)

What can I do? I'd like to help debug, if that's a bug I'm running into.

total 24G

-rw-rw-r-- 1 bro bro  84K Sep  1 01:57 capture_loss.log
-rw-rw-r-- 1 bro bro 5.6M Sep  1 01:58 communication.log
-rw-rw-r-- 1 bro bro 1.2G Sep  1 01:58 conn.log
-rw-rw-r-- 1 bro bro 476M Sep  1 01:58 conn-noise.log
-rw-rw-r-- 1 bro bro 1.8M Sep  1 01:58 dhcp.log
-rw-rw-r-- 1 bro bro 309M Sep  1 01:58 dns.log
-rw-rw-r-- 1 bro bro 609M Sep  1 01:58 dns-noise.log
-rw-rw-r-- 1 bro bro 115K Sep  1 01:58 dpd.log
-rw-rw-r-- 1 bro bro 1.5G Sep  1 01:58 files.log
-rw-rw-r-- 1 bro bro 1.5G Sep  1 01:58 http.log
-rw-rw-r-- 1 bro bro  65M Sep  1 01:58 http-noise.log
-rw-rw-r-- 1 bro bro 1.6M Sep  1 01:58 intel.log
-rw-rw-r-- 1 bro bro  37K Sep  1 01:54 intel-noise.log
-rw-rw-r-- 1 bro bro  68K Sep  1 01:58 irc.log
-rw-rw-r-- 1 bro bro 7.8M Sep  1 01:58 kerberos.log
-rw-rw-r-- 1 bro bro 566K Sep  1 01:58 known_certs.log
-rw-rw-r-- 1 bro bro  41K Sep  1 01:58 known_devices.log
-rw-rw-r-- 1 bro bro 244K Sep  1 01:58 known_hosts.log
-rw-rw-r-- 1 bro bro 330K Sep  1 01:58 known_services.log
-rw-rw-r-- 1 bro bro 4.9G Sep  1 01:58 mysql.log
-rw-rw-r-- 1 bro bro 636K Sep  1 01:58 notice.log
-rw-rw-r-- 1 bro bro 6.0K Sep  1 01:58 pe.log
-rw-rw-r-- 1 bro bro  559 Sep  1 01:31 reporter.log
-rw-rw-r-- 1 bro bro 168K Sep  1 01:57 sip.log
-rw-rw-r-- 1 bro bro  12M Sep  1 01:58 smtp.log
-rw-rw-r-- 1 bro bro  25M Sep  1 01:58 snmp.log
-rw-rw-r-- 1 bro bro  73M Sep  1 01:58 software.log
-rw-rw-r-- 1 bro bro 3.9M Sep  1 01:58 ssh.log
-rw-rw-r-- 1 bro bro  23K Sep  1 01:57 sslcipherstat_log1.log
-rw-rw-r-- 1 bro bro 766K Sep  1 01:58 sslcipherstat_log2.log
-rw-rw-r-- 1 bro bro 783M Sep  1 01:58 ssl.log
-rw-rw-r-- 1 bro bro  17K Sep  1 01:57 sslprotostat_log1.log
-rw-rw-r-- 1 bro bro 773K Sep  1 01:58 sslprotostat_log2.log
-rw-rw-r-- 1 bro bro  492 Sep  1 00:12 stderr.log
-rw-rw-r-- 1 bro bro  188 Sep  1 00:12 stdout.log
-rw-rw-r-- 1 bro bro 7.8K Sep  1 01:12 subnet.log
-rw-rw-r-- 1 bro bro 3.9G Sep  1 01:58 syslog.log
-rw-rw-r-- 1 bro bro 1.6M Sep  1 01:58 tunnel.log
-rw-rw-r-- 1 bro bro  46M Sep  1 01:58 weird.log
-rw-rw-r-- 1 bro bro 1.6G Sep  1 01:58 x509.log
-rw-rw-r-- 1 bro bro  683 Sep  1 01:31 xss.log

Logs aren't really helpful.

cat post-terminate-2015-09-01-00-12-15-61637-crash/.crash-diag.log


Bro 2.4

Linux 3.19.0-26-generic



==== No reporter.log


==== stderr.log

warning in /opt/bro/share/bro/brozilla/./intel-dns.bro, line 99:
deprecated (join_string_array)

warning in /nsm/bro/spool/installed-scripts-do-not-touch/site/local.bro,
line 176: multiple initializations for index (10.248.75.6)

warning in /nsm/bro/spool/installed-scripts-do-not-touch/site/local.bro,
line 176: multiple initializations for index (10.248.75.7)

warning in /nsm/bro/spool/installed-scripts-do-not-touch/site/local.bro,
line 177: multiple initializations for index (10.248.22.1)


==== stdout.log

max memory size         (kbytes, -m) unlimited

data seg size           (kbytes, -d) unlimited

virtual memory          (kbytes, -v) unlimited

core file size          (blocks, -c) unlimited


==== .cmdline

-U .status -p broctl -p broctl-live -p local -p nsmserver1-manager
local.bro broctl base/frameworks/cluster local-manager.bro broctl/auto


==== .env_vars

PATH=/opt/bro/bin:/opt/bro/share/broctl/scripts:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games

BROPATH=/nsm/bro/spool/installed-scripts-do-not-touch/site::/nsm/bro/spool/installed-scripts-do-not-touch/auto:/opt/bro/share/bro:/opt/bro/share/bro/policy:/opt/bro/share/bro/site

CLUSTER_NODE=nsmserver1-manager


==== .status

TERMINATED [atexit]


==== No prof.log


==== No packet_filter.log


==== No loaded_scripts.log

bro at nsmserver1:/nsm/bro/spool/tmp$