From franky.meier.1 at gmx.de  Wed Apr  1 02:01:44 2015
From: franky.meier.1 at gmx.de (Frank Meier)
Date: Wed, 01 Apr 2015 11:01:44 +0200
Subject: [Bro] http incomplete file extraction (Files::ANALYZER_EXTRACT)
In-Reply-To: <7A038474-58CC-43D7-B337-7B7C4024C248@illinois.edu>
References: <trinity-9655f832-a2a5-4afe-9ffa-746c4a6672ae-1427463329271@3capp-gmx-bs42>
	<,>
Message-ID: <1427878904.7027.0@mail.gmx.net>

Hi!

On Mo, M?r 30, 2015 at 6:54 , Siwek, Jon <jsiwek at illinois.edu> wrote:

> In files.log, the value of total_bytes is just taken from the HTTP 
> Content-Length header.  Since the value of seen_bytes is less than 
> total_bytes, you can suspect Bro didn?t see the full file for some 
> reason.  Do you have a weird.log containing any obvious clues?  Else, 
> I may need the original pcap to understand what went wrong.

The weird.log states some "above_hole_data_without_any_acks", but why 
does it work with tcpflow?

Here is what I did:

1) I downloaded the test file: wget 
http://ipv4.download.thinkbroadband.com/5MB.zip
2) Gathered the pcap: tcpdump -s0 -i eth0 -w download.pcap port http
3) checked if the file was completely captured with tcpflow:
tcpflow -FT -e http -r download.pcap
md5sums do match: 
~/bro-liste$ md5sum 
2015-04-01T07:45:00Z080.249.099.148.00080-192.168.002.103.42716-HTTPBODY-001.zip
b3215c06647bc550406a9c8ccc378756  
2015-04-01T07:45:00Z080.249.099.148.00080-192.168.002.103.42716-HTTPBODY-001.zip
~/bro-liste$ md5sum 5MB.zip 
b3215c06647bc550406a9c8ccc378756  5MB.zip

4) run bro (revision 32ae94de9ae36060651240a0ee11838e3e572223) with 
simple bro-file:
~/bro-liste$ cat extract.bro 
event file_new(f: fa_file)
{
                Files::add_analyzer(f, Files::ANALYZER_EXTRACT);
}

~/bro-liste$ /usr/local/bro/bin/bro -r download.pcap extract.bro 
1427874309.892545 warning in 
/usr/local/bro/share/bro/base/misc/find-checksum-offloading.bro, line 
54: Your trace file likely has invalid TCP checksums, most likely from 
NIC checksum offloading.

5) Logs from bro and the pcap: (14mb)
http://www.xup.in/dl,19594721/extract.tar.bz2/

Thanks!

Franky


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150401/a7a965d4/attachment.html 

From tim.molter at gmail.com  Wed Apr  1 06:52:09 2015
From: tim.molter at gmail.com (Tim Molter)
Date: Wed, 01 Apr 2015 15:52:09 +0200
Subject: [Bro] Does files.log generator recognize application/json file
	types?
Message-ID: <551BF809.7020406@gmail.com>

Does `files.log` generator recognize application/json file types, and if
not, what could I modify to get that to work? What does the generator
key into to make the decision?

From Joey.Barkley at ingramcontent.com  Wed Apr  1 07:52:05 2015
From: Joey.Barkley at ingramcontent.com (Barkley, Joey)
Date: Wed, 1 Apr 2015 09:52:05 -0500
Subject: [Bro] Detect failed HTTPS logins and get the source IP?
Message-ID: <040A357C-9F05-4C54-AE00-D3A440749617@ingramcontent.com>

All,

Our organization recently began to experience an attack where they are trying to log into a 3rd party web application that we have exposed externally. They appear to be using a bonnet that is spread across the globe, so our efforts to block country segments are only marginally effective. The real issue is that the application does not provide source IP addresses in the logs for the failed login attempts because it is being served up through nginx.

We are just now starting to dabble with Bro, but still have very little ?real world? experience with writing our own scripts. Has anyone written a script that might give us a little help in this direction? We need to be able to track a session (login request), wait for a failure to return through, and then apply a threshold. So for example, I want to log an ?alert? (sorry, forget what the terminology is in Bro) if an IP address fails login n times in x seconds. We will then use that log to block that IP address for a period of time.

Is this something that Bro can do, and if so, where do I start? Any guidance would be greatly appreciated.

Thanks,
Joey


From liam.randall at gmail.com  Wed Apr  1 08:30:42 2015
From: liam.randall at gmail.com (Liam Randall)
Date: Wed, 1 Apr 2015 11:30:42 -0400
Subject: [Bro] Detect failed HTTPS logins and get the source IP?
In-Reply-To: <040A357C-9F05-4C54-AE00-D3A440749617@ingramcontent.com>
References: <040A357C-9F05-4C54-AE00-D3A440749617@ingramcontent.com>
Message-ID: <CAC8+5eORPO1+HJWDaYycy0v7nG1xd3f4LJEb82JCjizFjsEGCA@mail.gmail.com>

Hey Joey,

There are endless derivations you can do of this, however here's an
approach assuming you are speaking of http basic authorization:

https://github.com/criticalstack/bro-scripts/blob/master/http-basic-auth-bruteforcing/detect-http-basic-auth-bruteforcer.bro

I noted a number of derivations of the script in the comments at the top to
give you some thoughts.

Have you looked at the characteristics of the attacking hosts?  Are they
members of a specific botnet?

V/r,

Liam Randall


On Wed, Apr 1, 2015 at 10:52 AM, Barkley, Joey <
Joey.Barkley at ingramcontent.com> wrote:

> All,
>
> Our organization recently began to experience an attack where they are
> trying to log into a 3rd party web application that we have exposed
> externally. They appear to be using a bonnet that is spread across the
> globe, so our efforts to block country segments are only marginally
> effective. The real issue is that the application does not provide source
> IP addresses in the logs for the failed login attempts because it is being
> served up through nginx.
>
> We are just now starting to dabble with Bro, but still have very little
> ?real world? experience with writing our own scripts. Has anyone written a
> script that might give us a little help in this direction? We need to be
> able to track a session (login request), wait for a failure to return
> through, and then apply a threshold. So for example, I want to log an
> ?alert? (sorry, forget what the terminology is in Bro) if an IP address
> fails login n times in x seconds. We will then use that log to block that
> IP address for a period of time.
>
> Is this something that Bro can do, and if so, where do I start? Any
> guidance would be greatly appreciated.
>
> Thanks,
> Joey
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150401/07f79a16/attachment.html 

From jsiwek at illinois.edu  Wed Apr  1 08:26:22 2015
From: jsiwek at illinois.edu (Siwek, Jon)
Date: Wed, 1 Apr 2015 15:26:22 +0000
Subject: [Bro] http incomplete file extraction (Files::ANALYZER_EXTRACT)
In-Reply-To: <1427878904.7027.0@mail.gmx.net>
References: <trinity-9655f832-a2a5-4afe-9ffa-746c4a6672ae-1427463329271@3capp-gmx-bs42>
	<,> <1427878904.7027.0@mail.gmx.net>
Message-ID: <CB1A4F82-989B-4021-9810-C4633CEE3606@illinois.edu>


> ~/bro-liste$ /usr/local/bro/bin/bro -r download.pcap extract.bro 
> 1427874309.892545 warning in /usr/local/bro/share/bro/base/misc/find-checksum-offloading.bro, line 54: Your trace file likely has invalid TCP checksums, most likely from NIC checksum offloading.

You?ll have to address this problem to get the results you expect.  See:

https://www.bro.org/documentation/faq.html#why-isn-t-bro-producing-the-logs-i-expect-a-note-about-checksums

> The weird.log states some ?above_hole_data_without_any_acks"

In this case, this seems like it?s just a side effect of the bad checksums, but in case you?re interested on how that type of situation can effect file extraction in Bro there?s discussion of how/why here:

https://bro-tracker.atlassian.net/browse/BIT-1255

- Jon


From dopheide at gmail.com  Wed Apr  1 10:03:03 2015
From: dopheide at gmail.com (Mike Dopheide)
Date: Wed, 1 Apr 2015 12:03:03 -0500
Subject: [Bro] string to address issue w/ is_v6_addr
Message-ID: <CAPy2kFZa0mPc4mvjeX6RztuV3+W09E_yktzhc8gcZa2gmxFZiQ@mail.gmail.com>

This confused me for quite some time this morning so I thought I'd share.
The script should make it clear, but when attempting to take a url string
and test to see if it's a valid address, the output from to_addr creates a
'valid' ipv6 address.

Is that a requirement for some reason internally?

-Dop

http://try.bro.org/#/trybro/saved/1313

event bro_init() {

    # is_v6_addr isn't giving the correct result because is_addr returns ::

    # assume url extracted from http:// or ftp:// string via regex:
    local url: string = "www.es.net";

    if(is_v4_addr(to_addr(url))){
        print fmt("hostname is v4 addr");
    }else{
        print fmt("hostname is not v4 addr");
    }

    if(is_v6_addr(to_addr(url))){
        print fmt("hostname is v6 addr");
    }else{
        print fmt("hostname is not v6 addr");
    }

    print fmt("Why? %s",to_addr(url));

}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150401/77d7f69b/attachment-0001.html 

From jsiwek at illinois.edu  Wed Apr  1 11:08:07 2015
From: jsiwek at illinois.edu (Siwek, Jon)
Date: Wed, 1 Apr 2015 18:08:07 +0000
Subject: [Bro] string to address issue w/ is_v6_addr
In-Reply-To: <CAPy2kFZa0mPc4mvjeX6RztuV3+W09E_yktzhc8gcZa2gmxFZiQ@mail.gmail.com>
References: <CAPy2kFZa0mPc4mvjeX6RztuV3+W09E_yktzhc8gcZa2gmxFZiQ@mail.gmail.com>
Message-ID: <A96B6C78-88F2-486A-9F86-739F8D212ED0@illinois.edu>


> On Apr 1, 2015, at 12:03 PM, Mike Dopheide <dopheide at gmail.com> wrote:
> 
> This confused me for quite some time this morning so I thought I'd share.  The script should make it clear, but when attempting to take a url string and test to see if it's a valid address, the output from to_addr creates a 'valid' ipv6 address.  
> 
> Is that a requirement for some reason internally?

to_addr() returning the unspecified IPv6 address on failure to convert an IP string to Bro?s address type is just an arbitrary choice.  Alternatively, it could return the unspecified IPv4 address, 0.0.0.0, but that doesn?t really save anything ? internally Bro?s address values all use a full 128 bits (IPv4 uses the "IPv4-mapped IPv6? representation).  It could also return a record type:

	type opt_addr: record { a: addr &optional; };

or

	type opt_addr: record { a: addr; success: bool; };

Where in the first, it only sets the field if the conversion succeeded, but failure to check for that fields existence before accessing is potentially more problematic than failure to check for [::].   In either, it?s adding another data type the user has to remember or lookup how to use.

So that?s the backstory of why [::] is the failure indicator.  One could also argue that using the unspecified IPv6 address (or IPv4) as a return value makes it ambiguous to try to parse ?::? (or ?0.0.0.0?) as the input string and I?d be on board w/ that and vote to switch to one of the return-a-record styles.

Anyway, from the example you gave, did you just mean to use ?lookup_hostname? instead of ?to_addr? ?

- Jon


From dopheide at gmail.com  Wed Apr  1 11:17:15 2015
From: dopheide at gmail.com (Mike Dopheide)
Date: Wed, 1 Apr 2015 13:17:15 -0500
Subject: [Bro] string to address issue w/ is_v6_addr
In-Reply-To: <A96B6C78-88F2-486A-9F86-739F8D212ED0@illinois.edu>
References: <CAPy2kFZa0mPc4mvjeX6RztuV3+W09E_yktzhc8gcZa2gmxFZiQ@mail.gmail.com>
	<A96B6C78-88F2-486A-9F86-739F8D212ED0@illinois.edu>
Message-ID: <CAPy2kFbPsLFTrTteJid4EUxPtw3G0_Jx2opn5x1H=g+Uh+PUjA@mail.gmail.com>

Thanks for the background!   Looks like what I need is is_valid_ip() from
base/utils/addrs.bro.

Problem was I was starting with a string that could be an IP or could be a
hostname.

-Dop

On Wed, Apr 1, 2015 at 1:08 PM, Siwek, Jon <jsiwek at illinois.edu> wrote:

>
> > On Apr 1, 2015, at 12:03 PM, Mike Dopheide <dopheide at gmail.com> wrote:
> >
> > This confused me for quite some time this morning so I thought I'd
> share.  The script should make it clear, but when attempting to take a url
> string and test to see if it's a valid address, the output from to_addr
> creates a 'valid' ipv6 address.
> >
> > Is that a requirement for some reason internally?
>
> to_addr() returning the unspecified IPv6 address on failure to convert an
> IP string to Bro?s address type is just an arbitrary choice.
> Alternatively, it could return the unspecified IPv4 address, 0.0.0.0, but
> that doesn?t really save anything ? internally Bro?s address values all use
> a full 128 bits (IPv4 uses the "IPv4-mapped IPv6? representation).  It
> could also return a record type:
>
>         type opt_addr: record { a: addr &optional; };
>
> or
>
>         type opt_addr: record { a: addr; success: bool; };
>
> Where in the first, it only sets the field if the conversion succeeded,
> but failure to check for that fields existence before accessing is
> potentially more problematic than failure to check for [::].   In either,
> it?s adding another data type the user has to remember or lookup how to use.
>
> So that?s the backstory of why [::] is the failure indicator.  One could
> also argue that using the unspecified IPv6 address (or IPv4) as a return
> value makes it ambiguous to try to parse ?::? (or ?0.0.0.0?) as the input
> string and I?d be on board w/ that and vote to switch to one of the
> return-a-record styles.
>
> Anyway, from the example you gave, did you just mean to use
> ?lookup_hostname? instead of ?to_addr? ?
>
> - Jon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150401/85683cb0/attachment.html 

From gfaulkner.nsm at gmail.com  Wed Apr  1 17:25:59 2015
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Wed, 01 Apr 2015 19:25:59 -0500
Subject: [Bro] Collecting and Graphing Various Bro Statistics
Message-ID: <551C8C97.2080108@gmail.com>

I'm looking to update some log and event management front-end 
infrastructure and as many of you know properly sizing them requires 
getting a handle on event rates, uncompressed log sizes etc. I seem to 
recall someone was working on a plugin to instrument the sending of Bro 
stats to some combination of statsd(not sure which version), Graphite 
and Grafana. I don't think the presentation ever got posted, so I don't 
know if this was just conceptual or actually in use somewhere, but I'm 
curious how far this got and if it would be a good fit for doing things 
like measuring event logging rates for the various Bro log files, and 
possibly file sizes over time? Any other tidbits about architecture, 
event pipeline, interesting use cases etc would be a bonus. If folks 
have any other automated ways they are doing this I'd be curious as well.

~Gary

From hhoffman at ip-solutions.net  Wed Apr  1 17:52:14 2015
From: hhoffman at ip-solutions.net (Harry Hoffman)
Date: Wed, 01 Apr 2015 20:52:14 -0400
Subject: [Bro] To proxy or not to proxy...
Message-ID: <20150402005219.57C4644E18@pb-smtp1.pobox.com>

Hi folks,

So in my continuing pursuit of perfecting my Bro setup I found that adding a proxy on every box that also runs workers keeps bro much happier then a single manager/proxy box with one or more worker(s) boxes.

Prior to adding the additional proxies bro workers would die due to resource constraints.

Are other folks doing this?

Cheers,
Harry




From franky.meier.1 at gmx.de  Thu Apr  2 00:23:36 2015
From: franky.meier.1 at gmx.de (Frank Meier)
Date: Thu, 02 Apr 2015 09:23:36 +0200
Subject: [Bro] http incomplete file extraction (Files::ANALYZER_EXTRACT)
 / solved
In-Reply-To: <CB1A4F82-989B-4021-9810-C4633CEE3606@illinois.edu>
References: <trinity-9655f832-a2a5-4afe-9ffa-746c4a6672ae-1427463329271@3capp-gmx-bs42>
	<,>
Message-ID: <1427959416.26929.0@mail.gmx.net>

Thanks to all who answered!

The -C switch did the trick. Sometimes warnings should be taken 
seriously...

Have a nice day!

Franky


On Mi, Apr 1, 2015 at 5:26 , Siwek, Jon <jsiwek at illinois.edu> wrote:
> 
>>  ~/bro-liste$ /usr/local/bro/bin/bro -r download.pcap extract.bro 
>>  1427874309.892545 warning in 
>> /usr/local/bro/share/bro/base/misc/find-checksum-offloading.bro, 
>> line 54: Your trace file likely has invalid TCP checksums, most 
>> likely from NIC checksum offloading.
> 
> You?ll have to address this problem to get the results you expect.  
> See:
> 
> https://www.bro.org/documentation/faq.html#why-isn-t-bro-producing-the-logs-i-expect-a-note-about-checksums
> 
>>  The weird.log states some ?above_hole_data_without_any_acks"
> 
> In this case, this seems like it?s just a side effect of the bad 
> checksums, but in case you?re interested on how that type of 
> situation can effect file extraction in Bro there?s discussion of 
> how/why here:
> 
> https://bro-tracker.atlassian.net/browse/BIT-1255
> 
> - Jon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150402/6ddb25b5/attachment.html 

From gfaulkner.nsm at gmail.com  Thu Apr  2 09:24:31 2015
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Thu, 02 Apr 2015 11:24:31 -0500
Subject: [Bro] To proxy or not to proxy...
In-Reply-To: <20150402005219.57C4644E18@pb-smtp1.pobox.com>
References: <20150402005219.57C4644E18@pb-smtp1.pobox.com>
Message-ID: <551D6D3F.1090109@gmail.com>

I'm currently running a separate box that has the manager and proxies on 
it, but I did just as you describe at one point and it seemed to work 
fairly well. You may want to reduce your worker count a bit to leave 
enough CPUs for the proxies. Out of curiosity are you pinning your 
workers to dedicated CPU cores? If you are not it could be that your 
workers are bouncing between cores due to hyper-threading which can 
cause them to stomp all over each other. I found pinning workers to 
cores helped tremendously when it came to worker health.

~Gary

On 4/1/2015 7:52 PM, Harry Hoffman wrote:
> Hi folks,
>
> So in my continuing pursuit of perfecting my Bro setup I found that adding a proxy on every box that also runs workers keeps bro much happier then a single manager/proxy box with one or more worker(s) boxes.
>
> Prior to adding the additional proxies bro workers would die due to resource constraints.
>
> Are other folks doing this?
>
> Cheers,
> Harry
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From asharma at lbl.gov  Thu Apr  2 10:17:11 2015
From: asharma at lbl.gov (Aashish Sharma)
Date: Thu, 2 Apr 2015 10:17:11 -0700
Subject: [Bro] To proxy or not to proxy...
In-Reply-To: <551D6D3F.1090109@gmail.com>
References: <20150402005219.57C4644E18@pb-smtp1.pobox.com>
	<551D6D3F.1090109@gmail.com>
Message-ID: <20150402171710.GB14690@yaksha.lbl.gov>

Same here: I have a proxy for every 10 workers on each of the physical box (which runs workers) in the cluster.

Ah! regarding CPU pinning: 

> fairly well. You may want to reduce your worker count a bit to leave 
> enough CPUs for the proxies. Out of curiosity are you pinning your 
> workers to dedicated CPU cores? If you are not it could be that your 
> workers are bouncing between cores due to hyper-threading which can 
> cause them to stomp all over each other. I found pinning workers to 
> cores helped tremendously when it came to worker health.

I agree completely! 

Also, Make sure that you have enough cores to run workers on. 

With respect to CPU pinning, on *FreeBSD*, CPUs are numbered as :
P = physical core 
H = Hyperthread core

0/1 = P/H 
2/3 = P/H 
4/5 = P/H 
...
...
11/12=P/H

You certainly don't want to pin_cpu on FreeBSD as 0,1,2,3 but instead pin_cpu=0,2,4,6,8.... (or 1,3,5,7...)

However, I beleive Linux does it different. While I have not yet looked at a Linux's box, I believe its scheme for hex-core processor is 

0/6=P/H
1/7=P/H
2/8=P/H
..
..
5/12=P/H

so you might want to pin_cpu on linux as: pin_cpu=01,2,3,4,5 or (6,7,8,9,10,11,12)

Make sure you leave a few cores alone for proxy and other tasks when pinning. 

Oh, btw, we have found no noticible difference in performance at all, when you pin a bro process on only physical core vs only hyperthreded cores. But make sure you don't pin bro processes on both P/H at the same time.

Now, it would be great if someone can confirm the linux side of the story. or shed more light on cpu_pinning. 




Aashish 


On Thu, Apr 02, 2015 at 11:24:31AM -0500, Gary Faulkner wrote:
> I'm currently running a separate box that has the manager and proxies on 
> it, but I did just as you describe at one point and it seemed to work 
> fairly well. You may want to reduce your worker count a bit to leave 
> enough CPUs for the proxies. Out of curiosity are you pinning your 
> workers to dedicated CPU cores? If you are not it could be that your 
> workers are bouncing between cores due to hyper-threading which can 
> cause them to stomp all over each other. I found pinning workers to 
> cores helped tremendously when it came to worker health.
> 
> ~Gary
> 
> On 4/1/2015 7:52 PM, Harry Hoffman wrote:
> > Hi folks,
> >
> > So in my continuing pursuit of perfecting my Bro setup I found that adding a proxy on every box that also runs workers keeps bro much happier then a single manager/proxy box with one or more worker(s) boxes.
> >
> > Prior to adding the additional proxies bro workers would die due to resource constraints.
> >
> > Are other folks doing this?
> >
> > Cheers,
> > Harry
> >
> >
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-- 
Aashish Sharma	(asharma at lbl.gov) 				 
Cyber Security, 
Lawrence Berkeley National Laboratory  
http://go.lbl.gov/pgp-aashish 
Office: (510)-495-2680  Cell: (510)-612-7971

From apumphrey at ivsec.com  Thu Apr  2 14:22:50 2015
From: apumphrey at ivsec.com (Adam Pumphrey)
Date: Thu, 2 Apr 2015 17:22:50 -0400
Subject: [Bro] To proxy or not to proxy...
In-Reply-To: <20150402171710.GB14690@yaksha.lbl.gov>
References: <20150402005219.57C4644E18@pb-smtp1.pobox.com>
	<551D6D3F.1090109@gmail.com>
	<20150402171710.GB14690@yaksha.lbl.gov>
Message-ID: <1CBA2E4A-12A9-4DAA-83D5-F11D5250806E@ivsec.com>

Great discussion and pointers.  I?m working on a similar performance tuning and stabilization effort.  

I took a closer look to verify and I can confirm Aashish?s statement about the numbering of cores on Linux.  We?re running CENTOS 6.2.  This box has 2 hyperthreaded hex-core procs.   All physical cores are assigned sequential ID?s in socket/core order, then hyperthreaded cores are assigned sequential ID?s in socket/core order.   

Here?s what we end up with:

__socket0 (P/H)__
0/12
1/13
2/14
3/15
4/16
5/17

__socket1 (P/H)__
6/18
7/19
8/20
9/21
10/22
11/23

Adam


> On Apr 2, 2015, at 1:17 PM, Aashish Sharma <asharma at lbl.gov> wrote:
> 
> Same here: I have a proxy for every 10 workers on each of the physical box (which runs workers) in the cluster.
> 
> Ah! regarding CPU pinning: 
> 
>> fairly well. You may want to reduce your worker count a bit to leave 
>> enough CPUs for the proxies. Out of curiosity are you pinning your 
>> workers to dedicated CPU cores? If you are not it could be that your 
>> workers are bouncing between cores due to hyper-threading which can 
>> cause them to stomp all over each other. I found pinning workers to 
>> cores helped tremendously when it came to worker health.
> 
> I agree completely! 
> 
> Also, Make sure that you have enough cores to run workers on. 
> 
> With respect to CPU pinning, on *FreeBSD*, CPUs are numbered as :
> P = physical core 
> H = Hyperthread core
> 
> 0/1 = P/H 
> 2/3 = P/H 
> 4/5 = P/H 
> ...
> ...
> 11/12=P/H
> 
> You certainly don't want to pin_cpu on FreeBSD as 0,1,2,3 but instead pin_cpu=0,2,4,6,8.... (or 1,3,5,7...)
> 
> However, I beleive Linux does it different. While I have not yet looked at a Linux's box, I believe its scheme for hex-core processor is 
> 
> 0/6=P/H
> 1/7=P/H
> 2/8=P/H
> ..
> ..
> 5/12=P/H
> 
> so you might want to pin_cpu on linux as: pin_cpu=01,2,3,4,5 or (6,7,8,9,10,11,12)
> 
> Make sure you leave a few cores alone for proxy and other tasks when pinning. 
> 
> Oh, btw, we have found no noticible difference in performance at all, when you pin a bro process on only physical core vs only hyperthreded cores. But make sure you don't pin bro processes on both P/H at the same time.
> 
> Now, it would be great if someone can confirm the linux side of the story. or shed more light on cpu_pinning. 
> 
> 
> 
> 
> Aashish 
> 
> 
> On Thu, Apr 02, 2015 at 11:24:31AM -0500, Gary Faulkner wrote:
>> I'm currently running a separate box that has the manager and proxies on 
>> it, but I did just as you describe at one point and it seemed to work 
>> fairly well. You may want to reduce your worker count a bit to leave 
>> enough CPUs for the proxies. Out of curiosity are you pinning your 
>> workers to dedicated CPU cores? If you are not it could be that your 
>> workers are bouncing between cores due to hyper-threading which can 
>> cause them to stomp all over each other. I found pinning workers to 
>> cores helped tremendously when it came to worker health.
>> 
>> ~Gary
>> 
>> On 4/1/2015 7:52 PM, Harry Hoffman wrote:
>>> Hi folks,
>>> 
>>> So in my continuing pursuit of perfecting my Bro setup I found that adding a proxy on every box that also runs workers keeps bro much happier then a single manager/proxy box with one or more worker(s) boxes.
>>> 
>>> Prior to adding the additional proxies bro workers would die due to resource constraints.
>>> 
>>> Are other folks doing this?
>>> 
>>> Cheers,
>>> Harry
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> 
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> -- 
> Aashish Sharma	(asharma at lbl.gov) 				 
> Cyber Security, 
> Lawrence Berkeley National Laboratory  
> http://go.lbl.gov/pgp-aashish 
> Office: (510)-495-2680  Cell: (510)-612-7971
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From hhoffman at ip-solutions.net  Thu Apr  2 14:42:27 2015
From: hhoffman at ip-solutions.net (Harry Hoffman)
Date: Thu, 02 Apr 2015 17:42:27 -0400
Subject: [Bro] To proxy or not to proxy...
Message-ID: <20150402214234.174D143B3D@pb-smtp1.pobox.com>

So, I disable hyperthreading by default and yes, I leave a the CPUs for workers and CPU for proxy.

It's a fairly remarkable difference.

Cheers,
HarryOn Apr 2, 2015 5:22 PM, Adam Pumphrey <apumphrey at ivsec.com> wrote:
>
> Great discussion and pointers.? I?m working on a similar performance tuning and stabilization effort.?
>
> I took a closer look to verify and I can confirm Aashish?s statement about the numbering of cores on Linux.? We?re running CENTOS 6.2.? This box has 2 hyperthreaded hex-core procs.?? All physical cores are assigned sequential ID?s in socket/core order, then hyperthreaded cores are assigned sequential ID?s in socket/core order.??
>
> Here?s what we end up with:
>
> __socket0 (P/H)__
> 0/12
> 1/13
> 2/14
> 3/15
> 4/16
> 5/17
>
> __socket1 (P/H)__
> 6/18
> 7/19
> 8/20
> 9/21
> 10/22
> 11/23
>
> Adam
>
> > On Apr 2, 2015, at 1:17 PM, Aashish Sharma <asharma at lbl.gov> wrote:
> > 
> > Same here: I have a proxy for every 10 workers on each of the physical box (which runs workers) in the cluster.
> > 
> > Ah! regarding CPU pinning: 
> > 
> >> fairly well. You may want to reduce your worker count a bit to leave 
> >> enough CPUs for the proxies. Out of curiosity are you pinning your 
> >> workers to dedicated CPU cores? If you are not it could be that your 
> >> workers are bouncing between cores due to hyper-threading which can 
> >> cause them to stomp all over each other. I found pinning workers to 
> >> cores helped tremendously when it came to worker health.
> > 
> > I agree completely! 
> > 
> > Also, Make sure that you have enough cores to run workers on. 
> > 
> > With respect to CPU pinning, on *FreeBSD*, CPUs are numbered as :
> > P = physical core 
> > H = Hyperthread core
> > 
> > 0/1 = P/H 
> > 2/3 = P/H 
> > 4/5 = P/H 
> > ...
> > ...
> > 11/12=P/H
> > 
> > You certainly don't want to pin_cpu on FreeBSD as 0,1,2,3 but instead pin_cpu=0,2,4,6,8.... (or 1,3,5,7...)
> > 
> > However, I beleive Linux does it different. While I have not yet looked at a Linux's box, I believe its scheme for hex-core processor is 
> > 
> > 0/6=P/H
> > 1/7=P/H
> > 2/8=P/H
> > ..
> > ..
> > 5/12=P/H
> > 
> > so you might want to pin_cpu on linux as: pin_cpu=01,2,3,4,5 or (6,7,8,9,10,11,12)
> > 
> > Make sure you leave a few cores alone for proxy and other tasks when pinning. 
> > 
> > Oh, btw, we have found no noticible difference in performance at all, when you pin a bro process on only physical core vs only hyperthreded cores. But make sure you don't pin bro processes on both P/H at the same time.
> > 
> > Now, it would be great if someone can confirm the linux side of the story. or shed more light on cpu_pinning. 
> > 
> > 
> > 
> > 
> > Aashish 
> > 
> > 
> > On Thu, Apr 02, 2015 at 11:24:31AM -0500, Gary Faulkner wrote:
> >> I'm currently running a separate box that has the manager and proxies on 
> >> it, but I did just as you describe at one point and it seemed to work 
> >> fairly well. You may want to reduce your worker count a bit to leave 
> >> enough CPUs for the proxies. Out of curiosity are you pinning your 
> >> workers to dedicated CPU cores? If you are not it could be that your 
> >> workers are bouncing between cores due to hyper-threading which can 
> >> cause them to stomp all over each other. I found pinning workers to 
> >> cores helped tremendously when it came to worker health.
> >> 
> >> ~Gary
> >> 
> >> On 4/1/2015 7:52 PM, Harry Hoffman wrote:
> >>> Hi folks,
> >>> 
> >>> So in my continuing pursuit of perfecting my Bro setup I found that adding a proxy on every box that also runs workers keeps bro much happier then a single manager/proxy box with one or more worker(s) boxes.
> >>> 
> >>> Prior to adding the additional proxies bro workers would die due to resource constraints.
> >>> 
> >>> Are other folks doing this?
> >>> 
> >>> Cheers,
> >>> Harry
> >>> 
> >>> 
> >>> 
> >>> _______________________________________________
> >>> Bro mailing list
> >>> bro at bro-ids.org
> >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >> 
> >> _______________________________________________
> >> Bro mailing list
> >> bro at bro-ids.org
> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> > 
> > -- 
> > Aashish Sharma (asharma at lbl.gov) 
> > Cyber Security, 
> > Lawrence Berkeley National Laboratory? 
> > http://go.lbl.gov/pgp-aashish 
> > Office: (510)-495-2680? Cell: (510)-612-7971
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From earl.eiland at root9b.com  Thu Apr  2 14:52:46 2015
From: earl.eiland at root9b.com (Earl Eiland)
Date: Thu, 2 Apr 2015 21:52:46 +0000
Subject: [Bro] 10 Gb capable Bro cluster
Message-ID: <CY1PR0401MB13872D25D729E53BBAC618BED7F20@CY1PR0401MB1387.namprd04.prod.outlook.com>

Has anyone built a Bro cluster capable of processing packets at 10Gb/sec?  If so, what hardware did you use?

Best Regards,

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity named.  If you are not the named addressee you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.  Please notify the sender immediately by email if you received this email in error and delete this email from your system. Any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of root9B LLC.?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150402/04b2b739/attachment.html 

From seth at icir.org  Thu Apr  2 21:21:07 2015
From: seth at icir.org (Seth Hall)
Date: Fri, 3 Apr 2015 00:21:07 -0400
Subject: [Bro] Does files.log generator recognize application/json file
	types?
In-Reply-To: <551BF809.7020406@gmail.com>
References: <551BF809.7020406@gmail.com>
Message-ID: <0DBC0963-490C-4A8B-B964-F7234D438847@icir.org>


> On Apr 1, 2015, at 9:52 AM, Tim Molter <tim.molter at gmail.com> wrote:
> 
> Does `files.log` generator recognize application/json file types, and if
> not, what could I modify to get that to work? What does the generator
> key into to make the decision?

It does not.  Unfortunately JSON detection would be pretty difficult and fraught with false positives.  If you have a regex that you?d like to suggest for matching JSON I?m sure some people could try it and we could find out if it?s resilient enough.

With Bro 2.3+ you can write your own file signatures that you can load in to try and sniff file mime types.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From gediminas.margis at gmail.com  Fri Apr  3 01:18:57 2015
From: gediminas.margis at gmail.com (Gediminas Margis)
Date: Fri, 03 Apr 2015 11:18:57 +0300
Subject: [Bro] HTTP traffic logging
Message-ID: <551E4CF1.7050005@gmail.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
 
Hello,

I am interested in logging full* HTTP traffic content into log
files/SIEM solution for inspection on later date.

Scenario would be to parse plaintext/decrypted HTTP traffic with Bro and
store source/dest, uri, POST/GET data values. This is for historical
search for malicious content on later date in the SIEM solution.

Critical parts are src, dst, URI, POST/GET data that is submitted.

I am currently going through Bro documentation but cant find any info on
how can I do this. I am looking at
https://www.bro.org/sphinx/scripts/base/bif/plugins/Bro_HTTP.events.bif.bro.html

As I understand the content of POST data is stored in HTTP request so I
would need to use http_request or http_entity_data.
Also I am pretty new to Bro so I'm not even sure how to start with
this.My end goal would be to have a log that looks something like this:

timestamp, method, src_ip, src_port, dst_ip, dst_port, uri,
data(GET/POST, key value pairs like name=mike&occupation=driver).

- -- 
Best Regards,

Gediminas Margis,

PGP Key-ID: 0xE6D92FE2FA3AD133
<http://keyserver1.pgp.com/vkd/DownloadKey.event?keyid=0xE6D92FE2FA3AD133>
77BD 9F67 F1CF 72B0 7273 E086 E6D9 2FE2 FA3A D133
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
 
iQIcBAEBCgAGBQJVHkzqAAoJEObZL+L6OtEzBooP/jwqf91ZTPMkdIHneR9+ZzXr
fes1p12cnDtTYMMXz2fJ+lw/Ne1JJ7HaAyK0xvRykzXUi7JmFmTA5YXEkRL8sAwA
Wf4y5/E3ER/QTkuCUaOEKEnustqkcnDdp3HPRuXCgbGUQGRch53FeFDHpYDvsEPh
84pVT8/hGJzuR92iUJePf1rdCL0FVp0Pak2yN73UBFepmdV+IvVaGx/dfL66UJ7X
gYzaqBDnKCoiU4tITc+s93gsQgPZOBsVKq6krb/nIXElkWQRn9CuLf/43G1ik8JB
SIVs0ZEdTchccfs5iG7wvE1xoyEkX+/+e+gPR+1mEN0jh8OkyJzeaCC6r4Ne44mf
kxHwyuTUF48pvtPQ9iyCOZqqUd4StZ1NzpIO+99hNkHZxYKEpOVccQp9UTiTQTnR
d8lvTJ4J9kQHyvnFMNIduHGmrkIDkkah7ayd3d5LbD+9W0V8G4VZmx3tVUAf/3mO
cIpmQNY0dnA3/XPbPLBYRgb175g3CsL2q04S/NJXVh1RIKIwv/CgsewVEOhfrr5V
IQJsU22B0cWplyfvjOrfrb1iyQxIMVqnHpNc76JgB/lSqhDD3Nba/D56k+ly6/U2
roalFw8umZDSMR8nPoN/nX/I8rWM0ReIqTI5who9Nulj3TFmVqdy9JYKFCJiFVbn
MOSZJ/MqYtRNsn3eq0eN
=4/gH
-----END PGP SIGNATURE-----

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150403/67937192/attachment-0001.html 

From hosom at battelle.org  Fri Apr  3 04:32:33 2015
From: hosom at battelle.org (Hosom, Stephen M)
Date: Fri, 3 Apr 2015 11:32:33 +0000
Subject: [Bro] HTTP traffic logging
In-Reply-To: <551E4CF1.7050005@gmail.com>
References: <551E4CF1.7050005@gmail.com>
Message-ID: <E5A2071C6FFFF640B714C224139C9B3301A59299@WP-MBX2B.milky-way.battelle.org>

Gediminas,

The folks at Broala have written a script that logs POST data. I think this does most of what you?re looking for:

https://github.com/broala/bro-snippets/blob/master/http-add-post-bodies.bro


From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Gediminas Margis
Sent: Friday, April 03, 2015 4:19 AM
To: bro at bro.org
Subject: [Bro] HTTP traffic logging


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hello,

I am interested in logging full* HTTP traffic content into log files/SIEM solution for inspection on later date.

Scenario would be to parse plaintext/decrypted HTTP traffic with Bro and store source/dest, uri, POST/GET data values. This is for historical search for malicious content on later date in the SIEM solution.

Critical parts are src, dst, URI, POST/GET data that is submitted.

I am currently going through Bro documentation but cant find any info on how can I do this. I am looking at https://www.bro.org/sphinx/scripts/base/bif/plugins/Bro_HTTP.events.bif.bro.html

As I understand the content of POST data is stored in HTTP request so I would need to use http_request or http_entity_data.
Also I am pretty new to Bro so I'm not even sure how to start with this.My end goal would be to have a log that looks something like this:

timestamp, method, src_ip, src_port, dst_ip, dst_port, uri, data(GET/POST, key value pairs like name=mike&occupation=driver).

- --
Best Regards,

Gediminas Margis,

PGP Key-ID: 0xE6D92FE2FA3AD133 <http://keyserver1.pgp.com/vkd/DownloadKey.event?keyid=0xE6D92FE2FA3AD133><http://keyserver1.pgp.com/vkd/DownloadKey.event?keyid=0xE6D92FE2FA3AD133>
77BD 9F67 F1CF 72B0 7273 E086 E6D9 2FE2 FA3A D133
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJVHkzqAAoJEObZL+L6OtEzBooP/jwqf91ZTPMkdIHneR9+ZzXr
fes1p12cnDtTYMMXz2fJ+lw/Ne1JJ7HaAyK0xvRykzXUi7JmFmTA5YXEkRL8sAwA
Wf4y5/E3ER/QTkuCUaOEKEnustqkcnDdp3HPRuXCgbGUQGRch53FeFDHpYDvsEPh
84pVT8/hGJzuR92iUJePf1rdCL0FVp0Pak2yN73UBFepmdV+IvVaGx/dfL66UJ7X
gYzaqBDnKCoiU4tITc+s93gsQgPZOBsVKq6krb/nIXElkWQRn9CuLf/43G1ik8JB
SIVs0ZEdTchccfs5iG7wvE1xoyEkX+/+e+gPR+1mEN0jh8OkyJzeaCC6r4Ne44mf
kxHwyuTUF48pvtPQ9iyCOZqqUd4StZ1NzpIO+99hNkHZxYKEpOVccQp9UTiTQTnR
d8lvTJ4J9kQHyvnFMNIduHGmrkIDkkah7ayd3d5LbD+9W0V8G4VZmx3tVUAf/3mO
cIpmQNY0dnA3/XPbPLBYRgb175g3CsL2q04S/NJXVh1RIKIwv/CgsewVEOhfrr5V
IQJsU22B0cWplyfvjOrfrb1iyQxIMVqnHpNc76JgB/lSqhDD3Nba/D56k+ly6/U2
roalFw8umZDSMR8nPoN/nX/I8rWM0ReIqTI5who9Nulj3TFmVqdy9JYKFCJiFVbn
MOSZJ/MqYtRNsn3eq0eN
=4/gH
-----END PGP SIGNATURE-----
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150403/a0967b41/attachment.html 

From dj.root at netronome.com  Fri Apr  3 06:32:51 2015
From: dj.root at netronome.com (DJ Root)
Date: Fri, 3 Apr 2015 09:32:51 -0400
Subject: [Bro] 10 Gb capable Bro cluster
In-Reply-To: <CY1PR0401MB13872D25D729E53BBAC618BED7F20@CY1PR0401MB1387.namprd04.prod.outlook.com>
References: <CY1PR0401MB13872D25D729E53BBAC618BED7F20@CY1PR0401MB1387.namprd04.prod.outlook.com>
Message-ID: <DEAA8C4B-27EF-4F4F-BDD3-AD891F737181@netronome.com>

Hi Earl,

Yes, we have built a cluster capable of processing 10Gbps.  We have this running in our lab and also have reference-able customers.

Hardware:
 - Dual socket Ivy Bridge X86 with cores running at 2.8GHZ
-  Netronome PCIe acceleration technology 

2U appliance is capable of 8.5 - 9.0Gbps Bro processing in steady state, peak 20Gbps+ with few packet drops (~2%).  Our 4U cluster can process 17-18Gbps of Bro steady state, peak 40-45Gbps.

Resources used:  32 worker threads on each 2U appliance / 4 threads for NIC management / 12 are unused and can be utilized for other applications.  Bro is not modified.

Let me know if you need anything else.

Thanks,
DJ Root
Netronome, Inc.
(617)686-0253

> On Apr 2, 2015, at 5:52 PM, Earl Eiland <earl.eiland at root9b.com> wrote:
> 
> Has anyone built a Bro cluster capable of processing packets at 10Gb/sec?  If so, what hardware did you use?
>  
> Best Regards,
>  
> Earl Eiland,
> Sr. Cyber Security Engineer,
> Emerging Technologies, root9B,
> San Antonio, Texas
>  
> This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity named.  If you are not the named addressee you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.  Please notify the sender immediately by email if you received this email in error and delete this email from your system. Any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of root9B LLC.?
>  
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org <mailto:bro at bro-ids.org>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro <http://mailman.icsi.berkeley.edu/mailman/listinfo/bro>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150403/9b21b6ba/attachment.html 

From anthony.kasza at gmail.com  Fri Apr  3 08:14:00 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Fri, 3 Apr 2015 08:14:00 -0700
Subject: [Bro] HTTP traffic logging
In-Reply-To: <E5A2071C6FFFF640B714C224139C9B3301A59299@WP-MBX2B.milky-way.battelle.org>
References: <551E4CF1.7050005@gmail.com>
	<E5A2071C6FFFF640B714C224139C9B3301A59299@WP-MBX2B.milky-way.battelle.org>
Message-ID: <CAEZw2bxxzKdkAv2t4XFJABZkMkOvuc=EzWT_LVXM9Lojxd+3bg@mail.gmail.com>

You might want to consider adding additional logic to that script to
selectively log POST bodies. Depending on your environment POST can get big.

-AK
On Apr 3, 2015 4:36 AM, "Hosom, Stephen M" <hosom at battelle.org> wrote:

>  Gediminas,
>
>
>
> The folks at Broala have written a script that logs POST data. I think
> this does most of what you?re looking for:
>
>
>
> https://github.com/broala/bro-snippets/blob/master/http-add-post-bodies.bro
>
>
>
>
>
> *From:* bro-bounces at bro.org [mailto:bro-bounces at bro.org] *On Behalf Of *Gediminas
> Margis
> *Sent:* Friday, April 03, 2015 4:19 AM
> *To:* bro at bro.org
> *Subject:* [Bro] HTTP traffic logging
>
>
>
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> Hello,
>
> I am interested in logging full* HTTP traffic content into log files/SIEM
> solution for inspection on later date.
>
> Scenario would be to parse plaintext/decrypted HTTP traffic with Bro and
> store source/dest, uri, POST/GET data values. This is for historical search
> for malicious content on later date in the SIEM solution.
>
> Critical parts are src, dst, URI, POST/GET data that is submitted.
>
> I am currently going through Bro documentation but cant find any info on
> how can I do this. I am looking at
> https://www.bro.org/sphinx/scripts/base/bif/plugins/Bro_HTTP.events.bif.bro.html
>
> As I understand the content of POST data is stored in HTTP request so I
> would need to use http_request or http_entity_data.
> Also I am pretty new to Bro so I'm not even sure how to start with this.My
> end goal would be to have a log that looks something like this:
>
> timestamp, method, src_ip, src_port, dst_ip, dst_port, uri, data(GET/POST,
> key value pairs like name=mike&occupation=driver).
>
> - --
> Best Regards,
>
> Gediminas Margis,
>
> PGP Key-ID: 0xE6D92FE2FA3AD133
> <http://keyserver1.pgp.com/vkd/DownloadKey.event?keyid=0xE6D92FE2FA3AD133>
> <http://keyserver1.pgp.com/vkd/DownloadKey.event?keyid=0xE6D92FE2FA3AD133>
> 77BD 9F67 F1CF 72B0 7273 E086 E6D9 2FE2 FA3A D133
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQIcBAEBCgAGBQJVHkzqAAoJEObZL+L6OtEzBooP/jwqf91ZTPMkdIHneR9+ZzXr
> fes1p12cnDtTYMMXz2fJ+lw/Ne1JJ7HaAyK0xvRykzXUi7JmFmTA5YXEkRL8sAwA
> Wf4y5/E3ER/QTkuCUaOEKEnustqkcnDdp3HPRuXCgbGUQGRch53FeFDHpYDvsEPh
> 84pVT8/hGJzuR92iUJePf1rdCL0FVp0Pak2yN73UBFepmdV+IvVaGx/dfL66UJ7X
> gYzaqBDnKCoiU4tITc+s93gsQgPZOBsVKq6krb/nIXElkWQRn9CuLf/43G1ik8JB
> SIVs0ZEdTchccfs5iG7wvE1xoyEkX+/+e+gPR+1mEN0jh8OkyJzeaCC6r4Ne44mf
> kxHwyuTUF48pvtPQ9iyCOZqqUd4StZ1NzpIO+99hNkHZxYKEpOVccQp9UTiTQTnR
> d8lvTJ4J9kQHyvnFMNIduHGmrkIDkkah7ayd3d5LbD+9W0V8G4VZmx3tVUAf/3mO
> cIpmQNY0dnA3/XPbPLBYRgb175g3CsL2q04S/NJXVh1RIKIwv/CgsewVEOhfrr5V
> IQJsU22B0cWplyfvjOrfrb1iyQxIMVqnHpNc76JgB/lSqhDD3Nba/D56k+ly6/U2
> roalFw8umZDSMR8nPoN/nX/I8rWM0ReIqTI5who9Nulj3TFmVqdy9JYKFCJiFVbn
> MOSZJ/MqYtRNsn3eq0eN
> =4/gH
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150403/0aab6197/attachment-0001.html 

From storm at tacc.utexas.edu  Fri Apr  3 08:40:56 2015
From: storm at tacc.utexas.edu (Patrick Storm)
Date: Fri, 3 Apr 2015 15:40:56 +0000
Subject: [Bro] 10 Gb capable Bro cluster
Message-ID: <52738DB1-0486-4C1E-BED2-93BD1B3C8D18@tacc.utexas.edu>

Earl,

We have a cluster that is capable of processing 10Gb/sec.

3x worker nodes:
- Dell R730
- Dual E5-2690 v3 CPUs
- 128Gb of RAM
- 1x Myricom card with Sniffer license ( 10G-PCIE2-8C2-2S+SNF3)

We run a proxy on our master node, and on each of the worker nodes. On each of the worker nodes we run 22 worker threads.

We found that setting a 32Gb buffer for Myricom to use does a lot to help out with bursts when Bro can?t keep up.

Let me know if you have any other questions.

Patrick Storm
Texas Advanced Computing Center ? UT Austin


From: Earl Eiland
Date: Thursday, April 2, 2015 at 4:52 PM
To: "bro at bro.org<mailto:bro at bro.org>"
Subject: [Bro] 10 Gb capable Bro cluster

Has anyone built a Bro cluster capable of processing packets at 10Gb/sec?  If so, what hardware did you use?

Best Regards,

Earl Eiland,
Sr. Cyber Security Engineer,
Emerging Technologies, root9B,
San Antonio, Texas

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity named.  If you are not the named addressee you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.  Please notify the sender immediately by email if you received this email in error and delete this email from your system. Any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of root9B LLC.?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150403/9e4b5aca/attachment.html 

From jtaylor1024 at yahoo.com  Wed Apr  8 12:07:22 2015
From: jtaylor1024 at yahoo.com (Jerome Taylor)
Date: Wed, 8 Apr 2015 19:07:22 +0000 (UTC)
Subject: [Bro] Error: cannot determine Bro version
Message-ID: <677744488.2675371.1428520042383.JavaMail.yahoo@mail.yahoo.com>

Hello all,
I just completed a new Bro installation. I receive the following error message when I try to start Bro (i.e. "broctl | install").
????Error: cannot determine Bro version
My Setup:=======
bro version 2.3-680
minimum configuration => (a single Bro instance on the localhost)
Note: "bro -v" will return the version number but it also produces a segmentation fault..

I do not see this issue in older releases of Bro. Is there a known issue with this latest release?
?Regards,
Jerome Taylor
M: 978-764-1269
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150408/30a97176/attachment.html 

From asharma at lbl.gov  Wed Apr  8 12:37:42 2015
From: asharma at lbl.gov (Aashish Sharma)
Date: Wed, 8 Apr 2015 12:37:42 -0700
Subject: [Bro] Error: cannot determine Bro version
In-Reply-To: <677744488.2675371.1428520042383.JavaMail.yahoo@mail.yahoo.com>
References: <677744488.2675371.1428520042383.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <20150408193740.GA52542@Aashishs-Mac-Pro.local>

I think you should try make && make install again! 

Lets see if the error still stays! 

On Wed, Apr 08, 2015 at 07:07:22PM +0000, Jerome Taylor wrote:
> Hello all,
> I just completed a new Bro installation. I receive the following error message when I try to start Bro (i.e. "broctl | install").
> ????Error: cannot determine Bro version
> My Setup:=======
> bro version 2.3-680
> minimum configuration => (a single Bro instance on the localhost)
> Note: "bro -v" will return the version number but it also produces a segmentation fault..
> 
> I do not see this issue in older releases of Bro. Is there a known issue with this latest release?
> ?Regards,
> Jerome Taylor
> M: 978-764-1269

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From jtaylor1024 at yahoo.com  Wed Apr  8 12:59:03 2015
From: jtaylor1024 at yahoo.com (Jerome Taylor)
Date: Wed, 8 Apr 2015 19:59:03 +0000 (UTC)
Subject: [Bro] Error: cannot determine Bro version
In-Reply-To: <20150408193740.GA52542@Aashishs-Mac-Pro.local>
References: <20150408193740.GA52542@Aashishs-Mac-Pro.local>
Message-ID: <233004250.2725349.1428523143179.JavaMail.yahoo@mail.yahoo.com>

Re-running "make; make install" does not fix the problem. The error remains..
In what I believe is an unrelated issue, I notice that the file "/usr/local/bro/spool/broctl-config.sh" does not exist; which in turn causes a broken link for "/usr/local/bro/share/broctl/scripts/broctl-config.sh". I temporarily worked around this problem by importing "/usr/local/bro/spool/broctl-config.sh" from a remote serer that was similarly configured. I mention this hoping that it might point to an underlying issue with the build.
?Regards,
Jerome Taylor

 


     On Wednesday, April 8, 2015 3:37 PM, Aashish Sharma <asharma at lbl.gov> wrote:
   

 I think you should try make && make install again! 

Lets see if the error still stays! 

On Wed, Apr 08, 2015 at 07:07:22PM +0000, Jerome Taylor wrote:
> Hello all,
> I just completed a new Bro installation. I receive the following error message when I try to start Bro (i.e. "broctl | install").
> ????Error: cannot determine Bro version
> My Setup:=======
> bro version 2.3-680
> minimum configuration => (a single Bro instance on the localhost)
> Note: "bro -v" will return the version number but it also produces a segmentation fault..
> 
> I do not see this issue in older releases of Bro. Is there a known issue with this latest release?
> ?Regards,
> Jerome Taylor
> M: 978-764-1269

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150408/2e02cd58/attachment.html 

From jsiwek at illinois.edu  Thu Apr  9 07:48:41 2015
From: jsiwek at illinois.edu (Siwek, Jon)
Date: Thu, 9 Apr 2015 14:48:41 +0000
Subject: [Bro] Error: cannot determine Bro version
In-Reply-To: <677744488.2675371.1428520042383.JavaMail.yahoo@mail.yahoo.com>
References: <677744488.2675371.1428520042383.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <F7D0CD42-C88B-4C24-9E04-CC3598434C8E@illinois.edu>


> On Apr 8, 2015, at 2:07 PM, Jerome Taylor <jtaylor1024 at yahoo.com> wrote:
> 
> I just completed a new Bro installation. I receive the following error message when I try to start Bro (i.e. "broctl | install").
> 
>     Error: cannot determine Bro version
> 
> My Setup:
> =======
> bro version 2.3-680
> minimum configuration => (a single Bro instance on the localhost)
> 
> Note: "bro -v" will return the version number but it also produces a segmentation fault..
> 
> I do not see this issue in older releases of Bro. Is there a known issue with this latest release?

No, that?s not been seen before and things appear to work fine on various platforms Bro is tested against.  Can you give more details?  What platform/OS/compiler?  Did you give any special ./configure flags?  Can you post a backtrace from gdb/lldb ?

> In what I believe is an unrelated issue, I notice that the file "/usr/local/bro/spool/broctl-config.sh" does not exist; 

I think that file is generated by `broctl install`, so if that doesn?t complete due to the issue you mention above, that could explain why it?s missing.

- Jon


From jtaylor1024 at yahoo.com  Thu Apr  9 08:45:09 2015
From: jtaylor1024 at yahoo.com (Jerome Taylor)
Date: Thu, 9 Apr 2015 15:45:09 +0000 (UTC)
Subject: [Bro] Error: cannot determine Bro version
In-Reply-To: <F7D0CD42-C88B-4C24-9E04-CC3598434C8E@illinois.edu>
References: <F7D0CD42-C88B-4C24-9E04-CC3598434C8E@illinois.edu>
Message-ID: <78985605.3465969.1428594309158.JavaMail.yahoo@mail.yahoo.com>

Jon,
Thanks for taking a look a this..
I have a very basic node.cfg file????[bro]
????type=standalone
????host=localhost
????interface=eth0

I am running CentOS 6.6 kernel 2.6.32-504. The diag report from broctl is listed below.????Bro 2.3-680
????Linux 2.6.32-504.12.2.el6.x86_64
????==== No reporter.log
????==== No stderr.log
????==== No stdout.log
????==== No .cmdline
????==== No .env_vars
????==== No .status
????==== No prof.log
????==== No packet_filter.log
????==== No loaded_scripts.log
# bro -v??????? ! reports the following
bro version 2.3-680Segmentation fault
I originally compile bro with a config option "configure --with-pcap=/..." but I have since removed this. The fault still occurs. I will provide a backtrace report shortly.

Regards,
Jerome Taylor
M: 978-764-1269
 


     On Thursday, April 9, 2015 10:50 AM, "Siwek, Jon" <jsiwek at illinois.edu> wrote:
   

 
> On Apr 8, 2015, at 2:07 PM, Jerome Taylor <jtaylor1024 at yahoo.com> wrote:
> 
> I just completed a new Bro installation. I receive the following error message when I try to start Bro (i.e. "broctl | install").
> 
>? ? Error: cannot determine Bro version
> 
> My Setup:
> =======
> bro version 2.3-680
> minimum configuration => (a single Bro instance on the localhost)
> 
> Note: "bro -v" will return the version number but it also produces a segmentation fault..
> 
> I do not see this issue in older releases of Bro. Is there a known issue with this latest release?

No, that?s not been seen before and things appear to work fine on various platforms Bro is tested against.? Can you give more details?? What platform/OS/compiler?? Did you give any special ./configure flags?? Can you post a backtrace from gdb/lldb ?

> In what I believe is an unrelated issue, I notice that the file "/usr/local/bro/spool/broctl-config.sh" does not exist; 

I think that file is generated by `broctl install`, so if that doesn?t complete due to the issue you mention above, that could explain why it?s missing.

- Jon

  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150409/a9f363bd/attachment-0001.html 

From jtaylor1024 at yahoo.com  Thu Apr  9 10:53:15 2015
From: jtaylor1024 at yahoo.com (Jerome Taylor)
Date: Thu, 9 Apr 2015 17:53:15 +0000 (UTC)
Subject: [Bro] Error: cannot determine Bro version
In-Reply-To: <78985605.3465969.1428594309158.JavaMail.yahoo@mail.yahoo.com>
References: <F7D0CD42-C88B-4C24-9E04-CC3598434C8E@illinois.edu>
	<78985605.3465969.1428594309158.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <2041104955.3582271.1428601995077.JavaMail.yahoo@mail.yahoo.com>

Jon,
I reconfigured Bro with debug enabled. The core dump is listed below.
# gdb bro core.10188
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.? Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/bro/bin/bro...done.
[New Thread 10188]
Missing separate debuginfo for /opt/netronome/lib/libpcap.so.1
Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/91/abf66eb676e4e7a8c82352de10d0dceb4b3a90
Missing separate debuginfo for /opt/netronome/lib/libnfe.so.2
Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/b8/da49e35d5c3d83af69bec52e94b95d7e848771
Missing separate debuginfo for /opt/netronome/lib/libns_armctrl.so.2
Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/e0/79905105f0bc1def30279830e021f5e27d20f5
Missing separate debuginfo for /opt/netronome/lib/libnfm.so.2
Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/fe/db0ff744c092fdf3ec2314563083d8ce06d6f9
Missing separate debuginfo for /opt/netronome/lib/libns_msg.so.2
Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/1d/1a3c0cc45eb6bdfca2f0f63bb12342093d11f5
Missing separate debuginfo for /opt/netronome/lib/libnfp.so.2
Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/a7/c27d95c1c926a73e00a41a3e6fe9502829c1f6
Missing separate debuginfo for
Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/4b/8e1260ccd7c3d8cc131e84001f7220651617ec
Reading symbols from /opt/netronome/lib/libpcap.so.1...(no debugging symbols found)...done.
Loaded symbols for /opt/netronome/lib/libpcap.so.1
Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libssl.so.10
Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libcrypto.so.10
Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libz.so.1
Reading symbols from /usr/lib64/libGeoIP.so.1...done.
Loaded symbols for /usr/lib64/libGeoIP.so.1
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libstdc++.so.6
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgcc_s.so.1
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /opt/netronome/lib/libnfe.so.2...(no debugging symbols found)...done.
Loaded symbols for /opt/netronome/lib/libnfe.so.2
Reading symbols from /opt/netronome/lib/libns_armctrl.so.2...(no debugging symbols found)...done.
Loaded symbols for /opt/netronome/lib/libns_armctrl.so.2
Reading symbols from /opt/netronome/lib/libnfm.so.2...done.
Loaded symbols for /opt/netronome/lib/libnfm.so.2
Reading symbols from /opt/netronome/lib/libns_msg.so.2...(no debugging symbols found)...done.
Loaded symbols for /opt/netronome/lib/libns_msg.so.2
Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgssapi_krb5.so.2
Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib64/libk5crypto.so.3
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /opt/netronome/lib/libnfp.so.2...done.
Loaded symbols for /opt/netronome/lib/libnfp.so.2
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libselinux.so.1
Core was generated by `bro -v'.
Program terminated with signal 11, Segmentation fault.
#0? 0x00000000006fc405 in plugin::Manager::HavePluginForHook (this=0x0, hook=plugin::HOOK_BRO_OBJ_DTOR) at /root/Bro/bro/src/plugin/Manager.h:163
163??? ??? ??? return hooks[hook] != 0;
Missing separate debuginfos, use: debuginfo-install geoip-1.4.6-1.el6.rf.x86_64 glibc-2.12-1.149.el6_6.5.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-33.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libgcc-4.4.7-11.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 libstdc++-4.4.7-11.el6.x86_64 openssl-1.0.1e-30.el6.8.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb)
?Regards,
Jerome Taylor
M: 978-764-1269
 


     On Thursday, April 9, 2015 11:45 AM, Jerome Taylor <jtaylor1024 at yahoo.com> wrote:
   

 Jon,
Thanks for taking a look a this..
I have a very basic node.cfg file????[bro]
????type=standalone
????host=localhost
????interface=eth0

I am running CentOS 6.6 kernel 2.6.32-504. The diag report from broctl is listed below.????Bro 2.3-680
????Linux 2.6.32-504.12.2.el6.x86_64
????==== No reporter.log
????==== No stderr.log
????==== No stdout.log
????==== No .cmdline
????==== No .env_vars
????==== No .status
????==== No prof.log
????==== No packet_filter.log
????==== No loaded_scripts.log
# bro -v??????? ! reports the following
bro version 2.3-680Segmentation fault
I originally compile bro with a config option "configure --with-pcap=/..." but I have since removed this. The fault still occurs. I will provide a backtrace report shortly.

Regards,
Jerome Taylor
M: 978-764-1269
 




        On Thursday, April 9, 2015 10:50 AM, "Siwek, Jon" <jsiwek at illinois.edu> wrote:
   

 
> On Apr 8, 2015, at 2:07 PM, Jerome Taylor <jtaylor1024 at yahoo.com> wrote:
> 
> I just completed a new Bro installation. I receive the following error message when I try to start Bro (i.e. "broctl | install").
> 
>? ? Error: cannot determine Bro version
> 
> My Setup:
> =======
> bro version 2.3-680
> minimum configuration => (a single Bro instance on the localhost)
> 
> Note: "bro -v" will return the version number but it also produces a segmentation fault..
> 
> I do not see this issue in older releases of Bro. Is there a known issue with this latest release?

No, that?s not been seen before and things appear to work fine on various platforms Bro is tested against.? Can you give more details?? What platform/OS/compiler?? Did you give any special ./configure flags?? Can you post a backtrace from gdb/lldb ?

> In what I believe is an unrelated issue, I notice that the file "/usr/local/bro/spool/broctl-config.sh" does not exist; 

I think that file is generated by `broctl install`, so if that doesn?t complete due to the issue you mention above, that could explain why it?s missing.

- Jon

  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150409/27d32ba1/attachment-0001.html 

From jsiwek at illinois.edu  Thu Apr  9 11:52:03 2015
From: jsiwek at illinois.edu (Siwek, Jon)
Date: Thu, 9 Apr 2015 18:52:03 +0000
Subject: [Bro] Error: cannot determine Bro version
In-Reply-To: <2041104955.3582271.1428601995077.JavaMail.yahoo@mail.yahoo.com>
References: <F7D0CD42-C88B-4C24-9E04-CC3598434C8E@illinois.edu>
	<78985605.3465969.1428594309158.JavaMail.yahoo@mail.yahoo.com>
	<2041104955.3582271.1428601995077.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <4B9A1D40-8FC7-40FA-B72A-F0CD0D1F52CE@illinois.edu>


> On Apr 9, 2015, at 12:53 PM, Jerome Taylor <jtaylor1024 at yahoo.com> wrote:
> 
> Core was generated by `bro -v'.
> Program terminated with signal 11, Segmentation fault.
> #0  0x00000000006fc405 in plugin::Manager::HavePluginForHook (this=0x0, hook=plugin::HOOK_BRO_OBJ_DTOR) at /root/Bro/bro/src/plugin/Manager.h:163
> 163            return hooks[hook] != 0;
> Missing separate debuginfos, use: debuginfo-install geoip-1.4.6-1.el6.rf.x86_64 glibc-2.12-1.149.el6_6.5.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-33.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libgcc-4.4.7-11.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 libstdc++-4.4.7-11.el6.x86_64 openssl-1.0.1e-30.el6.8.x86_64 zlib-1.2.3-29.el6.x86_64
> (gdb)

Can you enter ?backtrace? at the gdb prompt here and show the output of that as well?

- Jon


From abhall1 at yahoo.com  Thu Apr  9 21:31:22 2015
From: abhall1 at yahoo.com (abhall1)
Date: Fri, 10 Apr 2015 00:31:22 -0400
Subject: [Bro] Bro Digest, Vol 108, Issue 7
Message-ID: <34fo4w1dnj5kqtfume7lcu11.1428640281620@email.android.com>


    
The broctl-config.sh is generated when you run 'broctl install'. ?That is why you don't have the file.
A suggestion was made to reinstall bro. ?Please run 'make distclean' first before you run configure, make, make install. ?I would recommend trying with no options.
Does any other commands segfault at this point such as Broctl update, broctl check, broctl stats?
I have used up to version 2.3-411 on both gentoo and ubuntu and have not ran into that issue before.
Adam Hall

Sent via the Samsung Galaxy Note? 4, an AT&T 4G LTE smartphone

-------- Original message --------
From: bro-request at bro.org 
Date: 04/09/2015  11:45 AM  (GMT-05:00) 
To: bro at bro.org 
Subject: Bro Digest, Vol 108, Issue 7 

Send Bro mailing list submissions to
	bro at bro.org

To subscribe or unsubscribe via the World Wide Web, visit
	http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
or, via email, send a message with subject or body 'help' to
	bro-request at bro.org

You can reach the person managing the list at
	bro-owner at bro.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Bro digest..."


Today's Topics:

?? 1. Error: cannot determine Bro version (Jerome Taylor)
?? 2. Re: Error: cannot determine Bro version (Aashish Sharma)
?? 3. Re: Error: cannot determine Bro version (Jerome Taylor)
?? 4. Re: Error: cannot determine Bro version (Siwek, Jon)
?? 5. Re: Error: cannot determine Bro version (Jerome Taylor)


----------------------------------------------------------------------

Message: 1
Date: Wed, 8 Apr 2015 19:07:22 +0000 (UTC)
From: Jerome Taylor <jtaylor1024 at yahoo.com>
Subject: [Bro] Error: cannot determine Bro version
To: Bro <bro at bro.org>
Message-ID:
	<677744488.2675371.1428520042383.JavaMail.yahoo at mail.yahoo.com>
Content-Type: text/plain; charset="utf-8"

Hello all,
I just completed a new Bro installation. I receive the following error message when I try to start Bro (i.e. "broctl | install").
????Error: cannot determine Bro version
My Setup:=======
bro version 2.3-680
minimum configuration => (a single Bro instance on the localhost)
Note: "bro -v" will return the version number but it also produces a segmentation fault..

I do not see this issue in older releases of Bro. Is there a known issue with this latest release?
?Regards,
Jerome Taylor
M: 978-764-1269
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150408/30a97176/attachment-0001.html 

------------------------------

Message: 2
Date: Wed, 8 Apr 2015 12:37:42 -0700
From: Aashish Sharma <asharma at lbl.gov>
Subject: Re: [Bro] Error: cannot determine Bro version
To: Jerome Taylor <jtaylor1024 at yahoo.com>
Cc: Bro <bro at bro.org>
Message-ID: <20150408193740.GA52542 at Aashishs-Mac-Pro.local>
Content-Type: text/plain; charset=iso-8859-1

I think you should try make && make install again! 

Lets see if the error still stays! 

On Wed, Apr 08, 2015 at 07:07:22PM +0000, Jerome Taylor wrote:
> Hello all,
> I just completed a new Bro installation. I receive the following error message when I try to start Bro (i.e. "broctl | install").
> ????Error: cannot determine Bro version
> My Setup:=======
> bro version 2.3-680
> minimum configuration => (a single Bro instance on the localhost)
> Note: "bro -v" will return the version number but it also produces a segmentation fault..
> 
> I do not see this issue in older releases of Bro. Is there a known issue with this latest release?
> ?Regards,
> Jerome Taylor
> M: 978-764-1269

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



------------------------------

Message: 3
Date: Wed, 8 Apr 2015 19:59:03 +0000 (UTC)
From: Jerome Taylor <jtaylor1024 at yahoo.com>
Subject: Re: [Bro] Error: cannot determine Bro version
To: Aashish Sharma <asharma at lbl.gov>
Cc: Bro <bro at bro.org>
Message-ID:
	<233004250.2725349.1428523143179.JavaMail.yahoo at mail.yahoo.com>
Content-Type: text/plain; charset="utf-8"

Re-running "make; make install" does not fix the problem. The error remains..
In what I believe is an unrelated issue, I notice that the file "/usr/local/bro/spool/broctl-config.sh" does not exist; which in turn causes a broken link for "/usr/local/bro/share/broctl/scripts/broctl-config.sh". I temporarily worked around this problem by importing "/usr/local/bro/spool/broctl-config.sh" from a remote serer that was similarly configured. I mention this hoping that it might point to an underlying issue with the build.
?Regards,
Jerome Taylor

 


???? On Wednesday, April 8, 2015 3:37 PM, Aashish Sharma <asharma at lbl.gov> wrote:
?? 

 I think you should try make && make install again! 

Lets see if the error still stays! 

On Wed, Apr 08, 2015 at 07:07:22PM +0000, Jerome Taylor wrote:
> Hello all,
> I just completed a new Bro installation. I receive the following error message when I try to start Bro (i.e. "broctl | install").
> ????Error: cannot determine Bro version
> My Setup:=======
> bro version 2.3-680
> minimum configuration => (a single Bro instance on the localhost)
> Note: "bro -v" will return the version number but it also produces a segmentation fault..
> 
> I do not see this issue in older releases of Bro. Is there a known issue with this latest release?
> ?Regards,
> Jerome Taylor
> M: 978-764-1269

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



? 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150408/2e02cd58/attachment-0001.html 

------------------------------

Message: 4
Date: Thu, 9 Apr 2015 14:48:41 +0000
From: "Siwek, Jon" <jsiwek at illinois.edu>
Subject: Re: [Bro] Error: cannot determine Bro version
To: Jerome Taylor <jtaylor1024 at yahoo.com>
Cc: Bro <bro at bro.org>
Message-ID: <F7D0CD42-C88B-4C24-9E04-CC3598434C8E at illinois.edu>
Content-Type: text/plain; charset="utf-8"


> On Apr 8, 2015, at 2:07 PM, Jerome Taylor <jtaylor1024 at yahoo.com> wrote:
> 
> I just completed a new Bro installation. I receive the following error message when I try to start Bro (i.e. "broctl | install").
> 
>???? Error: cannot determine Bro version
> 
> My Setup:
> =======
> bro version 2.3-680
> minimum configuration => (a single Bro instance on the localhost)
> 
> Note: "bro -v" will return the version number but it also produces a segmentation fault..
> 
> I do not see this issue in older releases of Bro. Is there a known issue with this latest release?

No, that?s not been seen before and things appear to work fine on various platforms Bro is tested against.? Can you give more details?? What platform/OS/compiler?? Did you give any special ./configure flags?? Can you post a backtrace from gdb/lldb ?

> In what I believe is an unrelated issue, I notice that the file "/usr/local/bro/spool/broctl-config.sh" does not exist; 

I think that file is generated by `broctl install`, so if that doesn?t complete due to the issue you mention above, that could explain why it?s missing.

- Jon



------------------------------

Message: 5
Date: Thu, 9 Apr 2015 15:45:09 +0000 (UTC)
From: Jerome Taylor <jtaylor1024 at yahoo.com>
Subject: Re: [Bro] Error: cannot determine Bro version
To: "Siwek, Jon" <jsiwek at illinois.edu>
Cc: Bro <bro at bro.org>
Message-ID:
	<78985605.3465969.1428594309158.JavaMail.yahoo at mail.yahoo.com>
Content-Type: text/plain; charset="utf-8"

Jon,
Thanks for taking a look a this..
I have a very basic node.cfg file????[bro]
????type=standalone
????host=localhost
????interface=eth0

I am running CentOS 6.6 kernel 2.6.32-504. The diag report from broctl is listed below.????Bro 2.3-680
????Linux 2.6.32-504.12.2.el6.x86_64
????==== No reporter.log
????==== No stderr.log
????==== No stdout.log
????==== No .cmdline
????==== No .env_vars
????==== No .status
????==== No prof.log
????==== No packet_filter.log
????==== No loaded_scripts.log
# bro -v??????? ! reports the following
bro version 2.3-680Segmentation fault
I originally compile bro with a config option "configure --with-pcap=/..." but I have since removed this. The fault still occurs. I will provide a backtrace report shortly.

Regards,
Jerome Taylor
M: 978-764-1269
 


???? On Thursday, April 9, 2015 10:50 AM, "Siwek, Jon" <jsiwek at illinois.edu> wrote:
?? 

 
> On Apr 8, 2015, at 2:07 PM, Jerome Taylor <jtaylor1024 at yahoo.com> wrote:
> 
> I just completed a new Bro installation. I receive the following error message when I try to start Bro (i.e. "broctl | install").
> 
>? ? Error: cannot determine Bro version
> 
> My Setup:
> =======
> bro version 2.3-680
> minimum configuration => (a single Bro instance on the localhost)
> 
> Note: "bro -v" will return the version number but it also produces a segmentation fault..
> 
> I do not see this issue in older releases of Bro. Is there a known issue with this latest release?

No, that?s not been seen before and things appear to work fine on various platforms Bro is tested against.? Can you give more details?? What platform/OS/compiler?? Did you give any special ./configure flags?? Can you post a backtrace from gdb/lldb ?

> In what I believe is an unrelated issue, I notice that the file "/usr/local/bro/spool/broctl-config.sh" does not exist; 

I think that file is generated by `broctl install`, so if that doesn?t complete due to the issue you mention above, that could explain why it?s missing.

- Jon

? 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150409/a9f363bd/attachment.html 

------------------------------

_______________________________________________
Bro mailing list
Bro at bro.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


End of Bro Digest, Vol 108, Issue 7
***********************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150410/d94611d5/attachment-0001.html 

From damian.gerow at shopify.com  Mon Apr 13 13:57:04 2015
From: damian.gerow at shopify.com (Damian Gerow)
Date: Mon, 13 Apr 2015 16:57:04 -0400
Subject: [Bro] Bro processes not stopping on 'broctl restart'
Message-ID: <CAJ0FatQq04cvxjW0_bbyQm7uB+wh=TbbP_deU=KsNhX_ryte=Q@mail.gmail.com>

I've been playing with some scripts in a local testing environment, and
I've noticed that sometimes, a given Bro process won't restart properly.
I've seen this before in the past, but I've only started paying attention
to it recently.  In all cases where I've paid attention it's been the
worker process.

Below is a process listing I've just taken, where a 'broctl restart' was
issued earlier in the day.  This is running Bro 2.3.2, in a very simple
configuration: one node is running manager, proxy, and worker; and there
are two additional worker nodes in the cluster.  Installation is done from
source.  In this case, none of the workers in the cluster were stopped
properly.

Is this a known issue?  Is there some kind of logging somewhere I can look
at to see what's going on?  As far as I can tell, the `broctl restart` is
succeeding without issue, but the old bro workers just aren't exiting.

```
# ps auxw | grep -i bro
bro       6590  0.0  0.0  19596  1700 ?        S    Apr02   0:00 bash /opt/
bro/share/broctl/scripts/run-bro -1 -i eth0 -U .status -p broctl -p broctl-live
-p local -p worker-0 local.bro broctl base/frameworks/cluster local-worker.
bro broctl/auto
bro       6605  5.2  3.1 154976 122300 ?       S    Apr02 829:09 /opt/bro
/bin/bro -i eth0 -U .status -p broctl -p broctl-live -p local -p worker-0
local.bro broctl base/frameworks/cluster local-worker.bro broctl/auto
bro       6610  4.2  1.3  90300 53320 ?        SN   Apr02 672:40 /opt/bro
/bin/bro -i eth0 -U .status -p broctl -p broctl-live -p local -p worker-0
local.bro broctl base/frameworks/cluster local-worker.bro broctl/auto
root     12066  0.0  0.0  10464   948 pts/0    S+   20:50   0:00 grep
--color=auto -i bro
bro      22790  0.0  0.0  17992  1592 ?        S    15:47   0:00 bash /opt/
bro/share/broctl/scripts/run-bro -1 -U .status -p broctl -p broctl-live -p
local -p manager local.bro broctl base/frameworks/cluster local-manager.bro
broctl/auto
bro      22799  2.3  1.6 767060 62336 ?        Sl   15:47   7:04 /opt/bro
/bin/bro -U .status -p broctl -p broctl-live -p local -p manager local.bro
broctl base/frameworks/cluster local-manager.bro broctl/auto
bro      22805  8.5  1.6  98588 61928 ?        SN   15:47  25:47 /opt/bro
/bin/bro -U .status -p broctl -p broctl-live -p local -p manager local.bro
broctl base/frameworks/cluster local-manager.bro broctl/auto
bro      22822  0.0  0.0  17992  1596 ?        S    15:47   0:00 bash /opt/
bro/share/broctl/scripts/run-bro -1 -U .status -p broctl -p broctl-live -p
local -p proxy-0 local.bro broctl base/frameworks/cluster local-proxy bro
ctl/auto
bro      22862  2.1  1.4  87424 54852 ?        R    15:47   6:40 /opt/bro
/bin/bro -U .status -p broctl -p broctl-live -p local -p proxy-0 local.bro
broctl base/frameworks/cluster local-proxy broctl/auto
bro      22863  8.5  1.6 100556 63864 ?        RN   15:47  26:02 /opt/bro
/bin/bro -U .status -p broctl -p broctl-live -p local -p proxy-0 local.bro
broctl base/frameworks/cluster local-proxy broctl/auto
bro      22890  0.0  0.0  17996  1600 ?        S    15:47   0:00 bash /opt/
bro/share/broctl/scripts/run-bro -1 -i eth0 -U .status -p broctl -p broctl-live
-p local -p worker-0 local.bro broctl base/frameworks/cluster local-worker.
bro broctl/auto
bro      22900  6.0  2.1 115420 82684 ?        S    15:47  18:28 /opt/bro
/bin/bro -i eth0 -U .status -p broctl -p broctl-live -p local -p worker-0
local.bro broctl base/frameworks/cluster local-worker.bro broctl/auto
bro      22905  4.2  1.3  90444 53700 ?        SN   15:47  12:44 /opt/bro
/bin/bro -i eth0 -U .status -p broctl -p broctl-live -p local -p worker-0
local.bro broctl base/frameworks/cluster local-worker.bro broctl/auto
#
```
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150413/356f6771/attachment.html 

From daytona.ryu at gmail.com  Mon Apr 13 14:50:13 2015
From: daytona.ryu at gmail.com (Daytona Leo)
Date: Mon, 13 Apr 2015 14:50:13 -0700
Subject: [Bro] ACTION_EMAIL and ACTION_EMAIL_ADMIN not working
Message-ID: <CAAaFQ+b96vxHoeuPWbBG26hWsZ5i_2eRV2ouoYJLJybVdmBZ8Q@mail.gmail.com>

Hello Bros,

This message is in regards to getting the notification types ACTION_EMAIL
and ACTION_EMAIL_ADMIN to actually send an email.

I tried getting on IRC and noone replied, and I've tried everything.

First, let me say that I know bro can send emails with sendmail because
when bro crashes I get messages from my server and I've also tested
manually sending an email with sendmail.

I've been testing this with the Weak_Keys bro script to detect any SSL/TLS
keys that are less than 4096 length (so that it triggers on pretty much
every website)

./share/bro/policy/protocols/ssl/weak-keys.bro

my local.bro only contains:

@load policy/protocols/ssl/weak-keys.bro

The code added to weak-keys.bro at the end of the export section to enable
the email action is as follows:

        hook Notice::policy(n: Notice::Info)
        {
        if ( n$note == SSL::Weak_Key )
        add n$actions[Notice::ACTION_EMAIL_ADMIN];
        }

I can see in the notice.log that one of the listed actions for these
notices. Example from notice.log:

1428960187.772499       Cec6cr4QGk6SIcnxdb      192.168.1.15    60350
64.233.177.113  443     -       -       -       tcp     SSL::Weak_Key
Host uses weak certificate with 256 bit key        -       192.168.1.15
 64.233.177.113  443     -       bro
Notice::ACTION_EMAIL_ADMIN,Notice::ACTION_LOG   86400.000000    F       -
    -       -       - -

I've also tried this with ACTION_EMAIL and it still doesn't work.

Checking the /var/log/mail.log and mail.info looks like it is trying to
send emails but they aren't reaching my gmail.

Apr 13 14:40:59 brotector sendmail[21412]: t3DLewwo021412: to=
daytona.leo at gmail.com, ctladdr=root (0/0), delay=00:00:01, xdelay=00:00:00,
mailer=relay, pri=38033, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0,
stat=Sent (t3DLexYr021419 Message accepted for delivery)
Apr 13 14:41:00 brotector sm-mta[21426]: STARTTLS=client, relay=
aspmx.l.google.com., version=TLSv1/SSLv3, verify=FAIL,
cipher=ECDHE-RSA-AES128-GCM-SHA256, bits=128/128
Apr 13 14:41:01 brotector sm-mta[21426]: t3DLexYr021419: to=<
daytona.leo at gmail.com>, ctladdr=<root at brotector.brotector.bro> (0/0),
delay=00:00:02, xdelay=00:00:02, mailer=esmtp, pri=128310, relay=
aspmx.l.google.com. [64.233.177.26], dsn=2.0.0, stat=Sent (OK 1428961259
y62si5959254yhc.175 - gsmtp)

So perhaps it's being dropped for seeming like spam, but it does not arrive
in the spam folder. What I really don't understand is why the crash notices
will reach my inbox without issue.

Is there any way to fix this or maybe use an external SMTP authenticated
solution like mandrill? https://mandrillapp.com

I've tried everything and looked up so much information and watched tons of
videos. Countless hours spent. I really cant get the email alerts to work.

Any help is appreciated.

Thanks,

Daytona
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150413/71c74378/attachment.html 

From giedrius.ramas at gmail.com  Tue Apr 14 07:57:40 2015
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Tue, 14 Apr 2015 17:57:40 +0300
Subject: [Bro] erspan decapsulation
Message-ID: <CAHr6V5o+K2=daVqPBxigkwHGaKi7MTMvcZoe8qXQs-4riy_LNg@mail.gmail.com>

Hello, we have problems with ERSPAN package. Is there anyway BRO could
understand them ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150414/e610310a/attachment.html 

From luuk.hendriks at utwente.nl  Tue Apr 14 04:39:29 2015
From: luuk.hendriks at utwente.nl (Luuk Hendriks)
Date: Tue, 14 Apr 2015 13:39:29 +0200
Subject: [Bro] Incorrect orig_bytes after ICMP6 reassembly
Message-ID: <20150414113929.GB8352@corley.shackle.nl>

Hi,

When analyzing a pcap containing fragmented ICMP6 packets, the resulting size (i.e. orig_ip_bytes) in conn.log is not the expected size. For example, a pcap containing only 46 fragments forming a single large ping request of ~65k bytes will result in a orig_ip_bytes of only 376. With some additional debug output in the code (Frag.cc) it seems that the reassembly does take place, and the offset reaches near the 65k mark. However, I was not able to figure out where things do go wrong. Is this a possible bug, or am I misinterpreting/misunderstanding things?

NB: My script contains 'redef ignore_checksums=T;', as I'm working with a subset (via editcap) of a real capture. 

Version information:
bro 2.3.2, compiled from stable tarball
(Arch) Linux, kernel 3.19


Thanks,
 Luuk

From edthoma at sandia.gov  Tue Apr 14 09:59:58 2015
From: edthoma at sandia.gov (Thomas, Eric D)
Date: Tue, 14 Apr 2015 16:59:58 +0000
Subject: [Bro] Logging VLAN IDs
Message-ID: <D151969B.170CE%edthoma@sandia.gov>

Dear Bro developers,

I've been tasked with trying to modify the Bro source code so that
conn.log includes the VLAN IDs (including 802.1ah) that have been observed
in packets associated with that connection. I've scoped out a solution,
but I want to run it by you first before I start to go for it, in case I'm
missing something really big.

PktSrc::Process() does processing of VLAN and 802.1ah, but it just skips
over them by advancing the data pointer. I will, in addition, store those
VLAN IDs in a new member of the modified PktSrc class. This gets passed on
through net_packet_dispatch() and NetSessions::DispatchPacket(). At this
point NetSessions::NextPacket() gets called, but since the PktSrc doesn't
get passed to it, I'd need another way to pass it the VLAN ID. I am
considering two options:

1. duplicate NextPacket() adding a new parameter to pass it the VLAN IDs,
and call that instead, or
2. store the VLAN IDs in the NetSessions class, in DispatchPacket() so
it?s available to NextPacket() and DoNextPacket() <- Is there a reason
this wouldn?t work, e.g. issues with multi-threading/multi-processing?

Is there one option that seems better to you?

NetSessions::DoNextPacket() is called next and I would also need a
modification to pass it VLAN IDs, using one of the options above. In this
method we finally get access to the appropriate Connection instance, so I
would store the VLAN IDs in that instance in DoNextPacket().

I'd need to modify the Connection class in Conn.h to include a new member
for tracking VLAN IDs. I'd modify Connection::BuildConnVal() and
scripts/base/init-bare.bro's connection record to make the VLAN IDs
available to scripts. Lastly, I'd write a script to redef the conn Info
structure and handle one or more connection events (perhaps
connection_state_remove) to copy the VLAN IDs from the connection record
to the Info record.

Is there anything I'm missing? Is there a better way to approach this?

-- 
Eric Thomas

edthoma at sandia.gov



From jtaylor1024 at yahoo.com  Tue Apr 14 10:20:42 2015
From: jtaylor1024 at yahoo.com (Jerome Taylor)
Date: Tue, 14 Apr 2015 17:20:42 +0000 (UTC)
Subject: [Bro] Error: cannot determine Bro version
In-Reply-To: <4B9A1D40-8FC7-40FA-B72A-F0CD0D1F52CE@illinois.edu>
References: <4B9A1D40-8FC7-40FA-B72A-F0CD0D1F52CE@illinois.edu>
Message-ID: <212203436.3487615.1429032042494.JavaMail.yahoo@mail.yahoo.com>

Jon,
Sorry for the delay. I got sidetracked by another project.

Core was generated by `bro -v'.
Program terminated with signal 11, Segmentation fault.
#0? HavePluginForHook (this=0x7fec89226410, __in_chrg=<value optimized out>) at /root/Bro/bro/src/plugin/Manager.h:163
163??? ??? ??? return hooks[hook] != 0;
Missing separate debuginfos, use: debuginfo-install geoip-1.4.6-1.el6.rf.x86_64 glibc-2.12-1.149.el6_6.5.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-33.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libgcc-4.4.7-11.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 libstdc++-4.4.7-11.el6.x86_64 openssl-1.0.1e-30.el6.8.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) backtrace
#0? HavePluginForHook (this=0x7fec89226410, __in_chrg=<value optimized out>) at /root/Bro/bro/src/plugin/Manager.h:163
#1? BroObj::~BroObj (this=0x7fec89226410, __in_chrg=<value optimized out>) at /root/Bro/bro/src/Obj.cc:97
#2? 0x00007fec89669ebd in __cxa_finalize () from /lib64/libc.so.6
#3? 0x00007fec88fe5b96 in __do_global_dtors_aux () from /opt/netronome/lib/libnfm.so.2
#4? 0x0000000000000000 in ?? ()?Regards,
Jerome Taylor
M: 978-764-1269
 


     On Thursday, April 9, 2015 2:54 PM, "Siwek, Jon" <jsiwek at illinois.edu> wrote:
   

 
> On Apr 9, 2015, at 12:53 PM, Jerome Taylor <jtaylor1024 at yahoo.com> wrote:
> 
> Core was generated by `bro -v'.
> Program terminated with signal 11, Segmentation fault.
> #0? 0x00000000006fc405 in plugin::Manager::HavePluginForHook (this=0x0, hook=plugin::HOOK_BRO_OBJ_DTOR) at /root/Bro/bro/src/plugin/Manager.h:163
> 163? ? ? ? ? ? return hooks[hook] != 0;
> Missing separate debuginfos, use: debuginfo-install geoip-1.4.6-1.el6.rf.x86_64 glibc-2.12-1.149.el6_6.5.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-33.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libgcc-4.4.7-11.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 libstdc++-4.4.7-11.el6.x86_64 openssl-1.0.1e-30.el6.8.x86_64 zlib-1.2.3-29.el6.x86_64
> (gdb)

Can you enter ?backtrace? at the gdb prompt here and show the output of that as well?

- Jon


  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150414/63f4531a/attachment.html 

From jlay at slave-tothe-box.net  Tue Apr 14 10:43:14 2015
From: jlay at slave-tothe-box.net (James Lay)
Date: Tue, 14 Apr 2015 11:43:14 -0600
Subject: [Bro] Logging VLAN IDs
In-Reply-To: <D151969B.170CE%edthoma@sandia.gov>
References: <D151969B.170CE%edthoma@sandia.gov>
Message-ID: <d7d5ac63c0518a4f5e734042c84c77ac@localhost>

On 2015-04-14 10:59 AM, Thomas, Eric D wrote:
> Dear Bro developers,
>
> I've been tasked with trying to modify the Bro source code so that
> conn.log includes the VLAN IDs (including 802.1ah) that have been 
> observed
> in packets associated with that connection. I've scoped out a 
> solution,
> but I want to run it by you first before I start to go for it, in 
> case I'm
> missing something really big.
>
> PktSrc::Process() does processing of VLAN and 802.1ah, but it just 
> skips
> over them by advancing the data pointer. I will, in addition, store 
> those
> VLAN IDs in a new member of the modified PktSrc class. This gets 
> passed on
> through net_packet_dispatch() and NetSessions::DispatchPacket(). At 
> this
> point NetSessions::NextPacket() gets called, but since the PktSrc 
> doesn't
> get passed to it, I'd need another way to pass it the VLAN ID. I am
> considering two options:
>
> 1. duplicate NextPacket() adding a new parameter to pass it the VLAN 
> IDs,
> and call that instead, or
> 2. store the VLAN IDs in the NetSessions class, in DispatchPacket() 
> so
> it?s available to NextPacket() and DoNextPacket() <- Is there a 
> reason
> this wouldn?t work, e.g. issues with 
> multi-threading/multi-processing?
>
> Is there one option that seems better to you?
>
> NetSessions::DoNextPacket() is called next and I would also need a
> modification to pass it VLAN IDs, using one of the options above. In 
> this
> method we finally get access to the appropriate Connection instance, 
> so I
> would store the VLAN IDs in that instance in DoNextPacket().
>
> I'd need to modify the Connection class in Conn.h to include a new 
> member
> for tracking VLAN IDs. I'd modify Connection::BuildConnVal() and
> scripts/base/init-bare.bro's connection record to make the VLAN IDs
> available to scripts. Lastly, I'd write a script to redef the conn 
> Info
> structure and handle one or more connection events (perhaps
> connection_state_remove) to copy the VLAN IDs from the connection 
> record
> to the Info record.
>
> Is there anything I'm missing? Is there a better way to approach 
> this?

And a big +1 to this.  Would love to be able to filter VLAN's as well 
as we have listen to physical interfaces that have other interfaces 
mirrored that include some unwanted VLAN's.

James

From edthoma at sandia.gov  Tue Apr 14 12:35:37 2015
From: edthoma at sandia.gov (Thomas, Eric D)
Date: Tue, 14 Apr 2015 19:35:37 +0000
Subject: [Bro] [EXTERNAL] Bro Digest, Vol 108, Issue 12
In-Reply-To: <mailman.1.1429038001.7096.bro@bro.org>
References: <mailman.1.1429038001.7096.bro@bro.org>
Message-ID: <D152B992.171EC%edthoma@sandia.gov>

>
>And a big +1 to this.  Would love to be able to filter VLAN's as well
>as we have listen to physical interfaces that have other interfaces
>mirrored that include some unwanted VLAN's.
>
>James
>

I thought, if you are hoping to filter out traffic by VLANs, you can do a
PCAP filter. Is that not true?

In my case, we do want to process different VLANs, but we need to log
through which links the traffic was observed.



From wren3 at illinois.edu  Tue Apr 14 15:20:19 2015
From: wren3 at illinois.edu (Ren, Wenyu)
Date: Tue, 14 Apr 2015 22:20:19 +0000
Subject: [Bro] Question about processing network traffic
Message-ID: <D1DD2D198D059443B177F461727B01591A30748F@CHIMBX6.ad.uillinois.edu>

Dear all,

I have a very basic question about how bro Bro handles network traffic. I am doing some processing on each packet that Bro sees. If the processing time is longer than the packet arriving interval, will Bro block the new packet or buffer the new packet event and deal with it later? If it is buffered, will the event be dropped if the buffer is full?

Thanks,

Wenyu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150414/b51d16d4/attachment.html 

From franky.meier.1 at gmx.de  Wed Apr 15 00:45:01 2015
From: franky.meier.1 at gmx.de (Frank Meier)
Date: Wed, 15 Apr 2015 09:45:01 +0200
Subject: [Bro] postprocessing extracted files
Message-ID: <1429083901.3034.0@mail.gmx.net>

Hi.

I want to use Bro to extract files. After extraction these files will 
undergo some post-processing (e.g. lookup in a db of known files). Can 
I be sure, that a file logged in files.log with its hash has been 
written to disk completely? 

If not, I have two ideas how to solve this:
1) use a temporary filename until the file is completely written (like 
a prefix/postfix). exclude temporary files from post-processing.
2) emit an appropriate signal from Extract::~Extract() after the file 
is closed.

I would be happy to implement a solution.

Franky




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150415/d3ffef22/attachment.html 

From Emmanuel.TORQUATO at monext.net  Wed Apr 15 02:57:22 2015
From: Emmanuel.TORQUATO at monext.net (Emmanuel TORQUATO)
Date: Wed, 15 Apr 2015 11:57:22 +0200
Subject: [Bro] Detection of SSL clients which invalidate server cert
Message-ID: <DBABBC600D7B9C4FB2051B03AAA9213B0251CE3A@MAILBOXCCR.monext.net>

Hello,

I am searching a way to detect SSL clients which invalidate the server Cert during the SSL Handshake and refused the SSL connection establishment. Is there a way to do this with Bro? Does ssl_alert event can catch this ?

Thanks,

Regards,

Emmanuel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150415/968b71e8/attachment.html 

From seth at icir.org  Wed Apr 15 05:09:08 2015
From: seth at icir.org (Seth Hall)
Date: Wed, 15 Apr 2015 08:09:08 -0400
Subject: [Bro] erspan decapsulation
In-Reply-To: <CAHr6V5o+K2=daVqPBxigkwHGaKi7MTMvcZoe8qXQs-4riy_LNg@mail.gmail.com>
References: <CAHr6V5o+K2=daVqPBxigkwHGaKi7MTMvcZoe8qXQs-4riy_LNg@mail.gmail.com>
Message-ID: <FFDC5F68-88BC-48B7-9B62-EB94627A3A01@icir.org>


> On Apr 14, 2015, at 10:57 AM, Giedrius Ramas <giedrius.ramas at gmail.com> wrote:
> 
> Hello, we have problems with ERSPAN package. Is there anyway BRO could understand them ?

Could you privately provide us a small packet capture of ERSPAN packets?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Wed Apr 15 05:17:24 2015
From: seth at icir.org (Seth Hall)
Date: Wed, 15 Apr 2015 08:17:24 -0400
Subject: [Bro] postprocessing extracted files
In-Reply-To: <1429083901.3034.0@mail.gmx.net>
References: <1429083901.3034.0@mail.gmx.net>
Message-ID: <184C959B-8449-4EB5-A738-1B32F109E8C5@icir.org>


> On Apr 15, 2015, at 3:45 AM, Frank Meier <franky.meier.1 at gmx.de> wrote:
> 
> I want to use Bro to extract files. After extraction these files will undergo some post-processing (e.g. lookup in a db of known files). Can I be sure, that a file logged in files.log with its hash has been written to disk completely? 

You can handle the file_state_remove event.  At that point, everything about the file is complete and it?s being flushed from memory.

event file_state_remove(f: fa_file)
	{
	# Do what you need.
	}

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Wed Apr 15 05:18:43 2015
From: seth at icir.org (Seth Hall)
Date: Wed, 15 Apr 2015 08:18:43 -0400
Subject: [Bro] Question about processing network traffic
In-Reply-To: <D1DD2D198D059443B177F461727B01591A30748F@CHIMBX6.ad.uillinois.edu>
References: <D1DD2D198D059443B177F461727B01591A30748F@CHIMBX6.ad.uillinois.edu>
Message-ID: <8278AC20-D463-493E-9839-CC89F2C77350@icir.org>


> On Apr 14, 2015, at 6:20 PM, Ren, Wenyu <wren3 at illinois.edu> wrote:
> 
> I have a very basic question about how bro Bro handles network traffic. I am doing some processing on each packet that Bro sees. If the processing time is longer than the packet arriving interval, will Bro block the new packet or buffer the new packet event and deal with it later? If it is buffered, will the event be dropped if the buffer is full?

The packet will be lost once the NIC buffers are exceeded.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From franky.meier.1 at gmx.de  Wed Apr 15 06:04:19 2015
From: franky.meier.1 at gmx.de (Frank Meier)
Date: Wed, 15 Apr 2015 15:04:19 +0200
Subject: [Bro] postprocessing extracted files
In-Reply-To: <184C959B-8449-4EB5-A738-1B32F109E8C5@icir.org>
References: <1429083901.3034.0@mail.gmx.net>
	<184C959B-8449-4EB5-A738-1B32F109E8C5@icir.org>
Message-ID: <1429103059.3502.0@mail.gmx.net>

Hi,

On Mi, Apr 15, 2015 at 2:17 , Seth Hall <seth at icir.org> wrote:

> You can handle the file_state_remove event.  At that point, 
> everything about the file is complete and it?s being flushed from 
> memory.
> 
> event file_state_remove(f: fa_file)
> 	{
> 	# Do what you need.
> 	}

Thanks! That was easy.

Franky
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150415/ff6e065b/attachment-0001.html 

From johanna at icir.org  Wed Apr 15 06:17:29 2015
From: johanna at icir.org (Johanna Amann)
Date: Wed, 15 Apr 2015 06:17:29 -0700
Subject: [Bro] Detection of SSL clients which invalidate server cert
In-Reply-To: <DBABBC600D7B9C4FB2051B03AAA9213B0251CE3A@MAILBOXCCR.monext.net>
References: <DBABBC600D7B9C4FB2051B03AAA9213B0251CE3A@MAILBOXCCR.monext.net>
Message-ID: <20150415131723.GA2168@Beezling.local>

Hello Emmanuel,

On Wed, Apr 15, 2015 at 11:57:22AM +0200, Emmanuel TORQUATO wrote:
> I am searching a way to detect SSL clients which invalidate the server
> Cert during the SSL Handshake and refused the SSL connection
> establishment. Is there a way to do this with Bro? Does ssl_alert event
> can catch this ?

Yes, there probably is a way to do it, but it will involve quite a bit of
manual work.

First - I do not really think that the way that clients handle this case
is completely specified. You will probably have to examine the network
traffic of different clients, determine how exactly they are ending the
connection in this case, identify the client in your Bro script (usually
using the cipher suite they send), and then watch for that kind of
traffic.

E.g., at least ~1 year ago, Chrome just used to send TCP resets in
connections where it did not accept the server certificate. Hence, if you
have a client where the cipher suites it specifies match chrome, and you
see a certificate exchange followed by a client-initiated reset, it
probably rejected the certificate.

Sadly I do not think there is a trivial way to measuer that.

I hope this helps,
 Johanna

From apumphrey at ivsec.com  Wed Apr 15 06:36:11 2015
From: apumphrey at ivsec.com (Adam Pumphrey)
Date: Wed, 15 Apr 2015 09:36:11 -0400
Subject: [Bro] [EXTERNAL] Bro Digest, Vol 108, Issue 12
In-Reply-To: <D152B992.171EC%edthoma@sandia.gov>
References: <mailman.1.1429038001.7096.bro@bro.org>
	<D152B992.171EC%edthoma@sandia.gov>
Message-ID: <0C021EF0-4BC1-471C-81B0-D09850CF7380@ivsec.com>

> I thought, if you are hoping to filter out traffic by VLANs, you can do a
> PCAP filter. Is that not true?


That?s correct, you can include or exclude VLAN traffic using Bro capture filters.  I believe PacketFilter::restricted_filter from the PacketFilter framework is for doing just that (https://www.bro.org/sphinx/scripts/base/frameworks/packet-filter/main.bro.html <https://www.bro.org/sphinx/scripts/base/frameworks/packet-filter/main.bro.html>).   

In fact, you need to use the VLAN keyword to do any IP filtering of VLAN tagged (802.1q) traffic.  It won?t work correctly otherwise.  

You can specify VLAN ID?s also, ex:  not (van 100 or vlan 101).  

You may also encounter multiple VLAN tags on a single packet, there you?ll need the correct number of VLAN keywords, ex: (vlan 100 and vlan 101).  

You could see asynchronous tagging where traffic from a particular stream heading in one direction has 1 tag and traffic going in the opposite direction has 2 or more, so directionality matters also. 

This is all true for MPLS encapsulated traffic too.  The filtering behavior is basically the same from what I?ve seen.  

> In my case, we do want to process different VLANs, but we need to log
> through which links the traffic was observed.


I agree, it would be helpful to have VLAN ID(?s) available in conn records if they are present.  Its valuable info when your troubleshooting tap and traffic mirroring configurations in complex environments.  

Adam

> On Apr 14, 2015, at 3:35 PM, Thomas, Eric D <edthoma at sandia.gov> wrote:
> 
>> 
>> And a big +1 to this.  Would love to be able to filter VLAN's as well
>> as we have listen to physical interfaces that have other interfaces
>> mirrored that include some unwanted VLAN's.
>> 
>> James
>> 
> 
> I thought, if you are hoping to filter out traffic by VLANs, you can do a
> PCAP filter. Is that not true?
> 
> In my case, we do want to process different VLANs, but we need to log
> through which links the traffic was observed.
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150415/acb76f4e/attachment.html 

From Emmanuel.TORQUATO at monext.net  Wed Apr 15 08:00:52 2015
From: Emmanuel.TORQUATO at monext.net (Emmanuel TORQUATO)
Date: Wed, 15 Apr 2015 17:00:52 +0200
Subject: [Bro] Detection of SSL clients which invalidate server cert
In-Reply-To: <20150415131723.GA2168@Beezling.local>
References: <DBABBC600D7B9C4FB2051B03AAA9213B0251CE3A@MAILBOXCCR.monext.net>
	<20150415131723.GA2168@Beezling.local>
Message-ID: <DBABBC600D7B9C4FB2051B03AAA9213B0251D022@MAILBOXCCR.monext.net>

Hello Johanna,

Thanks for your reply. We have done a test with BRO and a tcpdump and in fact client send a SSL alert with "bad certificate". And Bro do the job beautifully and insert "bad_certificate" in the "last_alert" field.

Reagrds,

Emmanuel.

-----Message d'origine-----
De?: Johanna Amann [mailto:johanna at icir.org] 
Envoy??: mercredi 15 avril 2015 15:17
??: Emmanuel TORQUATO
Cc?: bro at bro.org
Objet?: Re: [Bro] Detection of SSL clients which invalidate server cert

Hello Emmanuel,

On Wed, Apr 15, 2015 at 11:57:22AM +0200, Emmanuel TORQUATO wrote:
> I am searching a way to detect SSL clients which invalidate the server 
> Cert during the SSL Handshake and refused the SSL connection 
> establishment. Is there a way to do this with Bro? Does ssl_alert 
> event can catch this ?

Yes, there probably is a way to do it, but it will involve quite a bit of manual work.

First - I do not really think that the way that clients handle this case is completely specified. You will probably have to examine the network traffic of different clients, determine how exactly they are ending the connection in this case, identify the client in your Bro script (usually using the cipher suite they send), and then watch for that kind of traffic.

E.g., at least ~1 year ago, Chrome just used to send TCP resets in connections where it did not accept the server certificate. Hence, if you have a client where the cipher suites it specifies match chrome, and you see a certificate exchange followed by a client-initiated reset, it probably rejected the certificate.

Sadly I do not think there is a trivial way to measuer that.

I hope this helps,
 Johanna


From blackhole.em at gmail.com  Wed Apr 15 08:05:56 2015
From: blackhole.em at gmail.com (Joe Blow)
Date: Wed, 15 Apr 2015 11:05:56 -0400
Subject: [Bro] Multi-Thread bro with pcap file?
Message-ID: <CAMCSCoWwZZC3XkZ0AHznw4hYkLKLWWy_s7tR6dS-Zj_VGyNNoA@mail.gmail.com>

Hey everyone,

I was wondering if anyone knows if it is possible to multi-thread BRO when
you are reading the input from a file like this:

/bro/bin/bro -r "$1" /bro/share/bro/site/local.bro "Site::local_nets = {
10.0.0.0/8, 192.168.0.0/16, 216.46.96.0/19, 172.16.0.0/12 }"

Can this only be done with one thread?

Cheers,

JB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150415/872bd729/attachment.html 

From wren3 at illinois.edu  Wed Apr 15 09:36:21 2015
From: wren3 at illinois.edu (Ren, Wenyu)
Date: Wed, 15 Apr 2015 16:36:21 +0000
Subject: [Bro] Question about processing network traffic
In-Reply-To: <8278AC20-D463-493E-9839-CC89F2C77350@icir.org>
References: <D1DD2D198D059443B177F461727B01591A30748F@CHIMBX6.ad.uillinois.edu>,
	<8278AC20-D463-493E-9839-CC89F2C77350@icir.org>
Message-ID: <D1DD2D198D059443B177F461727B01591A307781@CHIMBX6.ad.uillinois.edu>

Hi Seth,

Thanks for the answer. I still have some confusion about this. So the next packet will be buffered at the NIC before Bro finishes processing the current one? Are there chances that two or more packets are processed concurrently? Is this still true if I am using captured traffic traces? 
And if Bro runs a periodic job consuming non-negligible cpu power, how will that affect the packet processing? Will that block the packets from being processed?

Thanks a lot,

Wenyu
________________________________________
From: Seth Hall [seth at icir.org]
Sent: Wednesday, April 15, 2015 7:18 AM
To: Ren, Wenyu
Cc: bro at bro.org
Subject: Re: [Bro] Question about processing network traffic

> On Apr 14, 2015, at 6:20 PM, Ren, Wenyu <wren3 at illinois.edu> wrote:
>
> I have a very basic question about how bro Bro handles network traffic. I am doing some processing on each packet that Bro sees. If the processing time is longer than the packet arriving interval, will Bro block the new packet or buffer the new packet event and deal with it later? If it is buffered, will the event be dropped if the buffer is full?

The packet will be lost once the NIC buffers are exceeded.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From alexwis at gmail.com  Wed Apr 15 11:44:07 2015
From: alexwis at gmail.com (Alex Waher)
Date: Wed, 15 Apr 2015 11:44:07 -0700
Subject: [Bro] Multi-Thread bro with pcap file?
In-Reply-To: <CAMCSCoWwZZC3XkZ0AHznw4hYkLKLWWy_s7tR6dS-Zj_VGyNNoA@mail.gmail.com>
References: <CAMCSCoWwZZC3XkZ0AHznw4hYkLKLWWy_s7tR6dS-Zj_VGyNNoA@mail.gmail.com>
Message-ID: <CAGURB2fYrmKxuV_pxMDGGZCz=_vL4Ubh58vnDHGx53BcUm1D9Q@mail.gmail.com>

tracereplay [http://www.wand.net.nz/trac/libtrace/wiki/TraceReplay] will
let you loop packets around into a dummy network interface from an existing
pcap/erf. It wont be a problem for bro to attach onto the dummy interface
via pf_ring and cluster away at the traffic.

-Alex

On Wed, Apr 15, 2015 at 8:05 AM, Joe Blow <blackhole.em at gmail.com> wrote:

> Hey everyone,
>
> I was wondering if anyone knows if it is possible to multi-thread BRO when
> you are reading the input from a file like this:
>
> /bro/bin/bro -r "$1" /bro/share/bro/site/local.bro "Site::local_nets = {
> 10.0.0.0/8, 192.168.0.0/16, 216.46.96.0/19, 172.16.0.0/12 }"
>
> Can this only be done with one thread?
>
> Cheers,
>
> JB
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150415/cd90ef54/attachment-0001.html 

From luuk.hendriks at utwente.nl  Wed Apr 15 12:09:37 2015
From: luuk.hendriks at utwente.nl (Luuk Hendriks)
Date: Wed, 15 Apr 2015 21:09:37 +0200
Subject: [Bro] Multi-Thread bro with pcap file?
In-Reply-To: <CAMCSCoWwZZC3XkZ0AHznw4hYkLKLWWy_s7tR6dS-Zj_VGyNNoA@mail.gmail.com>
References: <CAMCSCoWwZZC3XkZ0AHznw4hYkLKLWWy_s7tR6dS-Zj_VGyNNoA@mail.gmail.com>
Message-ID: <20150415190937.GB26668@corley.shackle.nl>

Depending on your analysis, splitting the pcap with editcap (or splitting it at capture time already) and using GNU Parallel [0] can be a way of using multiple cores. I've successfully used this like so:

parallel --gnu --bar 'bro -r {} my-script.bro' ::: pcaps/capture.pcap*


The pcaps directory contains multiple pcap files (note the asterisk), and the output is written to stdout. You might want to capture the output to separate files and analyse those afterwards, again depending on what you want to do.


[0] http://www.gnu.org/software/parallel/

Hope this helps, 
 luuk


On Wed 15 Apr 2015, 11:05, Joe Blow wrote:
> Hey everyone,
> 
> I was wondering if anyone knows if it is possible to multi-thread BRO when
> you are reading the input from a file like this:
> 
> /bro/bin/bro -r "$1" /bro/share/bro/site/local.bro "Site::local_nets = {
> 10.0.0.0/8, 192.168.0.0/16, 216.46.96.0/19, 172.16.0.0/12 }"
> 
> Can this only be done with one thread?
> 
> Cheers,
> 
> JB

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From giedrius.ramas at gmail.com  Thu Apr 16 00:57:13 2015
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Thu, 16 Apr 2015 10:57:13 +0300
Subject: [Bro] erspan decapsulation
In-Reply-To: <FFDC5F68-88BC-48B7-9B62-EB94627A3A01@icir.org>
References: <CAHr6V5o+K2=daVqPBxigkwHGaKi7MTMvcZoe8qXQs-4riy_LNg@mail.gmail.com>
	<FFDC5F68-88BC-48B7-9B62-EB94627A3A01@icir.org>
Message-ID: <CAHr6V5pEkz00mpFkWB698_zAv_vxnJmav_UhuPvbYUtyj3eQpA@mail.gmail.com>

Thanks for reply,
I just figure out that I need to skip some bytes of package header. In my
current case I need to skip 22 bytes. So I edited ini-bare.bro file and
changed encap_hdr_size = 0 line to encap_hdr_size = 22 . BRO can now
understand traffic. Do not know  if I made a correct fix. Let me know if it
is not a right way to do .


On Wed, Apr 15, 2015 at 3:09 PM, Seth Hall <seth at icir.org> wrote:

>
> > On Apr 14, 2015, at 10:57 AM, Giedrius Ramas <giedrius.ramas at gmail.com>
> wrote:
> >
> > Hello, we have problems with ERSPAN package. Is there anyway BRO could
> understand them ?
>
> Could you privately provide us a small packet capture of ERSPAN packets?
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150416/376804d4/attachment.html 

From jlay at slave-tothe-box.net  Thu Apr 16 04:55:44 2015
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 16 Apr 2015 05:55:44 -0600
Subject: [Bro] An assist with file extraction
Message-ID: <1429185344.4032.18.camel@JamesiMac>

Hey all,

The topic pretty much says it...I've done a fair amount of reading
trying to determine the best way to extract file attachments in smtp
traffic.  Most of the information I've found is related to older
versions of bro.  Can someone point me to a current resource that will
work with the current version of bro?  Thank you.

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150416/bdd85692/attachment.html 

From hosom at battelle.org  Thu Apr 16 06:04:40 2015
From: hosom at battelle.org (Hosom, Stephen M)
Date: Thu, 16 Apr 2015 13:04:40 +0000
Subject: [Bro] An assist with file extraction
In-Reply-To: <1429185344.4032.18.camel@JamesiMac>
References: <1429185344.4032.18.camel@JamesiMac>
Message-ID: <E5A2071C6FFFF640B714C224139C9B3301A5D782@WP-MBX2B.milky-way.battelle.org>

For 2.3.2 (current release) you?ll want to use the event file_new.

Note that in 2.3.2 if you are extracting based on mime_type (most people do) you will want to verify that the field exists before you actually use it.

For master, which is what you are likely referring to? you?ll want the event file_mime_type.



From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of James Lay
Sent: Thursday, April 16, 2015 7:56 AM
To: Bro-IDS
Subject: [Bro] An assist with file extraction

Hey all,

The topic pretty much says it...I've done a fair amount of reading trying to determine the best way to extract file attachments in smtp traffic.  Most of the information I've found is related to older versions of bro.  Can someone point me to a current resource that will work with the current version of bro?  Thank you.

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150416/1ccc80da/attachment.html 

From lists at g-clef.net  Thu Apr 16 04:33:36 2015
From: lists at g-clef.net (Aaron Gee-Clough)
Date: Thu, 16 Apr 2015 07:33:36 -0400
Subject: [Bro] working with MS15-034
Message-ID: <552F9E10.5000806@g-clef.net>


All,

I'm working on a bro script to detect attempts for the
recently-announced IIS attack. I've hit an interesting issue: There's a
magic number that gets sent in the HTTP "RANGE" header to trigger the
vulnerability, and that number is 2^64. This is right at the edge of
what a "count" variable can hold, and it wraps around a regular "int"
variable.

I'd like to be able to detect anyone sending any number >= 2^64 in a
RANGE header, but I don't see how to do that with count variables in
bro. Does anyone have any ideas of how I can do this? Right now I'm
looking at doing something truly nasty, like comparing the length of the
strings holding the Range values. I'm *really* not happy with that,
though...it feels like a really ugly hack.

aaron

From kristoffer.bjork at gmail.com  Thu Apr 16 07:46:05 2015
From: kristoffer.bjork at gmail.com (=?UTF-8?Q?Kristoffer_Bj=C3=B6rk?=)
Date: Thu, 16 Apr 2015 16:46:05 +0200
Subject: [Bro] erspan decapsulation
In-Reply-To: <CAHr6V5pEkz00mpFkWB698_zAv_vxnJmav_UhuPvbYUtyj3eQpA@mail.gmail.com>
References: <CAHr6V5o+K2=daVqPBxigkwHGaKi7MTMvcZoe8qXQs-4riy_LNg@mail.gmail.com>
	<FFDC5F68-88BC-48B7-9B62-EB94627A3A01@icir.org>
	<CAHr6V5pEkz00mpFkWB698_zAv_vxnJmav_UhuPvbYUtyj3eQpA@mail.gmail.com>
Message-ID: <CAEGuVoLRf0xb7pY7wdCmQZ_CC7DM++qc4u3dqAZpi9M9DWyq0A@mail.gmail.com>

It should be ok to chop off the first bytes.
ERSPAN is basically cisco rspan with a GRE encapsulation.
I have been using GULP (https://staff.washington.edu/corey/gulp/) and
piping from gulp to bro -r - but your method is much cleaner way of doing
it.
Beware of not MTU issues though, since packets might get chopped off at the
end if they do not fit after the GRE encapsulation

//K

On Thu, Apr 16, 2015 at 9:57 AM, Giedrius Ramas <giedrius.ramas at gmail.com>
wrote:

> Thanks for reply,
> I just figure out that I need to skip some bytes of package header. In my
> current case I need to skip 22 bytes. So I edited ini-bare.bro file and
> changed encap_hdr_size = 0 line to encap_hdr_size = 22 . BRO can now
> understand traffic. Do not know  if I made a correct fix. Let me know if it
> is not a right way to do .
>
>
> On Wed, Apr 15, 2015 at 3:09 PM, Seth Hall <seth at icir.org> wrote:
>
>>
>> > On Apr 14, 2015, at 10:57 AM, Giedrius Ramas <giedrius.ramas at gmail.com>
>> wrote:
>> >
>> > Hello, we have problems with ERSPAN package. Is there anyway BRO could
>> understand them ?
>>
>> Could you privately provide us a small packet capture of ERSPAN packets?
>>
>>   .Seth
>>
>> --
>> Seth Hall
>> International Computer Science Institute
>> (Bro) because everyone has a network
>> http://www.bro.org/
>>
>>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150416/5c5b1b4f/attachment-0001.html 

From kristoffer.bjork at gmail.com  Thu Apr 16 08:15:47 2015
From: kristoffer.bjork at gmail.com (=?UTF-8?Q?Kristoffer_Bj=C3=B6rk?=)
Date: Thu, 16 Apr 2015 17:15:47 +0200
Subject: [Bro] erspan decapsulation
In-Reply-To: <CAEGuVoLRf0xb7pY7wdCmQZ_CC7DM++qc4u3dqAZpi9M9DWyq0A@mail.gmail.com>
References: <CAHr6V5o+K2=daVqPBxigkwHGaKi7MTMvcZoe8qXQs-4riy_LNg@mail.gmail.com>
	<FFDC5F68-88BC-48B7-9B62-EB94627A3A01@icir.org>
	<CAHr6V5pEkz00mpFkWB698_zAv_vxnJmav_UhuPvbYUtyj3eQpA@mail.gmail.com>
	<CAEGuVoLRf0xb7pY7wdCmQZ_CC7DM++qc4u3dqAZpi9M9DWyq0A@mail.gmail.com>
Message-ID: <CAEGuVoKFEgRPbbO7=6e5L1H31d59DJB-Ynnu9VEYnNawAcVWOw@mail.gmail.com>

However, to me it looks like 50bytes instead of 22 bytes? Like in this:
https://staff.washington.edu/corey/gulp/conv.c
But i guess bro deencapsulates the gre tunnel for you?

//K

On Thu, Apr 16, 2015 at 4:46 PM, Kristoffer Bj?rk <
kristoffer.bjork at gmail.com> wrote:

> It should be ok to chop off the first bytes.
> ERSPAN is basically cisco rspan with a GRE encapsulation.
> I have been using GULP (https://staff.washington.edu/corey/gulp/) and
> piping from gulp to bro -r - but your method is much cleaner way of doing
> it.
> Beware of not MTU issues though, since packets might get chopped off at
> the end if they do not fit after the GRE encapsulation
>
> //K
>
> On Thu, Apr 16, 2015 at 9:57 AM, Giedrius Ramas <giedrius.ramas at gmail.com>
> wrote:
>
>> Thanks for reply,
>> I just figure out that I need to skip some bytes of package header. In my
>> current case I need to skip 22 bytes. So I edited ini-bare.bro file and
>> changed encap_hdr_size = 0 line to encap_hdr_size = 22 . BRO can now
>> understand traffic. Do not know  if I made a correct fix. Let me know if it
>> is not a right way to do .
>>
>>
>> On Wed, Apr 15, 2015 at 3:09 PM, Seth Hall <seth at icir.org> wrote:
>>
>>>
>>> > On Apr 14, 2015, at 10:57 AM, Giedrius Ramas <giedrius.ramas at gmail.com>
>>> wrote:
>>> >
>>> > Hello, we have problems with ERSPAN package. Is there anyway BRO could
>>> understand them ?
>>>
>>> Could you privately provide us a small packet capture of ERSPAN packets?
>>>
>>>   .Seth
>>>
>>> --
>>> Seth Hall
>>> International Computer Science Institute
>>> (Bro) because everyone has a network
>>> http://www.bro.org/
>>>
>>>
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150416/b67a46a2/attachment.html 

From liburdi.joshua at gmail.com  Thu Apr 16 09:12:58 2015
From: liburdi.joshua at gmail.com (Josh Liburdi)
Date: Thu, 16 Apr 2015 09:12:58 -0700
Subject: [Bro] Fwd:  working with MS15-034
In-Reply-To: <CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
References: <552F9E10.5000806@g-clef.net>
	<CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
Message-ID: <CANrCRiFShJXTuGJD1x9+ic3_rPwskje1dp7EnKMAAas4HO8=1Q@mail.gmail.com>

Forwarding to the rest of the Bro list ...


---------- Forwarded message ----------
From: Josh Liburdi <liburdi.joshua at gmail.com>
Date: Thu, Apr 16, 2015 at 9:11 AM
Subject: Re: [Bro] working with MS15-034
To: Aaron Gee-Clough <lists at g-clef.net>


The Range header value in Bro should be a string-- if you're looking
to detect a specific magic number in this value, then instead of
converting the values to counts, you could match it like this by
leaving that magic number as a string:

if ( name == "RANGE" && "string" in value )

Josh

On Thu, Apr 16, 2015 at 4:33 AM, Aaron Gee-Clough <lists at g-clef.net> wrote:
>
> All,
>
> I'm working on a bro script to detect attempts for the
> recently-announced IIS attack. I've hit an interesting issue: There's a
> magic number that gets sent in the HTTP "RANGE" header to trigger the
> vulnerability, and that number is 2^64. This is right at the edge of
> what a "count" variable can hold, and it wraps around a regular "int"
> variable.
>
> I'd like to be able to detect anyone sending any number >= 2^64 in a
> RANGE header, but I don't see how to do that with count variables in
> bro. Does anyone have any ideas of how I can do this? Right now I'm
> looking at doing something truly nasty, like comparing the length of the
> strings holding the Range values. I'm *really* not happy with that,
> though...it feels like a really ugly hack.
>
> aaron
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

From lists at g-clef.net  Thu Apr 16 09:19:08 2015
From: lists at g-clef.net (Aaron Gee-Clough)
Date: Thu, 16 Apr 2015 12:19:08 -0400
Subject: [Bro] working with MS15-034
References: <552F9E10.5000806@g-clef.net>
	<CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
Message-ID: <552FE0FC.80604@g-clef.net>


True, but I was hoping to do more than just detect the magic number. I
was hoping to be able to say something along the lines of:

	if (name == "RANGE" && value > 2^64 )

My thinking here is that I don't want to play whack-a-mole with magic
numbers. I would like to flag any request for an offset that big as a
potential problem.

aaron

On 04/16/2015 12:11 PM, Josh Liburdi wrote:
> 
> The Range header value in Bro should be a string-- if you're looking
> to detect a specific magic number in this value, then instead of
> converting the values to counts, you could match it like this by
> leaving that magic number as a string:
> 
> if ( name == "RANGE" && "string" in value )
> 
> Josh
> 
> On Thu, Apr 16, 2015 at 4:33 AM, Aaron Gee-Clough <lists at g-clef.net> wrote:
>>
>> All,
>>
>> I'm working on a bro script to detect attempts for the
>> recently-announced IIS attack. I've hit an interesting issue: There's a
>> magic number that gets sent in the HTTP "RANGE" header to trigger the
>> vulnerability, and that number is 2^64. This is right at the edge of
>> what a "count" variable can hold, and it wraps around a regular "int"
>> variable.
>>
>> I'd like to be able to detect anyone sending any number >= 2^64 in a
>> RANGE header, but I don't see how to do that with count variables in
>> bro. Does anyone have any ideas of how I can do this? Right now I'm
>> looking at doing something truly nasty, like comparing the length of the
>> strings holding the Range values. I'm *really* not happy with that,
>> though...it feels like a really ugly hack.
>>
>> aaron
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

From vlad at grigorescu.org  Thu Apr 16 09:43:12 2015
From: vlad at grigorescu.org (Vlad Grigorescu)
Date: Thu, 16 Apr 2015 11:43:12 -0500
Subject: [Bro] working with MS15-034
In-Reply-To: <552FE0FC.80604@g-clef.net>
References: <552F9E10.5000806@g-clef.net>
	<CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
	<552FE0FC.80604@g-clef.net>
Message-ID: <CANUJ_FmYD2f-HcnNyGq_1TPiyyO0axYshGFWXx14B6zucMi2KQ@mail.gmail.com>

You can use to_double:

> $ bro -e 'print to_double("987654321123456789");'
> 9.876543e+17

  --Vlad

On Thu, Apr 16, 2015 at 11:19 AM, Aaron Gee-Clough <lists at g-clef.net> wrote:

>
> True, but I was hoping to do more than just detect the magic number. I
> was hoping to be able to say something along the lines of:
>
>         if (name == "RANGE" && value > 2^64 )
>
> My thinking here is that I don't want to play whack-a-mole with magic
> numbers. I would like to flag any request for an offset that big as a
> potential problem.
>
> aaron
>
> On 04/16/2015 12:11 PM, Josh Liburdi wrote:
> >
> > The Range header value in Bro should be a string-- if you're looking
> > to detect a specific magic number in this value, then instead of
> > converting the values to counts, you could match it like this by
> > leaving that magic number as a string:
> >
> > if ( name == "RANGE" && "string" in value )
> >
> > Josh
> >
> > On Thu, Apr 16, 2015 at 4:33 AM, Aaron Gee-Clough <lists at g-clef.net>
> wrote:
> >>
> >> All,
> >>
> >> I'm working on a bro script to detect attempts for the
> >> recently-announced IIS attack. I've hit an interesting issue: There's a
> >> magic number that gets sent in the HTTP "RANGE" header to trigger the
> >> vulnerability, and that number is 2^64. This is right at the edge of
> >> what a "count" variable can hold, and it wraps around a regular "int"
> >> variable.
> >>
> >> I'd like to be able to detect anyone sending any number >= 2^64 in a
> >> RANGE header, but I don't see how to do that with count variables in
> >> bro. Does anyone have any ideas of how I can do this? Right now I'm
> >> looking at doing something truly nasty, like comparing the length of the
> >> strings holding the Range values. I'm *really* not happy with that,
> >> though...it feels like a really ugly hack.
> >>
> >> aaron
> >> _______________________________________________
> >> Bro mailing list
> >> bro at bro-ids.org
> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150416/23918930/attachment.html 

From liburdi.joshua at gmail.com  Thu Apr 16 09:45:15 2015
From: liburdi.joshua at gmail.com (Josh Liburdi)
Date: Thu, 16 Apr 2015 09:45:15 -0700
Subject: [Bro] working with MS15-034
In-Reply-To: <CANUJ_FmYD2f-HcnNyGq_1TPiyyO0axYshGFWXx14B6zucMi2KQ@mail.gmail.com>
References: <552F9E10.5000806@g-clef.net>
	<CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
	<552FE0FC.80604@g-clef.net>
	<CANUJ_FmYD2f-HcnNyGq_1TPiyyO0axYshGFWXx14B6zucMi2KQ@mail.gmail.com>
Message-ID: <CANrCRiE5FrGoLFK4ekVNR8QBSajSuTg_H6Gd1uhiFfC+DSLvpg@mail.gmail.com>

I agree, I think double's are the way to go ... but the behavior is
odd: http://try.bro.org/#/trybro/saved/3780

It doesn't recognize the numbers as being equal.

Josh

On Thu, Apr 16, 2015 at 9:43 AM, Vlad Grigorescu <vlad at grigorescu.org> wrote:
> You can use to_double:
>
>> $ bro -e 'print to_double("987654321123456789");'
>> 9.876543e+17
>
>   --Vlad
>
> On Thu, Apr 16, 2015 at 11:19 AM, Aaron Gee-Clough <lists at g-clef.net> wrote:
>>
>>
>> True, but I was hoping to do more than just detect the magic number. I
>> was hoping to be able to say something along the lines of:
>>
>>         if (name == "RANGE" && value > 2^64 )
>>
>> My thinking here is that I don't want to play whack-a-mole with magic
>> numbers. I would like to flag any request for an offset that big as a
>> potential problem.
>>
>> aaron
>>
>> On 04/16/2015 12:11 PM, Josh Liburdi wrote:
>> >
>> > The Range header value in Bro should be a string-- if you're looking
>> > to detect a specific magic number in this value, then instead of
>> > converting the values to counts, you could match it like this by
>> > leaving that magic number as a string:
>> >
>> > if ( name == "RANGE" && "string" in value )
>> >
>> > Josh
>> >
>> > On Thu, Apr 16, 2015 at 4:33 AM, Aaron Gee-Clough <lists at g-clef.net>
>> > wrote:
>> >>
>> >> All,
>> >>
>> >> I'm working on a bro script to detect attempts for the
>> >> recently-announced IIS attack. I've hit an interesting issue: There's a
>> >> magic number that gets sent in the HTTP "RANGE" header to trigger the
>> >> vulnerability, and that number is 2^64. This is right at the edge of
>> >> what a "count" variable can hold, and it wraps around a regular "int"
>> >> variable.
>> >>
>> >> I'd like to be able to detect anyone sending any number >= 2^64 in a
>> >> RANGE header, but I don't see how to do that with count variables in
>> >> bro. Does anyone have any ideas of how I can do this? Right now I'm
>> >> looking at doing something truly nasty, like comparing the length of
>> >> the
>> >> strings holding the Range values. I'm *really* not happy with that,
>> >> though...it feels like a really ugly hack.
>> >>
>> >> aaron
>> >> _______________________________________________
>> >> Bro mailing list
>> >> bro at bro-ids.org
>> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>

From liburdi.joshua at gmail.com  Thu Apr 16 09:47:10 2015
From: liburdi.joshua at gmail.com (Josh Liburdi)
Date: Thu, 16 Apr 2015 09:47:10 -0700
Subject: [Bro] working with MS15-034
In-Reply-To: <CANrCRiE5FrGoLFK4ekVNR8QBSajSuTg_H6Gd1uhiFfC+DSLvpg@mail.gmail.com>
References: <552F9E10.5000806@g-clef.net>
	<CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
	<552FE0FC.80604@g-clef.net>
	<CANUJ_FmYD2f-HcnNyGq_1TPiyyO0axYshGFWXx14B6zucMi2KQ@mail.gmail.com>
	<CANrCRiE5FrGoLFK4ekVNR8QBSajSuTg_H6Gd1uhiFfC+DSLvpg@mail.gmail.com>
Message-ID: <CANrCRiHTbCfvSq+hnf3WT4bv1-CTHmcGNsHUPppdskPSdQBoBQ@mail.gmail.com>

Better version here: http://try.bro.org/#/trybro/saved/3782

On Thu, Apr 16, 2015 at 9:45 AM, Josh Liburdi <liburdi.joshua at gmail.com> wrote:
> I agree, I think double's are the way to go ... but the behavior is
> odd: http://try.bro.org/#/trybro/saved/3780
>
> It doesn't recognize the numbers as being equal.
>
> Josh
>
> On Thu, Apr 16, 2015 at 9:43 AM, Vlad Grigorescu <vlad at grigorescu.org> wrote:
>> You can use to_double:
>>
>>> $ bro -e 'print to_double("987654321123456789");'
>>> 9.876543e+17
>>
>>   --Vlad
>>
>> On Thu, Apr 16, 2015 at 11:19 AM, Aaron Gee-Clough <lists at g-clef.net> wrote:
>>>
>>>
>>> True, but I was hoping to do more than just detect the magic number. I
>>> was hoping to be able to say something along the lines of:
>>>
>>>         if (name == "RANGE" && value > 2^64 )
>>>
>>> My thinking here is that I don't want to play whack-a-mole with magic
>>> numbers. I would like to flag any request for an offset that big as a
>>> potential problem.
>>>
>>> aaron
>>>
>>> On 04/16/2015 12:11 PM, Josh Liburdi wrote:
>>> >
>>> > The Range header value in Bro should be a string-- if you're looking
>>> > to detect a specific magic number in this value, then instead of
>>> > converting the values to counts, you could match it like this by
>>> > leaving that magic number as a string:
>>> >
>>> > if ( name == "RANGE" && "string" in value )
>>> >
>>> > Josh
>>> >
>>> > On Thu, Apr 16, 2015 at 4:33 AM, Aaron Gee-Clough <lists at g-clef.net>
>>> > wrote:
>>> >>
>>> >> All,
>>> >>
>>> >> I'm working on a bro script to detect attempts for the
>>> >> recently-announced IIS attack. I've hit an interesting issue: There's a
>>> >> magic number that gets sent in the HTTP "RANGE" header to trigger the
>>> >> vulnerability, and that number is 2^64. This is right at the edge of
>>> >> what a "count" variable can hold, and it wraps around a regular "int"
>>> >> variable.
>>> >>
>>> >> I'd like to be able to detect anyone sending any number >= 2^64 in a
>>> >> RANGE header, but I don't see how to do that with count variables in
>>> >> bro. Does anyone have any ideas of how I can do this? Right now I'm
>>> >> looking at doing something truly nasty, like comparing the length of
>>> >> the
>>> >> strings holding the Range values. I'm *really* not happy with that,
>>> >> though...it feels like a really ugly hack.
>>> >>
>>> >> aaron
>>> >> _______________________________________________
>>> >> Bro mailing list
>>> >> bro at bro-ids.org
>>> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>

From vlad at grigorescu.org  Thu Apr 16 09:52:38 2015
From: vlad at grigorescu.org (Vlad Grigorescu)
Date: Thu, 16 Apr 2015 11:52:38 -0500
Subject: [Bro] working with MS15-034
In-Reply-To: <CANrCRiHTbCfvSq+hnf3WT4bv1-CTHmcGNsHUPppdskPSdQBoBQ@mail.gmail.com>
References: <552F9E10.5000806@g-clef.net>
	<CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
	<552FE0FC.80604@g-clef.net>
	<CANUJ_FmYD2f-HcnNyGq_1TPiyyO0axYshGFWXx14B6zucMi2KQ@mail.gmail.com>
	<CANrCRiE5FrGoLFK4ekVNR8QBSajSuTg_H6Gd1uhiFfC+DSLvpg@mail.gmail.com>
	<CANrCRiHTbCfvSq+hnf3WT4bv1-CTHmcGNsHUPppdskPSdQBoBQ@mail.gmail.com>
Message-ID: <CANUJ_FkdhomoC+GtaGGrn4i4B_k06fUJ6sXNdF0MkM5-uLa9nw@mail.gmail.com>

Well... they're not equal. :-)

magic is 18446744073709551615,
while d is 18446740000000000000

See this:

http://try.bro.org/#/trybro/saved/3786

On Thu, Apr 16, 2015 at 11:47 AM, Josh Liburdi <liburdi.joshua at gmail.com>
wrote:

> Better version here: http://try.bro.org/#/trybro/saved/3782
>
> On Thu, Apr 16, 2015 at 9:45 AM, Josh Liburdi <liburdi.joshua at gmail.com>
> wrote:
> > I agree, I think double's are the way to go ... but the behavior is
> > odd: http://try.bro.org/#/trybro/saved/3780
> >
> > It doesn't recognize the numbers as being equal.
> >
> > Josh
> >
> > On Thu, Apr 16, 2015 at 9:43 AM, Vlad Grigorescu <vlad at grigorescu.org>
> wrote:
> >> You can use to_double:
> >>
> >>> $ bro -e 'print to_double("987654321123456789");'
> >>> 9.876543e+17
> >>
> >>   --Vlad
> >>
> >> On Thu, Apr 16, 2015 at 11:19 AM, Aaron Gee-Clough <lists at g-clef.net>
> wrote:
> >>>
> >>>
> >>> True, but I was hoping to do more than just detect the magic number. I
> >>> was hoping to be able to say something along the lines of:
> >>>
> >>>         if (name == "RANGE" && value > 2^64 )
> >>>
> >>> My thinking here is that I don't want to play whack-a-mole with magic
> >>> numbers. I would like to flag any request for an offset that big as a
> >>> potential problem.
> >>>
> >>> aaron
> >>>
> >>> On 04/16/2015 12:11 PM, Josh Liburdi wrote:
> >>> >
> >>> > The Range header value in Bro should be a string-- if you're looking
> >>> > to detect a specific magic number in this value, then instead of
> >>> > converting the values to counts, you could match it like this by
> >>> > leaving that magic number as a string:
> >>> >
> >>> > if ( name == "RANGE" && "string" in value )
> >>> >
> >>> > Josh
> >>> >
> >>> > On Thu, Apr 16, 2015 at 4:33 AM, Aaron Gee-Clough <lists at g-clef.net>
> >>> > wrote:
> >>> >>
> >>> >> All,
> >>> >>
> >>> >> I'm working on a bro script to detect attempts for the
> >>> >> recently-announced IIS attack. I've hit an interesting issue:
> There's a
> >>> >> magic number that gets sent in the HTTP "RANGE" header to trigger
> the
> >>> >> vulnerability, and that number is 2^64. This is right at the edge of
> >>> >> what a "count" variable can hold, and it wraps around a regular
> "int"
> >>> >> variable.
> >>> >>
> >>> >> I'd like to be able to detect anyone sending any number >= 2^64 in a
> >>> >> RANGE header, but I don't see how to do that with count variables in
> >>> >> bro. Does anyone have any ideas of how I can do this? Right now I'm
> >>> >> looking at doing something truly nasty, like comparing the length of
> >>> >> the
> >>> >> strings holding the Range values. I'm *really* not happy with that,
> >>> >> though...it feels like a really ugly hack.
> >>> >>
> >>> >> aaron
> >>> >> _______________________________________________
> >>> >> Bro mailing list
> >>> >> bro at bro-ids.org
> >>> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >>> _______________________________________________
> >>> Bro mailing list
> >>> bro at bro-ids.org
> >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >>
> >>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150416/10d7b776/attachment.html 

From lists at g-clef.net  Thu Apr 16 09:58:11 2015
From: lists at g-clef.net (Aaron Gee-Clough)
Date: Thu, 16 Apr 2015 12:58:11 -0400
Subject: [Bro] working with MS15-034
References: <552F9E10.5000806@g-clef.net>
	<CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
	<552FE0FC.80604@g-clef.net>
	<CANUJ_FmYD2f-HcnNyGq_1TPiyyO0axYshGFWXx14B6zucMi2KQ@mail.gmail.com>
	<CANrCRiE5FrGoLFK4ekVNR8QBSajSuTg_H6Gd1uhiFfC+DSLvpg@mail.gmail.com>
	<CANrCRiHTbCfvSq+hnf3WT4bv1-CTHmcGNsHUPppdskPSdQBoBQ@mail.gmail.com>
	<CANUJ_FkdhomoC+GtaGGrn4i4B_k06fUJ6sXNdF0MkM5-uLa9nw@mail.gmail.com>
Message-ID: <552FEA23.40208@g-clef.net>


Thanks for the info. One question: if I declare:

const max_val = 1.8446744073709551615e+19;

Will the double keep all that precision?

aaron


On 04/16/2015 12:52 PM, Vlad Grigorescu wrote:
> Well... they're not equal. :-)
> 
> magic is 18446744073709551615, 
> while d is 18446740000000000000
> 
> See this:
> 
> http://try.bro.org/#/trybro/saved/3786
> 
> On Thu, Apr 16, 2015 at 11:47 AM, Josh Liburdi <liburdi.joshua at gmail.com
> <mailto:liburdi.joshua at gmail.com>> wrote:
> 
>     Better version here: http://try.bro.org/#/trybro/saved/3782
> 
>     On Thu, Apr 16, 2015 at 9:45 AM, Josh Liburdi
>     <liburdi.joshua at gmail.com <mailto:liburdi..joshua at gmail.com>> wrote:
>     > I agree, I think double's are the way to go ... but the behavior is
>     > odd: http://try.bro.org/#/trybro/saved/3780
>     >
>     > It doesn't recognize the numbers as being equal.
>     >
>     > Josh
>     >
>     > On Thu, Apr 16, 2015 at 9:43 AM, Vlad Grigorescu
>     <vlad at grigorescu.org <mailto:vlad at grigorescu.org>> wrote:
>     >> You can use to_double:
>     >>
>     >>> $ bro -e 'print to_double("987654321123456789");'
>     >>> 9.876543e+17
>     >>
>     >>   --Vlad
>     >>
>     >> On Thu, Apr 16, 2015 at 11:19 AM, Aaron Gee-Clough
>     <lists at g-clef.net <mailto:lists at g-clef.net>> wrote:
>     >>>
>     >>>
>     >>> True, but I was hoping to do more than just detect the magic
>     number. I
>     >>> was hoping to be able to say something along the lines of:
>     >>>
>     >>>         if (name == "RANGE" && value > 2^64 )
>     >>>
>     >>> My thinking here is that I don't want to play whack-a-mole with
>     magic
>     >>> numbers. I would like to flag any request for an offset that big
>     as a
>     >>> potential problem.
>     >>>
>     >>> aaron
>     >>>
>     >>> On 04/16/2015 12:11 PM, Josh Liburdi wrote:
>     >>> >
>     >>> > The Range header value in Bro should be a string-- if you're
>     looking
>     >>> > to detect a specific magic number in this value, then instead of
>     >>> > converting the values to counts, you could match it like this by
>     >>> > leaving that magic number as a string:
>     >>> >
>     >>> > if ( name == "RANGE" && "string" in value )
>     >>> >
>     >>> > Josh
>     >>> >
>     >>> > On Thu, Apr 16, 2015 at 4:33 AM, Aaron Gee-Clough
>     <lists at g-clef.net <mailto:lists at g-clef.net>>
>     >>> > wrote:
>     >>> >>
>     >>> >> All,
>     >>> >>
>     >>> >> I'm working on a bro script to detect attempts for the
>     >>> >> recently-announced IIS attack. I've hit an interesting issue:
>     There's a
>     >>> >> magic number that gets sent in the HTTP "RANGE" header to
>     trigger the
>     >>> >> vulnerability, and that number is 2^64. This is right at the
>     edge of
>     >>> >> what a "count" variable can hold, and it wraps around a
>     regular "int"
>     >>> >> variable.
>     >>> >>
>     >>> >> I'd like to be able to detect anyone sending any number >=
>     2^64 in a
>     >>> >> RANGE header, but I don't see how to do that with count
>     variables in
>     >>> >> bro. Does anyone have any ideas of how I can do this? Right
>     now I'm
>     >>> >> looking at doing something truly nasty, like comparing the
>     length of
>     >>> >> the
>     >>> >> strings holding the Range values. I'm *really* not happy with
>     that,
>     >>> >> though...it feels like a really ugly hack.
>     >>> >>
>     >>> >> aaron
>     >>> >> _______________________________________________
>     >>> >> Bro mailing list
>     >>> >> bro at bro-ids.org <mailto:bro at bro-ids.org>
>     >>> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>     >>> _______________________________________________
>     >>> Bro mailing list
>     >>> bro at bro-ids.org <mailto:bro at bro-ids.org>
>     >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>     >>
>     >>
> 
> 

From tim.wier at cuchicago.edu  Thu Apr 16 10:10:31 2015
From: tim.wier at cuchicago.edu (Wier, Timothy A.)
Date: Thu, 16 Apr 2015 12:10:31 -0500
Subject: [Bro] working with MS15-034
In-Reply-To: <CANUJ_FkdhomoC+GtaGGrn4i4B_k06fUJ6sXNdF0MkM5-uLa9nw@mail.gmail.com>
References: <552F9E10.5000806@g-clef.net>
	<CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
	<552FE0FC.80604@g-clef.net>
	<CANUJ_FmYD2f-HcnNyGq_1TPiyyO0axYshGFWXx14B6zucMi2KQ@mail.gmail.com>
	<CANrCRiE5FrGoLFK4ekVNR8QBSajSuTg_H6Gd1uhiFfC+DSLvpg@mail.gmail.com>
	<CANrCRiHTbCfvSq+hnf3WT4bv1-CTHmcGNsHUPppdskPSdQBoBQ@mail.gmail.com>
	<CANUJ_FkdhomoC+GtaGGrn4i4B_k06fUJ6sXNdF0MkM5-uLa9nw@mail.gmail.com>
Message-ID: <E7C6DFDE498C8A4BA466F17123F8CE6E02323676CF8F@E2K7MBX1.CNET.local>

This is what I?ve been playing with: http://try.bro.org/#/trybro/saved/3789.

Not sure how it will run in production.
Tim

From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Vlad Grigorescu
Sent: Thursday, April 16, 2015 11:53 AM
To: Josh Liburdi
Cc: bro at bro.org
Subject: Re: [Bro] working with MS15-034

Well... they're not equal. :-)

magic is 18446744073709551615,
while d is 18446740000000000000

See this:

http://try.bro.org/#/trybro/saved/3786

On Thu, Apr 16, 2015 at 11:47 AM, Josh Liburdi <liburdi.joshua at gmail.com<mailto:liburdi.joshua at gmail.com>> wrote:
Better version here: http://try.bro.org/#/trybro/saved/3782

On Thu, Apr 16, 2015 at 9:45 AM, Josh Liburdi <liburdi.joshua at gmail.com<mailto:liburdi.joshua at gmail.com>> wrote:
> I agree, I think double's are the way to go ... but the behavior is
> odd: http://try.bro.org/#/trybro/saved/3780
>
> It doesn't recognize the numbers as being equal.
>
> Josh
>
> On Thu, Apr 16, 2015 at 9:43 AM, Vlad Grigorescu <vlad at grigorescu.org<mailto:vlad at grigorescu.org>> wrote:
>> You can use to_double:
>>
>>> $ bro -e 'print to_double("987654321123456789");'
>>> 9.876543e+17
>>
>>   --Vlad
>>
>> On Thu, Apr 16, 2015 at 11:19 AM, Aaron Gee-Clough <lists at g-clef.net<mailto:lists at g-clef.net>> wrote:
>>>
>>>
>>> True, but I was hoping to do more than just detect the magic number. I
>>> was hoping to be able to say something along the lines of:
>>>
>>>         if (name == "RANGE" && value > 2^64 )
>>>
>>> My thinking here is that I don't want to play whack-a-mole with magic
>>> numbers. I would like to flag any request for an offset that big as a
>>> potential problem.
>>>
>>> aaron
>>>
>>> On 04/16/2015 12:11 PM, Josh Liburdi wrote:
>>> >
>>> > The Range header value in Bro should be a string-- if you're looking
>>> > to detect a specific magic number in this value, then instead of
>>> > converting the values to counts, you could match it like this by
>>> > leaving that magic number as a string:
>>> >
>>> > if ( name == "RANGE" && "string" in value )
>>> >
>>> > Josh
>>> >
>>> > On Thu, Apr 16, 2015 at 4:33 AM, Aaron Gee-Clough <lists at g-clef.net<mailto:lists at g-clef.net>>
>>> > wrote:
>>> >>
>>> >> All,
>>> >>
>>> >> I'm working on a bro script to detect attempts for the
>>> >> recently-announced IIS attack. I've hit an interesting issue: There's a
>>> >> magic number that gets sent in the HTTP "RANGE" header to trigger the
>>> >> vulnerability, and that number is 2^64. This is right at the edge of
>>> >> what a "count" variable can hold, and it wraps around a regular "int"
>>> >> variable.
>>> >>
>>> >> I'd like to be able to detect anyone sending any number >= 2^64 in a
>>> >> RANGE header, but I don't see how to do that with count variables in
>>> >> bro. Does anyone have any ideas of how I can do this? Right now I'm
>>> >> looking at doing something truly nasty, like comparing the length of
>>> >> the
>>> >> strings holding the Range values. I'm *really* not happy with that,
>>> >> though...it feels like a really ugly hack.
>>> >>
>>> >> aaron
>>> >> _______________________________________________
>>> >> Bro mailing list
>>> >> bro at bro-ids.org<mailto:bro at bro-ids.org>
>>> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org<mailto:bro at bro-ids.org>
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150416/c01eeecf/attachment-0001.html 

From awells at digiumcloud.com  Thu Apr 16 10:14:56 2015
From: awells at digiumcloud.com (Aubrey Wells)
Date: Thu, 16 Apr 2015 13:14:56 -0400
Subject: [Bro] working with MS15-034
In-Reply-To: <CANrCRiE5FrGoLFK4ekVNR8QBSajSuTg_H6Gd1uhiFfC+DSLvpg@mail.gmail.com>
References: <552F9E10.5000806@g-clef.net>
	<CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
	<552FE0FC.80604@g-clef.net>
	<CANUJ_FmYD2f-HcnNyGq_1TPiyyO0axYshGFWXx14B6zucMi2KQ@mail.gmail.com>
	<CANrCRiE5FrGoLFK4ekVNR8QBSajSuTg_H6Gd1uhiFfC+DSLvpg@mail.gmail.com>
Message-ID: <CAJ+z88ocP7F=kwi=1=p1jnH196BS08pmTAoLEbFnAfHaJFB1jA@mail.gmail.com>

Probably when it does the comparison, bro is internally using the full
value not the scientific notation. If you turn 1.844674e+19 into a number
you get 18446740000000000000 which is indeed less
than 18446744073709551615. The fmt just truncates it down and prints the
scientific notation while the mathematical comparison uses the full value.


---------------------
Aubrey Wells
Manager, Network Operations
Digium Cloud Services
Main: 888.305.3850
Support: 877.344.4861 or http://www.digium.com/en/support
<http://www.digium.com/en/support?elq=65516445a5964d3597e25eaf566bc2cf&elqCampaignId=>

On Thu, Apr 16, 2015 at 12:45 PM, Josh Liburdi <liburdi.joshua at gmail.com>
wrote:

> I agree, I think double's are the way to go ... but the behavior is
> odd: http://try.bro.org/#/trybro/saved/3780
>
> It doesn't recognize the numbers as being equal.
>
> Josh
>
> On Thu, Apr 16, 2015 at 9:43 AM, Vlad Grigorescu <vlad at grigorescu.org>
> wrote:
> > You can use to_double:
> >
> >> $ bro -e 'print to_double("987654321123456789");'
> >> 9.876543e+17
> >
> >   --Vlad
> >
> > On Thu, Apr 16, 2015 at 11:19 AM, Aaron Gee-Clough <lists at g-clef.net>
> wrote:
> >>
> >>
> >> True, but I was hoping to do more than just detect the magic number. I
> >> was hoping to be able to say something along the lines of:
> >>
> >>         if (name == "RANGE" && value > 2^64 )
> >>
> >> My thinking here is that I don't want to play whack-a-mole with magic
> >> numbers. I would like to flag any request for an offset that big as a
> >> potential problem.
> >>
> >> aaron
> >>
> >> On 04/16/2015 12:11 PM, Josh Liburdi wrote:
> >> >
> >> > The Range header value in Bro should be a string-- if you're looking
> >> > to detect a specific magic number in this value, then instead of
> >> > converting the values to counts, you could match it like this by
> >> > leaving that magic number as a string:
> >> >
> >> > if ( name == "RANGE" && "string" in value )
> >> >
> >> > Josh
> >> >
> >> > On Thu, Apr 16, 2015 at 4:33 AM, Aaron Gee-Clough <lists at g-clef.net>
> >> > wrote:
> >> >>
> >> >> All,
> >> >>
> >> >> I'm working on a bro script to detect attempts for the
> >> >> recently-announced IIS attack. I've hit an interesting issue:
> There's a
> >> >> magic number that gets sent in the HTTP "RANGE" header to trigger the
> >> >> vulnerability, and that number is 2^64. This is right at the edge of
> >> >> what a "count" variable can hold, and it wraps around a regular "int"
> >> >> variable.
> >> >>
> >> >> I'd like to be able to detect anyone sending any number >= 2^64 in a
> >> >> RANGE header, but I don't see how to do that with count variables in
> >> >> bro. Does anyone have any ideas of how I can do this? Right now I'm
> >> >> looking at doing something truly nasty, like comparing the length of
> >> >> the
> >> >> strings holding the Range values. I'm *really* not happy with that,
> >> >> though...it feels like a really ugly hack.
> >> >>
> >> >> aaron
> >> >> _______________________________________________
> >> >> Bro mailing list
> >> >> bro at bro-ids.org
> >> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >> _______________________________________________
> >> Bro mailing list
> >> bro at bro-ids.org
> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >
> >
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150416/bb88cede/attachment.html 

From liburdi.joshua at gmail.com  Thu Apr 16 10:20:51 2015
From: liburdi.joshua at gmail.com (Josh Liburdi)
Date: Thu, 16 Apr 2015 10:20:51 -0700
Subject: [Bro] working with MS15-034
In-Reply-To: <E7C6DFDE498C8A4BA466F17123F8CE6E02323676CF8F@E2K7MBX1.CNET.local>
References: <552F9E10.5000806@g-clef.net>
	<CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
	<552FE0FC.80604@g-clef.net>
	<CANUJ_FmYD2f-HcnNyGq_1TPiyyO0axYshGFWXx14B6zucMi2KQ@mail.gmail.com>
	<CANrCRiE5FrGoLFK4ekVNR8QBSajSuTg_H6Gd1uhiFfC+DSLvpg@mail.gmail.com>
	<CANrCRiHTbCfvSq+hnf3WT4bv1-CTHmcGNsHUPppdskPSdQBoBQ@mail.gmail.com>
	<CANUJ_FkdhomoC+GtaGGrn4i4B_k06fUJ6sXNdF0MkM5-uLa9nw@mail.gmail.com>
	<E7C6DFDE498C8A4BA466F17123F8CE6E02323676CF8F@E2K7MBX1.CNET.local>
Message-ID: <CANrCRiE_3Tmja2065ybJM0RU5JiMXCE-VJ-eRpULnybpsah9UA@mail.gmail.com>

Seems like it would be fine in production. There are multiple ways to
detect this vulnerability (including one not yet mentioned here that
was posted in the Bro IRC channel the other day). Personally I use a
script that looks for inbound connections that have a RANGE header,
save the RANGE value, then checks to see if the internal server
responded with the status code 416 (at which point the notice is fired
with the RANGE value included in it) ... this is vulnerability does a
good job of showing how flexible Bro can be with detection.

On Thu, Apr 16, 2015 at 10:10 AM, Wier, Timothy A.
<tim.wier at cuchicago.edu> wrote:
> This is what I?ve been playing with: http://try.bro.org/#/trybro/saved/3789.
>
>
>
> Not sure how it will run in production.
>
> Tim
>
>
>
> From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Vlad
> Grigorescu
> Sent: Thursday, April 16, 2015 11:53 AM
> To: Josh Liburdi
> Cc: bro at bro.org
> Subject: Re: [Bro] working with MS15-034
>
>
>
> Well... they're not equal. :-)
>
>
>
> magic is 18446744073709551615,
>
> while d is 18446740000000000000
>
>
>
> See this:
>
>
>
> http://try.bro.org/#/trybro/saved/3786
>
>
>
> On Thu, Apr 16, 2015 at 11:47 AM, Josh Liburdi <liburdi.joshua at gmail.com>
> wrote:
>
> Better version here: http://try.bro.org/#/trybro/saved/3782
>
>
> On Thu, Apr 16, 2015 at 9:45 AM, Josh Liburdi <liburdi.joshua at gmail.com>
> wrote:
>> I agree, I think double's are the way to go ... but the behavior is
>> odd: http://try.bro.org/#/trybro/saved/3780
>>
>> It doesn't recognize the numbers as being equal.
>>
>> Josh
>>
>> On Thu, Apr 16, 2015 at 9:43 AM, Vlad Grigorescu <vlad at grigorescu.org>
>> wrote:
>>> You can use to_double:
>>>
>>>> $ bro -e 'print to_double("987654321123456789");'
>>>> 9.876543e+17
>>>
>>>   --Vlad
>>>
>>> On Thu, Apr 16, 2015 at 11:19 AM, Aaron Gee-Clough <lists at g-clef.net>
>>> wrote:
>>>>
>>>>
>>>> True, but I was hoping to do more than just detect the magic number. I
>>>> was hoping to be able to say something along the lines of:
>>>>
>>>>         if (name == "RANGE" && value > 2^64 )
>>>>
>>>> My thinking here is that I don't want to play whack-a-mole with magic
>>>> numbers. I would like to flag any request for an offset that big as a
>>>> potential problem.
>>>>
>>>> aaron
>>>>
>>>> On 04/16/2015 12:11 PM, Josh Liburdi wrote:
>>>> >
>>>> > The Range header value in Bro should be a string-- if you're looking
>>>> > to detect a specific magic number in this value, then instead of
>>>> > converting the values to counts, you could match it like this by
>>>> > leaving that magic number as a string:
>>>> >
>>>> > if ( name == "RANGE" && "string" in value )
>>>> >
>>>> > Josh
>>>> >
>>>> > On Thu, Apr 16, 2015 at 4:33 AM, Aaron Gee-Clough <lists at g-clef.net>
>>>> > wrote:
>>>> >>
>>>> >> All,
>>>> >>
>>>> >> I'm working on a bro script to detect attempts for the
>>>> >> recently-announced IIS attack. I've hit an interesting issue: There's
>>>> >> a
>>>> >> magic number that gets sent in the HTTP "RANGE" header to trigger the
>>>> >> vulnerability, and that number is 2^64. This is right at the edge of
>>>> >> what a "count" variable can hold, and it wraps around a regular "int"
>>>> >> variable.
>>>> >>
>>>> >> I'd like to be able to detect anyone sending any number >= 2^64 in a
>>>> >> RANGE header, but I don't see how to do that with count variables in
>>>> >> bro. Does anyone have any ideas of how I can do this? Right now I'm
>>>> >> looking at doing something truly nasty, like comparing the length of
>>>> >> the
>>>> >> strings holding the Range values. I'm *really* not happy with that,
>>>> >> though...it feels like a really ugly hack.
>>>> >>
>>>> >> aaron
>>>> >> _______________________________________________
>>>> >> Bro mailing list
>>>> >> bro at bro-ids.org
>>>> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>> _______________________________________________
>>>> Bro mailing list
>>>> bro at bro-ids.org
>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>
>>>
>
>


From nweaver at ICSI.Berkeley.EDU  Thu Apr 16 10:23:12 2015
From: nweaver at ICSI.Berkeley.EDU (Nicholas Weaver)
Date: Thu, 16 Apr 2015 10:23:12 -0700
Subject: [Bro] working with MS15-034
In-Reply-To: <CANUJ_FkdhomoC+GtaGGrn4i4B_k06fUJ6sXNdF0MkM5-uLa9nw@mail.gmail.com>
References: <552F9E10.5000806@g-clef.net>
	<CANrCRiF9ABgW=Vs4h-E-VtADhM1w5NRzz6u6raSXrBfF=mJvnw@mail.gmail.com>
	<552FE0FC.80604@g-clef.net>
	<CANUJ_FmYD2f-HcnNyGq_1TPiyyO0axYshGFWXx14B6zucMi2KQ@mail.gmail.com>
	<CANrCRiE5FrGoLFK4ekVNR8QBSajSuTg_H6Gd1uhiFfC+DSLvpg@mail.gmail.com>
	<CANrCRiHTbCfvSq+hnf3WT4bv1-CTHmcGNsHUPppdskPSdQBoBQ@mail.gmail.com>
	<CANUJ_FkdhomoC+GtaGGrn4i4B_k06fUJ6sXNdF0MkM5-uLa9nw@mail.gmail.com>
Message-ID: <A5793207-3828-4A9E-821B-5CA3C989E373@icsi.berkeley.edu>


> On Apr 16, 2015, at 9:52 AM, Vlad Grigorescu <vlad at grigorescu.org> wrote:
> 
> Well... they're not equal. :-)
> 
> magic is 18446744073709551615,
> while d is 18446740000000000000

Stupid question:  Since the goal is an overflow of a 64 bit number, why not

to_double("Range_targe") > 9.1e+17?

You could theoretically have someone who really really wants byte 18446740000000000000 in a file, but really, how does the semantics work for "normal"?

--
Nicholas Weaver                  it is a tale, told by an idiot,
nweaver at icsi.berkeley.edu                full of sound and fury,
510-666-2903                                 .signifying nothing
PGP: http://www1.icsi.berkeley.edu/~nweaver/data/nweaver_pub.asc

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150416/5e845711/attachment.bin 

From lists at g-clef.net  Thu Apr 16 10:38:19 2015
From: lists at g-clef.net (Aaron Gee-Clough)
Date: Thu, 16 Apr 2015 13:38:19 -0400
Subject: [Bro] working with MS15-034
References: <552F9E10.5000806@g-clef.net>
Message-ID: <552FF38B.9000107@g-clef.net>


Many thanks for the help, everyone.

In case others are interested, here's where I ended up for the MS15-034
detector script. The reason for all the string splits is that I noticed
some legit "range" requests on my network included multiple ranges,
separated by commas. So, I had to check all of them.

https://gist.github.com/g-clef/063592400b111c76f760#file-ms15-034-bro

Comments/speed improvements always welcome.

aaron

On 04/16/2015 07:33 AM, Aaron Gee-Clough wrote:
> 
> 
> All,
> 
> I'm working on a bro script to detect attempts for the
> recently-announced IIS attack. I've hit an interesting issue: There's a
> magic number that gets sent in the HTTP "RANGE" header to trigger the
> vulnerability, and that number is 2^64. This is right at the edge of
> what a "count" variable can hold, and it wraps around a regular "int"
> variable.
> 
> I'd like to be able to detect anyone sending any number >= 2^64 in a
> RANGE header, but I don't see how to do that with count variables in
> bro. Does anyone have any ideas of how I can do this? Right now I'm
> looking at doing something truly nasty, like comparing the length of the
> strings holding the Range values. I'm *really* not happy with that,
> though...it feels like a really ugly hack.
> 
> aaron
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 

From seth at icir.org  Thu Apr 16 13:14:57 2015
From: seth at icir.org (Seth Hall)
Date: Thu, 16 Apr 2015 16:14:57 -0400
Subject: [Bro] erspan decapsulation
In-Reply-To: <CAEGuVoKFEgRPbbO7=6e5L1H31d59DJB-Ynnu9VEYnNawAcVWOw@mail.gmail.com>
References: <CAHr6V5o+K2=daVqPBxigkwHGaKi7MTMvcZoe8qXQs-4riy_LNg@mail.gmail.com>
	<FFDC5F68-88BC-48B7-9B62-EB94627A3A01@icir.org>
	<CAHr6V5pEkz00mpFkWB698_zAv_vxnJmav_UhuPvbYUtyj3eQpA@mail.gmail.com>
	<CAEGuVoLRf0xb7pY7wdCmQZ_CC7DM++qc4u3dqAZpi9M9DWyq0A@mail.gmail.com>
	<CAEGuVoKFEgRPbbO7=6e5L1H31d59DJB-Ynnu9VEYnNawAcVWOw@mail.gmail.com>
Message-ID: <EAF5D1E9-D0D3-43BE-B2BE-D5C292426A1C@icir.org>


> On Apr 16, 2015, at 11:15 AM, Kristoffer Bj?rk <kristoffer.bjork at gmail.com> wrote:
> 
> However, to me it looks like 50bytes instead of 22 bytes? Like in this: https://staff.washington.edu/corey/gulp/conv.c
> But i guess bro deencapsulates the gre tunnel for you?

Yep.  Bro supports GRE.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Thu Apr 16 13:15:25 2015
From: seth at icir.org (Seth Hall)
Date: Thu, 16 Apr 2015 16:15:25 -0400
Subject: [Bro] erspan decapsulation
In-Reply-To: <CAHr6V5pEkz00mpFkWB698_zAv_vxnJmav_UhuPvbYUtyj3eQpA@mail.gmail.com>
References: <CAHr6V5o+K2=daVqPBxigkwHGaKi7MTMvcZoe8qXQs-4riy_LNg@mail.gmail.com>
	<FFDC5F68-88BC-48B7-9B62-EB94627A3A01@icir.org>
	<CAHr6V5pEkz00mpFkWB698_zAv_vxnJmav_UhuPvbYUtyj3eQpA@mail.gmail.com>
Message-ID: <4C5CD74C-C21C-438A-8BF2-D85313627E55@icir.org>


> On Apr 16, 2015, at 3:57 AM, Giedrius Ramas <giedrius.ramas at gmail.com> wrote:
> 
> Thanks for reply, 
> I just figure out that I need to skip some bytes of package header. In my current case I need to skip 22 bytes. So I edited ini-bare.bro file and changed encap_hdr_size = 0 line to encap_hdr_size = 22 . BRO can now understand traffic. Do not know  if I made a correct fix. Let me know if it is not a right way to do . 

Ideally we?d just support ERSPAN natively, but there?s nothing wrong with your solution.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From robin at icir.org  Fri Apr 17 08:55:24 2015
From: robin at icir.org (Robin Sommer)
Date: Fri, 17 Apr 2015 08:55:24 -0700
Subject: [Bro] Logging VLAN IDs
In-Reply-To: <D151969B.170CE%edthoma@sandia.gov>
References: <D151969B.170CE%edthoma@sandia.gov>
Message-ID: <20150417155524.GO55440@icir.org>

(Cc'ing bro-dev, I suggest we continue the thread there).

This sounds generally reasonable, however I think we could take the 
opportunity here to generalize this a bit more for generally including
link-layer information into connection handling.

One thing that I didn't quite get form your description is if the
objective is really just to get the VLAN ID into conn.log, or whether
you also want to use it for defining what constitutes a connection in
the first place. The latter would aim at the situations where the same
IP addresses can appear on different VLANs for independent
connections. Right now, Bro can't keep them apart, but if we made the
VLAN part of the connection index into the session table, it would
treat them separately. Same applies to other link-level features that
could sometimes be useful to be a part of a connection's ID (like MAC
addresses).

With that in mind, some thoughts on generalizing this (note, I not
sureif you're working from 2.3 or git master. The PktSrc API has
changed a bit recently, I'll take git as my starting point).

    - One challenge is passing the the VLAN ID through to the various
      packet-related methods. You're suggesting additional parameters,
      which would work. However, these methods are already taking a
      bunch of parameters, and if in the future we wanted to pass
      through further link-layer info, we'd have to add even more. A
      more flexible alternative would be switching to simply passing a
      Packet structure around that encapsulates all the information,
      including what's already there (e.g., timestmap, pcap_hdr,
      payload, etc.). The new PktSrc API already has such a class:
      PktSrc::Packet; from a quick look I think we could elevate that
      to be something passed around more generally, and then extend it
      accordingly.

    - For the connections, I would store the VLAN inside the ConnID
      struct, and then modify BuildConnIDHashKey() to take it into
      account. That way, the session table will make it part of its
      index. Same for the script-land conn_id record; that will then
      make script-level tables work that index by conn_id.

    - Extending the ConnID like this could actually be made a run-time
      option: I believe it shouldn't be too difficult to let users
      chose the fields defining a ConnID, so that they can decide if,
      say, they want to VLAN to be in there or not. We could predefine
      a set of potential features to choose from, along with some
      script-land API to pick the set to use, with the current 4-tuple
      being the default. (This could be a 2nd step for later; if the
      first two points above were in place, this extension should
      become mainly a question of finding the right configuration
      interface.)

I haven't thought this thruogh too carefully, so it's conceivable that
I'm missing something. But I think it would be really helpful for many
folks to get more flexibility into the definition of what consitutes a
connection, with VLANs being a good initial target to support.

Robin

On Tue, Apr 14, 2015 at 16:59 +0000, you wrote:

> Dear Bro developers,
> 
> I've been tasked with trying to modify the Bro source code so that
> conn.log includes the VLAN IDs (including 802.1ah) that have been observed
> in packets associated with that connection. I've scoped out a solution,
> but I want to run it by you first before I start to go for it, in case I'm
> missing something really big.
> 
> PktSrc::Process() does processing of VLAN and 802.1ah, but it just skips
> over them by advancing the data pointer. I will, in addition, store those
> VLAN IDs in a new member of the modified PktSrc class. This gets passed on
> through net_packet_dispatch() and NetSessions::DispatchPacket(). At this
> point NetSessions::NextPacket() gets called, but since the PktSrc doesn't
> get passed to it, I'd need another way to pass it the VLAN ID. I am
> considering two options:
> 
> 1. duplicate NextPacket() adding a new parameter to pass it the VLAN IDs,
> and call that instead, or
> 2. store the VLAN IDs in the NetSessions class, in DispatchPacket() so
> it?s available to NextPacket() and DoNextPacket() <- Is there a reason
> this wouldn?t work, e.g. issues with multi-threading/multi-processing?
> 
> Is there one option that seems better to you?
> 
> NetSessions::DoNextPacket() is called next and I would also need a
> modification to pass it VLAN IDs, using one of the options above. In this
> method we finally get access to the appropriate Connection instance, so I
> would store the VLAN IDs in that instance in DoNextPacket().
> 
> I'd need to modify the Connection class in Conn.h to include a new member
> for tracking VLAN IDs. I'd modify Connection::BuildConnVal() and
> scripts/base/init-bare.bro's connection record to make the VLAN IDs
> available to scripts. Lastly, I'd write a script to redef the conn Info
> structure and handle one or more connection events (perhaps
> connection_state_remove) to copy the VLAN IDs from the connection record
> to the Info record.
> 
> Is there anything I'm missing? Is there a better way to approach this?
> 



-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From jlay at slave-tothe-box.net  Fri Apr 17 10:26:52 2015
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 17 Apr 2015 11:26:52 -0600
Subject: [Bro] An assist with file extraction
In-Reply-To: <E5A2071C6FFFF640B714C224139C9B3301A5D782@WP-MBX2B.milky-way.battelle.org>
References: <1429185344.4032.18.camel@JamesiMac>
	<E5A2071C6FFFF640B714C224139C9B3301A5D782@WP-MBX2B.milky-way.battelle.org>
Message-ID: <f42f8b146de9912e8c38583d0defac6d@localhost>

 

On 2015-04-16 07:04 AM, Hosom, Stephen M wrote: 

> For 2.3.2
(current release) you'll want to use the event file_new. 
> 
> Note that
in 2.3.2 if you are extracting based on mime_type (most people do) you
will want to verify that the field exists before you actually use it. 
>

> For master, which is what you are likely referring to? you'll want
the event file_mime_type. 
> 
> FROM: bro-bounces at bro.org
[mailto:bro-bounces at bro.org] ON BEHALF OF James Lay
> SENT: Thursday,
April 16, 2015 7:56 AM
> TO: Bro-IDS
> SUBJECT: [Bro] An assist with
file extraction 
> 
> Hey all,
> 
> The topic pretty much says it...I've
done a fair amount of reading trying to determine the best way to
extract file attachments in smtp traffic. Most of the information I've
found is related to older versions of bro. Can someone point me to a
current resource that will work with the current version of bro? Thank
you.
> 
> James

Thank you Stephen...I really appreciate the advice.


James 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150417/f59e2042/attachment.html 

From jlay at slave-tothe-box.net  Fri Apr 17 10:40:30 2015
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 17 Apr 2015 11:40:30 -0600
Subject: [Bro] An assist with file extraction
In-Reply-To: <1429185344.4032.18.camel@JamesiMac>
References: <1429185344.4032.18.camel@JamesiMac>
Message-ID: <a595193842c25eba89b4bff20e159715@localhost>

 

On 2015-04-16 05:55 AM, James Lay wrote: 

> Hey all,
> 
> The topic
pretty much says it...I've done a fair amount of reading trying to
determine the best way to extract file attachments in smtp traffic. Most
of the information I've found is related to older versions of bro. Can
someone point me to a current resource that will work with the current
version of bro? Thank you.
> 
> James

Well here's what I have: 

global
ext_map: table[string] of string = {
 ["application/x-dosexec"] =
"exe",
 ["application/zip"] = "zip",
 ["application/msword"] =
"xls",
};

event file_new(f: fa_file)
 {
 if ( f$source != "SMTP" )

return;

 if ( ! f?$mime_type || f$mime_type !in ext_map )
 return;


local ext = "";

 if ( f?$mime_type )
 ext = ext_map[f$mime_type];


local fname = fmt("%s-%s.%s", f$source, f$id, ext);

Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
[$extract_filename=fname]);
}

This appears to function ok....Office doc
XML format end up as zips, which is fine by me. Can anyone see anything
glaringly wrong with this? Also...I have bro log files zipped and
rotated at midnight..is there a way to include the extract_files
directory in that rotation, or, even better, have the extracted files go
into a directory name with say something like
/mnt/backup/extract_files/04-16-16 and change per day? Thank you.


James 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150417/8a17fe52/attachment.html 

From edthoma at sandia.gov  Fri Apr 17 12:00:04 2015
From: edthoma at sandia.gov (Thomas, Eric D)
Date: Fri, 17 Apr 2015 19:00:04 +0000
Subject: [Bro] [EXTERNAL] Re:  Logging VLAN IDs
In-Reply-To: <20150417155524.GO55440@icir.org>
References: <D151969B.170CE%edthoma@sandia.gov>
	<20150417155524.GO55440@icir.org>
Message-ID: <D1569FA0.17351%edthoma@sandia.gov>

Hi Robin, thanks for the reply. You can remove bro at bro.org if you want,
and I will follow your lead.

I was working from 2.3.2. I?ll take a look at git master next.

In my specific case, I?m just interested in logging VLAN IDs. For each
connection it could be a set of VLAN IDs, perhaps multiple IDs tracked in
each direction. I?m dealing with a scenario where multiple streams of
traffic are coming into my Bro sensor?s interface. I want a way to
demultiplex them, without relying upon IP address blocks. My ultimate goal
is to be able to identify which networks a connection is associated with.
If the VLAN IDs are just encapsulated in the connection ID, that would not
gain me anything.


Also, having link-level features be part of the connection index may be
dubious, but I?ll let the experts decide. With MAC addresses, for example,
a packet might take a different route than other packets in the
connection, but I wouldn?t want Bro to treat it as a different connection.
Perhaps the same thought applies to VLANs, I don?t know. I?m also having
difficulty coming up with a security scenario where it provides additional
benefit to have link-level features in the connection index, except
perhaps detecting packet spoofing.


Looking forward to your thoughts on this,
-- 
Eric Thomas
edthoma at sandia.gov




On 4/17/15, 8:55 AM, "Robin Sommer" <robin at icir.org> wrote:

>(Cc'ing bro-dev, I suggest we continue the thread there).
>
>This sounds generally reasonable, however I think we could take the
>opportunity here to generalize this a bit more for generally including
>link-layer information into connection handling.



From seth at icir.org  Fri Apr 17 13:20:57 2015
From: seth at icir.org (Seth Hall)
Date: Fri, 17 Apr 2015 16:20:57 -0400
Subject: [Bro] An assist with file extraction
In-Reply-To: <a595193842c25eba89b4bff20e159715@localhost>
References: <1429185344.4032.18.camel@JamesiMac>
	<a595193842c25eba89b4bff20e159715@localhost>
Message-ID: <9CBD32B4-2AAC-4058-A548-3FA48DCFB77A@icir.org>


> On Apr 17, 2015, at 1:40 PM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> This appears to function ok....Office doc XML format end up as zips, which is fine by me.

This will be fixed in 2.4.  New xml Office files will be identified as....

application/vnd.openxmlformats-officedocument.presentationml.presentation
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
application/vnd.openxmlformats-officedocument.wordprocessingml.document
and...
application/vnd.openxmlformats-officedocument in case a better option wasn?t discovered.  And, yes, those are the *actual* mime types for MS Office documents.

> Also...I have bro log files zipped and rotated at midnight..is there a way to include the extract_files directory in that rotation, or, even better, have the extracted files go into a directory name with say something like /mnt/backup/extract_files/04-16-16 and change per day? 

Please feel free to file a ticket.  That would be a nice trick. :)
	http://tracker.bro.org

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Fri Apr 17 13:24:05 2015
From: seth at icir.org (Seth Hall)
Date: Fri, 17 Apr 2015 16:24:05 -0400
Subject: [Bro] Multi-Thread bro with pcap file?
In-Reply-To: <CAMCSCoWwZZC3XkZ0AHznw4hYkLKLWWy_s7tR6dS-Zj_VGyNNoA@mail.gmail.com>
References: <CAMCSCoWwZZC3XkZ0AHznw4hYkLKLWWy_s7tR6dS-Zj_VGyNNoA@mail.gmail.com>
Message-ID: <8E677002-00BA-4A98-9888-A04978494D02@icir.org>


> On Apr 15, 2015, at 11:05 AM, Joe Blow <blackhole.em at gmail.com> wrote:
> 
> Can this only be done with one thread?

Alex and Luuk gave answers that will work, but I wanted to throw one more that is coming into the mix too.  Eventually you should be able to use PacketBricks (https://github.com/bro/packet-bricks) to read in pcap files and distribute the traffic out to multiple workers. (just wanted to point out another direction we?re moving).

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From jlay at slave-tothe-box.net  Fri Apr 17 13:25:35 2015
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 17 Apr 2015 14:25:35 -0600
Subject: [Bro] An assist with file extraction
In-Reply-To: <9CBD32B4-2AAC-4058-A548-3FA48DCFB77A@icir.org>
References: <1429185344.4032.18.camel@JamesiMac>
	<a595193842c25eba89b4bff20e159715@localhost>
	<9CBD32B4-2AAC-4058-A548-3FA48DCFB77A@icir.org>
Message-ID: <25932623f36b9f89a6f72b2e55049acb@localhost>

On 2015-04-17 02:20 PM, Seth Hall wrote:
>> On Apr 17, 2015, at 1:40 PM, James Lay <jlay at slave-tothe-box.net> 
>> wrote:
>>
>> This appears to function ok....Office doc XML format end up as zips, 
>> which is fine by me.
>
> This will be fixed in 2.4.  New xml Office files will be identified 
> as....
>
> 
> application/vnd.openxmlformats-officedocument.presentationml.presentation
> application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
> 
> application/vnd.openxmlformats-officedocument.wordprocessingml.document
> and...
> application/vnd.openxmlformats-officedocument in case a better option
> wasn?t discovered.  And, yes, those are the *actual* mime types for 
> MS
> Office documents.
>
>> Also...I have bro log files zipped and rotated at midnight..is there 
>> a way to include the extract_files directory in that rotation, or, 
>> even better, have the extracted files go into a directory name with 
>> say something like /mnt/backup/extract_files/04-16-16 and change per 
>> day?
>
> Please feel free to file a ticket.  That would be a nice trick. :)
> 	http://tracker.bro.org
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/

I will file...looks like I'll have to 'roll my own' for the archiving.  
Thank you.

James

From jlay at slave-tothe-box.net  Fri Apr 17 13:40:59 2015
From: jlay at slave-tothe-box.net (James Lay)
Date: Fri, 17 Apr 2015 14:40:59 -0600
Subject: [Bro] An assist with file extraction
In-Reply-To: <9CBD32B4-2AAC-4058-A548-3FA48DCFB77A@icir.org>
References: <1429185344.4032.18.camel@JamesiMac>
	<a595193842c25eba89b4bff20e159715@localhost>
	<9CBD32B4-2AAC-4058-A548-3FA48DCFB77A@icir.org>
Message-ID: <2feabf2faf27175bd6d8e95c971b9d90@localhost>

On 2015-04-17 02:20 PM, Seth Hall wrote:
>> On Apr 17, 2015, at 1:40 PM, James Lay <jlay at slave-tothe-box.net> 
>> wrote:
>>
>> This appears to function ok....Office doc XML format end up as zips, 
>> which is fine by me.
>
> This will be fixed in 2.4.  New xml Office files will be identified 
> as....
>
> 
> application/vnd.openxmlformats-officedocument.presentationml.presentation
> application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
> 
> application/vnd.openxmlformats-officedocument.wordprocessingml.document
> and...
> application/vnd.openxmlformats-officedocument in case a better option
> wasn?t discovered.  And, yes, those are the *actual* mime types for 
> MS
> Office documents.
>
>> Also...I have bro log files zipped and rotated at midnight..is there 
>> a way to include the extract_files directory in that rotation, or, 
>> even better, have the extracted files go into a directory name with 
>> say something like /mnt/backup/extract_files/04-16-16 and change per 
>> day?
>
> Please feel free to file a ticket.  That would be a nice trick. :)
> 	http://tracker.bro.org
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/

And one last bit....could I theoretically redef extract_files?

share/bro/base/files/extract/main.bro:  const prefix = 
"./extract_files/" &redef;

I could always symlink that directory to a different drive but 
eh....the more I can shove into the script the better.  Thanks again.

James

From troyj at maine.edu  Sat Apr 18 19:45:33 2015
From: troyj at maine.edu (Troy Jordan)
Date: Sat, 18 Apr 2015 22:45:33 -0400
Subject: [Bro] binpac++ plugin error
Message-ID: <553316CD.6040803@maine.edu>

I am working through section 3 of the binpac++ documentation to
understand binpac++ analyzers are integrated into Bro.

(I'm running the HILTI/Binpac++ docker image, under Centos7.)

I am getting an error when confirming the binpac++ plugin is present:

root at df5e8fd99740:~# export BRO_PLUGIN_PATH=/opt/hilti/build/bro

root at df5e8fd99740:~# bro -NN ssh.evt

fatal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1:
cannot load plugin library
/opt/hilti/build/bro//lib/Bro-Hilti.linux-x86_64.so:
/opt/hilti/build/bro//lib/Bro-Hilti.linux-x86_64.so: undefined symbol:
_ZN6plugin6Plugin11MetaHookPreENS_8HookTypeERKNSt3__14listINS_12HookArgumentENS2_9allocatorIS4_EEEE


Perhaps I have set the BRO_PLUGIN_PATH incorrectly?

- Troy


-- 


                     	  Troy Jordan
                   t r o y j @ m a i n e . e d u
			   GIAC GCIH,GCIA
------------------------------------------------------------
                Network Systems Security Analyst
             Information Technology Security Office
                    University of Maine System
------------------------------------------------------------
233 Science Building           |     voice: 207.561.3590
Portland, ME 04103             |     fax:   509.351.3650



"As you all know, Security Is Mortals chiefest Enemy"
 William Shakespeare, Macbeth

From rdump at river.com  Sun Apr 19 11:25:09 2015
From: rdump at river.com (Richard Johnson)
Date: Sun, 19 Apr 2015 12:25:09 -0600
Subject: [Bro] restrict_filters not preventing logging of selected IP
	addresses
Message-ID: <5533F305.6010101@river.com>

I think I'm specifying restrict_filters correctly to stop some hosts from 
being logged, but it's not working as I intend/expect.

My local.bro redefinition of restrict_filters (below) is being recognized and 
propagated by broctl install, as confirmed by print restrict_filters after 
restarting.

As further confirmation that the redef is being noticed, if I specify a pcap 
syntax impossibility in restrict_filters, I get workers quitting with
"fatal error in /raid/bro/share/bro/base/frameworks/packet-filter/./main.bro,
line 282: Bad pcap filter ..." on a restart.

Yet when the restrict_filter is OK and is seemingly recognized, the IP 
addresses in the restrict_filters still appear in log entries.

This logging continues after a broctl install and update, after a broctl 
install and restart, as well as after a complete cluster reboot.

I'm seeing this under Bro 2.3-7 on CentOS 6.5 with pfring.  Whether the 
capture_filters are redef'ed as shown in the details below, or not, doesn't 
change the restrict_filters failure I'm seeing.

Any ideas for where to take this debugging odyssey?  What am I missing that's 
obvious?


Richard

-------
Details:

[manager-host ~]$ grep capture_filters /raid/bro/share/bro/site/local.bro
redef capture_filters = { ["all"] = "ip or not ip" };
[manager-host ~]$ grep restrict_filters /raid/bro/share/bro/site/local.bro
redef restrict_filters += { ["not-these-hosts"] = "not host 172.16.1.1 and not 
host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88" };

[lines condensed for this message by removing extra pretty printing <cr>s]
[BroControl] > print capture_filters
      manager   capture_filters = { [all] = ip or not ip }
      proxy-1   capture_filters = { [all] = ip or not ip }
      proxy-2   capture_filters = { [all] = ip or not ip }
   worker-1-1   capture_filters = { [all] = ip or not ip }
   worker-1-2   capture_filters = { [all] = ip or not ip }
   worker-1-3   capture_filters = { [all] = ip or not ip }
   worker-1-4   capture_filters = { [all] = ip or not ip }
   worker-2-1   capture_filters = { [all] = ip or not ip }
   worker-2-2   capture_filters = { [all] = ip or not ip }
   worker-2-3   capture_filters = { [all] = ip or not ip }
   worker-2-4   capture_filters = { [all] = ip or not ip }

[lines condensed for this message by removing extra pretty printing <cr>s]
[BroControl] > print restrict_filters
      manager   restrict_filters = { [not-these-hosts] = not host 172.16.1.1 
and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 }
      proxy-1   restrict_filters = { [not-these-hosts] = not host 172.16.1.1 
and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 }
      proxy-2   restrict_filters = { [not-these-hosts] = not host 172.16.1.1 
and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 }
   worker-1-1   restrict_filters = { [not-these-hosts] = not host 172.16.1.1 
and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 }
   worker-1-2   restrict_filters = { [not-these-hosts] = not host 172.16.1.1 
and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 }
   worker-1-3   restrict_filters = { [not-these-hosts] = not host 172.16.1.1 
and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 }
   worker-1-4   restrict_filters = { [not-these-hosts] = not host 172.16.1.1 
and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 }
   worker-2-1   restrict_filters = { [not-these-hosts] = not host 172.16.1.1 
and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 }
   worker-2-2   restrict_filters = { [not-these-hosts] = not host 172.16.1.1 
and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 }
   worker-2-3   restrict_filters = { [not-these-hosts] = not host 172.16.1.1 
and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 }
   worker-2-4   restrict_filters = { [not-these-hosts] = not host 172.16.1.1 
and not host 172.16.22.22 and not host 172.16.39.39 and not host 172.16.88.88 }

[manager-host current]$ grep 172.16.88.88 conn.log | tail -3
1429461245.805348       CpuepS3Ds2GYzABCtb      xx.xx.xx.xx   xxxxx 
172.16.88.88   443    tcp     ssl     4192.655995     14660   16441   S1 
F      0ShADda  50      17268   49      19001   (empty)
1429464730.699197       CqVMY53iVvTFSWclAi      xx.xx.xx.xx    xxxxx 
172.16.88.88   443    tcp     ssl     1002.988461     5491    4481    SF 
F      0ShADdaFf        21      6591    17      5377    (empty)
1429464286.982078       CUl3Cl24bUWkgbhAGd      xx.xx.xx.xx   xxxxx 
172.16.88.88   443     tcp     ssl     1447.315821     7095    5595    SF 
  F      0ShADdafF        25      8403    21      6699    (empty)

From robin at icir.org  Sun Apr 19 22:06:51 2015
From: robin at icir.org (Robin Sommer)
Date: Sun, 19 Apr 2015 22:06:51 -0700
Subject: [Bro] binpac++ plugin error
In-Reply-To: <553316CD.6040803@maine.edu>
References: <553316CD.6040803@maine.edu>
Message-ID: <20150420050651.GJ72050@icir.org>


On Sat, Apr 18, 2015 at 22:45 -0400, you wrote:

> /opt/hilti/build/bro//lib/Bro-Hilti.linux-x86_64.so: undefined symbol:
> _ZN6plugin6Plugin11MetaHookPreENS_8HookTypeERKNSt3__14listINS_12HookArgumentENS2_9allocatorIS4_EEEE

Oh, there's actually a problem in the Docker file I believe. I'll see
that I get that fixed.

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From chilton.brian at yahoo.com  Tue Apr 21 07:13:29 2015
From: chilton.brian at yahoo.com (Brian Chilton)
Date: Tue, 21 Apr 2015 14:13:29 +0000 (UTC)
Subject: [Bro] Bro script derived off of the referrer
Message-ID: <2075953432.1460373.1429625609424.JavaMail.yahoo@mail.yahoo.com>

All,?I am attempting to write a script that will key off of when the referrer is empty.? The problem with that right now is that when I do this I have to use c$http$referrer == "-" which it does not like as an actual value.? Is there another way to do this?? I tried escaping it with a \ but that didn't seem to work either.? Any assistance you and provide would be great.?also, does anyone know where I can get some more info on the input framework??Thanks,?BC?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150421/e0d77eb7/attachment.html 

From soehlert at illinois.edu  Tue Apr 21 09:27:00 2015
From: soehlert at illinois.edu (Sam Oehlert)
Date: Tue, 21 Apr 2015 11:27:00 -0500
Subject: [Bro] Bro script derived off of the referrer
In-Reply-To: <2075953432.1460373.1429625609424.JavaMail.yahoo@mail.yahoo.com>
References: <2075953432.1460373.1429625609424.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <55367A54.4040306@illinois.edu>

To check a field to see if it's empty, you would use c$http?$referrer

As for input framework stuff:

https://www.bro.org/sphinx-git/scripts/base/frameworks/input/main.bro.html 
(this is for version 2.3)
http://blog.bro.org/2012/06/upcoming-loading-data-into-bro-with.html 
(this blog post is a little older, but I *think* still accurate)

-Sam


On 4/21/15 9:13 AM, Brian Chilton wrote:
> All,
> I am attempting to write a script that will key off of when the 
> referrer is empty.  The problem with that right now is that when I do 
> this I have to use c$http$referrer == "-" which it does not like as an 
> actual value.  Is there another way to do this?  I tried escaping it 
> with a \ but that didn't seem to work either.  Any assistance you and 
> provide would be great.
> also, does anyone know where I can get some more info on the input 
> framework?
> Thanks,
> BC
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150421/bb5725a9/attachment.html 

From anthony.kasza at gmail.com  Tue Apr 21 10:44:03 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Tue, 21 Apr 2015 10:44:03 -0700
Subject: [Bro] Bro script derived off of the referrer
In-Reply-To: <55367A54.4040306@illinois.edu>
References: <2075953432.1460373.1429625609424.JavaMail.yahoo@mail.yahoo.com>
	<55367A54.4040306@illinois.edu>
Message-ID: <CAEZw2bypF9pJWr8CATu=Ox1SRNEALduwhXe+jxy8dSXKQqjJXQ@mail.gmail.com>

Be sure to use the correct HTTP event, too. You don't want to check for the
referer before Bro has had a chance to add it to the connection object.

-AK
On Apr 21, 2015 9:44 AM, "Sam Oehlert" <soehlert at illinois.edu> wrote:

>  To check a field to see if it's empty, you would use c$http?$referrer
>
> As for input framework stuff:
>
> https://www.bro.org/sphinx-git/scripts/base/frameworks/input/main.bro.html
> (this is for version 2.3)
> http://blog.bro.org/2012/06/upcoming-loading-data-into-bro-with.html
> (this blog post is a little older, but I *think* still accurate)
>
> -Sam
>
>
> On 4/21/15 9:13 AM, Brian Chilton wrote:
>
>  All,
>
> I am attempting to write a script that will key off of when the referrer
> is empty.  The problem with that right now is that when I do this I have to
> use c$http$referrer == "-" which it does not like as an actual value.  Is
> there another way to do this?  I tried escaping it with a \ but that didn't
> seem to work either.  Any assistance you and provide would be great.
>
> also, does anyone know where I can get some more info on the input
> framework?
>
> Thanks,
>
> BC
>
>
> _______________________________________________
> Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150421/df27e53d/attachment.html 

From johanna at icir.org  Tue Apr 21 11:27:15 2015
From: johanna at icir.org (Johanna Amann)
Date: Tue, 21 Apr 2015 11:27:15 -0700
Subject: [Bro] Bro script derived off of the referrer
In-Reply-To: <55367A54.4040306@illinois.edu>
References: <2075953432.1460373.1429625609424.JavaMail.yahoo@mail.yahoo.com>
	<55367A54.4040306@illinois.edu>
Message-ID: <20150421182715.GA24116@wifi86.sys.ICSI.Berkeley.EDU>

On Tue, Apr 21, 2015 at 11:27:00AM -0500, Sam Oehlert wrote:
> As for input framework stuff:
> 
> https://www.bro.org/sphinx-git/scripts/base/frameworks/input/main.bro.html
> (this is for version 2.3)
> http://blog.bro.org/2012/06/upcoming-loading-data-into-bro-with.html (this
> blog post is a little older, but I *think* still accurate)

There also is https://www.bro.org/sphinx/frameworks/input.html, which is
probably the best starting point.

Johanna

From rrbarbosa at gmail.com  Wed Apr 22 02:34:55 2015
From: rrbarbosa at gmail.com (Rafael Barbosa)
Date: Wed, 22 Apr 2015 11:34:55 +0200
Subject: [Bro] Triggering events on incomplete PDUs
Message-ID: <CADOayQ-Wt0q-9fK0TdobxUYUqfFyjQfcXx8wQqtVtAVoc7oiDQ@mail.gmail.com>

Hi,

I am implementing a simple protocol analyzer for DLMS (smart metering
protocol), and I am trying to understand how the events are triggered.

Basically, I am interested in the first few bytes of the PDU, which
identify the types of requests/responses (e.g.: read, write,
authentication, etc). I implemented an analyzer for these bytes based on
the other protocols available, and I am able to trigger some events with
the values I need when parsing an example file.

However, the event only seem to be triggered when the full PDU is
avaliable. This is a big problem because the `snaplen` used for the capture
was quite small, thus most of the PDUs are incomplete.

My question is:  Is there is a way that I can force an event to be
triggered as soon as the first few bytes are available?

Best,
Rafael Barbosa
Research Consultant
www.encs.eu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150422/66b8c0a1/attachment.html 

From franky.meier.1 at gmx.de  Thu Apr 23 03:40:18 2015
From: franky.meier.1 at gmx.de (Frank Meier)
Date: Thu, 23 Apr 2015 12:40:18 +0200
Subject: [Bro] minor segfault in nb_dns.cc
Message-ID: <1429785618.7234.0@mail.gmx.net>

Hi there!

While testing bro sniffing replayed PCAPs I noticed a case where it 
segfaults. 
Because of the uncommon network config this looks like a minor bug to 
me.

The segfault happens, if a nameserver is set in /etc/resolv.conf, but 
the network
of the nameserver is not reachable: 

$ cat /etc/resolv.conf 
nameserver 192.168.1.1
$ cat dns.bro 
event bro_init() {
	when ( local result = lookup_hostname("example.com") ) {
	}
}
$ bro -v
bro version 2.3-793
$ bro dns.bro
warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: 
problem initializing NB-DNS: connect(192.168.1.1): Network is 
unreachable
warning: can't issue DNS request
warning: can't issue DNS request
Segmentation fault (core dumped)

The segfault does not happen, if BRO_DNS_FAKE ist set to on or off:

$ BRO_DNS_FAKE=0 bro dns.bro
warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: 
problem initializing NB-DNS: connect(192.168.1.1): Network is 
unreachable
$ BRO_DNS_FAKE=1 bro dns.bro
warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: 
problem initializing NB-DNS: connect(192.168.1.1): Network is 
unreachable

Here is the backtrace:

$ gdb bro /tmp/core 
GNU gdb (Ubuntu 7.8-1ubuntu4) 7.8.0.20141001-cvs
[...]
Core was generated by `bro dns.bro'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  nb_dns_fd (nd=0x0) at /home/franky/bro-git/bro/src/nb_dns.c:176
176		return (nd->s);
(gdb) bt
#0  nb_dns_fd (nd=0x0) at /home/franky/bro-git/bro/src/nb_dns.c:176
#1  0x0000000000567c1d in DNS_Mgr::AnswerAvailable (this=<optimized 
out>, timeout=0) at /home/franky/bro-git/bro/src/DNS_Mgr.cc:1425
#2  0x000000000056c24a in DNS_Mgr::DoProcess (this=0x15c1410, 
flush=false) at /home/franky/bro-git/bro/src/DNS_Mgr.cc:1382
#3  0x000000000056c420 in DNS_Mgr::Flush (this=0x15c1410) at 
/home/franky/bro-git/bro/src/DNS_Mgr.cc:1334
#4  0x0000000000540126 in done_with_network () at 
/home/franky/bro-git/bro/src/main.cc:316
#5  0x000000000051f679 in main (argc=<optimized out>, argv=<optimized 
out>) at /home/franky/bro-git/bro/src/main.cc:1216

A fix could be:

diff --git a/src/DNS_Mgr.cc b/src/DNS_Mgr.cc
index 11fd258..08f76df 100644
--- a/src/DNS_Mgr.cc
+++ b/src/DNS_Mgr.cc
@@ -1422,6 +1422,10 @@ void DNS_Mgr::DoProcess(bool flush)
 
 int DNS_Mgr::AnswerAvailable(int timeout)
        {
+       if (!nb_dns) {
+               reporter->Warning("nb_dns_fd() failed in 
DNS_Mgr::WaitForReplies");
+               return -1;
+       }
        int fd = nb_dns_fd(nb_dns);
        if ( fd < 0 )
                {

The segfault occurs with 2.3.2 and with a recent version from git 
(6fb4b522c6b3f2094a2f35761d3c4f7022bc4013)
(The current git from today does not compile).

I know it's not the usual use case for bro, but should I open a ticket 
about this? 

Franky





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150423/e4a0f04c/attachment.html 

From robin at icir.org  Thu Apr 23 06:49:31 2015
From: robin at icir.org (Robin Sommer)
Date: Thu, 23 Apr 2015 06:49:31 -0700
Subject: [Bro] minor segfault in nb_dns.cc
In-Reply-To: <1429785618.7234.0@mail.gmx.net>
References: <1429785618.7234.0@mail.gmx.net>
Message-ID: <20150423134931.GA24132@icir.org>



On Thu, Apr 23, 2015 at 12:40 +0200, Frank Meier wrote:

> I know it's not the usual use case for bro, but should I open a ticket  
> about this?

Yes, please. Every segfault is worth fixing. Thanks,

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From jdopheid at illinois.edu  Thu Apr 23 10:18:35 2015
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Thu, 23 Apr 2015 17:18:35 +0000
Subject: [Bro] FW: Contact form message
In-Reply-To: <201504211705.t3LH5AJM032022@bro-ids.icir.org>
References: <201504211705.t3LH5AJM032022@bro-ids.icir.org>
Message-ID: <D15E930F.150F5%jdopheid@illinois.edu>

Forwarding to the Bro mailing list. Can someone help Stoakley with his
error?

------

Jeannette Dopheide
Bro Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign






On 4/21/15, 12:05 PM, "Apache" <noreply at bro.org> wrote:


Contact: Stoakley Lloyd (stoakleyl at gmail.com)
IP:      168.156.174.91
Server:  www.bro.org

Message
-------

I am trying to install Bro on Linux Mint for a class when I use &quot;sudo
gdebi Bro-*.deb&quot; I get the Error &quot;gdebi error, file not found:
Bro-*.deb&quot; I have installed all the dependencies what am I doing
wrong?





From franky.meier.1 at gmx.de  Fri Apr 24 02:16:11 2015
From: franky.meier.1 at gmx.de (Frank Meier)
Date: Fri, 24 Apr 2015 11:16:11 +0200
Subject: [Bro] delayed bro operation
Message-ID: <1429866971.5205.0@mail.gmx.net>

Hi.

A policy forces me to run bro in a separate network. So the captured 
PCAPs are 
transfered to the bro network for logging purposes. How would I handle 
delays
in feeding bro with the PCAPS? Would connections spanning multiple 
PCAPs be a
problem?

My first idea is to crank up all the timeouts like this:

redef tcp_inactivity_timeout        = 5 days;
redef udp_inactivity_timeout        = 5 days;
redef icmp_inactivity_timeout       = 5 days;
redef default_file_timeout_interval = 5 days;

What performance penalty will I suffer? I guess the RAM usage will 
grow, 
because connections, which were not cleanly terminated, would hang 
around 
for a long time.

Are there any examples for this kind of setup? How would you search for 
this?

Have a nice weekend!

Franky
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150424/40369fc7/attachment.html 

From seth at icir.org  Fri Apr 24 07:23:50 2015
From: seth at icir.org (Seth Hall)
Date: Fri, 24 Apr 2015 10:23:50 -0400
Subject: [Bro] delayed bro operation
In-Reply-To: <1429866971.5205.0@mail.gmx.net>
References: <1429866971.5205.0@mail.gmx.net>
Message-ID: <D4B8EAD4-0211-4FB0-9337-05057381C2B5@icir.org>


> On Apr 24, 2015, at 5:16 AM, Frank Meier <franky.meier.1 at gmx.de> wrote:
> 
> A policy forces me to run bro in a separate network. So the captured PCAPs are
> transfered to the bro network for logging purposes. How would I handle delays
> in feeding bro with the PCAPS? Would connections spanning multiple PCAPs be a
> problem?

This is a problem that PacketBricks[1] will be able to solve eventually.  It?s not there yet, but eventually you?ll be able to create a load balancing architecture with persistent Bro/Snort/Suricata/etc processes and tell PacketBricks to read PCAPs as you get them in place (and, yes, I did just say clustered PCAP processing!).  Unfortunately this scenario is not quite ready in PacketBricks.

> redef tcp_inactivity_timeout        = 5 days;
> redef udp_inactivity_timeout        = 5 days;
> redef icmp_inactivity_timeout       = 5 days;
> redef default_file_timeout_interval = 5 days;

You could always try, but I get the sense you won?t be terribly happy with the result.

1. https://github.com/bro/packet-bricks

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150424/c15c412e/attachment.bin 

From anthony.kasza at gmail.com  Fri Apr 24 22:16:04 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Fri, 24 Apr 2015 22:16:04 -0700
Subject: [Bro] HTTP plus Compression File Extraction
Message-ID: <CAEZw2bzGAmeahP5ijT2Bq1v+PPT_Bcss4-bPuQXcP-o6i7BqRA@mail.gmail.com>

This question is likely gear towards Seth but I thought others on the list
may have an answer or find the question useful too.

Is there a way to control how the file analysis framework handles HTTP
compression? For example, if a PNG is transferred over HTTP with gzip
compression, can I have Bro dump the gzip file instead of the PNG?
Thanks!

-AK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150424/e5ee28a3/attachment.html 

From seth at icir.org  Sat Apr 25 18:12:13 2015
From: seth at icir.org (Seth Hall)
Date: Sat, 25 Apr 2015 21:12:13 -0400
Subject: [Bro] HTTP plus Compression File Extraction
In-Reply-To: <CAEZw2bzGAmeahP5ijT2Bq1v+PPT_Bcss4-bPuQXcP-o6i7BqRA@mail.gmail.com>
References: <CAEZw2bzGAmeahP5ijT2Bq1v+PPT_Bcss4-bPuQXcP-o6i7BqRA@mail.gmail.com>
Message-ID: <D0A7E134-1D7C-4517-90D4-91D27E8AF3A1@icir.org>


> On Apr 25, 2015, at 1:16 AM, anthony kasza <anthony.kasza at gmail.com> wrote:
> 
> Is there a way to control how the file analysis framework handles HTTP compression? For example, if a PNG is transferred over HTTP with gzip compression, can I have Bro dump the gzip file instead of the PNG?

Are you sure that?s what you really want?  In the case of gzip, deflate, etc encoded content, that encoding is actually part of the protocol it?s not really an aspect of the file.  That?s how Bro currently handles things at least and seems to be the most sane option to me, but perhaps you have some compelling use case?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150425/5b2ce47b/attachment.bin 

From anthony.kasza at gmail.com  Sat Apr 25 21:54:46 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Sat, 25 Apr 2015 21:54:46 -0700
Subject: [Bro] HTTP plus Compression File Extraction
In-Reply-To: <D0A7E134-1D7C-4517-90D4-91D27E8AF3A1@icir.org>
References: <CAEZw2bzGAmeahP5ijT2Bq1v+PPT_Bcss4-bPuQXcP-o6i7BqRA@mail.gmail.com>
	<D0A7E134-1D7C-4517-90D4-91D27E8AF3A1@icir.org>
Message-ID: <CAEZw2bwcAvqpzsXpm0eoFmN9YaZ_XAgvu+eSV7hEXfZiQEsdMA@mail.gmail.com>

It's absolutely the most sane case. As usual I have a specific use case in
mind. When the gzip contents are corrupted I'd like to attempt to recover
portions of whatever was transferred. I'll try to find an example trace...

-AK
On Apr 25, 2015 6:12 PM, "Seth Hall" <seth at icir.org> wrote:

>
> > On Apr 25, 2015, at 1:16 AM, anthony kasza <anthony.kasza at gmail.com>
> wrote:
> >
> > Is there a way to control how the file analysis framework handles HTTP
> compression? For example, if a PNG is transferred over HTTP with gzip
> compression, can I have Bro dump the gzip file instead of the PNG?
>
> Are you sure that?s what you really want?  In the case of gzip, deflate,
> etc encoded content, that encoding is actually part of the protocol it?s
> not really an aspect of the file.  That?s how Bro currently handles things
> at least and seems to be the most sane option to me, but perhaps you have
> some compelling use case?
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150425/82f875ec/attachment.html 

From seth at icir.org  Sat Apr 25 22:08:52 2015
From: seth at icir.org (Seth Hall)
Date: Sun, 26 Apr 2015 01:08:52 -0400
Subject: [Bro] HTTP plus Compression File Extraction
In-Reply-To: <CAEZw2bwcAvqpzsXpm0eoFmN9YaZ_XAgvu+eSV7hEXfZiQEsdMA@mail.gmail.com>
References: <CAEZw2bzGAmeahP5ijT2Bq1v+PPT_Bcss4-bPuQXcP-o6i7BqRA@mail.gmail.com>
	<D0A7E134-1D7C-4517-90D4-91D27E8AF3A1@icir.org>
	<CAEZw2bwcAvqpzsXpm0eoFmN9YaZ_XAgvu+eSV7hEXfZiQEsdMA@mail.gmail.com>
Message-ID: <7C574D40-97CA-48FC-B275-395959ADC667@icir.org>


> On Apr 26, 2015, at 12:54 AM, anthony kasza <anthony.kasza at gmail.com> wrote:
> 
> It's absolutely the most sane case. As usual I have a specific use case in mind. When the gzip contents are corrupted I'd like to attempt to recover portions of whatever was transferred. I'll try to find an example trace...

I may have just fixed the problem recently that you?re seeing.  There is an issue where some web servers don?t set all of the deflate headers correctly and Bro has never dealt with this right, but I fixed it recently, I just have to find where I put it, I don?t think I ever pushed it out to git.

Does this sound like the problem you?re seeing?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150426/01dced69/attachment.bin 

From franky.meier.1 at gmx.de  Mon Apr 27 00:29:43 2015
From: franky.meier.1 at gmx.de (Frank Meier)
Date: Mon, 27 Apr 2015 09:29:43 +0200
Subject: [Bro] delayed bro operation
In-Reply-To: <D4B8EAD4-0211-4FB0-9337-05057381C2B5@icir.org>
References: <1429866971.5205.0@mail.gmx.net>
	<D4B8EAD4-0211-4FB0-9337-05057381C2B5@icir.org>
Message-ID: <1430119783.6473.0@mail.gmx.net>

Hi.

On Fr, Apr 24, 2015 at 4:23 , Seth Hall <seth at icir.org> wrote:
> 
>>  On Apr 24, 2015, at 5:16 AM, Frank Meier <franky.meier.1 at gmx.de> 
>> wrote:
>>  
>>  A policy forces me to run bro in a separate network. So the 
>> captured PCAPs are
>>  transfered to the bro network for logging purposes. How would I 
>> handle delays
>>  in feeding bro with the PCAPS? Would connections spanning multiple 
>> PCAPs be a
>>  problem?
> 
> This is a problem that PacketBricks[1] will be able to solve 
> eventually.  It?s not there yet, but eventually you?ll be able to 
> create a load balancing architecture with persistent 
> Bro/Snort/Suricata/etc processes and tell PacketBricks to read PCAPs 
> as you get them in place (and, yes, I did just say clustered PCAP 
> processing!).  Unfortunately this scenario is not quite ready in 
> PacketBricks.
> 

Thanks, I will have a look into that!

Franky
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150427/457862ab/attachment.html 

From franky.meier.1 at gmx.de  Mon Apr 27 08:32:49 2015
From: franky.meier.1 at gmx.de (Frank Meier)
Date: Mon, 27 Apr 2015 17:32:49 +0200
Subject: [Bro] log rotation leaving conn.log unrotated
Message-ID: <1430148769.23502.0@mail.gmx.net>

Hi.

Before I ask yet another question, I wanted to say thanks to all who 
helped me
during the last weeks. :) All tips where very helpful and fast! I hope 
I can repay 
for this by contributing in the future.

I have this simple bro file:

redef Log::default_rotation_postprocessor_cmd = "./postrotate.sh";
redef Log::default_rotation_interval = 10 sec;

with postrotate.sh just printing the parameters:

echo "-1-"
echo $1
echo $2
echo $2
echo $2
echo $2
echo "-2-"



Now when bro is terminated via CTRL-C, the script is called:
1430147916.038582 received termination signal
1430147916.038582 1865 packets received on interface eth0, 45 dropped

-1-
files.2015-04-27-17-18-30.log
files
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
http.2015-04-27-17-18-30.log
http
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
weird.2015-04-27-17-18-30.log
weird
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
conn.2015-04-27-17-18-30.log
conn
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
reporter.2015-04-27-17-18-36.log
reporter
15-04-27_17.18.36
15-04-27_17.18.36
1
ascii
-2-

After that there is still a conn.log around. Why is this file not 
rotated? 
When I restart bro now, the conn.log seems to be overwritten and
entries for example in files.log reference a uid not found in any of
the conn.logs.

Franky



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150427/c4c702ef/attachment.html 

From franky.meier.1 at gmx.de  Mon Apr 27 08:41:10 2015
From: franky.meier.1 at gmx.de (Frank Meier)
Date: Mon, 27 Apr 2015 17:41:10 +0200
Subject: [Bro] log rotation leaving conn.log unrotated
Message-ID: <1430149270.23502.1@mail.gmx.net>

Hi.

Before I ask yet another question, I wanted to say thanks to all who 
helped me
during the last weeks. :) All tips where very helpful and fast! I hope 
I can repay 
for this by contributing in the future.

I have this simple bro file:

redef Log::default_rotation_postprocessor_cmd = "./postrotate.sh";
redef Log::default_rotation_interval = 10 sec;

with postrotate.sh just printing the parameters:

#!/bin/sh
echo "-1-"
echo $1
echo $2
echo $3
echo $4
echo $5
echo $6
echo "-2-"

Now when bro is terminated via CTRL-C, the script is called:
1430147916.038582 received termination signal
1430147916.038582 1865 packets received on interface eth0, 45 dropped

-1-
files.2015-04-27-17-18-30.log
files
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
http.2015-04-27-17-18-30.log
http
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
weird.2015-04-27-17-18-30.log
weird
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
conn.2015-04-27-17-18-30.log
conn
15-04-27_17.18.30
15-04-27_17.18.36
1
ascii
-2-
-1-
reporter.2015-04-27-17-18-36.log
reporter
15-04-27_17.18.36
15-04-27_17.18.36
1
ascii
-2-

After that there is still a conn.log around. Why is this file not 
rotated? 
When I restart bro now, the conn.log seems to be overwritten and
entries for example in files.log reference a uid not found in any of
the conn.logs.

I could not figure out why rotation works for most of the logs, but
not for conn.log.

Franky



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150427/8e66077c/attachment.html 

From anthony.kasza at gmail.com  Mon Apr 27 10:57:57 2015
From: anthony.kasza at gmail.com (anthony kasza)
Date: Mon, 27 Apr 2015 10:57:57 -0700
Subject: [Bro] HTTP plus Compression File Extraction
In-Reply-To: <7C574D40-97CA-48FC-B275-395959ADC667@icir.org>
References: <CAEZw2bzGAmeahP5ijT2Bq1v+PPT_Bcss4-bPuQXcP-o6i7BqRA@mail.gmail.com>
	<D0A7E134-1D7C-4517-90D4-91D27E8AF3A1@icir.org>
	<CAEZw2bwcAvqpzsXpm0eoFmN9YaZ_XAgvu+eSV7hEXfZiQEsdMA@mail.gmail.com>
	<7C574D40-97CA-48FC-B275-395959ADC667@icir.org>
Message-ID: <CAEZw2byuP7xn2TaTQqfP9_F-rCS0Pcjr_UM3hhz1KwgXvJ_nrQ@mail.gmail.com>

Nevermind! I was looking at corrupted gzip content being transmitted over
HTTP with gzip encoding. The HTTP encoding is handled properly by Bro. My
content was the issue.

-AK
On Apr 25, 2015 10:08 PM, "Seth Hall" <seth at icir.org> wrote:

>
> > On Apr 26, 2015, at 12:54 AM, anthony kasza <anthony.kasza at gmail.com>
> wrote:
> >
> > It's absolutely the most sane case. As usual I have a specific use case
> in mind. When the gzip contents are corrupted I'd like to attempt to
> recover portions of whatever was transferred. I'll try to find an example
> trace...
>
> I may have just fixed the problem recently that you?re seeing.  There is
> an issue where some web servers don?t set all of the deflate headers
> correctly and Bro has never dealt with this right, but I fixed it recently,
> I just have to find where I put it, I don?t think I ever pushed it out to
> git.
>
> Does this sound like the problem you?re seeing?
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150427/b97bd0f4/attachment.html 

From dnthayer at illinois.edu  Mon Apr 27 11:34:52 2015
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Mon, 27 Apr 2015 13:34:52 -0500
Subject: [Bro] log rotation leaving conn.log unrotated
In-Reply-To: <1430149270.23502.1@mail.gmx.net>
References: <1430149270.23502.1@mail.gmx.net>
Message-ID: <553E814C.8090104@illinois.edu>

On 04/27/2015 10:41 AM, Frank Meier wrote:
> Hi.
>
> Before I ask yet another question, I wanted to say thanks to all who
> helped me
> during the last weeks. :) All tips where very helpful and fast! I hope I
> can repay
> for this by contributing in the future.
>
> I have this simple bro file:
>
> redef Log::default_rotation_postprocessor_cmd = "./postrotate.sh";
> redef Log::default_rotation_interval = 10 sec;
>
> with postrotate.sh just printing the parameters:
>
> #!/bin/sh
> echo "-1-"
> echo $1
> echo $2
> echo $3
> echo $4
> echo $5
> echo $6
> echo "-2-"
>
> Now when bro is terminated via CTRL-C, the script is called:
> 1430147916.038582 received termination signal
> 1430147916.038582 1865 packets received on interface eth0, 45 dropped
>
> -1-
> files.2015-04-27-17-18-30.log
> files
> 15-04-27_17.18.30
> 15-04-27_17.18.36
> 1
> ascii
> -2-
> -1-
> http.2015-04-27-17-18-30.log
> http
> 15-04-27_17.18.30
> 15-04-27_17.18.36
> 1
> ascii
> -2-
> -1-
> weird.2015-04-27-17-18-30.log
> weird
> 15-04-27_17.18.30
> 15-04-27_17.18.36
> 1
> ascii
> -2-
> -1-
> conn.2015-04-27-17-18-30.log
> conn
> 15-04-27_17.18.30
> 15-04-27_17.18.36
> 1
> ascii
> -2-
> -1-
> reporter.2015-04-27-17-18-36.log
> reporter
> 15-04-27_17.18.36
> 15-04-27_17.18.36
> 1
> ascii
> -2-
>
> After that there is still a conn.log around. Why is this file not rotated?
> When I restart bro now, the conn.log seems to be overwritten and
> entries for example in files.log reference a uid not found in any of
> the conn.logs.
>
> I could not figure out why rotation works for most of the logs, but
> not for conn.log.
>
> Franky

In your example, conn.log was rotated (the new filename was 
conn.2015-04-27-17-18-30.log).  The conn.log file you saw
after terminating Bro was most likely created in the short time span
between rotating conn.log and Bro termination.


From giedrius.ramas at gmail.com  Tue Apr 28 00:39:35 2015
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Tue, 28 Apr 2015 10:39:35 +0300
Subject: [Bro] BRO intel framework
Message-ID: <CAHr6V5rOg7LqSMqCdC2YfpKyCmyWqLmzJVYG-4eyA=r-5k-61A@mail.gmail.com>

Hi all ,
I am using BRO intel framework and have some doubts about intel.dat file .
Currently I have in my script following :

redef Intel::read_files += {
        "/opt/bro/share/bro/intel/intel.dat"
};


How can I append data to /intel.dat ? Can I just overwrite it by using mv
linux command ? Is it necessary to reload bro once /intel.dat changed  ?

Please shed some light on how bro works with that file .
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150428/cefd9d2f/attachment.html 

From franky.meier.1 at gmx.de  Tue Apr 28 01:46:10 2015
From: franky.meier.1 at gmx.de (Frank Meier)
Date: Tue, 28 Apr 2015 10:46:10 +0200
Subject: [Bro] log rotation leaving conn.log unrotated
In-Reply-To: <553E814C.8090104@illinois.edu>
References: <1430149270.23502.1@mail.gmx.net> <553E814C.8090104@illinois.edu>
Message-ID: <1430210770.23502.2@mail.gmx.net>

hi,

On Mo, Apr 27, 2015 at 8:34 , Daniel Thayer <dnthayer at illinois.edu> 
wrote:
> 
> In your example, conn.log was rotated (the new filename was 
> conn.2015-04-27-17-18-30.log).  The conn.log file you saw
> after terminating Bro was most likely created in the short time span
> between rotating conn.log and Bro termination.

I think there is more to this. If repeat the following steps I do loose 
some entries in conn.log:

1) start bro
2) produce some traffic
3) stop bro via CTRL-C
4) restart bro
5) wait for log rotation
6) stop bro via CTRL-C

grep for conn_uids from files.log. For some entries in files.log there 
will be no match in any 
of the conn.logs. My workaround for now is to append the stale
conn.log to the last rotated log on shutdown:

file_name=$1
base_name=$2
from=$3
to=$4
terminating=$5
writer=$6

echo "[+] LOG: Rotating $file_name (base: $base_name, from: $from, to: 
$to, terminating: $terminating, writer: $writer)" 

if [ $terminating -eq 1 -a -f $base_name.log ]; then
        echo "terminating. appending stale $base_name.log "     
        # cut timestamp
        head -n-1 $file_name > $file_name.tmp
        mv $file_name.tmp $file_name
	# cut header
        tail -n +9 $base_name.log >> $file_name
        mv $base_name.log done.log
fi

I don't have the time right now, but I will look further into this.

Franky
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150428/9674a745/attachment.html 

From rrbarbosa at gmail.com  Tue Apr 28 02:24:37 2015
From: rrbarbosa at gmail.com (Rafael Barbosa)
Date: Tue, 28 Apr 2015 11:24:37 +0200
Subject: [Bro] Triggering events on incomplete PDUs
In-Reply-To: <CADOayQ-Wt0q-9fK0TdobxUYUqfFyjQfcXx8wQqtVtAVoc7oiDQ@mail.gmail.com>
References: <CADOayQ-Wt0q-9fK0TdobxUYUqfFyjQfcXx8wQqtVtAVoc7oiDQ@mail.gmail.com>
Message-ID: <CADOayQ-Lk6XiM2i3WndzQGkWjg62hXg7S6dQRppniWJ5JXtLQQ@mail.gmail.com>

Hi,

I realize that I might not have included enough details. Attached I am
sending the dlms-protocol.pac and dlms-analyzer.pac I created to process
DLMS traffic.

My current goal is to extract the fields on the wrapper (DLMS_Wrapper) even
when the message body (DLMS_Request/DLMS_Reply) is not complete in the
captured traffic. As is, all events I defined are only triggered when a
full PDU is present.

I could not find any information on how to trigger events on incomplete PDUs
on the bro website or mailing list, so any help is welcome.

I can also send the other files in my DLMS analyzer, and generate an
example pcap file for testing, if necessary.

Thanks,
Rafael


Rafael Barbosa

On Wed, Apr 22, 2015 at 11:34 AM, Rafael Barbosa <rrbarbosa at gmail.com>
wrote:

> Hi,
>
> I am implementing a simple protocol analyzer for DLMS (smart metering
> protocol), and I am trying to understand how the events are triggered.
>
> Basically, I am interested in the first few bytes of the PDU, which
> identify the types of requests/responses (e.g.: read, write,
> authentication, etc). I implemented an analyzer for these bytes based on
> the other protocols available, and I am able to trigger some events with
> the values I need when parsing an example file.
>
> However, the event only seem to be triggered when the full PDU is
> avaliable. This is a big problem because the `snaplen` used for the capture
> was quite small, thus most of the PDUs are incomplete.
>
> My question is:  Is there is a way that I can force an event to be
> triggered as soon as the first few bytes are available?
>
> Best,
> Rafael Barbosa
> Research Consultant
> www.encs.eu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150428/66da2808/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dlms-analyzer.pac
Type: application/x-ns-proxy-autoconfig
Size: 2509 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150428/66da2808/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dlms-protocol.pac
Type: application/x-ns-proxy-autoconfig
Size: 770 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150428/66da2808/attachment-0001.bin 

From seth at icir.org  Tue Apr 28 09:10:00 2015
From: seth at icir.org (Seth Hall)
Date: Tue, 28 Apr 2015 12:10:00 -0400
Subject: [Bro] BRO intel framework
In-Reply-To: <CAHr6V5rOg7LqSMqCdC2YfpKyCmyWqLmzJVYG-4eyA=r-5k-61A@mail.gmail.com>
References: <CAHr6V5rOg7LqSMqCdC2YfpKyCmyWqLmzJVYG-4eyA=r-5k-61A@mail.gmail.com>
Message-ID: <D8004946-4BA0-4F38-83BD-EDBE2BEBBC72@icir.org>


> On Apr 28, 2015, at 3:39 AM, Giedrius Ramas <giedrius.ramas at gmail.com> wrote:
> 
> How can I append data to /intel.dat ? Can I just overwrite it by using mv linux command ?

Yes, that?s the best option.

> Is it necessary to reload bro once /intel.dat changed  ?

Nope. Bro will pick up the changes automatically.  If you are running on a cluster, it will pick them up on the manager and distribute them out to the workers.  Also, the internal intelligence representation is accumulative.  If you remove something from that file, Bro is still watching for it.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150428/cd51faaa/attachment.bin 

From chilton.brian at yahoo.com  Tue Apr 28 10:50:42 2015
From: chilton.brian at yahoo.com (Brian Chilton)
Date: Tue, 28 Apr 2015 17:50:42 +0000 (UTC)
Subject: [Bro] file hashing, and virustotal api
Message-ID: <453996112.7430308.1430243442477.JavaMail.yahoo@mail.yahoo.com>

All,?Thanks for all the help on my previous question, but now I have another.? I would like to take the hashed values of files that bro see's and check them against virustotal using their API Key.? Additionally id like to take the results of the lookup and determine if there are a certain number of hits then look to see if our current anti-virus is one of them and if not fire a notice.? logic looks like this?if?virus total hits?is > 8 ?????see if anti-virus vendor is?true?if yes fire notice, if not end script.? 
the problem I am running into here is making the request out with bro.? The current framework only returns the number of hits, I think this method could be more useful depending on the return, so long as I can make the outbound request. Any input or ideas would be very helpful, and once again if this is documented somewhere pointing me in that direction would be fine as well.??Thanks again for the help,?BC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150428/2d571662/attachment-0001.html 

From seth at icir.org  Tue Apr 28 13:17:28 2015
From: seth at icir.org (Seth Hall)
Date: Tue, 28 Apr 2015 16:17:28 -0400
Subject: [Bro] file hashing, and virustotal api
In-Reply-To: <453996112.7430308.1430243442477.JavaMail.yahoo@mail.yahoo.com>
References: <453996112.7430308.1430243442477.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <1166D719-324B-4DD6-A7D6-9C9A3C602C2E@icir.org>


> On Apr 28, 2015, at 1:50 PM, Brian Chilton <chilton.brian at yahoo.com> wrote:
> 
> Thanks for all the help on my previous question, but now I have another.  I would like to take the hashed values of files that bro see's and check them against virustotal using their API Key.

Here?s a script I presented at the recent Bro4Pros event.  virus-total.bro has the core code and vt-hashing.bro integrates with the file analysis.

  .Seth


-------------- next part --------------
A non-text attachment was scrubbed...
Name: virus-total.bro
Type: application/octet-stream
Size: 2809 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150428/e19f7b02/attachment.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: vt-hashing.bro
Type: application/octet-stream
Size: 1121 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150428/e19f7b02/attachment-0001.obj 
-------------- next part --------------

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150428/e19f7b02/attachment.bin 

From hckim at narusec.com  Tue Apr 28 18:08:29 2015
From: hckim at narusec.com (=?UTF-8?B?6rmA7Z2s7LKg?=)
Date: Wed, 29 Apr 2015 10:08:29 +0900
Subject: [Bro] file hashing, and virustotal api
Message-ID: <CAHs13Ahaim+V5oGopw9GsDsVXokBMUjEf7-2n2_T3z3Ne_aM3g@mail.gmail.com>

>the problem I am running into here is making the request out with bro.?
I think you are looking for this?
https://github.com/sooshie/bro-scripts/blob/master/2.2-scripts/vt_check.bro

code returns the number of hits
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150429/8975a8f6/attachment.html 

From giedrius.ramas at gmail.com  Tue Apr 28 22:43:02 2015
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Wed, 29 Apr 2015 08:43:02 +0300
Subject: [Bro] BRO intel framework
In-Reply-To: <D8004946-4BA0-4F38-83BD-EDBE2BEBBC72@icir.org>
References: <CAHr6V5rOg7LqSMqCdC2YfpKyCmyWqLmzJVYG-4eyA=r-5k-61A@mail.gmail.com>
	<D8004946-4BA0-4F38-83BD-EDBE2BEBBC72@icir.org>
Message-ID: <CAHr6V5qw8-+SKEvRkB8AJ_1kfiUNUfrN=41nMF8kMh_3DP90pw@mail.gmail.com>

Thanks for reply,
Could you please elaborate more on that point:" Also, the internal
intelligence representation is accumulative.  If you remove something from
that file, Bro is still watching for it." So,  for example if I will
overwrite the whole intel file with the new one, what happened to the
records from the old file ? Bro still watching for them ?

On Tue, Apr 28, 2015 at 7:10 PM, Seth Hall <seth at icir.org> wrote:

>
> > On Apr 28, 2015, at 3:39 AM, Giedrius Ramas <giedrius.ramas at gmail.com>
> wrote:
> >
> > How can I append data to /intel.dat ? Can I just overwrite it by using
> mv linux command ?
>
> Yes, that?s the best option.
>
> > Is it necessary to reload bro once /intel.dat changed  ?
>
> Nope. Bro will pick up the changes automatically.  If you are running on a
> cluster, it will pick them up on the manager and distribute them out to the
> workers.  Also, the internal intelligence representation is accumulative.
> If you remove something from that file, Bro is still watching for it.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150429/4cd20af6/attachment.html 

From npratley at redhat.com  Tue Apr 28 22:59:27 2015
From: npratley at redhat.com (Nick Pratley)
Date: Wed, 29 Apr 2015 15:59:27 +1000
Subject: [Bro] BRO intel framework
In-Reply-To: <CAHr6V5qw8-+SKEvRkB8AJ_1kfiUNUfrN=41nMF8kMh_3DP90pw@mail.gmail.com>
References: <CAHr6V5rOg7LqSMqCdC2YfpKyCmyWqLmzJVYG-4eyA=r-5k-61A@mail.gmail.com>
	<D8004946-4BA0-4F38-83BD-EDBE2BEBBC72@icir.org>
	<CAHr6V5qw8-+SKEvRkB8AJ_1kfiUNUfrN=41nMF8kMh_3DP90pw@mail.gmail.com>
Message-ID: <1430287167.26137.13.camel@redhat.com>

On Wed, 2015-04-29 at 08:43 +0300, Giedrius Ramas wrote:
> Thanks for reply,
> Could you please elaborate more on that point:" Also, the internal
> intelligence representation is accumulative.  If you remove something
> from that file, Bro is still watching for it." So,  for example if I
> will overwrite the whole intel file with the new one, what happened to
> the records from the old file ? Bro still watching for them ?  

Yes, Bro would still be watching for them, at least if
http://blog.bro.org/2014/01/intelligence-data-and-bro_4980.html is still
accurate:

"
 A restart is required if you want to purge entries that have been
removed from the feeds, but not if you only want the new entries because
Bro keeps the file open and will pick up any new additions.
"



From giedrius.ramas at gmail.com  Wed Apr 29 04:42:03 2015
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Wed, 29 Apr 2015 14:42:03 +0300
Subject: [Bro] BRO intel framework
In-Reply-To: <1430287167.26137.13.camel@redhat.com>
References: <CAHr6V5rOg7LqSMqCdC2YfpKyCmyWqLmzJVYG-4eyA=r-5k-61A@mail.gmail.com>
	<D8004946-4BA0-4F38-83BD-EDBE2BEBBC72@icir.org>
	<CAHr6V5qw8-+SKEvRkB8AJ_1kfiUNUfrN=41nMF8kMh_3DP90pw@mail.gmail.com>
	<1430287167.26137.13.camel@redhat.com>
Message-ID: <CAHr6V5rJLHvSDRz_AELDcYQoOZy2WY1xEG+zyn5+1=gYDJztCg@mail.gmail.com>

One more thing I need to clarify. I see in bro intel data file (generated
by CIF)  Intel::URL   url's have a prefix http:// . However when I visit
these URLs  BRO Intel do not trigger. I tried to remove prefix http:// from
url's in BRO intel file and BRO Intel works well then. So is there anything
wrong with CIF generated BRO intel file or elsewhere ?

On Wed, Apr 29, 2015 at 8:59 AM, Nick Pratley <npratley at redhat.com> wrote:

> On Wed, 2015-04-29 at 08:43 +0300, Giedrius Ramas wrote:
> > Thanks for reply,
> > Could you please elaborate more on that point:" Also, the internal
> > intelligence representation is accumulative.  If you remove something
> > from that file, Bro is still watching for it." So,  for example if I
> > will overwrite the whole intel file with the new one, what happened to
> > the records from the old file ? Bro still watching for them ?
>
> Yes, Bro would still be watching for them, at least if
> http://blog.bro.org/2014/01/intelligence-data-and-bro_4980.html is still
> accurate:
>
> "
>  A restart is required if you want to purge entries that have been
> removed from the feeds, but not if you only want the new entries because
> Bro keeps the file open and will pick up any new additions.
> "
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150429/d761a688/attachment.html 

From seth at icir.org  Wed Apr 29 06:03:24 2015
From: seth at icir.org (Seth Hall)
Date: Wed, 29 Apr 2015 09:03:24 -0400
Subject: [Bro] BRO intel framework
In-Reply-To: <CAHr6V5qw8-+SKEvRkB8AJ_1kfiUNUfrN=41nMF8kMh_3DP90pw@mail.gmail.com>
References: <CAHr6V5rOg7LqSMqCdC2YfpKyCmyWqLmzJVYG-4eyA=r-5k-61A@mail.gmail.com>
	<D8004946-4BA0-4F38-83BD-EDBE2BEBBC72@icir.org>
	<CAHr6V5qw8-+SKEvRkB8AJ_1kfiUNUfrN=41nMF8kMh_3DP90pw@mail.gmail.com>
Message-ID: <F2729B4F-711E-4C9B-BB65-CA8C3B517D15@icir.org>


> On Apr 29, 2015, at 1:43 AM, Giedrius Ramas <giedrius.ramas at gmail.com> wrote:
> 
> Could you please elaborate more on that point:" Also, the internal intelligence representation is accumulative.  If you remove something from that file, Bro is still watching for it." So,  for example if I will overwrite the whole intel file with the new one, what happened to the records from the old file ? Bro still watching for them ?

Yes, it was designed that way originally so we that could do some optimizations in the future.  I?m starting to see some more work that needs to be done on the Intelligence framework though so I?m sure that over the next release or two there will be improvements coming in this area and others.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150429/028d4d00/attachment-0001.bin 

From seth at icir.org  Wed Apr 29 06:32:48 2015
From: seth at icir.org (Seth Hall)
Date: Wed, 29 Apr 2015 09:32:48 -0400
Subject: [Bro] BRO intel framework
In-Reply-To: <CAHr6V5rJLHvSDRz_AELDcYQoOZy2WY1xEG+zyn5+1=gYDJztCg@mail.gmail.com>
References: <CAHr6V5rOg7LqSMqCdC2YfpKyCmyWqLmzJVYG-4eyA=r-5k-61A@mail.gmail.com>
	<D8004946-4BA0-4F38-83BD-EDBE2BEBBC72@icir.org>
	<CAHr6V5qw8-+SKEvRkB8AJ_1kfiUNUfrN=41nMF8kMh_3DP90pw@mail.gmail.com>
	<1430287167.26137.13.camel@redhat.com>
	<CAHr6V5rJLHvSDRz_AELDcYQoOZy2WY1xEG+zyn5+1=gYDJztCg@mail.gmail.com>
Message-ID: <135874BD-DD8A-4CD2-92B0-5F9D621EDB72@icir.org>


> On Apr 29, 2015, at 7:42 AM, Giedrius Ramas <giedrius.ramas at gmail.com> wrote:
> 
> One more thing I need to clarify. I see in bro intel data file (generated by CIF)  Intel::URL   url's have a prefix http:// . However when I visit these URLs  BRO Intel do not trigger. I tried to remove prefix http:// from url's in BRO intel file and BRO Intel works well then. So is there anything wrong with CIF generated BRO intel file or elsewhere ?

Oh, that?s not good.  I actually thought at some point that I started stripping prefixes off of urls as they came in, but I may not have gotten that out anywhere.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150429/8ea711fc/attachment.bin 

From iamreck at gmail.com  Wed Apr 29 06:42:33 2015
From: iamreck at gmail.com (Ryan)
Date: Wed, 29 Apr 2015 09:42:33 -0400
Subject: [Bro] Bro Log Analysis - by CIDR
Message-ID: <CAAWJUBdGS2cZoC9whtSHZfmY2i1zDZCJQYME34t-+Y0Ngatwfg@mail.gmail.com>

I'm looking at analyzing bro logs, filtering by an arbitrary CIDR.

Before I go write a Python script that will handle this - I was wondering
if something already existed.

As an example -

    zcat ssl.12\:00\:00-13\:00\:00.log.gz | bro-cut server_name id.orig_h |
by_CIDR.py 129.21.1.0/23

I was also contemplating modifying bro-cut to handle this.

Thanks,
Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150429/8183abb4/attachment.html 

From iamreck at gmail.com  Wed Apr 29 10:49:32 2015
From: iamreck at gmail.com (Ryan)
Date: Wed, 29 Apr 2015 13:49:32 -0400
Subject: [Bro] Bro Log Analysis - by CIDR
In-Reply-To: <CAAWJUBdGS2cZoC9whtSHZfmY2i1zDZCJQYME34t-+Y0Ngatwfg@mail.gmail.com>
References: <CAAWJUBdGS2cZoC9whtSHZfmY2i1zDZCJQYME34t-+Y0Ngatwfg@mail.gmail.com>
Message-ID: <CAAWJUBdxvZRvqG+FtYnrEVJixcrztSi_NNcpQ6mJmvxA+PA0pg@mail.gmail.com>

If I do write a Python script to do this - I'm heavily inclined to use
Python 3 (for the ipaddress Module).

Ryan Peck


On Wed, Apr 29, 2015 at 9:42 AM, Ryan <iamreck at gmail.com> wrote:

> I'm looking at analyzing bro logs, filtering by an arbitrary CIDR.
>
> Before I go write a Python script that will handle this - I was wondering
> if something already existed.
>
> As an example -
>
>     zcat ssl.12\:00\:00-13\:00\:00.log.gz | bro-cut server_name id.orig_h
> | by_CIDR.py 129.21.1.0/23
>
> I was also contemplating modifying bro-cut to handle this.
>
> Thanks,
> Ryan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150429/93c444b1/attachment.html 

From paul.halliday at gmail.com  Wed Apr 29 11:14:15 2015
From: paul.halliday at gmail.com (Paul Halliday)
Date: Wed, 29 Apr 2015 15:14:15 -0300
Subject: [Bro] Bro Log Analysis - by CIDR
In-Reply-To: <CAAWJUBdxvZRvqG+FtYnrEVJixcrztSi_NNcpQ6mJmvxA+PA0pg@mail.gmail.com>
References: <CAAWJUBdGS2cZoC9whtSHZfmY2i1zDZCJQYME34t-+Y0Ngatwfg@mail.gmail.com>
	<CAAWJUBdxvZRvqG+FtYnrEVJixcrztSi_NNcpQ6mJmvxA+PA0pg@mail.gmail.com>
Message-ID: <CAJfn-RFpmJt1xWUQe-NouYMEguv1siLAWp9R1obTA0+EswncyA@mail.gmail.com>

Not sure if this helps:
https://www.bro.org/sphinx/components/pysubnettree/README.html

On Wed, Apr 29, 2015 at 2:49 PM, Ryan <iamreck at gmail.com> wrote:
> If I do write a Python script to do this - I'm heavily inclined to use
> Python 3 (for the ipaddress Module).
>
> Ryan Peck
>
>
> On Wed, Apr 29, 2015 at 9:42 AM, Ryan <iamreck at gmail.com> wrote:
>>
>> I'm looking at analyzing bro logs, filtering by an arbitrary CIDR.
>>
>> Before I go write a Python script that will handle this - I was wondering
>> if something already existed.
>>
>> As an example -
>>
>>     zcat ssl.12\:00\:00-13\:00\:00.log.gz | bro-cut server_name id.orig_h
>> | by_CIDR.py 129.21.1.0/23
>>
>> I was also contemplating modifying bro-cut to handle this.
>>
>> Thanks,
>> Ryan
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



-- 
Paul Halliday
http://www.pintumbler.org/

From iamreck at gmail.com  Wed Apr 29 11:17:21 2015
From: iamreck at gmail.com (Ryan)
Date: Wed, 29 Apr 2015 14:17:21 -0400
Subject: [Bro] Bro Log Analysis - by CIDR
In-Reply-To: <CAJfn-RFpmJt1xWUQe-NouYMEguv1siLAWp9R1obTA0+EswncyA@mail.gmail.com>
References: <CAAWJUBdGS2cZoC9whtSHZfmY2i1zDZCJQYME34t-+Y0Ngatwfg@mail.gmail.com>
	<CAAWJUBdxvZRvqG+FtYnrEVJixcrztSi_NNcpQ6mJmvxA+PA0pg@mail.gmail.com>
	<CAJfn-RFpmJt1xWUQe-NouYMEguv1siLAWp9R1obTA0+EswncyA@mail.gmail.com>
Message-ID: <CAAWJUBcbVVsgWQy_kjOD0d97gXobAEFhKPb9crD6rB2PN=KNXA@mail.gmail.com>

Ah - Very nice. Thank you for pointing that out Paul.

Ryan Peck


On Wed, Apr 29, 2015 at 2:14 PM, Paul Halliday <paul.halliday at gmail.com>
wrote:

> Not sure if this helps:
> https://www.bro.org/sphinx/components/pysubnettree/README.html
>
> On Wed, Apr 29, 2015 at 2:49 PM, Ryan <iamreck at gmail.com> wrote:
> > If I do write a Python script to do this - I'm heavily inclined to use
> > Python 3 (for the ipaddress Module).
> >
> > Ryan Peck
> >
> >
> > On Wed, Apr 29, 2015 at 9:42 AM, Ryan <iamreck at gmail.com> wrote:
> >>
> >> I'm looking at analyzing bro logs, filtering by an arbitrary CIDR.
> >>
> >> Before I go write a Python script that will handle this - I was
> wondering
> >> if something already existed.
> >>
> >> As an example -
> >>
> >>     zcat ssl.12\:00\:00-13\:00\:00.log.gz | bro-cut server_name
> id.orig_h
> >> | by_CIDR.py 129.21.1.0/23
> >>
> >> I was also contemplating modifying bro-cut to handle this.
> >>
> >> Thanks,
> >> Ryan
> >
> >
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
>
> --
> Paul Halliday
> http://www.pintumbler.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150429/d922236f/attachment.html 

From matt at monaco.cx  Wed Apr 29 20:08:16 2015
From: matt at monaco.cx (Matthew Monaco)
Date: Wed, 29 Apr 2015 21:08:16 -0600
Subject: [Bro] script/cluster management practices
Message-ID: <55419CA0.7040508@monaco.cx>

Hello,

My colleagues and I are interested in hearing about how some of you manage your
clusters and scripts.

Are most of your scripts from the Bro git repo? Or have you collected/developed
a lot over time? Especially for the latter, how many are you running in production?

Is it typical to worry about the performance impact of adding scripts; do you
ever remove things because packet drops grow too high? Or is it just time for
more hardware?

Along those lines, how big is your cluster (nodes/workers)? I've heard roughly
100 Mbps/core. Does this mean it's not uncommon to have a 400-core cluster for 40G?

How do you test your scripts? Are you really attentive about keeping PCAPs to
trigger alerts, etc?


Many thanks!
Matt

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150429/dc838bca/attachment.bin 

From life.130815 at gmail.com  Wed Apr 29 22:17:12 2015
From: life.130815 at gmail.com (Mo Jia)
Date: Thu, 30 Apr 2015 13:17:12 +0800
Subject: [Bro] send logs to custom server by socket
Message-ID: <CAJ17HSfHjgKYpx660FXq7ZpC_bXd8bNNKCwbp+_SpmXxvbNy6A@mail.gmail.com>

Hello:

If I don't want log to disk, and want send json logs to a remote
server. When some code like this Log::write(HTTP::LOG, c$http); it was
send http log to my server. Dose this mean I need change
src/logging/writters/ascii ? Or I should add a new writer something
like socket? I don't want change the bro scripts already have, so
Log:write(HTTP::LOG, c$http) should don't change. Or I think is
add a config like

LOG_SERVER_IP = 192.168.100
LOG_SERVER_PORT = 8087

and all the http , notice and so on all send to the server.
Any suggest? Or does somebody already done before?

From life.130815 at gmail.com  Thu Apr 30 09:27:33 2015
From: life.130815 at gmail.com (Mo Jia)
Date: Fri, 1 May 2015 00:27:33 +0800
Subject: [Bro] loging to elasticsearch git clone
Message-ID: <CAJ17HSfBLFy5OpbY+J7TZT-0PJUMexdPGaE_-fE5aLQua-8whQ@mail.gmail.com>

Hi :

I follow the https://www.bro.org/sphinx/frameworks/logging-elasticsearch.html
with git clone latest source, seem it can't take effect to find it
should build elasticsearch.  So how can I build elasticsearch with
latest source?

From hosom at battelle.org  Thu Apr 30 18:38:15 2015
From: hosom at battelle.org (Hosom, Stephen M)
Date: Fri, 1 May 2015 01:38:15 +0000
Subject: [Bro] send logs to custom server by socket
In-Reply-To: <CAJ17HSfHjgKYpx660FXq7ZpC_bXd8bNNKCwbp+_SpmXxvbNy6A@mail.gmail.com>
References: <CAJ17HSfHjgKYpx660FXq7ZpC_bXd8bNNKCwbp+_SpmXxvbNy6A@mail.gmail.com>
Message-ID: <E5A2071C6FFFF640B714C224139C9B3301A7CD10@WP-MBX2B.milky-way.battelle.org>

I believe you likely want functionality that technically exists in Master. 

Check out remote logging with Broker... https://www.bro.org/sphinx-git/frameworks/broker.html#remote-logging

I haven't played with that yet, so I can't be certain it does precisely what you want... 

Alternatively, you could just delete the logs after they rotate and send the logs via syslog with rsyslog, or your syslog daemon of choice. 

Let me know if that helps!
________________________________________
From: bro-bounces at bro.org [bro-bounces at bro.org] on behalf of Mo Jia [life.130815 at gmail.com]
Sent: Thursday, April 30, 2015 1:17 AM
To: bro at bro.org
Subject: [Bro] send logs to custom server by socket

Hello:

If I don't want log to disk, and want send json logs to a remote
server. When some code like this Log::write(HTTP::LOG, c$http); it was
send http log to my server. Dose this mean I need change
src/logging/writters/ascii ? Or I should add a new writer something
like socket? I don't want change the bro scripts already have, so
Log:write(HTTP::LOG, c$http) should don't change. Or I think is
add a config like

LOG_SERVER_IP = 192.168.100
LOG_SERVER_PORT = 8087

and all the http , notice and so on all send to the server.
Any suggest? Or does somebody already done before?
_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From daniel.guerra69 at gmail.com  Thu Apr 30 19:41:26 2015
From: daniel.guerra69 at gmail.com (Daniel Guerra)
Date: Fri, 1 May 2015 04:41:26 +0200
Subject: [Bro] loging to elasticsearch git clone
In-Reply-To: <CAJ17HSfBLFy5OpbY+J7TZT-0PJUMexdPGaE_-fE5aLQua-8whQ@mail.gmail.com>
References: <CAJ17HSfBLFy5OpbY+J7TZT-0PJUMexdPGaE_-fE5aLQua-8whQ@mail.gmail.com>
Message-ID: <40FFA35A-0161-4E0A-B573-6278DF689ED2@gmail.com>

I log to json files. After this I use logstash to store it in elasticsearch.
Logstash has an embeded elasicsearch + kibana

in bro edit init-default.bro and add @load policy/tuning/json-logs

a config i use for logstash might be handy for you 

Regards,
Daniel

input {
  file {
    codec => json
    path => "/input/*.log"
    type => "bro_log"
  }
}

filter {
  # Parse the `time` attribute as a UNIX timestamp (seconds since epoch)
  # and store it in `@timestamp` attribute. This will be used in Kibana later on.
  date {
    match => [ "ts", "UNIX" ]
  }
  translate {
      field => "conn_state"
      destination => "conn_state_full"
      dictionary => [
        "S0", "Attempt",
        "S1", "Established",
        "S2", "Originator close only",
        "S3", "Responder close only",
        "SF", "SYN/FIN completion",
        "REJ", "Rejected",
        "RSTO", "Originator aborted",
        "RSTR", "Responder aborted",
        "RSTOS0", "Originator SYN +  RST",
        "RSTRH", "Responder SYN ACK + RST",
        "SH", "Originator SYN + FIN",
        "SHR", "Responder SYN ACK + FIN",
        "OTH", "Midstream traffic"
      ]
    }
    grok {
        match => { "path" => ".*\/(?<bro_type>[a-zA-Z0-9]+)\.log$" }
    }
}


output {
  elasticsearch {
    embedded => true
  }
}

> On 30 Apr 2015, at 18:27, Mo Jia <life.130815 at gmail.com> wrote:
> 
> Hi :
> 
> I follow the https://www.bro.org/sphinx/frameworks/logging-elasticsearch.html
> with git clone latest source, seem it can't take effect to find it
> should build elasticsearch.  So how can I build elasticsearch with
> latest source?
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150501/57d5d47f/attachment.html