From vladg at illinois.edu  Mon Feb  1 07:30:42 2016
From: vladg at illinois.edu (Vlad Grigorescu)
Date: Mon, 01 Feb 2016 09:30:42 -0600
Subject: [Bro] Binpac analyzer error
In-Reply-To: <CAAHbCpFEcPjuKwvoji+9VztENu5QgNk2V8huzqaR=XzzAOhRzw@mail.gmail.com>
References: <CAAHbCpFEcPjuKwvoji+9VztENu5QgNk2V8huzqaR=XzzAOhRzw@mail.gmail.com>
Message-ID: <m0twls30t9.fsf@vladg.net>

You're not doing anything wrong, this was a bug in
binpac_quickstart. Two people submitted pull requests fixing this issue,
I just hadn't gotten around to merging it in yet.

I just merged a fix for this into master and it should work for you now. Please let
me know if you run into any more issues.

Thanks,

  --Vlad

Matias Davaro <matiasdavaro at gmail.com> writes:

> Hello,
>
> I am trying to learn the bro programming language and following along with
> Jon Schipp's youtube video bro - writing an analyzer. When I attempt to
> generate the files for the protocol I receive the following error:
>
> /binpac_quickstart$ ./start.py RIP "Routing Internet Protocol" ../bro --udp
> Traceback (most recent call last):
>   File "./start.py", line 177, in <module>
>     main(arguments)
>   File "./start.py", line 59, in main
>     if do_plugin:
> UnboundLocalError: local variable 'do_plugin' referenced before assignment
> elcabezon at elcabezon:~/binpac_quickstart$
>
>
> However when I append plugin at the end of the command, the files are
> generated in the src and scripts directory. I just want to know what I am
> doing wrong. Thank you very much.
>
> Very respectfully,
>
> Matias
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 800 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160201/64cbb33d/attachment.bin 

From jlay at slave-tothe-box.net  Mon Feb  1 15:31:39 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Mon, 01 Feb 2016 16:31:39 -0700
Subject: [Bro] Lying about DNS yields interesting bro entries
Message-ID: <586fc929bfa8d4c338a30d1a8fd15716@localhost>

Curious.  I'll show the data first:

2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53      udp 
     dns     -       -       -       SHR     T      F0       d       0    
    0       1       73      (empty)
2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53      udp 
     21365   -       -       -       -       -      2SERVFAIL        F    
    F       F       F       0       -       -       T
2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53      
dns_unmatched_reply     -       F       bro


Packet capture listening to udp port 420 (no other match for 
65.113.230.90):
2016-02-01 08:48:12.311086633 65.113.230.90 -> x.x.x.x DNS 89 Standard 
query response 0x5375 Server failure A otqxwnenalwb.www.1818my[.]com < 
[] added by me


Syslog:
Feb  1 08:48:12 kernel: [157335.232732] IN=ppp0 OUT= MAC= 
SRC=65.113.230.90 DST=x.x.x.x LEN=73 TOS=0x00 PREC=0x00 TTL=249 ID=23786 
PROTO=UDP SPT=53 DPT=420 LEN=53

I guess my question is, is this desired behavior?  I see the 
dns_unmatched_reply, but it seems the first two entries never 
happened...so should they be there?  Thanks...more of a curious question 
more than anything else.

James

From vincent at ragosta.net  Tue Feb  2 07:21:21 2016
From: vincent at ragosta.net (vincent at ragosta.net)
Date: Tue, 02 Feb 2016 10:21:21 -0500
Subject: [Bro] credit-card-exposure script
Message-ID: <cdcfb9e245efdc737ed0f16108e47b9a@ragosta.net>

Hello,

I'm running Security Onion under Ubuntu 14.04 and am trying to use Seth 
Hall's credit-card-exposure script. I installed it as follows:

cd /opt/bro/share/bro
git clone git://github.com/sethhall/credit-card-exposure.git
echo "@load credit-card-exposure" >> local.bro

I then restarted Bro and see the following in the stderr.log file:

warning in /opt/bro/share/bro/credit-card-exposure/./main.bro, line 81: 
deprecated (split_all)
warning in /opt/bro/share/bro/credit-card-exposure/./main.bro, line 91: 
deprecated (join_string_array)


I tried triggering a notification by emailing myself a fake credit card 
number and visiting web sites with test numbers on them, but no alert is 
triggered. Can anyone provide some assistance?

Thank you!

From seth at icir.org  Tue Feb  2 08:20:02 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 2 Feb 2016 11:20:02 -0500
Subject: [Bro] Lying about DNS yields interesting bro entries
In-Reply-To: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
References: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
Message-ID: <1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>


> On Feb 1, 2016, at 6:31 PM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> I guess my question is, is this desired behavior?  I see the 
> dns_unmatched_reply, but it seems the first two entries never 
> happened...so should they be there?  Thanks...more of a curious question 
> more than anything else.

Which two entries are you referring to?  This looks correct to me.  It looks like you saw a stray DNS response message, but there was no query.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Tue Feb  2 08:38:21 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 2 Feb 2016 11:38:21 -0500
Subject: [Bro] credit-card-exposure script
In-Reply-To: <cdcfb9e245efdc737ed0f16108e47b9a@ragosta.net>
References: <cdcfb9e245efdc737ed0f16108e47b9a@ragosta.net>
Message-ID: <984DA2CD-7662-4644-8C68-F5108E4F1D87@icir.org>


> On Feb 2, 2016, at 10:21 AM, vincent at ragosta.net wrote:
> 
> I tried triggering a notification by emailing myself a fake credit card 
> number and visiting web sites with test numbers on them, but no alert is 
> triggered. Can anyone provide some assistance?

Sorry about that.  No one had commented that that script didn't work in 2.4.  It works now.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From vincent at ragosta.net  Tue Feb  2 08:53:19 2016
From: vincent at ragosta.net (vincent at ragosta.net)
Date: Tue, 02 Feb 2016 11:53:19 -0500
Subject: [Bro] credit-card-exposure script
Message-ID: <564806f03bc4a38b34d846d576296074@ragosta.net>

> Sorry about that.  No one had commented that that script didn't work in 
> 2.4.  It works now.

Bro loads the script without generating an error. However, I still do 
not see any alerts appearing in notice.log.

Anything special I need to do to have it log this alert?

Thanks!

From seth at icir.org  Tue Feb  2 08:56:37 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 2 Feb 2016 11:56:37 -0500
Subject: [Bro] credit-card-exposure script
In-Reply-To: <564806f03bc4a38b34d846d576296074@ragosta.net>
References: <564806f03bc4a38b34d846d576296074@ragosta.net>
Message-ID: <969C513A-9502-48CF-B008-77182395DF7E@icir.org>

Could you send me some traffic that you think it should catch but isn't? (you can send privately)

  .Seth

> On Feb 2, 2016, at 11:53 AM, vincent at ragosta.net wrote:
> 
>> Sorry about that.  No one had commented that that script didn't work in 
>> 2.4.  It works now.
> 
> Bro loads the script without generating an error. However, I still do 
> not see any alerts appearing in notice.log.
> 
> Anything special I need to do to have it log this alert?
> 
> Thanks!
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From jlay at slave-tothe-box.net  Tue Feb  2 09:01:45 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Tue, 02 Feb 2016 10:01:45 -0700
Subject: [Bro] Lying about DNS yields interesting bro entries
In-Reply-To: <1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>
References: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
	<1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>
Message-ID: <951e64537520fa0d9dc728de2e636024@localhost>

On 2016-02-02 09:20, Seth Hall wrote:
>> On Feb 1, 2016, at 6:31 PM, James Lay <jlay at slave-tothe-box.net> 
>> wrote:
>> 
>> I guess my question is, is this desired behavior?  I see the
>> dns_unmatched_reply, but it seems the first two entries never
>> happened...so should they be there?  Thanks...more of a curious 
>> question
>> more than anything else.
> 
> Which two entries are you referring to?  This looks correct to me.  It
> looks like you saw a stray DNS response message, but there was no
> query.
> 
>   .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/

Hi Seth,

Pretty sure this is me missing something first off.  But to be honest 
all the entries:

2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53      udp 
     dns     -       -       -       SHR     T      F0       d       0    
    0       1       73      (empty)
2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53      udp 
     21365   -       -       -       -       -      2SERVFAIL        F    
    F       F       F       0       -       -       T
2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53      
dns_unmatched_reply     -       F       bro


The above says to me "x.x.x.x sent data to 65.113.230.90 on port 53, and 
got a servfail response, and this was actually an unmatched dns 
response".  But in reality, this is what happened:

2016-02-01T08:48:12-0700  65.113.230.90    53   x.x.x.x    420      
dns_unmatched_reply     -       F       bro

65.113.230.90 was the id.orig_h, not x.x.x.x, but as I read the first 
three they state that x.x.x.x was the id.orig_h.  But in fact per this 
drop:

Feb  1 08:48:12 kernel: [157335.232732] IN=ppp0 OUT= MAC= 
SRC=65.113.230.90 DST=x.x.x.x LEN=73 TOS=0x00 PREC=0x00 TTL=249 ID=23786 
PROTO=UDP SPT=53 DPT=420 LEN=53

x.x.x.x did not send any traffic to 65.113.230.90, even though conn, 
dns, and weird.  As I look at it though, I think it's me needing to get 
over reading left to right with Bro :)  Thanks Seth...hope that makes 
sense.

James

From anthony.kasza at gmail.com  Tue Feb  2 10:56:11 2016
From: anthony.kasza at gmail.com (anthony kasza)
Date: Tue, 2 Feb 2016 10:56:11 -0800
Subject: [Bro] Lying about DNS yields interesting bro entries
In-Reply-To: <951e64537520fa0d9dc728de2e636024@localhost>
References: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
	<1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>
	<951e64537520fa0d9dc728de2e636024@localhost>
Message-ID: <CAEZw2bynr2e89UAoyKKYBitrGfKKp8jSjtb+YQS1oQosrWfQAg@mail.gmail.com>

It sounds like the oddness is around the orig_h and resp_h of unmatched
replies.
Which system originated a connection of an unmatched DNS reply? That begs
the question: was the reply unsolicited or did Bro miss the request?

-AK
On Feb 2, 2016 9:18 AM, "James Lay" <jlay at slave-tothe-box.net> wrote:

> On 2016-02-02 09:20, Seth Hall wrote:
> >> On Feb 1, 2016, at 6:31 PM, James Lay <jlay at slave-tothe-box.net>
> >> wrote:
> >>
> >> I guess my question is, is this desired behavior?  I see the
> >> dns_unmatched_reply, but it seems the first two entries never
> >> happened...so should they be there?  Thanks...more of a curious
> >> question
> >> more than anything else.
> >
> > Which two entries are you referring to?  This looks correct to me.  It
> > looks like you saw a stray DNS response message, but there was no
> > query.
> >
> >   .Seth
> >
> > --
> > Seth Hall
> > International Computer Science Institute
> > (Bro) because everyone has a network
> > http://www.bro.org/
>
> Hi Seth,
>
> Pretty sure this is me missing something first off.  But to be honest
> all the entries:
>
> 2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53      udp
>      dns     -       -       -       SHR     T      F0       d       0
>     0       1       73      (empty)
> 2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53      udp
>      21365   -       -       -       -       -      2SERVFAIL        F
>     F       F       F       0       -       -       T
> 2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53
> dns_unmatched_reply     -       F       bro
>
>
> The above says to me "x.x.x.x sent data to 65.113.230.90 on port 53, and
> got a servfail response, and this was actually an unmatched dns
> response".  But in reality, this is what happened:
>
> 2016-02-01T08:48:12-0700  65.113.230.90    53   x.x.x.x    420
> dns_unmatched_reply     -       F       bro
>
> 65.113.230.90 was the id.orig_h, not x.x.x.x, but as I read the first
> three they state that x.x.x.x was the id.orig_h.  But in fact per this
> drop:
>
> Feb  1 08:48:12 kernel: [157335.232732] IN=ppp0 OUT= MAC=
> SRC=65.113.230.90 DST=x.x.x.x LEN=73 TOS=0x00 PREC=0x00 TTL=249 ID=23786
> PROTO=UDP SPT=53 DPT=420 LEN=53
>
> x.x.x.x did not send any traffic to 65.113.230.90, even though conn,
> dns, and weird.  As I look at it though, I think it's me needing to get
> over reading left to right with Bro :)  Thanks Seth...hope that makes
> sense.
>
> James
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160202/6164d011/attachment-0001.html 

From jlay at slave-tothe-box.net  Tue Feb  2 11:16:34 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Tue, 02 Feb 2016 12:16:34 -0700
Subject: [Bro] Lying about DNS yields interesting bro entries
In-Reply-To: <CAEZw2bynr2e89UAoyKKYBitrGfKKp8jSjtb+YQS1oQosrWfQAg@mail.gmail.com>
References: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
	<1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>
	<951e64537520fa0d9dc728de2e636024@localhost>
	<CAEZw2bynr2e89UAoyKKYBitrGfKKp8jSjtb+YQS1oQosrWfQAg@mail.gmail.com>
Message-ID: <e051948cde8ee147cba0e9fafda9dacb@localhost>

Hi Anthony, 

I should first preface the whole reason I started down this path was
because I find that a fair amount of firewall hits I see are on port
420, so I started packet capturing for udp port 420, and that's when I
started to notice this.  So here's what Bro showed for the IP of
65.113.230.90: 

2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53      udp
    dns     -       -       -       SHR     T      F0       d       0   
   0       1       73      (empty)
 2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53     
udp     21365   -       -       -       -       -      2SERVFAIL       
F       F       F       F       0       -       -       T
 2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53     
dns_unmatched_reply     -       F       bro 

 The packet capture showed this: 
2016-02-01 08:48:12.311086633 65.113.230.90 -> x.x.x.x DNS 89 Standard
query response 0x5375 Server failure A otqxwnenalwb.www.1818my[.]com [1]
< [] added by me

 And syslog shows the dropped packet:
 Feb  1 08:48:12 kernel: [157335.232732] IN=ppp0 OUT= MAC=
SRC=65.113.230.90 DST=x.x.x.x LEN=73 TOS=0x00 PREC=0x00 TTL=249 ID=23786
PROTO=UDP SPT=53 DPT=420 LEN=53 

I believe all these replies are unsolicited..here's more info from the
pcap: 

  1 2016-02-01 07:27:04.816315071 162.227.35.49 -> x.x.x.x DNS 80
Standard query response 0x5375 Server failure A xec.www.1818my.com
  2 2016-02-01 07:29:06.691950594 69.165.170.13 -> x.x.x.x DNS 86
Standard query response 0x5375 Server failure A hdjxvefrc.www.1818my.com
  3 2016-02-01 07:43:15.851708630 77.245.146.9 -> x.x.x.x DNS 78
Standard query response 0x5375 Server failure A x.www.1818my.com
  4 2016-02-01 07:58:16.362832986 185.37.170.75 -> x.x.x.x DNS 84
Standard query response 0x5375 Server failure A mvaakbx.www.1818my.com
  5 2016-02-01 08:21:34.161432864 200.66.71.204 -> x.x.x.x DNS 91
Standard query response 0x5375 Server failure A
qpohqpuhyhydkx.www.1818my.com
  6 2016-02-01 08:34:15.116312435 177.38.182.14 -> x.x.x.x DNS 80
Standard query response 0x5375 Server failure A xba.www.1818my.com
  7 2016-02-01 08:46:06.804898711 180.166.211.154 -> x.x.x.x DNS 89
Standard query response 0x5375 Server failure A
cnmxiditebuh.www.1818my.com
  8 2016-02-01 08:48:12.311086633 65.113.230.90 -> x.x.x.x DNS 89
Standard query response 0x5375 Server failure A
otqxwnenalwb.www.1818my.com
  9 2016-02-01 09:50:28.034713107 86.53.30.145 -> x.x.x.x DNS 80
Standard query response 0x5375 Server failure A xbd.www.1818my.com
 10 2016-02-01 10:45:33.406354013 189.36.206.166 -> x.x.x.x DNS 82
Standard query response 0x5375 Server failure A zavyx.www.1818my.com
 11 2016-02-01 10:51:52.599903143 188.64.112.107 -> x.x.x.x DNS 78
Standard query response 0x5375 Server failure A x.www.1818my.com
 12 2016-02-01 10:55:32.267107992  66.79.47.97 -> x.x.x.x DNS 90
Standard query response 0x5375 Server failure A
nocdefghvwxym.www.1818my.com
 13 2016-02-01 11:30:22.634179669 41.223.176.254 -> x.x.x.x DNS 88
Standard query response 0x5375 Server failure A
yzxeuexhebp.www.1818my.com
 14 2016-02-01 11:37:14.661145722 209.200.125.113 -> x.x.x.x DNS 91
Standard query response 0x5375 Server failure A
edgjgdehunglcx.www.1818my.com
 15 2016-02-01 12:27:23.082648045 110.45.190.78 -> x.x.x.x DNS 93
Standard query response 0x5375 Server failure A
epgxahchmlejcbgn.www.1818my.com
 16 2016-02-01 12:48:26.354106148 195.208.51.100 -> x.x.x.x DNS 90
Standard query response 0x5375 Server failure A
nbcdefguvwxlz.www.1818my.com
 17 2016-02-01 13:21:15.031248372 38.100.165.4 -> x.x.x.x DNS 90
Standard query response 0x5375 Server failure A
nbcqesguijxlm.www.1818my.com
 18 2016-02-01 13:24:04.308343363 117.204.35.201 -> x.x.x.x DNS 78
Standard query response 0x5375 Server failure A x.www.1818my.com
 19 2016-02-01 13:47:54.805174964 216.227.105.205 -> x.x.x.x DNS 158
Standard query response 0x5375 No such name A gaxavkciclt.www.1818my.com
SOA rad0.elltel.net
 20 2016-02-01 13:55:47.810864539 208.106.155.144 -> x.x.x.x DNS 82
Standard query response 0x5375 Server failure A craxx.www.1818my.com
 21 2016-02-01 14:03:28.220225897 120.151.219.248 -> x.x.x.x DNS 89
Standard query response 0x5375 Server failure A
uzuxwlodybgn.www.1818my.com
 22 2016-02-01 14:30:31.689614428 40.134.192.119 -> x.x.x.x DNS 108
Standard query response 0x5375 A mamzjnvojxeopzr.www.1818my.com A
127.123.45.67
 23 2016-02-01 15:04:36.756997582 219.163.72.226 -> x.x.x.x DNS 80
Standard query response 0x5375 Refused A xoo.www.1818my.com
 24 2016-02-01 16:10:37.693277685 198.71.54.86 -> x.x.x.x DNS 78
Standard query response 0x5375 Refused A x.www.1818my.com
 25 2016-02-01 16:23:06.072114319 95.79.36.228 -> x.x.x.x DNS 86
Standard query response 0x5375 Server failure A xqzxqckbl.www.1818my.com
 26 2016-02-01 16:44:48.085836197 86.62.78.132 -> x.x.x.x DNS 93
Standard query response 0x5375 Server failure A
cjmxkxwvixoxmdoj.www.1818my.com
 27 2016-02-01 16:45:53.057522907 110.137.88.41 -> x.x.x.x DNS 80
Standard query response 0x5375 Server failure A xzi.www.1818my.com
 28 2016-02-01 17:49:34.389252897 46.10.71.248 -> x.x.x.x DNS 80
Standard query response 0x5375 Server failure A xrz.www.1818my.com
 29 2016-02-01 18:13:04.732047378 192.198.208.163 -> x.x.x.x DNS 90
Standard query response 0x5375 Server failure A
abpdrsguijxlm.www.1818my.com
 31 2016-02-01 20:07:53.110496413  60.6.223.26 -> x.x.x.x DNS 107
Standard query response 0x5375 A ifoxmncxspsxkx.www.1818my.com A
127.0.0.1
 32 2016-02-01 23:41:16.181497169 84.120.111.129 -> x.x.x.x DNS 89
Standard query response 0x5375 Server failure A
ahcxsfedofit.www.1818my.com
 33 2016-02-02 00:04:54.704232888 68.153.208.145 -> x.x.x.x DNS 92
Standard query response 0x5375 Server failure A
zscwrvdouxlhhkb.www.1818my.com
 34 2016-02-02 00:22:04.204288331 188.171.5.71 -> x.x.x.x DNS 91
Standard query response 0xc737 Server failure A
cxyvqralibkjux.www.1818my.com[Malformed Packet]
 35 2016-02-02 00:23:21.751179271  60.2.46.214 -> x.x.x.x DNS 94
Standard query response 0x5375 A x.www.1818my.com A 127.0.0.1
 36 2016-02-02 00:31:50.457266962 202.4.227.99 -> x.x.x.x DNS 82
Standard query response 0x5375 Server failure A goylx.www.1818my.com
 37 2016-02-02 01:25:38.118660850 77.85.169.52 -> x.x.x.x DNS 91
Standard query response 0x5375 Server failure A
czipcjorufyxgx.www.1818my.com
 38 2016-02-02 01:28:33.637461325 210.227.116.101 -> x.x.x.x DNS 92
Standard query response 0x5375 Server failure A
ezcuarfwvxdotau.www.1818my.com
 39 2016-02-02 01:55:14.214997203 220.157.103.38 -> x.x.x.x DNS 88
Standard query response 0x5375 Server failure A
lixgvrowxjb.www.1818my.com
 40 2016-02-02 02:02:32.277401732 207.30.133.65 -> x.x.x.x DNS 92
Standard query response 0x5375 Server failure A
nfwnvkmqpxxjbof.www.1818my.com
 41 2016-02-02 02:55:41.387701175 50.240.13.113 -> x.x.x.x DNS 80
Standard query response 0x5375 Refused A xji.www.1818my.com
 42 2016-02-02 03:04:08.783792035 203.115.19.200 -> x.x.x.x DNS 92
Standard query response 0x5375 Server failure A
npcmbhqulxaxrzm.www.1818my.com
 44 2016-02-02 03:43:48.400021981 94.229.95.156 -> x.x.x.x DNS 88
Standard query response 0x5375 Server failure A
gdxypnfowcc.www.1818my.com
133 2016-02-02 10:24:15.482620371 101.99.20.163 -> x.x.x.x DNS 92
Standard query response 0x5375 Server failure A
jqysdilejxcznxr.www.1818my.com
134 2016-02-02 10:38:53.528569603 90.154.197.253 -> x.x.x.x DNS 82
Standard query response 0x5375 Server failure A uacvx.www.1818my.com
135 2016-02-02 10:51:19.133470580 86.102.168.66 -> x.x.x.x DNS 82
Standard query response 0x5375 Server failure A lxxax.www.1818my.com
136 2016-02-02 10:56:28.214752509 163.23.118.1 -> x.x.x.x DNS 86
Standard query response 0x5375 Refused A fmhxnyfya.www.1818my.com
137 2016-02-02 10:56:48.808973842  193.8.47.24 -> x.x.x.x DNS 82
Standard query response 0x5375 Server failure A mdhyx.www.1818my.com
138 2016-02-02 11:34:38.020879046 193.106.93.233 -> x.x.x.x DNS 78
Standard query response 0x5375 Server failure A x.www.1818my.com

Of interest is that every single one of these is source from 53, and
destination is 420, which is why they continue to get dropped at my
firewall.  Very curious.  I'm going to switch the capture to full on dns
and see what comes up.  More to come...thank you. 

James 

On 2016-02-02 11:56, anthony kasza wrote: 

> It sounds like the oddness is around the orig_h and resp_h of
> unmatched replies.
> Which system originated a connection of an unmatched DNS reply? That
> begs the question: was the reply unsolicited or did Bro miss the
> request? 
> 
> -AK
> 
> On Feb 2, 2016 9:18 AM, "James Lay" <jlay at slave-tothe-box.net> wrote:
> 
> On 2016-02-02 09:20, Seth Hall wrote: On Feb 1, 2016, at 6:31 PM, James Lay <jlay at slave-tothe-box.net>
> wrote:
> 
> I guess my question is, is this desired behavior?  I see the
> dns_unmatched_reply, but it seems the first two entries never
> happened...so should they be there?  Thanks...more of a curious
> question
> more than anything else. 
> Which two entries are you referring to?  This looks correct to me.
 It 

> looks like you saw a stray DNS response message, but there was no
> query.
> 
> .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/

Hi Seth,

Pretty sure this is me missing something first off.  But to be
honest
all the entries:

2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53
udp
dns     -       -       -       SHR     T      F0       d
0
0       1       73      (empty)
2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53
udp
21365   -       -       -       -       -      2SERVFAIL
F
F       F       F       0       -       -       T
2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53
dns_unmatched_reply     -       F       bro

The above says to me "x.x.x.x sent data to 65.113.230.90 on port 53,
and
got a servfail response, and this was actually an unmatched dns
response".  But in reality, this is what happened:

2016-02-01T08:48:12-0700  65.113.230.90    53   x.x.x.x    420
dns_unmatched_reply     -       F       bro

65.113.230.90 was the id.orig_h, not x.x.x.x, but as I read the
first
three they state that x.x.x.x was the id.orig_h.  But in fact per
this
drop:

Feb  1 08:48:12 kernel: [157335.232732] IN=ppp0 OUT= MAC=
SRC=65.113.230.90 DST=x.x.x.x LEN=73 TOS=0x00 PREC=0x00 TTL=249
ID=23786
PROTO=UDP SPT=53 DPT=420 LEN=53

x.x.x.x did not send any traffic to 65.113.230.90, even though conn,
dns, and weird.  As I look at it though, I think it's me needing to
get
over reading left to right with Bro :)  Thanks Seth...hope that
makes
sense.

James
_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro 

Links:
------
[1] http://www.1818my[.]com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160202/83948250/attachment-0001.html 

From andrew.william.smith at gmail.com  Tue Feb  2 11:31:36 2016
From: andrew.william.smith at gmail.com (Andrew Smith)
Date: Tue, 2 Feb 2016 13:31:36 -0600
Subject: [Bro] Lying about DNS yields interesting bro entries
In-Reply-To: <CAEZw2bynr2e89UAoyKKYBitrGfKKp8jSjtb+YQS1oQosrWfQAg@mail.gmail.com>
References: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
	<1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>
	<951e64537520fa0d9dc728de2e636024@localhost>
	<CAEZw2bynr2e89UAoyKKYBitrGfKKp8jSjtb+YQS1oQosrWfQAg@mail.gmail.com>
Message-ID: <CAPFwV1zbJG7SOCCuMUccDXENSCsEwerUPnyYQRaxtFY4Y_Pg0Q@mail.gmail.com>

It looks unlikely that Bro missed the request based on the hostname in the
query.

That looks like the DNS server is getting attacked with a spoofed DNS query
flood, and is sending DNS responses to the spoofed addresses, and one of
the spoofed addresses just happened to be one of James' IPs, so Bro is
really seeing a response that it didn't see a request for, because the
request came from some attacker out on the Internet. In other words, it's
backscatter from someone else being attacked.

The hostname in the query looks like it's has extra randomized text
prepended to an actual hostname to avoid caches and to cause as much load
on the DNS server as possible.

On Tue, Feb 2, 2016 at 12:56 PM, anthony kasza <anthony.kasza at gmail.com>
wrote:

> It sounds like the oddness is around the orig_h and resp_h of unmatched
> replies.
> Which system originated a connection of an unmatched DNS reply? That begs
> the question: was the reply unsolicited or did Bro miss the request?
>
> -AK
> On Feb 2, 2016 9:18 AM, "James Lay" <jlay at slave-tothe-box.net> wrote:
>
>> On 2016-02-02 09:20, Seth Hall wrote:
>> >> On Feb 1, 2016, at 6:31 PM, James Lay <jlay at slave-tothe-box.net>
>> >> wrote:
>> >>
>> >> I guess my question is, is this desired behavior?  I see the
>> >> dns_unmatched_reply, but it seems the first two entries never
>> >> happened...so should they be there?  Thanks...more of a curious
>> >> question
>> >> more than anything else.
>> >
>> > Which two entries are you referring to?  This looks correct to me.  It
>> > looks like you saw a stray DNS response message, but there was no
>> > query.
>> >
>> >   .Seth
>> >
>> > --
>> > Seth Hall
>> > International Computer Science Institute
>> > (Bro) because everyone has a network
>> > http://www.bro.org/
>>
>> Hi Seth,
>>
>> Pretty sure this is me missing something first off.  But to be honest
>> all the entries:
>>
>> 2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53      udp
>>      dns     -       -       -       SHR     T      F0       d       0
>>     0       1       73      (empty)
>> 2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53      udp
>>      21365   -       -       -       -       -      2SERVFAIL        F
>>     F       F       F       0       -       -       T
>> 2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53
>> dns_unmatched_reply     -       F       bro
>>
>>
>> The above says to me "x.x.x.x sent data to 65.113.230.90 on port 53, and
>> got a servfail response, and this was actually an unmatched dns
>> response".  But in reality, this is what happened:
>>
>> 2016-02-01T08:48:12-0700  65.113.230.90    53   x.x.x.x    420
>> dns_unmatched_reply     -       F       bro
>>
>> 65.113.230.90 was the id.orig_h, not x.x.x.x, but as I read the first
>> three they state that x.x.x.x was the id.orig_h.  But in fact per this
>> drop:
>>
>> Feb  1 08:48:12 kernel: [157335.232732] IN=ppp0 OUT= MAC=
>> SRC=65.113.230.90 DST=x.x.x.x LEN=73 TOS=0x00 PREC=0x00 TTL=249 ID=23786
>> PROTO=UDP SPT=53 DPT=420 LEN=53
>>
>> x.x.x.x did not send any traffic to 65.113.230.90, even though conn,
>> dns, and weird.  As I look at it though, I think it's me needing to get
>> over reading left to right with Bro :)  Thanks Seth...hope that makes
>> sense.
>>
>> James
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160202/7503f7bb/attachment.html 

From seth at icir.org  Tue Feb  2 11:42:44 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 2 Feb 2016 14:42:44 -0500
Subject: [Bro] credit-card-exposure script
In-Reply-To: <969C513A-9502-48CF-B008-77182395DF7E@icir.org>
References: <564806f03bc4a38b34d846d576296074@ragosta.net>
	<969C513A-9502-48CF-B008-77182395DF7E@icir.org>
Message-ID: <612AA4EF-2D86-4256-B81A-96B60676A883@icir.org>

Thanks for the traffic, there were some lingering bugs.  I also added an option to make it work for the traffic you sent me.

redef CreditCardExposure::use_cc_separators=F;

That will make the script validate long strings of digits (it only worked if the CC# had internal separators before).

Thanks!
  .Seth

> On Feb 2, 2016, at 11:56 AM, Seth Hall <seth at icir.org> wrote:
> 
> Could you send me some traffic that you think it should catch but isn't? (you can send privately)
> 
>  .Seth
> 
>> On Feb 2, 2016, at 11:53 AM, vincent at ragosta.net wrote:
>> 
>>> Sorry about that.  No one had commented that that script didn't work in 
>>> 2.4.  It works now.
>> 
>> Bro loads the script without generating an error. However, I still do 
>> not see any alerts appearing in notice.log.
>> 
>> Anything special I need to do to have it log this alert?
>> 
>> Thanks!
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> 
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Tue Feb  2 11:44:38 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 2 Feb 2016 14:44:38 -0500
Subject: [Bro] Lying about DNS yields interesting bro entries
In-Reply-To: <CAPFwV1zbJG7SOCCuMUccDXENSCsEwerUPnyYQRaxtFY4Y_Pg0Q@mail.gmail.com>
References: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
	<1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>
	<951e64537520fa0d9dc728de2e636024@localhost>
	<CAEZw2bynr2e89UAoyKKYBitrGfKKp8jSjtb+YQS1oQosrWfQAg@mail.gmail.com>
	<CAPFwV1zbJG7SOCCuMUccDXENSCsEwerUPnyYQRaxtFY4Y_Pg0Q@mail.gmail.com>
Message-ID: <BF69046C-AA9E-4710-9E09-FA14EBC64152@icir.org>


> On Feb 2, 2016, at 2:31 PM, Andrew Smith <andrew.william.smith at gmail.com> wrote:
> 
> That looks like the DNS server is getting attacked with a spoofed DNS query flood, and is sending DNS responses to the spoofed addresses, and one of the spoofed addresses just happened to be one of James' IPs, so Bro is really seeing a response that it didn't see a request for, because the request came from some attacker out on the Internet. In other words, it's backscatter from someone else being attacked.

Yep, I believe that's exactly right.  Bro is also (correctly) flipping the connection around which you can see in the conn.log because the originator of the "connection" never sent any packets.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From jlay at slave-tothe-box.net  Tue Feb  2 17:50:04 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Tue, 02 Feb 2016 18:50:04 -0700
Subject: [Bro] Lying about DNS yields interesting bro entries
In-Reply-To: <BF69046C-AA9E-4710-9E09-FA14EBC64152@icir.org>
References: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
	<1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>
	<951e64537520fa0d9dc728de2e636024@localhost>
	<CAEZw2bynr2e89UAoyKKYBitrGfKKp8jSjtb+YQS1oQosrWfQAg@mail.gmail.com>
	<CAPFwV1zbJG7SOCCuMUccDXENSCsEwerUPnyYQRaxtFY4Y_Pg0Q@mail.gmail.com>
	<BF69046C-AA9E-4710-9E09-FA14EBC64152@icir.org>
Message-ID: <1454464204.11343.4.camel@gamebox>

On Tue, 2016-02-02 at 14:44 -0500, Seth Hall wrote:

> > On Feb 2, 2016, at 2:31 PM, Andrew Smith <andrew.william.smith at gmail.com> wrote:
> > 
> > That looks like the DNS server is getting attacked with a spoofed DNS query flood, and is sending DNS responses to the spoofed addresses, and one of the spoofed addresses just happened to be one of James' IPs, so Bro is really seeing a response that it didn't see a request for, because the request came from some attacker out on the Internet. In other words, it's backscatter from someone else being attacked.
> 
> Yep, I believe that's exactly right.  Bro is also (correctly) flipping the connection around which you can see in the conn.log because the originator of the "connection" never sent any packets.
> 
>   .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/


Ok cool....we are all in agreement that this is an unsolicited DNS
response.  However...wouldn't this:

2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53
     udp     dns     -       -       -       SHR     T      F0       d
      0       0       1       73      (empty)
2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53
     udp     21365   -       -       -       -       -      2SERVFAIL
       F       F       F       F       0       -       -       T
2016-02-01T08:48:12-0700  x.x.x.x    420     65.113.230.90   53
dns_unmatched_reply     -       F       bro

be something instead like this (the below is a made up entry):

2016-02-01T08:48:12-0700  65.113.230.90    420     x.x.x.x  53
dns_unmatched_reply     -       F       bro

Not trying to beat a dead horse here...just trying to understand how Bro
is treating a DNS response that it never saw requested.  Thanks all.

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160202/14af547c/attachment.html 

From seth at icir.org  Tue Feb  2 20:59:40 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 2 Feb 2016 23:59:40 -0500
Subject: [Bro] Lying about DNS yields interesting bro entries
In-Reply-To: <1454464204.11343.4.camel@gamebox>
References: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
	<1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>
	<951e64537520fa0d9dc728de2e636024@localhost>
	<CAEZw2bynr2e89UAoyKKYBitrGfKKp8jSjtb+YQS1oQosrWfQAg@mail.gmail.com>
	<CAPFwV1zbJG7SOCCuMUccDXENSCsEwerUPnyYQRaxtFY4Y_Pg0Q@mail.gmail.com>
	<BF69046C-AA9E-4710-9E09-FA14EBC64152@icir.org>
	<1454464204.11343.4.camel@gamebox>
Message-ID: <259CE645-A45F-4213-A817-23385E0A49F1@icir.org>


> On Feb 2, 2016, at 8:50 PM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> 2016-02-01T08:48:12-0700  65.113.230.90    420     x.x.x.x  53      dns_unmatched_reply     -       F       bro
> 
> Not trying to beat a dead horse here...just trying to understand how Bro is treating a DNS response that it never saw requested.  Thanks all.

Hah, not a problem.  A lot of this stuff has so many edge cases and fairly arbitrary decisions on how to handle various situations deep down in scripts.

I am actually seeing the issue you're getting now.  It's like the IP addresses were flipped but the ports weren't.  To be completely honest, I don't know what's causing that without seeing the actual traffic.  Could you send a packet that causes this behavior?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From jan.grashoefer at gmail.com  Wed Feb  3 02:28:55 2016
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Wed, 3 Feb 2016 11:28:55 +0100
Subject: [Bro] Lying about DNS yields interesting bro entries
In-Reply-To: <259CE645-A45F-4213-A817-23385E0A49F1@icir.org>
References: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
	<1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>
	<951e64537520fa0d9dc728de2e636024@localhost>
	<CAEZw2bynr2e89UAoyKKYBitrGfKKp8jSjtb+YQS1oQosrWfQAg@mail.gmail.com>
	<CAPFwV1zbJG7SOCCuMUccDXENSCsEwerUPnyYQRaxtFY4Y_Pg0Q@mail.gmail.com>
	<BF69046C-AA9E-4710-9E09-FA14EBC64152@icir.org>
	<1454464204.11343.4.camel@gamebox>
	<259CE645-A45F-4213-A817-23385E0A49F1@icir.org>
Message-ID: <56B1D667.6020006@gmail.com>

Hi,

> I am actually seeing the issue you're getting now.  It's like the IP addresses were flipped but the ports weren't.  To be completely honest, I don't know what's causing that without seeing the actual traffic.  Could you send a packet that causes this behavior?

I think you are talking past each other. If I am not mistaken, James is
struggling with the originator/responder pattern of Bro. I guess he just
forgot to swap ports in his made up log line.

So the question would be: Why is the source IP logged as the responder's
IP for the unmatched reply?

That would be because source/destination is not equal to
originator/responder. At first Bro assumes the source is the originator.
But then Bro identifies the packet as a DNS response and therefore
determines the source IP as the responder's IP. So orig/resp get flipped
as Seth wrote:

> Bro is also (correctly) flipping the connection around which you can see in the conn.log because the originator of the "connection" never sent any packets.

Did I get this right, James, or are you really struggling with flipped
ports?

Jan

From vincent at ragosta.net  Wed Feb  3 04:46:30 2016
From: vincent at ragosta.net (vincent at ragosta.net)
Date: Wed, 03 Feb 2016 07:46:30 -0500
Subject: [Bro] credit-card-exposure script
In-Reply-To: <mailman.1.1454443201.23021.bro@bro.org>
References: <mailman.1.1454443201.23021.bro@bro.org>
Message-ID: <e3a4a3375ab2910cb6375f747785b3ce@ragosta.net>

Seth,

I appreciate you looking at this so quickly, but I still cannot get a 
notice to be raised with the latest changes. No error is reported and 
the script is loaded. Is there anything else I can check to debug this?

Thank you,

Vincent


> Thanks for the traffic, there were some lingering bugs.  I also added
> an option to make it work for the traffic you sent me.
> 
> redef CreditCardExposure::use_cc_separators=F;
> 
> That will make the script validate long strings of digits (it only
> worked if the CC# had internal separators before).
> 
> Thanks!
>   .Seth

From jlay at slave-tothe-box.net  Wed Feb  3 05:46:29 2016
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 03 Feb 2016 06:46:29 -0700
Subject: [Bro] Lying about DNS yields interesting bro entries
In-Reply-To: <56B1D667.6020006@gmail.com>
References: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
	<1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>
	<951e64537520fa0d9dc728de2e636024@localhost>
	<CAEZw2bynr2e89UAoyKKYBitrGfKKp8jSjtb+YQS1oQosrWfQAg@mail.gmail.com>
	<CAPFwV1zbJG7SOCCuMUccDXENSCsEwerUPnyYQRaxtFY4Y_Pg0Q@mail.gmail.com>
	<BF69046C-AA9E-4710-9E09-FA14EBC64152@icir.org>
	<1454464204.11343.4.camel@gamebox>
	<259CE645-A45F-4213-A817-23385E0A49F1@icir.org>
	<56B1D667.6020006@gmail.com>
Message-ID: <1454507189.11343.11.camel@gamebox>

On Wed, 2016-02-03 at 11:28 +0100, Jan Grash?fer wrote:

> Hi,
> 
> > I am actually seeing the issue you're getting now.  It's like the IP addresses were flipped but the ports weren't.  To be completely honest, I don't know what's causing that without seeing the actual traffic.  Could you send a packet that causes this behavior?
> 
> I think you are talking past each other. If I am not mistaken, James is
> struggling with the originator/responder pattern of Bro. I guess he just
> forgot to swap ports in his made up log line.
> 
> So the question would be: Why is the source IP logged as the responder's
> IP for the unmatched reply?
> 
> That would be because source/destination is not equal to
> originator/responder. At first Bro assumes the source is the originator.
> But then Bro identifies the packet as a DNS response and therefore
> determines the source IP as the responder's IP. So orig/resp get flipped
> as Seth wrote:
> 
> > Bro is also (correctly) flipping the connection around which you can see in the conn.log because the originator of the "connection" never sent any packets.
> 
> Did I get this right, James, or are you really struggling with flipped
> ports?
> 
> Jan
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


Thanks Jan...I think I finally explained it well enough that Seth is
able to look at it.  At the end of the day the question for me is when
an unsolicited dns response comes in from source port 53 to destination
port 420, why does bro show my machine as the originator of the traffic.
Guess I should have just said that in the first place 8-|

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160203/7c7429e6/attachment.html 

From seth at icir.org  Wed Feb  3 06:15:28 2016
From: seth at icir.org (Seth Hall)
Date: Wed, 3 Feb 2016 09:15:28 -0500
Subject: [Bro] credit-card-exposure script
In-Reply-To: <e3a4a3375ab2910cb6375f747785b3ce@ragosta.net>
References: <mailman.1.1454443201.23021.bro@bro.org>
	<e3a4a3375ab2910cb6375f747785b3ce@ragosta.net>
Message-ID: <B1E6BDA7-EA95-4997-B66E-63C2B3EDF56C@icir.org>


> On Feb 3, 2016, at 7:46 AM, vincent at ragosta.net wrote:
> 
> I appreciate you looking at this so quickly, but I still cannot get a 
> notice to be raised with the latest changes. No error is reported and 
> the script is loaded. Is there anything else I can check to debug this?

Did you change that setting that I mentioned?  By default, that script is only watching for numbers with separators in them (like 1234-5678-9012-3456) but in the traffic you sent me, those numbers don't have internal separators.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Wed Feb  3 06:19:00 2016
From: seth at icir.org (Seth Hall)
Date: Wed, 3 Feb 2016 09:19:00 -0500
Subject: [Bro] Lying about DNS yields interesting bro entries
In-Reply-To: <1454507189.11343.11.camel@gamebox>
References: <586fc929bfa8d4c338a30d1a8fd15716@localhost>
	<1A9CFAE9-CBB9-481C-B930-25FC67CD997F@icir.org>
	<951e64537520fa0d9dc728de2e636024@localhost>
	<CAEZw2bynr2e89UAoyKKYBitrGfKKp8jSjtb+YQS1oQosrWfQAg@mail.gmail.com>
	<CAPFwV1zbJG7SOCCuMUccDXENSCsEwerUPnyYQRaxtFY4Y_Pg0Q@mail.gmail.com>
	<BF69046C-AA9E-4710-9E09-FA14EBC64152@icir.org>
	<1454464204.11343.4.camel@gamebox>
	<259CE645-A45F-4213-A817-23385E0A49F1@icir.org>
	<56B1D667.6020006@gmail.com> <1454507189.11343.11.camel@gamebox>
Message-ID: <604325FA-4533-4349-9B1A-F5897CC40C5D@icir.org>


> On Feb 3, 2016, at 8:46 AM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> Thanks Jan...I think I finally explained it well enough that Seth is able to look at it.  At the end of the day the question for me is when an unsolicited dns response comes in from source port 53 to destination port 420, why does bro show my machine as the originator of the traffic.  Guess I should have just said that in the first place 8-|

Hah, I actually answered your question in the first reply, but then I got confused when I looked at your reply to that email.

Anyway, what Bro is doing is using port 53/udp as a heuristic and it's taking a guess that it may have the originator and responder backwards so it flips them.  One thing I've been meaning to add for a long time is an indicator in the conn.log for connections that were flipped.  At the very least it would be nice to know if Bro flipped the connection in it's attempt to analyze the traffic correctly.  

It does make for some subtlety in analyzing logs if you don't know that Bro is doing that.  Especially in cases like this where only a single packet was sent from outside your network.  The biggest thing to keep in mind that originator and responder are very different concepts than source and destination.  src and dst work perfectly if you're talking about individual packets, but when you're talking about a connection composed of two flows and many packets, originator and responder work much better.
 
 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From vincent at ragosta.net  Wed Feb  3 15:51:06 2016
From: vincent at ragosta.net (vincent at ragosta.net)
Date: Wed, 03 Feb 2016 18:51:06 -0500
Subject: [Bro] credit-card-exposure script
In-Reply-To: <B1E6BDA7-EA95-4997-B66E-63C2B3EDF56C@icir.org>
References: <mailman.1.1454443201.23021.bro@bro.org>
	<e3a4a3375ab2910cb6375f747785b3ce@ragosta.net>
	<B1E6BDA7-EA95-4997-B66E-63C2B3EDF56C@icir.org>
Message-ID: <0d130796df557a2fcca338dfc93eacbc@ragosta.net>

> Did you change that setting that I mentioned?

I failed to make the change. After making the change it is now firing a 
notice as intended.

Thank you!

On 2016-02-03 09:15, Seth Hall wrote:
>> On Feb 3, 2016, at 7:46 AM, vincent at ragosta.net wrote:
>> 
>> I appreciate you looking at this so quickly, but I still cannot get a
>> notice to be raised with the latest changes. No error is reported and
>> the script is loaded. Is there anything else I can check to debug 
>> this?
> 
> Did you change that setting that I mentioned?  By default, that script
> is only watching for numbers with separators in them (like
> 1234-5678-9012-3456) but in the traffic you sent me, those numbers
> don't have internal separators.
> 
>   .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/

From dnj0496 at gmail.com  Wed Feb  3 19:55:06 2016
From: dnj0496 at gmail.com (Dk Jack)
Date: Wed, 3 Feb 2016 19:55:06 -0800
Subject: [Bro] event q.
Message-ID: <CAGATen6xy5puFJ+_kyKc4JHoaQ=+9w1WiKzSNxyW7i6yaQiW4A@mail.gmail.com>

Hi,
I am generating an event from my plugin. I wrote a script to create a new
record and stream to log my event. I added my script to my local.bro file
and ran it against a pcap like this:

./bro -r <pcap> -C ../share/bro/site/local.bro

The event log foo.log gets created correctly i.e. the log has entries
corresponding to the events in
the pcap. However, when I run bro using broctl, and replay packets using
tcpreplay, it generates
an empty foo.log. That is, the log file only contains the headers. Is there
something special I need
to add to my script when running in cluster mode? Thanks.

Dnj.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160203/a82f7bae/attachment.html 

From vitologrillo at gmail.com  Thu Feb  4 10:11:12 2016
From: vitologrillo at gmail.com (Vito Logrillo)
Date: Thu, 4 Feb 2016 19:11:12 +0100
Subject: [Bro] Problems with tcprs plugin
Message-ID: <CANdYiFHpbSVHfrOcZTPrt82Q+SFO_+kTMpjReKG=He6YqeS9iw@mail.gmail.com>

Hi all,
i'm trying to compile the tcprs plugin with the latest bro version
(2.4-267): using the make command i have this error

/tcprs/src/TCPRS_Endpoint.cc: In member function
?analyzer::tcp::SCORE*
analyzer::tcp::TCPRS_Endpoint::scoreRetransmission(analyzer::tcp::SequenceRange*,
analyzer::tcp::Segment*, analyzer::tcp::RETRANSMISSION_REASON_CODE,
HashKey*)?:
/home/aramis/bro/aux/plugins/tcprs/src/TCPRS_Endpoint.cc:1180:47:
error: call of overloaded ?abs(unsigned int)? is ambiguous
      confidence_value *= 1.0 / (abs(i - TRIPLE) + 1.0); //ack distancing
                                               ^

what's wrong?
Thanks


From daniel.guerra69 at gmail.com  Sat Feb  6 13:34:44 2016
From: daniel.guerra69 at gmail.com (Daniel Guerra)
Date: Sat, 6 Feb 2016 22:34:44 +0100
Subject: [Bro] Problems with tcprs plugin
In-Reply-To: <CANdYiFHpbSVHfrOcZTPrt82Q+SFO_+kTMpjReKG=He6YqeS9iw@mail.gmail.com>
References: <CANdYiFHpbSVHfrOcZTPrt82Q+SFO_+kTMpjReKG=He6YqeS9iw@mail.gmail.com>
Message-ID: <805DEDEF-F73D-4DED-A94B-76054D322171@gmail.com>

Hi Vito,

Same problem here 

/tmp/bro/aux/plugins/tcprs/src/TCPRS_Endpoint.cc: In member function ?analyzer::tcp::SCORE* analyzer::tcp::TCPRS_Endpoint::scoreRetransmission(analyzer::tcp::SequenceRange*, analyzer::tcp::Segment*, analyzer::tcp::RETRANSMISSION_REASON_CODE, HashKey*)?:
/tmp/bro/aux/plugins/tcprs/src/TCPRS_Endpoint.cc:1180:47: error: call of overloaded ?abs(unsigned int)? is ambiguous
      confidence_value *= 1.0 / (abs(i - TRIPLE) + 1.0); //ack distancing
..
[0mCMakeFiles/Bro-TCPRS.linux-x86_64.dir/build.make:130: recipe for target 'CMakeFiles/Bro-TCPRS.linux-x86_64.dir/src/TCPRS_Endpoint.cc.o' failed
make[3]: *** [CMakeFiles/Bro-TCPRS.linux-x86_64.dir/src/TCPRS_Endpoint.cc.o] Error 1


> On 04 Feb 2016, at 19:11, Vito Logrillo <vitologrillo at gmail.com> wrote:
> 
> Hi all,
> i'm trying to compile the tcprs plugin with the latest bro version
> (2.4-267): using the make command i have this error
> 
> /tcprs/src/TCPRS_Endpoint.cc: In member function
> ?analyzer::tcp::SCORE*
> analyzer::tcp::TCPRS_Endpoint::scoreRetransmission(analyzer::tcp::SequenceRange*,
> analyzer::tcp::Segment*, analyzer::tcp::RETRANSMISSION_REASON_CODE,
> HashKey*)?:
> /home/aramis/bro/aux/plugins/tcprs/src/TCPRS_Endpoint.cc:1180:47:
> error: call of overloaded ?abs(unsigned int)? is ambiguous
>      confidence_value *= 1.0 / (abs(i - TRIPLE) + 1.0); //ack distancing
>                                               ^
> 
> what's wrong?
> Thanks
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160206/0f6eda2a/attachment.html 

From james.swaro at gmail.com  Mon Feb  8 14:59:36 2016
From: james.swaro at gmail.com (James Swaro)
Date: Mon, 8 Feb 2016 16:59:36 -0600
Subject: [Bro] Problems with tcprs plugin
In-Reply-To: <805DEDEF-F73D-4DED-A94B-76054D322171@gmail.com>
References: <CANdYiFHpbSVHfrOcZTPrt82Q+SFO_+kTMpjReKG=He6YqeS9iw@mail.gmail.com>
	<805DEDEF-F73D-4DED-A94B-76054D322171@gmail.com>
Message-ID: <CAAG4iqn5sju+mWg4XrON5eSLE_YyY9JPdTuZwQ9bTDi45fy-OQ@mail.gmail.com>

I'll take a look at that tonight. Can you tell me what environment you're
using?
On Feb 6, 2016 15:44, "Daniel Guerra" <daniel.guerra69 at gmail.com> wrote:

> Hi Vito,
>
> Same problem here
>
> [91m/tmp/bro/aux/plugins/tcprs/src/TCPRS_Endpoint.cc: In member function
> ?analyzer::tcp::SCORE*
> analyzer::tcp::TCPRS_Endpoint::scoreRetransmission(analyzer::tcp::SequenceRange*,
> analyzer::tcp::Segment*, analyzer::tcp::RETRANSMISSION_REASON_CODE,
> HashKey*)?: /tmp/bro/aux/plugins/tcprs/src/TCPRS_Endpoint.cc:1180:47:
> error: call of overloaded ?abs(unsigned int)? is ambiguous confidence_value
> *= 1.0 / (abs(i - TRIPLE) + 1.0); //ack distancing
> ..
> [0mCMakeFiles/Bro-TCPRS.linux-x86_64.dir/build.make:130: recipe for target
> 'CMakeFiles/Bro-TCPRS.linux-x86_64.dir/src/TCPRS_Endpoint.cc.o' failed
> [91mmake[3]: ***
> [CMakeFiles/Bro-TCPRS.linux-x86_64.dir/src/TCPRS_Endpoint.cc.o] Error 1
>
>
> On 04 Feb 2016, at 19:11, Vito Logrillo <vitologrillo at gmail.com> wrote:
>
> Hi all,
> i'm trying to compile the tcprs plugin with the latest bro version
> (2.4-267): using the make command i have this error
>
> /tcprs/src/TCPRS_Endpoint.cc: In member function
> ?analyzer::tcp::SCORE*
>
> analyzer::tcp::TCPRS_Endpoint::scoreRetransmission(analyzer::tcp::SequenceRange*,
> analyzer::tcp::Segment*, analyzer::tcp::RETRANSMISSION_REASON_CODE,
> HashKey*)?:
> /home/aramis/bro/aux/plugins/tcprs/src/TCPRS_Endpoint.cc:1180:47:
> error: call of overloaded ?abs(unsigned int)? is ambiguous
>      confidence_value *= 1.0 / (abs(i - TRIPLE) + 1.0); //ack distancing
>                                               ^
>
> what's wrong?
> Thanks
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160208/e900e551/attachment.html 

From hacecky at jlab.org  Tue Feb  9 08:06:14 2016
From: hacecky at jlab.org (Eric Hacecky)
Date: Tue, 9 Feb 2016 11:06:14 -0500 (EST)
Subject: [Bro] Basic Alerts/Email questions
Message-ID: <1908147847.2902285.1455033974906.JavaMail.zimbra@jlab.org>

I've been working with Bro for about a week focused on IDS/IPS functionality.

I'm starting small and took this snip of code from someone else asking how to get email alerts and put it in my local.bro

hook Notice::policy(n: Notice::Info) &priority=0
        { 
        add n$actions[Notice::ACTION_EMAIL]; 
        } 

I went through some documentation here:

https://www.bro.org/sphinx-git/scripts/base/frameworks/notice/main.bro.html

specifically the section labeled ?Notice::Type? with ~40 different types listed starting with Notice::Tally.

This seems to be what is now emailed, although there are very few email notices being generated, and only from a few of the categories.  Weird::Activitiy and Scan::Port_Scan

I also saw code like this somewhere

redef Notice::emailed_types += { 
#  FTP::Bruteforcing, 
  FTP::Site_Exec_Success, 
  HTTP::SQL_Injection_Attacker, 
  HTTP::SQL_Injection_Victim, 
#  SMTP::Blocklist_Error_Message, 
#  SMTP::Blocklist_Blocked_Host, 
#  SMTP::Suspicious_Origination, 
  SSH::Password_Guessing, 
  SSH::Login_By_Password_Guesser, 
  TeamCymruMalwareHashRegistry::Match, 
  Intel::Notice,
  Intel::DOMAIN,
  Intel::CERT_HASH,
  Intel::FILE_HASH,
 };

which seems to correlate to this documentation

https://www.bro.org/sphinx/bro-noticeindex.html

So I also threw that code into my local.bro

It doesn't seem to do anything.  Is there a way I can check?  Is it redundant with the hook code above to send an email for any notice?

===========

Next question

The modules from the previous snip I have commented out from give errors, example:

[BroControl] > check 
bro scripts failed. 
error in /usr/local/bro/share/bro/site/local.bro, line 100: unknown identifier FTP::Bruteforcing, at or near "FTP::Bruteforcing"

Ok.  I try to see why FTP::Bruteforcing errors while FTP::Site_Exec_Success doesn't.

This script seems to correspond to FTP::Bruteforcing

/usr/local/bro/share/bro/policy/protocols/ftp/detect-bruteforcing.bro

While this script corresponds to FTP::Site_Exec_Success

/usr/local/bro/share/bro/policy/protocols/ftp/detect.bro

Everything looks fine there to me....so why does FTP::Bruteforcing error but FTP::Site_Exec_Success not?

============

Finally, like I said my email alerts are VERY sparse.  After about a week I have the following:

Weird:Activity ? I have 25 SYN_after_partial alerts.  Not particularly useful

Scan::Port_Scan - 3 alerts.  Substantially less than are actually occurring.

Aside from that I have 1 SQL injection alert from Bro.

Meanwhile I have 100s of SQLi alerts registered in snort.

I check conn.log in bro and it's seeing the sessions that snort alerts on.

I looked in /http/detect-sqli and it appears that it's just a regex.  So the regex doesn't match 90+% of the sqli attacks seen on my network?

Thanks in advance for any help,
Eric










From sxz1069 at gmail.com  Tue Feb  9 09:51:10 2016
From: sxz1069 at gmail.com (xiaozhe shao)
Date: Tue, 9 Feb 2016 12:51:10 -0500
Subject: [Bro] Cluster Bro configuration
Message-ID: <4B40D0F6-3BA7-44F4-8054-3E829B1B699D@gmail.com>

Hi all,

	I?m a newbie of Bro. I?m trying to configure a cluster bro using PF_RING. But, I don?t have PF_RING+DNA license. I just want to use two bro instances (workers) running at the same physical machine. Whether PF_RING can load balance the traffic between two workers? Should I config the worker in node.cfg file as following:

[worker-1]
type=worker
host=10.0.0.50
interface=eth4
lb_method=pf_ring
lb_procs=2
[worker-2]
type=worker
host=10.0.0.50
interface=eth4
lb_method=pf_ring
lb_procs=2

Best,
-xiaozhe

From jazoff at illinois.edu  Tue Feb  9 09:58:17 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 9 Feb 2016 17:58:17 +0000
Subject: [Bro] Cluster Bro configuration
In-Reply-To: <4B40D0F6-3BA7-44F4-8054-3E829B1B699D@gmail.com>
References: <4B40D0F6-3BA7-44F4-8054-3E829B1B699D@gmail.com>
Message-ID: <EA7DD2E5-223F-4ED0-A340-9202773BAFB8@illinois.edu>

Hi,

You simply need

[worker]
type=worker
host=10.0.0.50
interface=eth4
lb_method=pf_ring
lb_procs=2


the lb_procs=2 will create the 2 workers for you.


-- 
- Justin Azoff

> On Feb 9, 2016, at 12:51 PM, xiaozhe shao <sxz1069 at gmail.com> wrote:
> 
> Hi all,
> 
> 	I?m a newbie of Bro. I?m trying to configure a cluster bro using PF_RING. But, I don?t have PF_RING+DNA license. I just want to use two bro instances (workers) running at the same physical machine. Whether PF_RING can load balance the traffic between two workers? Should I config the worker in node.cfg file as following:
> 
> [worker-1]
> type=worker
> host=10.0.0.50
> interface=eth4
> lb_method=pf_ring
> lb_procs=2
> [worker-2]
> type=worker
> host=10.0.0.50
> interface=eth4
> lb_method=pf_ring
> lb_procs=2
> 
> Best,
> -xiaozhe
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From sxz1069 at gmail.com  Tue Feb  9 13:01:12 2016
From: sxz1069 at gmail.com (xiaozhe shao)
Date: Tue, 9 Feb 2016 16:01:12 -0500
Subject: [Bro] Cluster Bro configuration
In-Reply-To: <EA7DD2E5-223F-4ED0-A340-9202773BAFB8@illinois.edu>
References: <4B40D0F6-3BA7-44F4-8054-3E829B1B699D@gmail.com>
	<EA7DD2E5-223F-4ED0-A340-9202773BAFB8@illinois.edu>
Message-ID: <4F61F0A9-DEBB-4729-AE80-73CB84DCBC18@gmail.com>

Thank you very much for your reply. 
Additionally, I want to know who splits the traffic into two parts. PF_RING or Bro worker? Is there any introduction (maybe some webpage) about this functionality?

Best,
-xiaozhe

> On Feb 9, 2016, at 12:58 PM, Azoff, Justin S <jazoff at illinois.edu> wrote:
> 
> Hi,
> 
> You simply need
> 
> [worker]
> type=worker
> host=10.0.0.50
> interface=eth4
> lb_method=pf_ring
> lb_procs=2
> 
> 
> the lb_procs=2 will create the 2 workers for you.
> 
> 
> -- 
> - Justin Azoff
> 
>> On Feb 9, 2016, at 12:51 PM, xiaozhe shao <sxz1069 at gmail.com> wrote:
>> 
>> Hi all,
>> 
>> 	I?m a newbie of Bro. I?m trying to configure a cluster bro using PF_RING. But, I don?t have PF_RING+DNA license. I just want to use two bro instances (workers) running at the same physical machine. Whether PF_RING can load balance the traffic between two workers? Should I config the worker in node.cfg file as following:
>> 
>> [worker-1]
>> type=worker
>> host=10.0.0.50
>> interface=eth4
>> lb_method=pf_ring
>> lb_procs=2
>> [worker-2]
>> type=worker
>> host=10.0.0.50
>> interface=eth4
>> lb_method=pf_ring
>> lb_procs=2
>> 
>> Best,
>> -xiaozhe
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 



From matiasdavaro at gmail.com  Tue Feb  9 16:34:01 2016
From: matiasdavaro at gmail.com (Matias Davaro)
Date: Tue, 9 Feb 2016 19:34:01 -0500
Subject: [Bro] Receiving no rule to make target error
Message-ID: <CAAHbCpFLfaez0-9cmkGOCTcgySQm5KZfSdhUKP2kjEEbJW7WQA@mail.gmail.com>

Hello,

I recently installed bro and have been trying to write a simple bro script
with binpac_analyzer. After generating the necessary files with binpac and
writing the small script I try to make the file and I receive the following
error:

~/bro$ make
make -C build all
make[1]: Entering directory `/home/elcabezon/bro/build'
make[2]: Entering directory `/home/elcabezon/bro/build'
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  4%] Built target broker
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  4%] Built target test_data
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  4%] Built target test_data_c
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  4%] Built target test_event
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  5%] Built target test_log
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  5%] Built target test_network_types
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  5%] Built target test_network_types_c
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  5%] Built target test_optional
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  5%] Built target test_print
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  5%] Built target test_print_c
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  5%] Built target test_print_remote
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  5%] Built target test_print_remote_c
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  5%] Built target test_radix_tree
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  6%] Built target test_store_backend
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  6%] Built target test_store_c
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  6%] Built target test_store_clone
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  6%] Built target test_store_expiry
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  6%] Built target test_store_frontend
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  6%] Built target test_store_master
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  6%] Built target test_store_remote
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  6%] Built target test_subscription_matching
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  7%] Built target test_variant
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  8%] Built target bifcl
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
[  8%] Built target bif-alt-broker-store.bif
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: Leaving directory `/home/elcabezon/bro/build'
make[3]: Entering directory `/home/elcabezon/bro/build'
make[3]: *** No rule to make target
`../src/analyzer/protocol/dce-rpc/binpac', needed by
`src/analyzer/protocol/dce-rpc/dce_rpc_pac.h'.  Stop.
make[3]: Leaving directory `/home/elcabezon/bro/build'
make[2]: ***
[src/analyzer/protocol/dce-rpc/CMakeFiles/pac-analyzer-protocol-dce-rpc-dce_rpc.pac.dir/all]
Error 2
make[2]: Leaving directory `/home/elcabezon/bro/build'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/elcabezon/bro/build'
make: *** [all] Error 2


Now I know sometimes this error appears when there is no source file
however I still I don't know why I am receiving an error for the dce-rpc
protocol when I have not tampered with it at all. I am on ubuntu 14.04. Any
help I could receive would be greatly appreciated. Thank you.

Matias
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160209/0a648d22/attachment.html 

From Blake.Mackey at rmc.ca  Tue Feb  9 18:33:38 2016
From: Blake.Mackey at rmc.ca (Blake Mackey)
Date: Wed, 10 Feb 2016 02:33:38 +0000
Subject: [Bro] File Extraction wierdness
Message-ID: <0355392EA7FA984BA70120F1A0D40AEE3CA1A32E@win55.rmc-cmr.rmc.ca>

I am having issues getting garbled file extractions from both live interfaces and traces.
Smaller files appear unaffected, but the larger the file, the greater chance of it being extracted incorrectly with Bro.

Is this normal behaviour? Or is Bro relatively bulletproof when it comes to file extraction?

Steps taken already:
Viewing wierd and notice logs, nothing stands out as abnormal.
Disabled all offloading of the NIC. No change.
Running a frameworks/files/extract-all-files.bro by itself. No change.
Running the packet loss script to determine if packets are being lost. 0.0% packet loss detected.

Could anyone suggest alternative things I can try to resolve this?

Thanks in advance!


Blake Mackey, CD
SLt | ens 1
Royal Military College of Canada | coll?ge militaire royal du Canada
(613)331-6438

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160210/1dbc3904/attachment.html 

From seth at icir.org  Tue Feb  9 20:09:13 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 9 Feb 2016 23:09:13 -0500
Subject: [Bro] File Extraction wierdness
In-Reply-To: <0355392EA7FA984BA70120F1A0D40AEE3CA1A32E@win55.rmc-cmr.rmc.ca>
References: <0355392EA7FA984BA70120F1A0D40AEE3CA1A32E@win55.rmc-cmr.rmc.ca>
Message-ID: <8CB7D61B-D60F-4581-AEC5-996CA9119B56@icir.org>


> On Feb 9, 2016, at 9:33 PM, Blake Mackey <Blake.Mackey at rmc.ca> wrote:
> 
> Is this normal behaviour? Or is Bro relatively bulletproof when it comes to file extraction?

Yes, it's generally functional.

> Could anyone suggest alternative things I can try to resolve this?

Could you capture some traffic that is giving you trouble and send it to me (offlist)?  It sounds to me like you're having packet loss issues, but I can't be sure without seeing the raw traffic.

Thanks
  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Tue Feb  9 20:30:23 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 9 Feb 2016 23:30:23 -0500
Subject: [Bro] Basic Alerts/Email questions
In-Reply-To: <1908147847.2902285.1455033974906.JavaMail.zimbra@jlab.org>
References: <1908147847.2902285.1455033974906.JavaMail.zimbra@jlab.org>
Message-ID: <D1A09BE2-F3F3-4963-9EDB-574ACAB38316@icir.org>


> On Feb 9, 2016, at 11:06 AM, Eric Hacecky <hacecky at jlab.org> wrote:
> 
> hook Notice::policy(n: Notice::Info) &priority=0
>        { 
>        add n$actions[Notice::ACTION_EMAIL]; 
>        } 

I don't think you want to do this.  It will result in all notices being emailed.

> It doesn't seem to do anything.  Is there a way I can check?  Is it redundant with the hook code above to send an email for any notice?

Notice::emailed_types is a pre-implemented mechanism if you only need coarse grained decisions about what to email.  You can see how it's implemented in base/notice/main.bro.  You can either easily use that or define your own Notice::policy hook to implement any other more complicated handling that you might want.

> /usr/local/bro/share/bro/policy/protocols/ftp/detect.bro
> 
> Everything looks fine there to me....so why does FTP::Bruteforcing error but FTP::Site_Exec_Success not?

Are you loading the policy/ftp/detect-bruteforcing.bro script?  If you aren't loading the script but you are trying to access identifiers defined in the script it won't work.

> Weird:Activity ? I have 25 SYN_after_partial alerts.  Not particularly useful

I wouldn't typically put any huge attention to Weird logs or notices.  They're helpful but typically not in day to day activity.  

> Scan::Port_Scan - 3 alerts.  Substantially less than are actually occurring.

Measured how?

> Aside from that I have 1 SQL injection alert from Bro.
> Meanwhile I have 100s of SQLi alerts registered in snort.

I would pay attention to that one from Bro.  It's likely higher value to you than the 100's from Snort.

> I looked in /http/detect-sqli and it appears that it's just a regex.  So the regex doesn't match 90+% of the sqli attacks seen on my network?

Bro is doing SQL injection detection based on a larger number of attacks either coming from an attacker or going to a victim.  It was originally written to find SQL injection based data extraction (and has worked phenomenally well for that at a number of large sites).  You can see individual requests that appear to be HTTP SQL injection requests in the "tags" field in the http.log.  If you grep your http.log for URI_SQLI you will see the individual requests.  

You seem to be approaching Bro from the perspective that it's a different version of Snort.  Please try to let go of thinking about network monitoring and intrusion detection with the mindset of Snort where one signature generates one notice for a sequence of bytes.  Spend a long time digging through the logs that it's generating, it's likely that you'll get a lot of pleasant surprises and you will learn a lot about your network that you didn't already know.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From tgdesrochers at gmail.com  Wed Feb 10 05:11:49 2016
From: tgdesrochers at gmail.com (Tim Desrochers)
Date: Wed, 10 Feb 2016 08:11:49 -0500
Subject: [Bro] [bro] FTP User Name
Message-ID: <CANrcqc0hBvG6aAKJ5dp+C9kfeURjUivG4=gQdmiGjqsP82crEg@mail.gmail.com>

Where does the username from FTP logs get derived from?

I have a use case where I see FTP traffic to a destination but my AD is
reporting the user originating the traffic as one name but the user field
of the FTP log shows a different name.

Why would this be?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160210/29878159/attachment.html 

From hacecky at jlab.org  Wed Feb 10 07:33:35 2016
From: hacecky at jlab.org (Eric Hacecky)
Date: Wed, 10 Feb 2016 10:33:35 -0500 (EST)
Subject: [Bro] Basic Alerts/Email questions
In-Reply-To: <1709353096.3074885.1455118365709.JavaMail.zimbra@jlab.org>
References: <1908147847.2902285.1455033974906.JavaMail.zimbra@jlab.org>
	<D1A09BE2-F3F3-4963-9EDB-574ACAB38316@icir.org>
Message-ID: <72540064.3075012.1455118415504.JavaMail.zimbra@jlab.org>

Hi Seth,

Thanks for the reply.


>> Aside from that I have 1 SQL injection alert from Bro.
>> Meanwhile I have 100s of SQLi alerts registered in snort.

- Found the problem here.

from detect-sqli.bro  const sqli_requests_threshold: double = 50.0 &redef;

50 is just too high for my environment as the attacks get shut down before they reach that threshold.  I redefined it to a lower value.

I'll skip the rest of the previous email and focus on the real message here.


>You seem to be approaching Bro from the perspective that it's a different version of Snort.  Please try to let go of thinking about network monitoring and intrusion detection with the mindset of Snort where one >signature generates one notice for a sequence of bytes.  Spend a long time digging through the logs that it's generating, it's likely that you'll get a lot of pleasant surprises and you will learn a lot about >your network that you didn't already know.

You are 100% correct.  I could list a number of different articles I've read recently, some of them by you in fact, that convey the same sentiments.

I'm already a believer and will continue to use Bro, but for management who will never directly interact with it, I need Bro to be meaningful to them.  Whether that's Snort-like alerts, metrics, heuristic data, etc.  I just went with alerts because it seemed like the easiest to get going straight away.

I've been combing through github.com/trending/bro for some scripts that I can add.  I've done as many training exercises as I can find, including the ones offered on bro.org as well as some from other sites, including ones that deal with logging like you mention https://www.bro.org/bro-workshop-2011/solutions/logs/

So that's where I am.  I will certainly continue working with the logs.  In addition to that if you or anyone else have ideas on where I should be focusing my time for a new Bro install please let me know.

Thanks,
Eric

From vladg at illinois.edu  Wed Feb 10 07:50:24 2016
From: vladg at illinois.edu (Vlad Grigorescu)
Date: Wed, 10 Feb 2016 09:50:24 -0600
Subject: [Bro] [bro] FTP User Name
In-Reply-To: <CANrcqc0hBvG6aAKJ5dp+C9kfeURjUivG4=gQdmiGjqsP82crEg@mail.gmail.com>
References: <CANrcqc0hBvG6aAKJ5dp+C9kfeURjUivG4=gQdmiGjqsP82crEg@mail.gmail.com>
Message-ID: <m060xwk1j3.fsf@vladg.net>

From the USER command. See:
https://github.com/bro/bro/blob/master/scripts/base/protocols/ftp/main.bro#L169

> 	if ( command == "USER" )
>		c$ftp$user = arg;

It's possible that the analyzer has a bug in it - if you could share
some more details or ideally a PCAP, we can look at getting it fixed.

Thanks,

  --Vlad

Tim Desrochers <tgdesrochers at gmail.com> writes:

> Where does the username from FTP logs get derived from?
>
> I have a use case where I see FTP traffic to a destination but my AD is
> reporting the user originating the traffic as one name but the user field
> of the FTP log shows a different name.
>
> Why would this be?
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 800 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160210/326678ec/attachment.bin 

From tgdesrochers at gmail.com  Wed Feb 10 07:58:50 2016
From: tgdesrochers at gmail.com (Tim Desrochers)
Date: Wed, 10 Feb 2016 10:58:50 -0500
Subject: [Bro] [bro] FTP User Name
In-Reply-To: <m060xwk1j3.fsf@vladg.net>
References: <CANrcqc0hBvG6aAKJ5dp+C9kfeURjUivG4=gQdmiGjqsP82crEg@mail.gmail.com>
	<m060xwk1j3.fsf@vladg.net>
Message-ID: <CANrcqc1RYjqyg3Rgd+_kzYc-zQ4HkCwZOktt42NMrGWgr+VO6g@mail.gmail.com>

Unfortunately I cannot share any pcap due to the network the device is on.
I can share that we believe FTP account accessed is in the name of the
"USER" field recorded by bro but the AD user who uploaded to item is a
different user.

So I guess my question should be, does bro pull the name from the the FTP
session or does it try to pull info from something like the devices log to
determine the user of the IP address who uploaded the file?

On Wed, Feb 10, 2016 at 10:50 AM, Vlad Grigorescu <vladg at illinois.edu>
wrote:

> From the USER command. See:
>
> https://github.com/bro/bro/blob/master/scripts/base/protocols/ftp/main.bro#L169
>
> >       if ( command == "USER" )
> >               c$ftp$user = arg;
>
> It's possible that the analyzer has a bug in it - if you could share
> some more details or ideally a PCAP, we can look at getting it fixed.
>
> Thanks,
>
>   --Vlad
>
> Tim Desrochers <tgdesrochers at gmail.com> writes:
>
> > Where does the username from FTP logs get derived from?
> >
> > I have a use case where I see FTP traffic to a destination but my AD is
> > reporting the user originating the traffic as one name but the user field
> > of the FTP log shows a different name.
> >
> > Why would this be?
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160210/c18dac92/attachment.html 

From seth at icir.org  Wed Feb 10 08:07:48 2016
From: seth at icir.org (Seth Hall)
Date: Wed, 10 Feb 2016 11:07:48 -0500
Subject: [Bro] File Extraction wierdness
In-Reply-To: <8CB7D61B-D60F-4581-AEC5-996CA9119B56@icir.org>
References: <0355392EA7FA984BA70120F1A0D40AEE3CA1A32E@win55.rmc-cmr.rmc.ca>
	<8CB7D61B-D60F-4581-AEC5-996CA9119B56@icir.org>
Message-ID: <478A775A-A745-4212-9464-964213F71454@icir.org>


> On Feb 9, 2016, at 11:09 PM, Seth Hall <seth at icir.org> wrote:
> 
> Could you capture some traffic that is giving you trouble and send it to me (offlist)?  It sounds to me like you're having packet loss issues, but I can't be sure without seeing the raw traffic.

Thanks for the data, I definitely see that it didn't extract correctly for you.  If I take the raw traffic and run Bro (git master) on it it extracts the file just fine.  What version of Bro are you running and what exactly is the command line you are running?  I'll show you what I ran...

bro -r bro.trace frameworks/files/extract-all-files

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Wed Feb 10 08:16:30 2016
From: seth at icir.org (Seth Hall)
Date: Wed, 10 Feb 2016 11:16:30 -0500
Subject: [Bro] Basic Alerts/Email questions
In-Reply-To: <72540064.3075012.1455118415504.JavaMail.zimbra@jlab.org>
References: <1908147847.2902285.1455033974906.JavaMail.zimbra@jlab.org>
	<D1A09BE2-F3F3-4963-9EDB-574ACAB38316@icir.org>
	<72540064.3075012.1455118415504.JavaMail.zimbra@jlab.org>
Message-ID: <AF7B7722-0238-44DA-8ABB-CF3A2BB23F35@icir.org>


> On Feb 10, 2016, at 10:33 AM, Eric Hacecky <hacecky at jlab.org> wrote:
> 
> 50 is just too high for my environment as the attacks get shut down before they reach that threshold.  I redefined it to a lower value.

You have something in place already actively watching for attacks and shutting off attackers?

> I'm already a believer and will continue to use Bro, but for management who will never directly interact with it, I need Bro to be meaningful to them.  Whether that's Snort-like alerts, metrics, heuristic data, etc.  I just went with alerts because it seemed like the easiest to get going straight away.

This is certainly where things get complicated because what management should be watching for is an engaged and vigilant incident response team.  If that team has tools that cause them to be more engaged and more vigilant then that's great.  Unfortunately most of the graphs that management will want to see won't actually reflect the reality of the activity for the incident responders and incident finders/hunters which most of the industry has split out from incident response at this point (which I still think is unfortunate).

> I've been combing through github.com/trending/bro for some scripts that I can add.  I've done as many training exercises as I can find, including the ones offered on bro.org as well as some from other sites, including ones that deal with logging like you mention https://www.bro.org/bro-workshop-2011/solutions/logs/

A fun one to load in case you haven't noticed it yet is the one that catalogs touches to Microsoft's Dr. Watson service.  It will log hardware getting attached to system and process crashes among some other things.
	https://github.com/broala/bro-drwatson

Make sure you follow the directions and clone that repository recursively because there is another repository that it needs to pull in for hardware information.

> So that's where I am.  I will certainly continue working with the logs.  In addition to that if you or anyone else have ideas on where I should be focusing my time for a new Bro install please let me know.

Great!  Definitely let us know if you have questions.  I know there are a lot of logs and it can take a long time to fully grok them all.  

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Wed Feb 10 08:19:52 2016
From: seth at icir.org (Seth Hall)
Date: Wed, 10 Feb 2016 11:19:52 -0500
Subject: [Bro] [bro] FTP User Name
In-Reply-To: <CANrcqc1RYjqyg3Rgd+_kzYc-zQ4HkCwZOktt42NMrGWgr+VO6g@mail.gmail.com>
References: <CANrcqc0hBvG6aAKJ5dp+C9kfeURjUivG4=gQdmiGjqsP82crEg@mail.gmail.com>
	<m060xwk1j3.fsf@vladg.net>
	<CANrcqc1RYjqyg3Rgd+_kzYc-zQ4HkCwZOktt42NMrGWgr+VO6g@mail.gmail.com>
Message-ID: <29313614-DCD0-41D0-8626-090705C0029F@icir.org>


> On Feb 10, 2016, at 10:58 AM, Tim Desrochers <tgdesrochers at gmail.com> wrote:
> 
> So I guess my question should be, does bro pull the name from the the FTP session or does it try to pull info from something like the devices log to determine the user of the IP address who uploaded the file?

>From the FTP session as Vlad indicated.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From tgdesrochers at gmail.com  Wed Feb 10 09:03:50 2016
From: tgdesrochers at gmail.com (Tim Desrochers)
Date: Wed, 10 Feb 2016 12:03:50 -0500
Subject: [Bro] [bro] FTP User Name
In-Reply-To: <29313614-DCD0-41D0-8626-090705C0029F@icir.org>
References: <CANrcqc0hBvG6aAKJ5dp+C9kfeURjUivG4=gQdmiGjqsP82crEg@mail.gmail.com>
	<m060xwk1j3.fsf@vladg.net>
	<CANrcqc1RYjqyg3Rgd+_kzYc-zQ4HkCwZOktt42NMrGWgr+VO6g@mail.gmail.com>
	<29313614-DCD0-41D0-8626-090705C0029F@icir.org>
Message-ID: <CANrcqc3Caa0KMB20jOib_FpOgf669vZQkC1DTm2hdprGEGH93w@mail.gmail.com>

I understand now. I am checking kerberos logs now to determine the user.

Thank you for responses.
On Feb 10, 2016 11:19 AM, "Seth Hall" <seth at icir.org> wrote:

>
> > On Feb 10, 2016, at 10:58 AM, Tim Desrochers <tgdesrochers at gmail.com>
> wrote:
> >
> > So I guess my question should be, does bro pull the name from the the
> FTP session or does it try to pull info from something like the devices log
> to determine the user of the IP address who uploaded the file?
>
> From the FTP session as Vlad indicated.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160210/d8b85893/attachment-0001.html 

From Blake.Mackey at rmc.ca  Wed Feb 10 09:36:53 2016
From: Blake.Mackey at rmc.ca (Blake Mackey)
Date: Wed, 10 Feb 2016 17:36:53 +0000
Subject: [Bro]  File Extraction wierdness
Message-ID: <0355392EA7FA984BA70120F1A0D40AEE3CA1B8C2@win55.rmc-cmr.rmc.ca>

> Thanks for the data, I definitely see that it didn't extract correctly for you.  If I take the raw traffic and run Bro (git master) on it it extracts the file just fine.  What version of Bro are you running and what exactly is the command line you are running?  I'll show you what I ran...
>
> bro -r bro.trace frameworks/files/extract-all-files

I am building my bro recursively from Git master as well, and using the same arguments as you.
For some reason I now extract only a single file using Bro (no longer any corrupted ones)... but there are over 30 files in that trace that are able to be extracted with Wireshark.

Is the format in this trace somehow preventing proper reassembly with Bro?


Blake Mackey, CD
SLt | ens 1
Royal Military College of Canada | coll?ge militaire royal du Canada
(613)331-6438





From seth at icir.org  Wed Feb 10 10:16:48 2016
From: seth at icir.org (Seth Hall)
Date: Wed, 10 Feb 2016 13:16:48 -0500
Subject: [Bro] File Extraction wierdness
In-Reply-To: <0355392EA7FA984BA70120F1A0D40AEE3CA1B8C2@win55.rmc-cmr.rmc.ca>
References: <0355392EA7FA984BA70120F1A0D40AEE3CA1B8C2@win55.rmc-cmr.rmc.ca>
Message-ID: <C886F951-6CB7-4DFB-B5BB-D5500E87110A@icir.org>


> On Feb 10, 2016, at 12:36 PM, Blake Mackey <Blake.Mackey at rmc.ca> wrote:
> 
> Is the format in this trace somehow preventing proper reassembly with Bro?

There are a number of things that could be causing the trouble.  You can send me some traffic off list again if you'd like me to take a look at it.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From shawn.homan at gmail.com  Wed Feb 10 13:55:02 2016
From: shawn.homan at gmail.com (Shawn Homan)
Date: Wed, 10 Feb 2016 16:55:02 -0500
Subject: [Bro] SHA256 Hash File Analyzer
Message-ID: <CACiJwL+Pzv0vA9pvsn7qj_5WT9v+udA9nVUmXQMz5yA7wX6BHQ@mail.gmail.com>

I was wondering if anyone can tell me why the sha256 hash functionality
isn't turned on by default for the files log.

I am working on something and needed to turn it on. I normally only use Bro
to process pcap files offline and have never used it on a live network.

Does it cause performance issues?

Thanks,

Shawn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160210/d251b7b9/attachment.html 

From grahambridgeland at yahoo.co.uk  Thu Feb 11 01:53:02 2016
From: grahambridgeland at yahoo.co.uk (Graham Bridgeland)
Date: Thu, 11 Feb 2016 09:53:02 +0000 (UTC)
Subject: [Bro] Framework Script Path Error
References: <25992675.4154582.1455184382370.JavaMail.yahoo.ref@mail.yahoo.com>
Message-ID: <25992675.4154582.1455184382370.JavaMail.yahoo@mail.yahoo.com>

Hello
Hoping someone could explain why when I'm using the sumstats-countconns.bro script provided on the?Summary Statistics page that I get an error from the first @load statement saying it can't find base/frameworks/sumstats.?
My $BROPATH looks ok as /usr/local/bro/share/bro is present. The permissions for the file are the same as the other .bro file.
I'm using a .pcap file rather than a .trace as indicated, not sure this matters.
Many thanks
Graham
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160211/e50d3234/attachment.html 

From jazoff at illinois.edu  Thu Feb 11 05:57:15 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Thu, 11 Feb 2016 13:57:15 +0000
Subject: [Bro] Framework Script Path Error
In-Reply-To: <25992675.4154582.1455184382370.JavaMail.yahoo@mail.yahoo.com>
References: <25992675.4154582.1455184382370.JavaMail.yahoo.ref@mail.yahoo.com>
	<25992675.4154582.1455184382370.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <6E6AD212-988B-4C0E-B9FE-12D34E352F66@illinois.edu>

Can you share the script you are running and the output you are getting?

You can paste it into http://try.bro.org/ to get a shareable link as well.


-- 
- Justin Azoff

> On Feb 11, 2016, at 4:53 AM, Graham Bridgeland <grahambridgeland at yahoo.co.uk> wrote:
> 
> Hello
> 
> Hoping someone could explain why when I'm using the sumstats-countconns.bro script provided on the Summary Statistics page that I get an error from the first @load statement saying it can't find base/frameworks/sumstats. 
> 
> My $BROPATH looks ok as /usr/local/bro/share/bro is present. The permissions for the file are the same as the other .bro file.
> 
> I'm using a .pcap file rather than a .trace as indicated, not sure this matters.
> 
> Many thanks
> 
> Graham
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From seth at icir.org  Thu Feb 11 07:40:50 2016
From: seth at icir.org (Seth Hall)
Date: Thu, 11 Feb 2016 10:40:50 -0500
Subject: [Bro] SHA256 Hash File Analyzer
In-Reply-To: <CACiJwL+Pzv0vA9pvsn7qj_5WT9v+udA9nVUmXQMz5yA7wX6BHQ@mail.gmail.com>
References: <CACiJwL+Pzv0vA9pvsn7qj_5WT9v+udA9nVUmXQMz5yA7wX6BHQ@mail.gmail.com>
Message-ID: <D1791FBE-9120-44DA-875D-1ED3E3F59197@icir.org>


> On Feb 10, 2016, at 4:55 PM, Shawn Homan <shawn.homan at gmail.com> wrote:
> 
> I was wondering if anyone can tell me why the sha256 hash functionality isn't turned on by default for the files log.
> 
> I am working on something and needed to turn it on. I normally only use Bro to process pcap files offline and have never used it on a live network. 
> 
> Does it cause performance issues?

When I was setting the default behavior a few years ago, I did some very weak testing and noticed that if I had md5 and sha1 turned on, the performance impact was ~1%, but it jumped up somewhere between 3-4% when I enabled SHA256.  That measurement should be revisited sometime soon though and perhaps even better measurements done to see if that performance impact is still there.

Generally though, there is nothing in place which is stopping you from enabling SHA256 file hashes.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From cbarbaro at cert.unlp.edu.ar  Thu Feb 11 10:53:07 2016
From: cbarbaro at cert.unlp.edu.ar (Cristian Daniel Barbaro)
Date: Thu, 11 Feb 2016 15:53:07 -0300
Subject: [Bro] Scan UDP
Message-ID: <56BCD893.9070705@cert.unlp.edu.ar>

Hi, Community.

i'm testing UDP ports scans with Nmap but Bro doesn't detect this scan
type.

Bro implements this scan type detect?

I'm using hooks (Scan::addr_scan_policy Scan::port_scan_policy) to
generate logs, with UDP the logs remain empty.
However, the UDP connections was stored in conn.log.

Thanks and sorry for my english.

-- 
Cristian Daniel Barbaro
CERT UNLP
--




From seth at icir.org  Thu Feb 11 12:58:33 2016
From: seth at icir.org (Seth Hall)
Date: Thu, 11 Feb 2016 15:58:33 -0500
Subject: [Bro] Scan UDP
In-Reply-To: <56BCD893.9070705@cert.unlp.edu.ar>
References: <56BCD893.9070705@cert.unlp.edu.ar>
Message-ID: <82CCEB61-C63B-49C8-8CDA-35DDB1D05B01@icir.org>


> On Feb 11, 2016, at 1:53 PM, Cristian Daniel Barbaro <cbarbaro at cert.unlp.edu.ar> wrote:
> 
> Bro implements this scan type detect?

There is a prototype script that we put together a while ago that detects UDP scans.  If you run it, I'd love to get any feedback that you have.
	https://github.com/sethhall/bro-junk-drawer/blob/master/scan_udp.bro

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From fmonsen at ucsc.edu  Thu Feb 11 14:48:14 2016
From: fmonsen at ucsc.edu (Forest Monsen)
Date: Thu, 11 Feb 2016 14:48:14 -0800
Subject: [Bro] Scan UDP
In-Reply-To: <82CCEB61-C63B-49C8-8CDA-35DDB1D05B01@icir.org>
References: <56BCD893.9070705@cert.unlp.edu.ar>
	<82CCEB61-C63B-49C8-8CDA-35DDB1D05B01@icir.org>
Message-ID: <56BD0FAE.2050709@ucsc.edu>

On 02/11/2016 12:58 PM, Seth Hall wrote:
> If you run it, I'd love to get any feedback that you have.

I like this script and am using it in production. The *nix traceroute
utility does trigger it, because it uses the sequential UDP port numbers
by default instead of ICMP... so I would like to exclude those.

Best,
Forest

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160211/ea431a92/attachment.bin 

From shawn.homan at gmail.com  Thu Feb 11 15:39:20 2016
From: shawn.homan at gmail.com (Shawn Homan)
Date: Thu, 11 Feb 2016 18:39:20 -0500
Subject: [Bro] SHA256 Hash File Analyzer
In-Reply-To: <D1791FBE-9120-44DA-875D-1ED3E3F59197@icir.org>
References: <CACiJwL+Pzv0vA9pvsn7qj_5WT9v+udA9nVUmXQMz5yA7wX6BHQ@mail.gmail.com>
	<D1791FBE-9120-44DA-875D-1ED3E3F59197@icir.org>
Message-ID: <CACiJwLLfqgmpEOEHZnSZOfuzbPw1aYngQ3YgTrrUi23GhTDyJQ@mail.gmail.com>

Thanks for the information. I have it turned on in my offline system, but
not sure how to measure performance.

On Thu, Feb 11, 2016 at 10:40 AM, Seth Hall <seth at icir.org> wrote:

>
> > On Feb 10, 2016, at 4:55 PM, Shawn Homan <shawn.homan at gmail.com> wrote:
> >
> > I was wondering if anyone can tell me why the sha256 hash functionality
> isn't turned on by default for the files log.
> >
> > I am working on something and needed to turn it on. I normally only use
> Bro to process pcap files offline and have never used it on a live network.
> >
> > Does it cause performance issues?
>
> When I was setting the default behavior a few years ago, I did some very
> weak testing and noticed that if I had md5 and sha1 turned on, the
> performance impact was ~1%, but it jumped up somewhere between 3-4% when I
> enabled SHA256.  That measurement should be revisited sometime soon though
> and perhaps even better measurements done to see if that performance impact
> is still there.
>
> Generally though, there is nothing in place which is stopping you from
> enabling SHA256 file hashes.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160211/95055aa1/attachment.html 

From tgdesrochers at gmail.com  Sun Feb 14 03:14:25 2016
From: tgdesrochers at gmail.com (Tim Desrochers)
Date: Sun, 14 Feb 2016 06:14:25 -0500
Subject: [Bro] [bro] Scanning IP's
Message-ID: <CANrcqc1eDODBxcfjAr6=MnHu3LrK23DJpwkkwJm8jBnJ5mKSgg@mail.gmail.com>

As with every infrastructure I am plagued with people scanning my external
edge.  I see little value in getting notices for scanning attempts and
password guessing attempts but I do see value in running monthly reports
and generating blocklists based on repeat offenders.

Is there a way to tell the notice framework to only create alarms (emails)
if it sees scans of any kind (address, port, password guessing, etc) if
they are from the IP's in my $HOME_NET defined in network.cfg?

Justification, If I

redef Notice::ignored_types += {
  SSH::Password_Guessing,
  Scan::Address_Scan,
  Scan::Port_Scan,
  HTTP::SQL_Injection_Attacker,
  ShellShock::Scanner,
  ScanUDP::Address_Scan,
  ScanUDP::Port_Scan,
};

Then I get no logging of the events anywhere.  Therefore I can't run
reports of offenders and build active blocklists or other intel gathering
activities.

If I:

# Set rule to only email specific notice types:
redef Notice::emailed_types += {
  Weird::Activity,
  Signatures::Sensitive_Signature,
  Signatures::Multiple_Signatures,
  Signatures::Multiple_Sig_Responders,
  Signatures::Count_Signature,
  Intel::Notice,
  TeamCymruMalwareHashRegistry::Match,
  Traceroute::Detected,
  FTP::Bruteforcing,
  FTP::Site_Exec_Success,
  HTTP::SQL_Injection_Victim,
  SMTP::Blocklist_Error_Message,
  SMTP::Blocklist_Blocked_Host,
  SMTP::Suspicious_Origination,
  SSH::Login_By_Password_Guesser,
  SSH::Interesting_Hostname_Login,
};

Then I get flooded with email from any of the guessing activity (Side note:
I find that the above logic doesn't restrict email notices to just those
listed in the defined email types above.  I still get plenty of notices
about events not listed in the list above).  If the redef
Notice::emailed_types worked it would be a start but I'd still like to get
emails about IP addresses in my internal net getting scanned by other IP's
in my internal net, that definitely an indicator of unwanted behavior.

Any assistance would be greatly appreciated.  Just trying to tune things to
a manageable level.

Thanks
Tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160214/870a4c41/attachment.html 

From jazoff at illinois.edu  Sun Feb 14 04:35:44 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Sun, 14 Feb 2016 12:35:44 +0000
Subject: [Bro] [bro] Scanning IP's
In-Reply-To: <CANrcqc1eDODBxcfjAr6=MnHu3LrK23DJpwkkwJm8jBnJ5mKSgg@mail.gmail.com>
References: <CANrcqc1eDODBxcfjAr6=MnHu3LrK23DJpwkkwJm8jBnJ5mKSgg@mail.gmail.com>
Message-ID: <BF0CCEB0-4CEC-43C3-BC8D-69BA562244D2@illinois.edu>

The thing to understand is that the ignored_types and emailed_types are just tables defined to make tweaking the base notice policy easier.

That default notice policy is:

hook Notice::policy(n: Notice::Info) &priority=10
        {
        if ( n$note in Notice::ignored_types )
                break;

        if ( n$note in Notice::not_suppressed_types )
                n$suppress_for=0secs;
        if ( n$note in Notice::alarmed_types )
                add n$actions[ACTION_ALARM];
        if ( n$note in Notice::emailed_types )
                add n$actions[ACTION_EMAIL];

        if ( n$note in Notice::type_suppression_intervals )
                n$suppress_for=Notice::type_suppression_intervals[n$note];

        # Logging is a default action.  It can be removed in a later hook if desired.
        add n$actions[ACTION_LOG];
        }

As you can see, adding notice types to those tables just tweaks the behavior of the default Notice::policy hook.  To do some of the things you want to do, you just need a hook like

hook Notice::policy(n: Notice::Info)
{
    if (n$note == Scan::Port_Scan && Site::is_local_addr(n$src))
        add n$actions[Notice::ACTION_EMAIL];
}

If that would get repetitive, you can create your own table like

const local_emailed_types: set[Notice::Type] = {} &redef; 

and have the policy be

hook Notice::policy(n: Notice::Info)
{
    if (n$note in local_emailed_types && Site::is_local_addr(n$src))
        add n$actions[Notice::ACTION_EMAIL];
}

-- 
- Justin Azoff

> On Feb 14, 2016, at 6:14 AM, Tim Desrochers <tgdesrochers at gmail.com> wrote:
> 
> As with every infrastructure I am plagued with people scanning my external edge.  I see little value in getting notices for scanning attempts and password guessing attempts but I do see value in running monthly reports and generating blocklists based on repeat offenders.  
> 
> Is there a way to tell the notice framework to only create alarms (emails) if it sees scans of any kind (address, port, password guessing, etc) if they are from the IP's in my $HOME_NET defined in network.cfg?
> 
> Justification, If I 
> 
> redef Notice::ignored_types += {
>   SSH::Password_Guessing,
>   Scan::Address_Scan,
>   Scan::Port_Scan,
>   HTTP::SQL_Injection_Attacker,
>   ShellShock::Scanner,
>   ScanUDP::Address_Scan,
>   ScanUDP::Port_Scan,
> };
> 
> Then I get no logging of the events anywhere.  Therefore I can't run reports of offenders and build active blocklists or other intel gathering activities.  
> 
> If I:
> 
> # Set rule to only email specific notice types:
> redef Notice::emailed_types += {
>   Weird::Activity,
>   Signatures::Sensitive_Signature,
>   Signatures::Multiple_Signatures,
>   Signatures::Multiple_Sig_Responders,
>   Signatures::Count_Signature,
>   Intel::Notice,
>   TeamCymruMalwareHashRegistry::Match,
>   Traceroute::Detected,
>   FTP::Bruteforcing,
>   FTP::Site_Exec_Success,
>   HTTP::SQL_Injection_Victim,
>   SMTP::Blocklist_Error_Message,
>   SMTP::Blocklist_Blocked_Host,
>   SMTP::Suspicious_Origination,
>   SSH::Login_By_Password_Guesser,
>   SSH::Interesting_Hostname_Login,
> };
> 
> Then I get flooded with email from any of the guessing activity (Side note: I find that the above logic doesn't restrict email notices to just those listed in the defined email types above.  I still get plenty of notices about events not listed in the list above).  If the redef Notice::emailed_types worked it would be a start but I'd still like to get emails about IP addresses in my internal net getting scanned by other IP's in my internal net, that definitely an indicator of unwanted behavior.  
> 
> Any assistance would be greatly appreciated.  Just trying to tune things to a manageable level.
> 
> Thanks
> Tim
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From tgdesrochers at gmail.com  Sun Feb 14 05:19:06 2016
From: tgdesrochers at gmail.com (Tim Desrochers)
Date: Sun, 14 Feb 2016 08:19:06 -0500
Subject: [Bro] [bro] Scanning IP's
In-Reply-To: <BF0CCEB0-4CEC-43C3-BC8D-69BA562244D2@illinois.edu>
References: <CANrcqc1eDODBxcfjAr6=MnHu3LrK23DJpwkkwJm8jBnJ5mKSgg@mail.gmail.com>
	<BF0CCEB0-4CEC-43C3-BC8D-69BA562244D2@illinois.edu>
Message-ID: <CANrcqc2FF=g1LB-87V7s7wVA4aOPfn8h99CtJTt=Zknvbbg-aA@mail.gmail.com>

FANTASTIC!!  Giving it a shot now

On Sun, Feb 14, 2016 at 7:35 AM, Azoff, Justin S <jazoff at illinois.edu>
wrote:

> The thing to understand is that the ignored_types and emailed_types are
> just tables defined to make tweaking the base notice policy easier.
>
> That default notice policy is:
>
> hook Notice::policy(n: Notice::Info) &priority=10
>         {
>         if ( n$note in Notice::ignored_types )
>                 break;
>
>         if ( n$note in Notice::not_suppressed_types )
>                 n$suppress_for=0secs;
>         if ( n$note in Notice::alarmed_types )
>                 add n$actions[ACTION_ALARM];
>         if ( n$note in Notice::emailed_types )
>                 add n$actions[ACTION_EMAIL];
>
>         if ( n$note in Notice::type_suppression_intervals )
>                 n$suppress_for=Notice::type_suppression_intervals[n$note];
>
>         # Logging is a default action.  It can be removed in a later hook
> if desired.
>         add n$actions[ACTION_LOG];
>         }
>
> As you can see, adding notice types to those tables just tweaks the
> behavior of the default Notice::policy hook.  To do some of the things you
> want to do, you just need a hook like
>
> hook Notice::policy(n: Notice::Info)
> {
>     if (n$note == Scan::Port_Scan && Site::is_local_addr(n$src))
>         add n$actions[Notice::ACTION_EMAIL];
> }
>
> If that would get repetitive, you can create your own table like
>
> const local_emailed_types: set[Notice::Type] = {} &redef;
>
> and have the policy be
>
> hook Notice::policy(n: Notice::Info)
> {
>     if (n$note in local_emailed_types && Site::is_local_addr(n$src))
>         add n$actions[Notice::ACTION_EMAIL];
> }
>
> --
> - Justin Azoff
>
> > On Feb 14, 2016, at 6:14 AM, Tim Desrochers <tgdesrochers at gmail.com>
> wrote:
> >
> > As with every infrastructure I am plagued with people scanning my
> external edge.  I see little value in getting notices for scanning attempts
> and password guessing attempts but I do see value in running monthly
> reports and generating blocklists based on repeat offenders.
> >
> > Is there a way to tell the notice framework to only create alarms
> (emails) if it sees scans of any kind (address, port, password guessing,
> etc) if they are from the IP's in my $HOME_NET defined in network.cfg?
> >
> > Justification, If I
> >
> > redef Notice::ignored_types += {
> >   SSH::Password_Guessing,
> >   Scan::Address_Scan,
> >   Scan::Port_Scan,
> >   HTTP::SQL_Injection_Attacker,
> >   ShellShock::Scanner,
> >   ScanUDP::Address_Scan,
> >   ScanUDP::Port_Scan,
> > };
> >
> > Then I get no logging of the events anywhere.  Therefore I can't run
> reports of offenders and build active blocklists or other intel gathering
> activities.
> >
> > If I:
> >
> > # Set rule to only email specific notice types:
> > redef Notice::emailed_types += {
> >   Weird::Activity,
> >   Signatures::Sensitive_Signature,
> >   Signatures::Multiple_Signatures,
> >   Signatures::Multiple_Sig_Responders,
> >   Signatures::Count_Signature,
> >   Intel::Notice,
> >   TeamCymruMalwareHashRegistry::Match,
> >   Traceroute::Detected,
> >   FTP::Bruteforcing,
> >   FTP::Site_Exec_Success,
> >   HTTP::SQL_Injection_Victim,
> >   SMTP::Blocklist_Error_Message,
> >   SMTP::Blocklist_Blocked_Host,
> >   SMTP::Suspicious_Origination,
> >   SSH::Login_By_Password_Guesser,
> >   SSH::Interesting_Hostname_Login,
> > };
> >
> > Then I get flooded with email from any of the guessing activity (Side
> note: I find that the above logic doesn't restrict email notices to just
> those listed in the defined email types above.  I still get plenty of
> notices about events not listed in the list above).  If the redef
> Notice::emailed_types worked it would be a start but I'd still like to get
> emails about IP addresses in my internal net getting scanned by other IP's
> in my internal net, that definitely an indicator of unwanted behavior.
> >
> > Any assistance would be greatly appreciated.  Just trying to tune things
> to a manageable level.
> >
> > Thanks
> > Tim
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160214/92aa0883/attachment.html 

From tgdesrochers at gmail.com  Sun Feb 14 05:42:21 2016
From: tgdesrochers at gmail.com (Tim Desrochers)
Date: Sun, 14 Feb 2016 08:42:21 -0500
Subject: [Bro] [bro] Scanning IP's
In-Reply-To: <BF0CCEB0-4CEC-43C3-BC8D-69BA562244D2@illinois.edu>
References: <CANrcqc1eDODBxcfjAr6=MnHu3LrK23DJpwkkwJm8jBnJ5mKSgg@mail.gmail.com>
	<BF0CCEB0-4CEC-43C3-BC8D-69BA562244D2@illinois.edu>
Message-ID: <CANrcqc19FBp4EVJgM6nm=wBjUMuckR46ufXx_OSo2xLbw1GDpg@mail.gmail.com>

Followup question:

If I set this will I still get the other notices emailed to me such as
items from the intel framework that I have set meta.do_notice and
meta.if_in.  Or will I have to make another notice hook to still allow for
those to send emails when observed.

Obviously I have some bro scripting classes to attend, but in the meanwhile
I am just trying to hack this together.

Tim

On Sun, Feb 14, 2016 at 7:35 AM, Azoff, Justin S <jazoff at illinois.edu>
wrote:

> The thing to understand is that the ignored_types and emailed_types are
> just tables defined to make tweaking the base notice policy easier.
>
> That default notice policy is:
>
> hook Notice::policy(n: Notice::Info) &priority=10
>         {
>         if ( n$note in Notice::ignored_types )
>                 break;
>
>         if ( n$note in Notice::not_suppressed_types )
>                 n$suppress_for=0secs;
>         if ( n$note in Notice::alarmed_types )
>                 add n$actions[ACTION_ALARM];
>         if ( n$note in Notice::emailed_types )
>                 add n$actions[ACTION_EMAIL];
>
>         if ( n$note in Notice::type_suppression_intervals )
>                 n$suppress_for=Notice::type_suppression_intervals[n$note];
>
>         # Logging is a default action.  It can be removed in a later hook
> if desired.
>         add n$actions[ACTION_LOG];
>         }
>
> As you can see, adding notice types to those tables just tweaks the
> behavior of the default Notice::policy hook.  To do some of the things you
> want to do, you just need a hook like
>
> hook Notice::policy(n: Notice::Info)
> {
>     if (n$note == Scan::Port_Scan && Site::is_local_addr(n$src))
>         add n$actions[Notice::ACTION_EMAIL];
> }
>
> If that would get repetitive, you can create your own table like
>
> const local_emailed_types: set[Notice::Type] = {} &redef;
>
> and have the policy be
>
> hook Notice::policy(n: Notice::Info)
> {
>     if (n$note in local_emailed_types && Site::is_local_addr(n$src))
>         add n$actions[Notice::ACTION_EMAIL];
> }
>
> --
> - Justin Azoff
>
> > On Feb 14, 2016, at 6:14 AM, Tim Desrochers <tgdesrochers at gmail.com>
> wrote:
> >
> > As with every infrastructure I am plagued with people scanning my
> external edge.  I see little value in getting notices for scanning attempts
> and password guessing attempts but I do see value in running monthly
> reports and generating blocklists based on repeat offenders.
> >
> > Is there a way to tell the notice framework to only create alarms
> (emails) if it sees scans of any kind (address, port, password guessing,
> etc) if they are from the IP's in my $HOME_NET defined in network.cfg?
> >
> > Justification, If I
> >
> > redef Notice::ignored_types += {
> >   SSH::Password_Guessing,
> >   Scan::Address_Scan,
> >   Scan::Port_Scan,
> >   HTTP::SQL_Injection_Attacker,
> >   ShellShock::Scanner,
> >   ScanUDP::Address_Scan,
> >   ScanUDP::Port_Scan,
> > };
> >
> > Then I get no logging of the events anywhere.  Therefore I can't run
> reports of offenders and build active blocklists or other intel gathering
> activities.
> >
> > If I:
> >
> > # Set rule to only email specific notice types:
> > redef Notice::emailed_types += {
> >   Weird::Activity,
> >   Signatures::Sensitive_Signature,
> >   Signatures::Multiple_Signatures,
> >   Signatures::Multiple_Sig_Responders,
> >   Signatures::Count_Signature,
> >   Intel::Notice,
> >   TeamCymruMalwareHashRegistry::Match,
> >   Traceroute::Detected,
> >   FTP::Bruteforcing,
> >   FTP::Site_Exec_Success,
> >   HTTP::SQL_Injection_Victim,
> >   SMTP::Blocklist_Error_Message,
> >   SMTP::Blocklist_Blocked_Host,
> >   SMTP::Suspicious_Origination,
> >   SSH::Login_By_Password_Guesser,
> >   SSH::Interesting_Hostname_Login,
> > };
> >
> > Then I get flooded with email from any of the guessing activity (Side
> note: I find that the above logic doesn't restrict email notices to just
> those listed in the defined email types above.  I still get plenty of
> notices about events not listed in the list above).  If the redef
> Notice::emailed_types worked it would be a start but I'd still like to get
> emails about IP addresses in my internal net getting scanned by other IP's
> in my internal net, that definitely an indicator of unwanted behavior.
> >
> > Any assistance would be greatly appreciated.  Just trying to tune things
> to a manageable level.
> >
> > Thanks
> > Tim
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160214/7d6a04bf/attachment-0001.html 

From peter.osterberg at hexbit.se  Mon Feb 15 07:29:13 2016
From: peter.osterberg at hexbit.se (=?UTF-8?Q?Peter_=c3=96sterberg?=)
Date: Mon, 15 Feb 2016 16:29:13 +0100
Subject: [Bro] TCP state change event, or other solution
Message-ID: <56C1EEC9.2020801@hexbit.se>

Hello

I am pretty new with BRO so I might have missed something. I have
however looked through the documentation and tried searching the mail
list for an answer before posting.

I'd like to log the payload of the first TCP packet in an established
session, without using the performance costly new_packet event.

I was hoping to an event that could trigger on an TCP state change, but
there don't seem to exist any such event.

I also want to do this for ICMP and UDP but that seems pretty straight
forward by combining the new_connection event and the get
get_current_packet function.

I did look at connection_established event but that one did never
trigger. I understand from the documentation that this event only
triggers if BRO sees packets that are established, but BRO missed the
handshake part.

I also looked at the Conn::log_conn event and the conn_state property,
but it doesn't seem that this one triggers for every packet.


My idea is to make some kind of list containing the connection's uid,
(add uid with the new_connection event for TCP packets only). On every
session iterate this list look for the first established packet for
sessions in the list. Save the contents and then remove the uid from the
list again. This should hopefully keep the list quite short enough to
not be to performance heavy.

The problem is that I don't find an event that seems appropriate to use
for this.

Are there better solutions to this? Any pointers are most welcome!

BR,
Peter



From nick at nickallen.org  Mon Feb 15 09:57:45 2016
From: nick at nickallen.org (Nick Allen)
Date: Mon, 15 Feb 2016 12:57:45 -0500
Subject: [Bro] Traversing 'Table' in C code of Bro Plugin
Message-ID: <CAHSJ8Nzs0EENkgZ-YzrHJ9kTyG=8w8gaeWUq5wd-vzbYhqmqfA@mail.gmail.com>

I am trying to create a plugin that allows a user to define a 'table'
of key/values that are all strings.  In my BIF file I have something
like the following:

  const config: table[string] of string;

It seems that a table in Bro maps to a TableType* in the C space of my plugin.

    TableType *config = BifConst::Example::config;

How do I go about iterating over the key/value pairs defined in
'config' in the C code of my plugin?  I am unable to find any good
examples of code doing that.

Many thanks in advance.

-- 
Nick Allen <nick at nickallen.org>

From johanna at icir.org  Mon Feb 15 11:03:14 2016
From: johanna at icir.org (Johanna Amann)
Date: Mon, 15 Feb 2016 11:03:14 -0800
Subject: [Bro] Traversing 'Table' in C code of Bro Plugin
In-Reply-To: <CAHSJ8Nzs0EENkgZ-YzrHJ9kTyG=8w8gaeWUq5wd-vzbYhqmqfA@mail.gmail.com>
References: <CAHSJ8Nzs0EENkgZ-YzrHJ9kTyG=8w8gaeWUq5wd-vzbYhqmqfA@mail.gmail.com>
Message-ID: <20160215190314.GA16114@Beezling.local>

Hi Nick,

https://github.com/bro/bro/blob/master/src/input/Manager.cc#L288-L306 is
the most succint example I could find after a quick look. You call
InitForIteration() and then get the individual TableEntryVal*s calling
NextEntry.

I hope this helps,
 Johanna

On Mon, Feb 15, 2016 at 12:57:45PM -0500, Nick Allen wrote:
> I am trying to create a plugin that allows a user to define a 'table'
> of key/values that are all strings.  In my BIF file I have something
> like the following:
> 
>   const config: table[string] of string;
> 
> It seems that a table in Bro maps to a TableType* in the C space of my plugin.
> 
>     TableType *config = BifConst::Example::config;
> 
> How do I go about iterating over the key/value pairs defined in
> 'config' in the C code of my plugin?  I am unable to find any good
> examples of code doing that.
> 
> Many thanks in advance.
> 
> -- 
> Nick Allen <nick at nickallen.org>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> 

From abhimantalwar.1992 at gmail.com  Tue Feb 16 01:51:39 2016
From: abhimantalwar.1992 at gmail.com (Abhiman talwar)
Date: Tue, 16 Feb 2016 15:21:39 +0530
Subject: [Bro] BRO:Logging all files
Message-ID: <CALcrXFzhQmmxThr5tBLYcX_x5czQ-Hx6vJ7a4Ntcq-VPkvLVqg@mail.gmail.com>

I want to log all the streams the BRO has to offer. Currently I am getting
only a given set BRO logs in default state. How can I get my way with
tweaking it so as to get all the log files possible?

Regards
Abhiman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160216/54f8054f/attachment.html 

From puntogtg at tiscali.it  Tue Feb 16 03:29:46 2016
From: puntogtg at tiscali.it (puntogtg at tiscali.it)
Date: Tue, 16 Feb 2016 12:29:46 +0100
Subject: [Bro] =?utf-8?q?_File_Extraction=3A_doc/xls=3Dok=2C_docx/xlsx=3Dk?=
	=?utf-8?q?o?=
Message-ID: <bfb1d702994a3494f781aca21ed81454@tiscali.it>

  

Hello,
I am trying to find out if I did some mistake in my
extract.bro script.
Basically I am able to extract the doc,pdf,xls but
not the docx,xlsx,xlsm etc (all new office files).
Script looks like
this:

global ext_map: table[string] of string = {

["application/msword"] = "doc",

["application/vnd.openxmlformats-officedocument.wordprocessingml.document"]
= "docx",

["application/vnd.openxmlformats-officedocument.wordprocessingml.template"]
= "dotx",
 ["application/vnd.ms-word.document.macroEnabled.12"] =
"docm",
 ["application/vnd.ms-word.template.macroEnabled.12"] = "dotm",

["application/vnd.ms-excel"] = "xls",

["application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"] =
"xlsx",

["application/vnd.openxmlformats-officedocument.spreadsheetml.template"]
= "xltx",
 ["application/vnd.ms-excel.sheet.macroEnabled.12"] = "xlsm",

["application/vnd.ms-excel.template.macroEnabled.12"] = "xltm",

["application/vnd.ms-excel.addin.macroEnabled.12"] = "xlam",

["application/vnd.ms-excel.sheet.binary.macroEnabled.12"] = "xlsb",

["application/vnd.ms-powerpoint"] = "ppt",

["application/vnd.openxmlformats-officedocument.presentationml.presentation"]
= "pptx",

["application/vnd.openxmlformats-officedocument.presentationml.template"]
= "potx",

["application/vnd.openxmlformats-officedocument.presentationml.slideshow"]
= "ppsx",
 ["application/vnd.ms-powerpoint.addin.macroEnabled.12"] =
"ppam",
 ["application/vnd.ms-powerpoint.presentation.macroEnabled.12"]
= "pptm",

["application/vnd.ms-powerpoint.presentation.macroEnabled.12"] =
"potm",
 ["application/vnd.ms-powerpoint.slideshow.macroEnabled.12"] =
"ppsm",
} &default ="";

event file_new(f: fa_file)
 {
 if ( !
f?$mime_type )
 return;
 local ext = "";
 if ( f?$mime_type )
 ext =
ext_map[f$mime_type];
 #if ( ext !="pdf" && ext !="exe" && ext !="swf"
)
 if ( ext !="doc" && ext !="docx" && ext !="dotx" && ext !="docm" &&
ext !="dotm" && ext !="xls" && ext !="xlsx" && ext !="xltx" && ext
!="xlsm" && ext !="xltm" && ext !="xlam" && ext !="xlsb" && ext !="ppt"
&& ext !="pptx" && ext !="potx" && ext !="ppsx" && ext !="ppam" && ext
!="pptm" && ext !="potm" && ext !="ppsm" )
 return;

 local fname =
fmt("/bro/extracted/%s-%s.%s", f$source, f$id, ext);

Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
[$extract_filename=fname]);
 break;
}

Into the files.log I can see when
extract matches:

1455104772.317535 FiWH8E2GK4LZmK8kYg 12.23.29.13
194.1.1.22 C9Ujyw1HodyV6hrs4f SMTP 5 DATA_EVENT,EXTRACT,SHA1,MD5
application/msword SPCH_100658601_1_Skillsupdatefebruary2016.doc
0.056888 T F 44973 - 2736 0 F - - - -
/bro/extracted/SMTP-FiWH8E2GK4LZmK8kYg.doc
1455105508.920691
FiqR9N1j5G1JlUlDe 12.23.29.13 12.3.16.5 COsYzjbE2bCVGewz1 SMTP 7
SHA1,DATA_EVENT,MD5,EXTRACT application/msword SCD List - SS101-612a.vsd
0.148642 T F 91656 - 2696 0 F - - - -
/bro/extracted/SMTP-FiqR9N1j5G1JlUlDe.doc
1455105575.354126
FmnQbA19ShsuCDh0bk 12.23.29.13 16.2.23.2 CXYSjQx0YmTqhDagf SMTP 3
DATA_EVENT,MD5,SHA1,EXTRACT application/msword 00336582.doc 0.378492 TF
177152 - 0 0 F - c7c213a316143494115c905fd28938f9
8b7d7c28b0d2c28ad1287db60e7c26925181ab07 -
/bro/extracted/SMTP-FmnQbA19ShsuCDh0bk.doc

But no matches for new
office files...

Do you have any idea?

I have another question: in
order to keep track of files extracted, how can I set the filename with
something trackable like realfilename ?

Thanks in advance.
 



Connetti gratis il mondo con la nuova indoona:  hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo.
E chiami gratis anche i numeri fissi e mobili nel mondo!
Scarica subito l?app Vai su https://www.indoona.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160216/d77478b0/attachment.html 

From peter.osterberg at hexbit.se  Tue Feb 16 04:11:25 2016
From: peter.osterberg at hexbit.se (=?UTF-8?Q?Peter_=c3=96sterberg?=)
Date: Tue, 16 Feb 2016 13:11:25 +0100
Subject: [Bro] File Extraction: doc/xls=ok, docx/xlsx=ko
In-Reply-To: <bfb1d702994a3494f781aca21ed81454@tiscali.it>
References: <bfb1d702994a3494f781aca21ed81454@tiscali.it>
Message-ID: <56C311ED.9060201@hexbit.se>

I have never done this myself but it seems like f$info$filename could be
a possible solution to your second question.

/Peter

Den 2016-02-16 kl. 12:29, skrev puntogtg at tiscali.it:
> Hello,
> I am trying to find out if I did some mistake in my extract.bro script.
> Basically   I am able to extract the doc,pdf,xls but not the docx,xlsx,xlsm etc (all new office files).
> Script looks like this:
>  
> global ext_map: table[string] of string = {
>     ["application/msword"] = "doc",
>     ["application/vnd.openxmlformats-officedocument.wordprocessingml.document"] = "docx",
>     ["application/vnd.openxmlformats-officedocument.wordprocessingml.template"] = "dotx",
>     ["application/vnd.ms-word.document.macroEnabled.12"] = "docm",
>     ["application/vnd.ms-word.template.macroEnabled.12"] = "dotm",
>     ["application/vnd.ms-excel"] = "xls",
>     ["application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"] = "xlsx",
>     ["application/vnd.openxmlformats-officedocument.spreadsheetml.template"] = "xltx",
>     ["application/vnd.ms-excel.sheet.macroEnabled.12"] = "xlsm",
>     ["application/vnd.ms-excel.template.macroEnabled.12"] = "xltm",
>     ["application/vnd.ms-excel.addin.macroEnabled.12"] = "xlam",
>     ["application/vnd.ms-excel.sheet.binary.macroEnabled.12"] = "xlsb",
>     ["application/vnd.ms-powerpoint"] = "ppt",
>     ["application/vnd.openxmlformats-officedocument.presentationml.presentation"] = "pptx",
>     ["application/vnd.openxmlformats-officedocument.presentationml.template"] = "potx",
>     ["application/vnd.openxmlformats-officedocument.presentationml.slideshow"] = "ppsx",
>     ["application/vnd.ms-powerpoint.addin.macroEnabled.12"] = "ppam",
>     ["application/vnd.ms-powerpoint.presentation.macroEnabled.12"] = "pptm",
>     ["application/vnd.ms-powerpoint.presentation.macroEnabled.12"] = "potm",
>     ["application/vnd.ms-powerpoint.slideshow.macroEnabled.12"] = "ppsm",
> } &default ="";
>
>  
> event file_new(f: fa_file)
>    {
>        if ( ! f?$mime_type  )
>         return;
>     local ext = "";
>  if ( f?$mime_type )
>         ext = ext_map[f$mime_type];
>     #if ( ext !="pdf" && ext !="exe" && ext !="swf" )
>     if ( ext !="doc" && ext !="docx" && ext !="dotx" && ext !="docm" && ext !="dotm" && ext !="xls" && ext !="xlsx" && ext !="xltx" && ext !="xlsm" && ext !="xltm" && ext !="xlam" && ext !="xlsb" && ext !="ppt" && ext !="pptx" && ext !="potx" && ext !="ppsx" && ext !="ppam" && ext !="pptm" && ext !="potm" && ext !="ppsm" )
>     return;
>
>       local fname = fmt("/bro/extracted/%s-%s.%s", f$source, f$id, ext);
>            Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
>            break;
> }
>
> Into the files.log I can see when extract matches:
>
> 1455104772.317535       FiWH8E2GK4LZmK8kYg      12.23.29.13  194.1.1.22     C9Ujyw1HodyV6hrs4f      SMTP    5       DATA_EVENT,EXTRACT,SHA1,MD5     application/msword      SPCH_100658601_1_Skillsupdatefebruary2016.doc  0.056888        T       F       44973   -       2736    0       F       -       -       -       -       /bro/extracted/SMTP-FiWH8E2GK4LZmK8kYg.doc
> 1455105508.920691       FiqR9N1j5G1JlUlDe       12.23.29.13  12.3.16.5   COsYzjbE2bCVGewz1       SMTP    7       SHA1,DATA_EVENT,MD5,EXTRACT     application/msword      SCD List - SS101-612a.vsd     0.148642 T       F       91656   -       2696    0       F       -       -       -       -       /bro/extracted/SMTP-FiqR9N1j5G1JlUlDe.doc
> 1455105575.354126       FmnQbA19ShsuCDh0bk      12.23.29.13  16.2.23.2   CXYSjQx0YmTqhDagf       SMTP    3       DATA_EVENT,MD5,SHA1,EXTRACT     application/msword      00336582.doc    0.378492      TF       177152  -       0       0       F       -       c7c213a316143494115c905fd28938f9        8b7d7c28b0d2c28ad1287db60e7c26925181ab07        -       /bro/extracted/SMTP-FmnQbA19ShsuCDh0bk.doc
>
>
> But no matches for new office files...
>
> Do you have any idea?
>
>
>
> I have another question: in order to keep track of files extracted, how can I set the filename with something trackable like realfilename ?
>
> Thanks in advance.
>
>
>
> Connetti gratis il mondo con la nuova indoona: hai la chat, le
> chiamate, le video chiamate e persino le chiamate di gruppo.
> E chiami gratis anche i numeri fissi e mobili nel mondo!
> Scarica subito l?app Vai su https://www.indoona.com/
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160216/efb718bc/attachment.html 

From jazoff at illinois.edu  Tue Feb 16 06:01:13 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 16 Feb 2016 14:01:13 +0000
Subject: [Bro] File Extraction: doc/xls=ok, docx/xlsx=ko
In-Reply-To: <bfb1d702994a3494f781aca21ed81454@tiscali.it>
References: <bfb1d702994a3494f781aca21ed81454@tiscali.it>
Message-ID: <1C8AB3C0-B15E-4D2F-AFF9-B6AE3824D76E@illinois.edu>


> On Feb 16, 2016, at 6:29 AM, puntogtg at tiscali.it wrote:
> 
> Hello,
> I am trying to find out if I did some mistake in my extract.bro script.
> Basically   I am able to extract the doc,pdf,xls but not the docx,xlsx,xlsm etc (all new office files).

I believe the problem is that as far as bro is concerned, new office files are really .zip archives.

> Script looks like this:
>  
> global ext_map: table[string] of string = {
>     ["application/msword"] = "doc",
>     ["application/vnd.openxmlformats-officedocument.wordprocessingml.document"] = "docx",
>     ["application/vnd.openxmlformats-officedocument.wordprocessingml.template"] = "dotx",
> 
[..]
> } &default ="";
> 
>  
> event file_new(f: fa_file)
>    {
>        if ( ! f?$mime_type  )
>         return;
>     local ext = "";
>  if ( f?$mime_type )
>         ext = ext_map[f$mime_type];
>     #if ( ext !="pdf" && ext !="exe" && ext !="swf" )
>     if ( ext !="doc" && ext !="docx" && ext !="dotx" && ext !="docm" && ext !="dotm" && ext !="xls" && ext !="xlsx" && ext !="xltx" && ext !="xlsm" && ext !="xltm" && ext !="xlam" && ext !="xlsb" && ext !="ppt" && ext !="pptx" && ext !="potx" && ext !="ppsx" && ext !="ppam" && ext !="pptm" && ext !="potm" && ext !="ppsm" )
>     return;
> 
>       local fname = fmt("/bro/extracted/%s-%s.%s", f$source, f$id, ext);
>            Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
>            break;
> }

 
Aside from the issue that docx shows up as a zip file, here is a fixed up version of that file_new event:

event file_new(f: fa_file)
   {
    if (!f?$mime_type)
        return;

    if (f$mime_type !in ext_map)
        return;

    ext = ext_map[f$mime_type];
    local fname = fmt("/bro/extracted/%s-%s.%s", f$source, f$id, ext);
    Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
    break;
}



-- 
- Justin Azoff




From martin.arlitt at ucalgary.ca  Tue Feb 16 06:50:23 2016
From: martin.arlitt at ucalgary.ca (Martin Arlitt)
Date: Tue, 16 Feb 2016 07:50:23 -0700
Subject: [Bro] event suppression
Message-ID: <56C3372F.4040809@ucalgary.ca>

hi

the event suppression in Bro does not appear to work the way I thought 
it would. For example, in my notice.log file, the suppress_for value 
always appears to be 3600. In misc/scan.bro (loaded in local.bro), 
addr_scan_interval and port_scan_interval both are set to 5min by 
default, yet still report 3600 in the suppress_for column of the log. Is 
there something obvious that I am overlooking?

thanks Martin

From jazoff at illinois.edu  Tue Feb 16 06:56:18 2016
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 16 Feb 2016 14:56:18 +0000
Subject: [Bro] event suppression
In-Reply-To: <56C3372F.4040809@ucalgary.ca>
References: <56C3372F.4040809@ucalgary.ca>
Message-ID: <D19AB7C9-EB53-400B-B949-BC2E27F9CABF@illinois.edu>

port_scan_interval is not the suppression interval:

        ## Port scans detect that an attacking host appears to be
        ## scanning a single victim host on several ports.  This notice
        ## is generated when an attacking host attempts to connect to
        ## :bro:id:`Scan::port_scan_threshold`
        ## unique ports on a single host over the previous
        ## :bro:id:`Scan::port_scan_interval` time range.

    ## Failed connection attempts are tracked over this time interval for
    ## the port scan detection.  A higher interval will detect slower
    ## scanners, but may also yield more false positives.

If you want to change the suppression interval, use:


redef Notice::type_suppression_intervals += {
    [Scan::Port_Scan]           = 300sec,
    [Scan::Address_Scan]        = 300sec,
};


-- 
- Justin Azoff

> On Feb 16, 2016, at 9:50 AM, Martin Arlitt <martin.arlitt at ucalgary.ca> wrote:
> 
> hi
> 
> the event suppression in Bro does not appear to work the way I thought 
> it would. For example, in my notice.log file, the suppress_for value 
> always appears to be 3600. In misc/scan.bro (loaded in local.bro), 
> addr_scan_interval and port_scan_interval both are set to 5min by 
> default, yet still report 3600 in the suppress_for column of the log. Is 
> there something obvious that I am overlooking?
> 
> thanks Martin
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From liburdi.joshua at gmail.com  Tue Feb 16 07:03:17 2016
From: liburdi.joshua at gmail.com (Josh Liburdi)
Date: Tue, 16 Feb 2016 10:03:17 -0500
Subject: [Bro] File Extraction: doc/xls=ok, docx/xlsx=ko
In-Reply-To: <1C8AB3C0-B15E-4D2F-AFF9-B6AE3824D76E@illinois.edu>
References: <bfb1d702994a3494f781aca21ed81454@tiscali.it>
	<1C8AB3C0-B15E-4D2F-AFF9-B6AE3824D76E@illinois.edu>
Message-ID: <CANrCRiEGKLSuOYuHnbC8iQZTQB4s5M3Ci4MZf0xR_jegqwvX7Q@mail.gmail.com>

Justin's correct that the header for new Office files are very similar
to Zip files, but they do differ slightly. If you look at the code
that identifies a file as
application/vnd.openxmlformats-officedocument.wordprocessingml.document,
it uses this regular expression:

/^PK\x03\x04.{26}(\[Content_Types\]\.xml|_rels\x2f\.rels|word\x2f).*PK\x03\x04.{26}word\x2f/

Here is the regular expression that identifies Zip files: /

^PK\x03\x04.{2}/

It's possible that your new office files aren't fitting this format.
In that case, I'd suggest adding this mime_type to your table:
application/vnd.openxmlformats-officedocument

It uses the basic new Office file header as a regular expression:

/^PK\x03\x04\x14\x00\x06\x00/

If you use application/vnd.openxmlformats-officedocument, then you
can't assume that the file has a specific extension-- all you know is
that the file is a new Office file.

Hope that clears up some confusion (and doesn't cause more)!

Josh

On Tue, Feb 16, 2016 at 9:01 AM, Azoff, Justin S <jazoff at illinois.edu> wrote:
>
>> On Feb 16, 2016, at 6:29 AM, puntogtg at tiscali.it wrote:
>>
>> Hello,
>> I am trying to find out if I did some mistake in my extract.bro script.
>> Basically   I am able to extract the doc,pdf,xls but not the docx,xlsx,xlsm etc (all new office files).
>
> I believe the problem is that as far as bro is concerned, new office files are really .zip archives.
>
>> Script looks like this:
>>
>> global ext_map: table[string] of string = {
>>     ["application/msword"] = "doc",
>>     ["application/vnd.openxmlformats-officedocument.wordprocessingml.document"] = "docx",
>>     ["application/vnd.openxmlformats-officedocument.wordprocessingml.template"] = "dotx",
>>
> [..]
>> } &default ="";
>>
>>
>> event file_new(f: fa_file)
>>    {
>>        if ( ! f?$mime_type  )
>>         return;
>>     local ext = "";
>>  if ( f?$mime_type )
>>         ext = ext_map[f$mime_type];
>>     #if ( ext !="pdf" && ext !="exe" && ext !="swf" )
>>     if ( ext !="doc" && ext !="docx" && ext !="dotx" && ext !="docm" && ext !="dotm" && ext !="xls" && ext !="xlsx" && ext !="xltx" && ext !="xlsm" && ext !="xltm" && ext !="xlam" && ext !="xlsb" && ext !="ppt" && ext !="pptx" && ext !="potx" && ext !="ppsx" && ext !="ppam" && ext !="pptm" && ext !="potm" && ext !="ppsm" )
>>     return;
>>
>>       local fname = fmt("/bro/extracted/%s-%s.%s", f$source, f$id, ext);
>>            Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
>>            break;
>> }
>
>
> Aside from the issue that docx shows up as a zip file, here is a fixed up version of that file_new event:
>
> event file_new(f: fa_file)
>    {
>     if (!f?$mime_type)
>         return;
>
>     if (f$mime_type !in ext_map)
>         return;
>
>     ext = ext_map[f$mime_type];
>     local fname = fmt("/bro/extracted/%s-%s.%s", f$source, f$id, ext);
>     Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
>     break;
> }
>
>
>
> --
> - Justin Azoff
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From seth at icir.org  Tue Feb 16 09:33:34 2016
From: seth at icir.org (Seth Hall)
Date: Tue, 16 Feb 2016 12:33:34 -0500
Subject: [Bro] File Extraction: doc/xls=ok, docx/xlsx=ko
In-Reply-To: <CANrCRiEGKLSuOYuHnbC8iQZTQB4s5M3Ci4MZf0xR_jegqwvX7Q@mail.gmail.com>
References: <bfb1d702994a3494f781aca21ed81454@tiscali.it>
	<1C8AB3C0-B15E-4D2F-AFF9-B6AE3824D76E@illinois.edu>
	<CANrCRiEGKLSuOYuHnbC8iQZTQB4s5M3Ci4MZf0xR_jegqwvX7Q@mail.gmail.com>
Message-ID: <2B7F3B90-BEEB-447B-BE6A-46094D6E4617@icir.org>


> On Feb 16, 2016, at 10:03 AM, Josh Liburdi <liburdi.joshua at gmail.com> wrote:
> 
> Hope that clears up some confusion (and doesn't cause more)!

Thanks Josh!

One more tiny note, if anyone discovers files that they think should be matching a particular type and they aren't, please reach out.  We are maintaining all of the file type identification ourselves at this point and I'd like to make sure that we're doing a really nice job at identifying file types.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From puntogtg at tiscali.it  Thu Feb 18 00:24:36 2016
From: puntogtg at tiscali.it (puntogtg at tiscali.it)
Date: Thu, 18 Feb 2016 09:24:36 +0100
Subject: [Bro]
 =?utf-8?q?File_Extraction=3A_doc/xls=3Dok=2C_docx/xlsx=3Dko?=
Message-ID: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>

  Hello,
regarding filename I'm trying something like this:

 local
fname = fmt("/bro/extracted/%s.%s", f$info$filename, ext);

Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
[$extract_filename=fname]);
 break;

No errors but files are not being
saved. :)
Can you give me some help?
Thanks

Il 16.02.2016 13:11 Peter
?sterberg ha scritto: 

> I have never done this myself but it seems
like f$info$filename could be a possible solution to your second
question.
> 
> /Peter
> 
> Den 2016-02-16 kl. 12:29, skrev
puntogtg at tiscali.it [4]: 
> 
>> Hello,
>> I am trying to find out if I
did some mistake in my extract.bro script.
>> Basically I am able to
extract the doc,pdf,xls but not the docx,xlsx,xlsm etc (all new office
files).
>> Script looks like this:
>> 
>> global ext_map: table[string]
of string = {
>> ["application/msword"] = "doc",
>>
["application/vnd.openxmlformats-officedocument.wordprocessingml.document"]
= "docx",
>>
["application/vnd.openxmlformats-officedocument.wordprocessingml.template"]
= "dotx",
>> ["application/vnd.ms-word.document.macroEnabled.12"] =
"docm",
>> ["application/vnd.ms-word.template.macroEnabled.12"] =
"dotm",
>> ["application/vnd.ms-excel"] = "xls",
>>
["application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"] =
"xlsx",
>>
["application/vnd.openxmlformats-officedocument.spreadsheetml.template"]
= "xltx",
>> ["application/vnd.ms-excel.sheet.macroEnabled.12"] =
"xlsm",
>> ["application/vnd.ms-excel.template.macroEnabled.12"] =
"xltm",
>> ["application/vnd.ms-excel.addin.macroEnabled.12"] =
"xlam",
>> ["application/vnd.ms-excel.sheet.binary.macroEnabled.12"] =
"xlsb",
>> ["application/vnd.ms-powerpoint"] = "ppt",
>>
["application/vnd.openxmlformats-officedocument.presentationml.presentation"]
= "pptx",
>>
["application/vnd.openxmlformats-officedocument.presentationml.template"]
= "potx",
>>
["application/vnd.openxmlformats-officedocument.presentationml.slideshow"]
= "ppsx",
>> ["application/vnd.ms-powerpoint.addin.macroEnabled.12"] =
"ppam",
>>
["application/vnd.ms-powerpoint.presentation.macroEnabled.12"] =
"pptm",
>>
["application/vnd.ms-powerpoint.presentation.macroEnabled.12"] =
"potm",
>> ["application/vnd.ms-powerpoint.slideshow.macroEnabled.12"] =
"ppsm",
>> } &default ="";
>> 
>> event file_new(f: fa_file)
>> {
>> if
( ! f?$mime_type )
>> return;
>> local ext = "";
>> if ( f?$mime_type
)
>> ext = ext_map[f$mime_type];
>> #if ( ext !="pdf" && ext !="exe" &&
ext !="swf" )
>> if ( ext !="doc" && ext !="docx" && ext !="dotx" && ext
!="docm" && ext !="dotm" && ext !="xls" && ext !="xlsx" && ext !="xltx"
&& ext !="xlsm" && ext !="xltm" && ext !="xlam" && ext !="xlsb" && ext
!="ppt" && ext !="pptx" && ext !="potx" && ext !="ppsx" && ext !="ppam"
&& ext !="pptm" && ext !="potm" && ext !="ppsm" )
>> return;
>> 
>>
local fname = fmt("/bro/extracted/%s-%s.%s", f$source, f$id, ext);
>>
Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
[$extract_filename=fname]);
>> break;
>> }
>> 
>> Into the files.log I
can see when extract matches:
>> 
>> 1455104772.317535
FiWH8E2GK4LZmK8kYg 12.23.29.13 194.1.1.22 C9Ujyw1HodyV6hrs4f SMTP 5
DATA_EVENT,EXTRACT,SHA1,MD5 application/msword
SPCH_100658601_1_Skillsupdatefebruary2016.doc 0.056888 T F 44973 - 2736
0 F - - - - /bro/extracted/SMTP-FiWH8E2GK4LZmK8kYg.doc
>>
1455105508.920691 FiqR9N1j5G1JlUlDe 12.23.29.13 12.3.16.5
COsYzjbE2bCVGewz1 SMTP 7 SHA1,DATA_EVENT,MD5,EXTRACT application/msword
SCD List - SS101-612a.vsd 0.148642 T F 91656 - 2696 0 F - - - -
/bro/extracted/SMTP-FiqR9N1j5G1JlUlDe.doc
>> 1455105575.354126
FmnQbA19ShsuCDh0bk 12.23.29.13 16.2.23.2 CXYSjQx0YmTqhDagf SMTP 3
DATA_EVENT,MD5,SHA1,EXTRACT application/msword 00336582.doc 0.378492 TF
177152 - 0 0 F - c7c213a316143494115c905fd28938f9
8b7d7c28b0d2c28ad1287db60e7c26925181ab07 -
/bro/extracted/SMTP-FmnQbA19ShsuCDh0bk.doc
>> 
>> But no matches for new
office files...
>> 
>> Do you have any idea?
>> 
>> I have another
question: in order to keep track of files extracted, how can I set the
filename with something trackable like realfilename ?
>> 
>> Thanks in
advance.
>> 
>> Connetti gratis il mondo con la nuova indoona: hai la
chat, le chiamate, le video chiamate e persino le chiamate di gruppo.
>>
E chiami gratis anche i numeri fissi e mobili nel mondo!
>> Scarica
subito l'app Vai su https://www.indoona.com/ [1]
>> 
>>
_______________________________________________
>> Bro mailing list
>>
bro at bro-ids.org [2]
>>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro [3]

Connetti
gratis il mondo con la nuova indoona: hai la chat, le chiamate, le video
chiamate e persino le chiamate di gruppo.
 E chiami gratis anche i
numeri fissi e mobili nel mondo!
 Scarica subito l'app Vai su
https://www.indoona.com/ [5]

  


Connetti gratis il mondo con la nuova indoona:  hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo.
E chiami gratis anche i numeri fissi e mobili nel mondo!
Scarica subito l?app Vai su https://www.indoona.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160218/d6aed76e/attachment.html 

From seth at icir.org  Thu Feb 18 05:46:25 2016
From: seth at icir.org (Seth Hall)
Date: Thu, 18 Feb 2016 08:46:25 -0500
Subject: [Bro] File Extraction: doc/xls=ok, docx/xlsx=ko
In-Reply-To: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
Message-ID: <AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>


> On Feb 18, 2016, at 3:24 AM, puntogtg at tiscali.it wrote:
> 
>       local fname = fmt("/bro/extracted/%s.%s", f$info$filename, ext);
>            Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
>            break;

Add this outside of any event handler:

redef FilesExtract::prefix = "/bro/extracted/";

Then change the code you gave to:

	local fname = fmt("%s.%s", f$info$filename, ext);
	Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]);
	break;

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From puntogtg at tiscali.it  Thu Feb 18 06:17:20 2016
From: puntogtg at tiscali.it (puntogtg at tiscali.it)
Date: Thu, 18 Feb 2016 15:17:20 +0100
Subject: [Bro]
 =?utf-8?q?File_Extraction=3A_doc/xls=3Dok=2C_docx/xlsx=3Dko?=
In-Reply-To: <AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
	<AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
Message-ID: <2ba2f04f325225ea8981aba9caf56372@tiscali.it>

  Hello,
I put 

redef FilesExtract::prefix = "/bro/extracted/";
 at
the beginning of the script but when run gave me an error:

 error in
./extract.bro, line 1: "redef" used but not previously defined
(FilesExtract::prefix)
internal warning in ./extract.bro, line 1: Can't
document redef of FilesExtract::prefix, identifier lookup failed

Il
18.02.2016 14:46 Seth Hall ha scritto: 

>> On Feb 18, 2016, at 3:24 AM,
puntogtg at tiscali.it [1] wrote: local fname = fmt("/bro/extracted/%s.%s",
f$info$filename, ext); Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
[$extract_filename=fname]); break;
> 
> Add this outside of any event
handler:
> 
> redef FilesExtract::prefix = "/bro/extracted/";
> 
> Then
change the code you gave to:
> 
> local fname = fmt("%s.%s",
f$info$filename, ext);
> Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
[$extract_filename=fname]);
> break;
> 
> .Seth
> 
> --
> Seth Hall
>
International Computer Science Institute
> (Bro) because everyone has a
network
> http://www.bro.org/ [2]
  


Connetti gratis il mondo con la nuova indoona:  hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo.
E chiami gratis anche i numeri fissi e mobili nel mondo!
Scarica subito l?app Vai su https://www.indoona.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160218/e801dbe2/attachment.html 

From seth at icir.org  Thu Feb 18 07:28:00 2016
From: seth at icir.org (Seth Hall)
Date: Thu, 18 Feb 2016 10:28:00 -0500
Subject: [Bro] File Extraction: doc/xls=ok, docx/xlsx=ko
In-Reply-To: <2ba2f04f325225ea8981aba9caf56372@tiscali.it>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
	<AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
	<2ba2f04f325225ea8981aba9caf56372@tiscali.it>
Message-ID: <3FC818AB-66E3-4906-A883-947092586A5F@icir.org>


> On Feb 18, 2016, at 9:17 AM, puntogtg at tiscali.it wrote:
> 
> Hello,
> I put 
> redef FilesExtract::prefix = "/bro/extracted/";

Sorry, typing mistake...

redef FileExtract::prefix = "/bro/extracted/";

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/


From puntogtg at tiscali.it  Thu Feb 18 07:57:50 2016
From: puntogtg at tiscali.it (puntogtg at tiscali.it)
Date: Thu, 18 Feb 2016 16:57:50 +0100
Subject: [Bro]
 =?utf-8?q?File_Extraction=3A_doc/xls=3Dok=2C_docx/xlsx=3Dko?=
In-Reply-To: <3FC818AB-66E3-4906-A883-947092586A5F@icir.org>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
	<AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
	<2ba2f04f325225ea8981aba9caf56372@tiscali.it>
	<3FC818AB-66E3-4906-A883-947092586A5F@icir.org>
Message-ID: <04f7c5947b7675cec0603edad9de5c5b@tiscali.it>

  Hello,
now no errors but behavior is like after first suggestion of
Peter: files not created.
Any idea?

Il 18.02.2016 16:28 Seth Hall ha
scritto: 

>> On Feb 18, 2016, at 9:17 AM, puntogtg at tiscali.it [1]
wrote: Hello, I put redef FilesExtract::prefix = "/bro/extracted/";
> 
>
Sorry, typing mistake...
> 
> redef FileExtract::prefix =
"/bro/extracted/";
> 
> .Seth
> 
> --
> Seth Hall
> International
Computer Science Institute
> (Bro) because everyone has a network
>
http://www.bro.org/ [2]
  


Connetti gratis il mondo con la nuova indoona:  hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo.
E chiami gratis anche i numeri fissi e mobili nel mondo!
Scarica subito l?app Vai su https://www.indoona.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160218/96055b38/attachment-0001.html 

From seth at icir.org  Thu Feb 18 11:42:39 2016
From: seth at icir.org (Seth Hall)
Date: Thu, 18 Feb 2016 14:42:39 -0500
Subject: [Bro] File Extraction: doc/xls=ok, docx/xlsx=ko
In-Reply-To: <04f7c5947b7675cec0603edad9de5c5b@tiscali.it>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
	<AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
	<2ba2f04f325225ea8981aba9caf56372@tiscali.it>
	<3FC818AB-66E3-4906-A883-947092586A5F@icir.org>
	<04f7c5947b7675cec0603edad9de5c5b@tiscali.it>
Message-ID: <285F97C0-7E82-4643-8EA9-9B8B4D642843@icir.org>

Does the user you are running Bro as have permission to write to that directory?  Does the directory exist?

  .Seth


> On Feb 18, 2016, at 10:57 AM, puntogtg at tiscali.it wrote:
> 
> Hello,
> now no errors but behavior is like after first suggestion of Peter: files not created.
> Any idea?
> 
> Il 18.02.2016 16:28 Seth Hall ha scritto:
> 
>>> On Feb 18, 2016, at 9:17 AM, puntogtg at tiscali.it wrote: Hello, I put redef FilesExtract::prefix = "/bro/extracted/";
>> Sorry, typing mistake...
>> 
>> redef FileExtract::prefix = "/bro/extracted/";
>> 
>>   .Seth
>> 
>> --
>> Seth Hall
>> International Computer Science Institute
>> (Bro) because everyone has a network
>> 
>> http://www.bro.org/
>> 
>> 
>> 
> 
> 
> 
> Connetti gratis il mondo con la nuova indoona: hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo.
> E chiami gratis anche i numeri fissi e mobili nel mondo!
> Scarica subito l?app Vai su https://www.indoona.com/
> 

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From puntogtg at tiscali.it  Fri Feb 19 00:05:14 2016
From: puntogtg at tiscali.it (puntogtg at tiscali.it)
Date: Fri, 19 Feb 2016 09:05:14 +0100
Subject: [Bro]
 =?utf-8?q?File_Extraction=3A_doc/xls=3Dok=2C_docx/xlsx=3Dko?=
In-Reply-To: <285F97C0-7E82-4643-8EA9-9B8B4D642843@icir.org>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
	<AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
	<2ba2f04f325225ea8981aba9caf56372@tiscali.it>
	<3FC818AB-66E3-4906-A883-947092586A5F@icir.org>
	<04f7c5947b7675cec0603edad9de5c5b@tiscali.it>
	<285F97C0-7E82-4643-8EA9-9B8B4D642843@icir.org>
Message-ID: <c43b519dd04cc824757426252d737bff@tiscali.it>

  Of course the user has rights to write in that folder and the folder
exist, in fact with previous conf everything is ok, apart the name of
the files..

Il 18.02.2016 20:42 Seth Hall ha scritto: 

> Does the user
you are running Bro as have permission to write to that directory? Does
the directory exist?
> 
> .Seth
> 
>> On Feb 18, 2016, at 10:57 AM,
puntogtg at tiscali.it [3]wrote: Hello, now no errors but behavior is like
after first suggestion of Peter: files not created. Any idea? Il
18.02.2016 16:28 Seth Hall ha scritto: 
>> 
>>>> On Feb 18, 2016, at
9:17 AM, puntogtg at tiscali.it [1] wrote: Hello, I put redef
FilesExtract::prefix = "/bro/extracted/";
>>> Sorry, typing mistake...
redef FileExtract::prefix = "/bro/extracted/"; .Seth -- Seth Hall
International Computer Science Institute (Bro) because everyone has a
network http://www.bro.org/ [2]
>> Connetti gratis il mondo con la nuova
indoona: hai la chat, le chiamate, le video chiamate e persino le
chiamate di gruppo. E chiami gratis anche i numeri fissi e mobili nel
mondo! Scarica subito l'app Vai su https://www.indoona.com/ [4]
> 
>
--
> Seth Hall
> International Computer Science Institute
> (Bro)
because everyone has a network
> http://www.bro.org/ [5]
 



Connetti gratis il mondo con la nuova indoona:  hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo.
E chiami gratis anche i numeri fissi e mobili nel mondo!
Scarica subito l?app Vai su https://www.indoona.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160219/ec21efb8/attachment.html 

From peter.osterberg at hexbit.se  Fri Feb 19 01:09:07 2016
From: peter.osterberg at hexbit.se (=?UTF-8?Q?Peter_=C3=96sterberg?=)
Date: Fri, 19 Feb 2016 10:09:07 +0100
Subject: [Bro]
 =?utf-8?q?File_Extraction=3A_doc/xls=3Dok=2C_docx/xlsx=3Dko?=
In-Reply-To: <c43b519dd04cc824757426252d737bff@tiscali.it>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
	<AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
	<2ba2f04f325225ea8981aba9caf56372@tiscali.it>
	<3FC818AB-66E3-4906-A883-947092586A5F@icir.org>
	<04f7c5947b7675cec0603edad9de5c5b@tiscali.it>
	<285F97C0-7E82-4643-8EA9-9B8B4D642843@icir.org>
	<c43b519dd04cc824757426252d737bff@tiscali.it>
Message-ID: <dc465373fb089686674d9e5a8af44a1c@hexbit.se>

 

Just an idea 

What would the output be if you try to print the
contents to stdout? Nice way of learing what the variable actually
contains... 

Put this somewhere just before the call to write the file.
Then you know if you have a valid filename or not. Good practise is to
use print when unsure about the contents of a variable, it will quickly
reveal stuff instead of fumbling in total darkness. 

print
f$info$filename 

As said earlier, I've never tried using that variable
but it should be there and hold the right value at least according to
the documentation. 

/Peter 

2016-02-19 09:05 skrev
puntogtg at tiscali.it: 

> Of course the user has rights to write in that
folder and the folder exist, in fact with previous conf everything is
ok, apart the name of the files..
> 
> Il 18.02.2016 20:42 Seth Hall ha
scritto: 
> 
>> Does the user you are running Bro as have permission to
write to that directory? Does the directory exist?
>> 
>> .Seth
>> 
>>>
On Feb 18, 2016, at 10:57 AM, puntogtg at tiscali.itwrote: Hello, now no
errors but behavior is like after first suggestion of Peter: files not
created. Any idea? Il 18.02.2016 16:28 Seth Hall ha scritto: 
>>> 
>>>>>
On Feb 18, 2016, at 9:17 AM, puntogtg at tiscali.it wrote: Hello, I put
redef FilesExtract::prefix = "/bro/extracted/";
>>>> Sorry, typing
mistake... redef FileExtract::prefix = "/bro/extracted/"; .Seth -- Seth
Hall International Computer Science Institute (Bro) because everyone has
a network http://www.bro.org/ [1]
>>> Connetti gratis il mondo con la
nuova indoona: hai la chat, le chiamate, le video chiamate e persino le
chiamate di gruppo. E chiami gratis anche i numeri fissi e mobili nel
mondo! Scarica subito l'app Vai su https://www.indoona.com/ [2]
>> 
>>
--
>> Seth Hall
>> International Computer Science Institute
>> (Bro)
because everyone has a network
>> http://www.bro.org/ [1]
> 
> Connetti
gratis il mondo con la nuova indoona: hai la chat, le chiamate, le video
chiamate e persino le chiamate di gruppo.
> E chiami gratis anche i
numeri fissi e mobili nel mondo!
> Scarica subito l'app Vai su
https://www.indoona.com/ [2]
> 
>
_______________________________________________
> Bro mailing list
>
bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
[3]

 

Links:
------
[1] http://www.bro.org/
[2]
https://www.indoona.com/
[3]
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160219/517ebd0b/attachment.html 

From seth at icir.org  Fri Feb 19 06:17:35 2016
From: seth at icir.org (Seth Hall)
Date: Fri, 19 Feb 2016 09:17:35 -0500
Subject: [Bro] File Extraction: doc/xls=ok, docx/xlsx=ko
In-Reply-To: <c43b519dd04cc824757426252d737bff@tiscali.it>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
	<AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
	<2ba2f04f325225ea8981aba9caf56372@tiscali.it>
	<3FC818AB-66E3-4906-A883-947092586A5F@icir.org>
	<04f7c5947b7675cec0603edad9de5c5b@tiscali.it>
	<285F97C0-7E82-4643-8EA9-9B8B4D642843@icir.org>
	<c43b519dd04cc824757426252d737bff@tiscali.it>
Message-ID: <578E504A-88BD-4124-84CA-073C8A5DE6BB@icir.org>


> On Feb 19, 2016, at 3:05 AM, puntogtg at tiscali.it wrote:
> 
> Of course the user has rights to write in that folder and the folder exist, in fact with previous conf everything is ok, apart the name of the files..

Ugh, I just realized the problem...

 if ( f?$mime_type )
        ext = ext_map[f$mime_type];

That code can't work in the file_new event.  In Bro 2.4, there is a new event named file_sniff.  It's at the point where some content from the file has been seen and Bro has had a chance to look at it and take a guess about the file type.  You aren't seeing any file extraction because you have a return statement that's returning if there is no known file extension (which there isn't at that point!).

event file_sniff(f: fa_file, meta: fa_metadata)
	{
	if ( meta?$mime_type )
		{
		# put your code here...
		}
	}

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From puntogtg at tiscali.it  Fri Feb 19 06:54:23 2016
From: puntogtg at tiscali.it (puntogtg at tiscali.it)
Date: Fri, 19 Feb 2016 15:54:23 +0100
Subject: [Bro]
 =?utf-8?q?File_Extraction=3A_doc/xls=3Dok=2C_docx/xlsx=3Dko?=
In-Reply-To: <578E504A-88BD-4124-84CA-073C8A5DE6BB@icir.org>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
	<AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
	<2ba2f04f325225ea8981aba9caf56372@tiscali.it>
	<3FC818AB-66E3-4906-A883-947092586A5F@icir.org>
	<04f7c5947b7675cec0603edad9de5c5b@tiscali.it>
	<285F97C0-7E82-4643-8EA9-9B8B4D642843@icir.org>
	<c43b519dd04cc824757426252d737bff@tiscali.it>
	<578E504A-88BD-4124-84CA-073C8A5DE6BB@icir.org>
Message-ID: <7b9bb8cbb8ec72fc947678d1386c6a02@tiscali.it>

  Hi,
I added but tells me 

identifier not defined: fa_metadata

Il
19.02.2016 15:17 Seth Hall ha scritto: 

>> On Feb 19, 2016, at 3:05 AM,
puntogtg at tiscali.it [1] wrote: Of course the user has rights to write in
that folder and the folder exist, in fact with previous conf everything
is ok, apart the name of the files..
> 
> Ugh, I just realized the
problem...
> 
> if ( f?$mime_type )
> ext = ext_map[f$mime_type];
> 
>
That code can't work in the file_new event. In Bro 2.4, there is a new
event named file_sniff. It's at the point where some content from the
file has been seen and Bro has had a chance to look at it and take a
guess about the file type. You aren't seeing any file extraction because
you have a return statement that's returning if there is no known file
extension (which there isn't at that point!).
> 
> event file_sniff(f:
fa_file, meta: fa_metadata)
> {
> if ( meta?$mime_type )
> {
> # put
your code here...
> }
> }
> 
> .Seth
> 
> --
> Seth Hall
> International
Computer Science Institute
> (Bro) because everyone has a network
>
http://www.bro.org/ [2]
  


Connetti gratis il mondo con la nuova indoona:  hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo.
E chiami gratis anche i numeri fissi e mobili nel mondo!
Scarica subito l?app Vai su https://www.indoona.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160219/e7b64e14/attachment.html 

From liburdi.joshua at gmail.com  Fri Feb 19 07:20:49 2016
From: liburdi.joshua at gmail.com (Josh Liburdi)
Date: Fri, 19 Feb 2016 10:20:49 -0500
Subject: [Bro] File Extraction: doc/xls=ok, docx/xlsx=ko
In-Reply-To: <7b9bb8cbb8ec72fc947678d1386c6a02@tiscali.it>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
	<AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
	<2ba2f04f325225ea8981aba9caf56372@tiscali.it>
	<3FC818AB-66E3-4906-A883-947092586A5F@icir.org>
	<04f7c5947b7675cec0603edad9de5c5b@tiscali.it>
	<285F97C0-7E82-4643-8EA9-9B8B4D642843@icir.org>
	<c43b519dd04cc824757426252d737bff@tiscali.it>
	<578E504A-88BD-4124-84CA-073C8A5DE6BB@icir.org>
	<7b9bb8cbb8ec72fc947678d1386c6a02@tiscali.it>
Message-ID: <CANrCRiFgWV7StvDvi4=uNxvysBUCBc-ZNE6gnhPPrK9JZdZJ5w@mail.gmail.com>

Apologies if I missed it, but which version of Bro are you running?

Josh

On Fri, Feb 19, 2016 at 9:54 AM,  <puntogtg at tiscali.it> wrote:
> Hi,
> I added  but tells me
>
> identifier not defined: fa_metadata
>
>
>
>
>
>
> Il 19.02.2016 15:17 Seth Hall ha scritto:
>
> On Feb 19, 2016, at 3:05 AM, puntogtg at tiscali.it wrote: Of course the user
> has rights to write in that folder and the folder exist, in fact with
> previous conf everything is ok, apart the name of the files..
>
> Ugh, I just realized the problem...
>
>  if ( f?$mime_type )
>         ext = ext_map[f$mime_type];
>
> That code can't work in the file_new event.  In Bro 2.4, there is a new
> event named file_sniff.  It's at the point where some content from the file
> has been seen and Bro has had a chance to look at it and take a guess about
> the file type.  You aren't seeing any file extraction because you have a
> return statement that's returning if there is no known file extension (which
> there isn't at that point!).
>
> event file_sniff(f: fa_file, meta: fa_metadata)
> 	{
> 	if ( meta?$mime_type )
> 		{
> 		# put your code here...
> 		}
> 	}
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
>
>
> Connetti gratis il mondo con la nuova indoona: hai la chat, le chiamate, le
> video chiamate e persino le chiamate di gruppo.
> E chiami gratis anche i numeri fissi e mobili nel mondo!
> Scarica subito l?app Vai su https://www.indoona.com/
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From puntogtg at tiscali.it  Fri Feb 19 08:46:09 2016
From: puntogtg at tiscali.it (puntogtg at tiscali.it)
Date: Fri, 19 Feb 2016 17:46:09 +0100
Subject: [Bro]
 =?utf-8?q?File_Extraction=3A_doc/xls=3Dok=2C_docx/xlsx=3Dko?=
In-Reply-To: <CANrCRiFgWV7StvDvi4=uNxvysBUCBc-ZNE6gnhPPrK9JZdZJ5w@mail.gmail.com>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
	<AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
	<2ba2f04f325225ea8981aba9caf56372@tiscali.it>
	<3FC818AB-66E3-4906-A883-947092586A5F@icir.org>
	<04f7c5947b7675cec0603edad9de5c5b@tiscali.it>
	<285F97C0-7E82-4643-8EA9-9B8B4D642843@icir.org>
	<c43b519dd04cc824757426252d737bff@tiscali.it>
	<578E504A-88BD-4124-84CA-073C8A5DE6BB@icir.org>
	<7b9bb8cbb8ec72fc947678d1386c6a02@tiscali.it>
	<CANrCRiFgWV7StvDvi4=uNxvysBUCBc-ZNE6gnhPPrK9JZdZJ5w@mail.gmail.com>
Message-ID: <65d9c9358651e597f442056c5ef8530a@tiscali.it>

  Josh,
I have to say apologies...
Was a good idea to check the
version: I was running 2.3!
Now compiled the new one: 2.4.1 ^__^
Into
extracted.bro put again: 

event file_sniff(f: fa_file, meta:
fa_metadata)

 {

 if ( meta?$mime_type )

 local fname =
fmt("/bro/extracted/%s.%s", f$info$filename, ext);

Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
[$extract_filename=fname]);
 return;
 break;

it is working and files
are coming with name!

A question now: previously I was using the
map

global ext_map: table[string] of string = {

["application/x-dosexec"] = "exe",

 ["application/vnd.ms-excel"] =
"xls",
} &default ="";

to select what file type to save, now it seems
all extensions are saved..
How can select what to keep?

Thanks

Il
19.02.2016 16:20 Josh Liburdi ha scritto: 

> Apologies if I missed it,
but which version of Bro are you running?
> 
> Josh
> 
> On Fri, Feb 19,
2016 at 9:54 AM, wrote:
> 
>> Hi, I added but tells me identifier not
defined: fa_metadata Il 19.02.2016 15:17 Seth Hall ha scritto: On Feb
19, 2016, at 3:05 AM, puntogtg at tiscali.it [1] wrote: Of course the user
has rights to write in that folder and the folder exist, in fact with
previous conf everything is ok, apart the name of the files.. Ugh, I
just realized the problem... if ( f?$mime_type ) ext =
ext_map[f$mime_type]; That code can't work in the file_new event. In Bro
2.4, there is a new event named file_sniff. It's at the point where some
content from the file has been seen and Bro has had a chance to look at
it and take a guess about the file type. You aren't seeing any file
extraction because you have a return statement that's returning if there
is no known file extension (which there isn't at that point!). event
file_sniff(f: fa_file, meta: fa_metadata) { if ( meta?$mime_type ) { #
put your code here... } } .Seth -- Seth Hall International Computer
Science Institute (Bro) because everyone has a network
http://www.bro.org/ [2] Connetti gratis il mondo con la nuova indoona:
hai la chat, le chiamate, le video chiamate e persino le chiamate di
gruppo. E chiami gratis anche i numeri fissi e mobili nel mondo! Scarica
subito l'app Vai su https://www.indoona.com/ [3]
_______________________________________________ Bro mailing list
bro at bro-ids.org [4]
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro [5]
 



Connetti gratis il mondo con la nuova indoona:  hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo.
E chiami gratis anche i numeri fissi e mobili nel mondo!
Scarica subito l?app Vai su https://www.indoona.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160219/20f0cc5e/attachment.html 

From seth at icir.org  Fri Feb 19 10:29:01 2016
From: seth at icir.org (Seth Hall)
Date: Fri, 19 Feb 2016 13:29:01 -0500
Subject: [Bro] File Extraction: doc/xls=ok, docx/xlsx=ko
In-Reply-To: <65d9c9358651e597f442056c5ef8530a@tiscali.it>
References: <cb12af6ade22eb2dadde338f2346ca55@tiscali.it>
	<AD4794A7-371F-4AFC-B689-7DE7980A10A0@icir.org>
	<2ba2f04f325225ea8981aba9caf56372@tiscali.it>
	<3FC818AB-66E3-4906-A883-947092586A5F@icir.org>
	<04f7c5947b7675cec0603edad9de5c5b@tiscali.it>
	<285F97C0-7E82-4643-8EA9-9B8B4D642843@icir.org>
	<c43b519dd04cc824757426252d737bff@tiscali.it>
	<578E504A-88BD-4124-84CA-073C8A5DE6BB@icir.org>
	<7b9bb8cbb8ec72fc947678d1386c6a02@tiscali.it>
	<CANrCRiFgWV7StvDvi4=uNxvysBUCBc-ZNE6gnhPPrK9JZdZJ5w@mail.gmail.com>
	<65d9c9358651e597f442056c5ef8530a@tiscali.it>
Message-ID: <2F3DD803-6532-416C-AAA2-CA4183FEA89C@icir.org>


> On Feb 19, 2016, at 11:46 AM, puntogtg at tiscali.it wrote:
> 
> to select what file type to save, now it seems all extensions are saved..
> How can select what to keep?

Your if statement isn't checking for a particular mime type before adding the extraction analyzer.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From bglaze at gmail.com  Fri Feb 19 16:06:48 2016
From: bglaze at gmail.com (Brandon Glaze)
Date: Fri, 19 Feb 2016 16:06:48 -0800
Subject: [Bro] delay compress bro log rotation
Message-ID: <CAM5AJtYb1ZQYx=kZ34aEZaM0B0UTYVbvwNWwBmGaGTFNdosPyQ@mail.gmail.com>

Good afternoon,

Is there a way to enable a "delay compress" type command (like in
logrotate) for bro/broctl cron? I want to post process log files and it
would be much more efficient if they were uncompressed.

=====================
Brandon Glaze
bglaze at gmail.com

"Lead me, follow me, or get the hell out of my way."
- General George Patton Jr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160219/5e0f3065/attachment.html 

From hhoffman at ip-solutions.net  Fri Feb 19 16:17:26 2016
From: hhoffman at ip-solutions.net (Harry Hoffman)
Date: Fri, 19 Feb 2016 19:17:26 -0500
Subject: [Bro] broctl errors related to replacement of ifconfig
Message-ID: <56C7B096.70401@ip-solutions.net>

Hi Folks,

On later versions of Linux distros iproute2 replaces ifconfig with ip

Starting at line 601 at 
https://github.com/bro/broctl/blob/master/BroControl/config.py

It looks like ifconfig is hard-written into the logic. Probably needs a 
patch to check for the ip command.

Cheers,
Harry

From beikejinmiao at gmail.com  Thu Feb 25 01:38:08 2016
From: beikejinmiao at gmail.com (=?UTF-8?B?5p2O6YeR6IuX?=)
Date: Thu, 25 Feb 2016 17:38:08 +0800
Subject: [Bro] How to update table automaticlly when reading from SQLite
	Databases?
Message-ID: <CADFdfdjTFj_gkE4Wki0oMO_bGxiDy+tpTiPUpbUFEc9PUWfTTw@mail.gmail.com>

I see the error of "error:
/root/bro-suricata/bro/intels/abnormal/Input::READER_SQLITE: SQLite only
supports manual reading mode."
Here is my bro script

export {
    type Idx_HOST: record {
        host: string;
    };

    type Val: record {
        target:         string &optional;
        start_times:    vector of string &optional;
        end_times:      vector of string &optional;
        nsrc_ips:       vector of string &optional;
#       reason:         string &optional;
    };

    global abnormal_host: table[string] of Val = table();
}

event bro_init()
{
    Input::add_table([
            $source=abnormal_db,
            $name="abnormal_host",
            $idx=Idx_HOST,
            $val=Val,
            $destination=abnormal_host,
            $reader=Input::READER_SQLITE,
            $config=table(["query"] = "select * from abnormal_host;")
            $mode=Input::REREAD
            ]);
}


How can i reread the data from sqlite automaticlly.?
Or how can i use the command of `Input::force_update("")` in python?
Thanks you very much
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160225/4832d532/attachment.html 

From zpravaiz at aus.edu  Thu Feb 25 22:00:25 2016
From: zpravaiz at aus.edu (Zafar Pravaiz)
Date: Fri, 26 Feb 2016 10:00:25 +0400
Subject: [Bro] NO DHCP.log
Message-ID: <0295DF8E-215A-42B2-BE76-A567B2BE7C45@aus.edu>

Hi , 

I am running SO 14.04. This is just capturing DNS and DHCP traffic on a span port. Recently i ran soup and reboot the box. After that i have noticed no DHCP log is showing up in bro log. i can see known_services shows DHCP as service but there no dhcp.log file being generate.  Any clue what went wrong? 


I would appreciate any help 

Thanks 
Zafar 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160226/f205911c/attachment.html 

From johanna at icir.org  Fri Feb 26 09:04:17 2016
From: johanna at icir.org (Johanna Amann)
Date: Fri, 26 Feb 2016 09:04:17 -0800
Subject: [Bro] How to update table automaticlly when reading from SQLite
 Databases?
In-Reply-To: <CADFdfdjTFj_gkE4Wki0oMO_bGxiDy+tpTiPUpbUFEc9PUWfTTw@mail.gmail.com>
References: <CADFdfdjTFj_gkE4Wki0oMO_bGxiDy+tpTiPUpbUFEc9PUWfTTw@mail.gmail.com>
Message-ID: <20160226170407.GA60195@wifi110.sys.ICSI.Berkeley.EDU>

Hello,

On Thu, Feb 25, 2016 at 05:38:08PM +0800, ??? wrote:
> I see the error of "error:
> /root/bro-suricata/bro/intels/abnormal/Input::READER_SQLITE: SQLite only
> supports manual reading mode."

[...]

> How can i reread the data from sqlite automaticlly.?

That is not supported at the moment, sorry. The current implementation
does not support any kind of automatic notification upon changes.

> Or how can i use the command of `Input::force_update("")` in python?

There also is no direct way to do this - you have to trigger the command
within Bro. If you want to trigger it in a python script, you have to send
a notification to Bro that it should execute Input::force_update. The
easiest way to do that probably is to use Broker (the new bro
communication library) in python, send an event to Bro, catch that event
in the master, and let that event call force_update.

I hope this helps,
 Johanna

From johanna at icir.org  Fri Feb 26 09:08:25 2016
From: johanna at icir.org (Johanna Amann)
Date: Fri, 26 Feb 2016 09:08:25 -0800
Subject: [Bro] broctl errors related to replacement of ifconfig
In-Reply-To: <56C7B096.70401@ip-solutions.net>
References: <56C7B096.70401@ip-solutions.net>
Message-ID: <20160226170825.GB60195@wifi110.sys.ICSI.Berkeley.EDU>

Hello Harry,

thanks, we should fix this before the next release. I created a bug to
track this at https://bro-tracker.atlassian.net/browse/BIT-1540

Johanna

On Fri, Feb 19, 2016 at 07:17:26PM -0500, Harry Hoffman wrote:
> Hi Folks,
> 
> On later versions of Linux distros iproute2 replaces ifconfig with ip
> 
> Starting at line 601 at 
> https://github.com/bro/broctl/blob/master/BroControl/config.py
> 
> It looks like ifconfig is hard-written into the logic. Probably needs a 
> patch to check for the ip command.
> 
> Cheers,
> Harry
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 
> 

From johanna at icir.org  Fri Feb 26 09:18:35 2016
From: johanna at icir.org (Johanna Amann)
Date: Fri, 26 Feb 2016 09:18:35 -0800
Subject: [Bro] NO DHCP.log
In-Reply-To: <0295DF8E-215A-42B2-BE76-A567B2BE7C45@aus.edu>
References: <0295DF8E-215A-42B2-BE76-A567B2BE7C45@aus.edu>
Message-ID: <20160226171835.GC60195@wifi110.sys.ICSI.Berkeley.EDU>

Hello,

On Fri, Feb 26, 2016 at 10:00:25AM +0400, Zafar Pravaiz wrote:
> I am running SO 14.04. This is just capturing DNS and DHCP traffic on a
> span port. Recently i ran soup and reboot the box. After that i have
> noticed no DHCP log is showing up in bro log. i can see known_services
> shows DHCP as service but there no dhcp.log file being generate.  Any
> clue what went wrong? 

On a first glance I do not really have any idea what went wrong, but there
are a few things to check -

* just to verify, dns.log is still being written correctly?

* could you check that you see dhcp connections in conn.log? They should
  be tagged with dhcp in the service field.

and

* could you verify that loaded_scripts.log contains
  scripts/base/protocols/dhcp?

Johanna

From johanna at icir.org  Fri Feb 26 09:23:48 2016
From: johanna at icir.org (Johanna Amann)
Date: Fri, 26 Feb 2016 09:23:48 -0800
Subject: [Bro] delay compress bro log rotation
In-Reply-To: <CAM5AJtYb1ZQYx=kZ34aEZaM0B0UTYVbvwNWwBmGaGTFNdosPyQ@mail.gmail.com>
References: <CAM5AJtYb1ZQYx=kZ34aEZaM0B0UTYVbvwNWwBmGaGTFNdosPyQ@mail.gmail.com>
Message-ID: <20160226172348.GD60195@wifi110.sys.ICSI.Berkeley.EDU>

Hello Brandon,

On Fri, Feb 19, 2016 at 04:06:48PM -0800, Brandon Glaze wrote:
> Is there a way to enable a "delay compress" type command (like in
> logrotate) for bro/broctl cron? I want to post process log files and it
> would be much more efficient if they were uncompressed.

As far as I am aware, there is no command that delays compression of the
logs. However, you should be able to install custom postprocessing scripts
into broctl, which will be run on the uncompressed log files - this is how
the default connection summary reports are generated.

I never tried this, but I think you should just be able to add a script to
the "postprocessors" directory in broctl, and it should be called on
log-rotation for every log-file. You can use the implementation of the
script that generates the connection summary as a guideline on how to
implement this:
https://github.com/bro/broctl/tree/master/bin/postprocessors

I hope this helps,
 Johanna

From zpravaiz at aus.edu  Fri Feb 26 09:32:46 2016
From: zpravaiz at aus.edu (Zafar Pravaiz)
Date: Fri, 26 Feb 2016 21:32:46 +0400
Subject: [Bro] NO DHCP.log
In-Reply-To: <20160226171835.GC60195@wifi110.sys.ICSI.Berkeley.EDU>
References: <0295DF8E-215A-42B2-BE76-A567B2BE7C45@aus.edu>
	<20160226171835.GC60195@wifi110.sys.ICSI.Berkeley.EDU>
Message-ID: <30069FAE-3BC2-42A2-AE72-F8E0CCE2F855@aus.edu>


> On Feb 26, 2016, at 9:18 PM, Johanna Amann <johanna at icir.org> wrote:
> 
> Hello,
> 
> On Fri, Feb 26, 2016 at 10:00:25AM +0400, Zafar Pravaiz wrote:
>> I am running SO 14.04. This is just capturing DNS and DHCP traffic on a
>> span port. Recently i ran soup and reboot the box. After that i have
>> noticed no DHCP log is showing up in bro log. i can see known_services
>> shows DHCP as service but there no dhcp.log file being generate.  Any
>> clue what went wrong? 
> 
> On a first glance I do not really have any idea what went wrong, but there
> are a few things to check -
> 
> * just to verify, dns.log is still being written correctly?
> 

Yes dns.log being update as expected. 

> * could you check that you see dhcp connections in conn.log? They should
>  be tagged with dhcp in the service field.
> 

yes i can see conn.log getting entries for DHCP 

> and
> 
> * could you verify that loaded_scripts.log contains
>  scripts/base/protocols/dhcp?
> 

These are the scripts are being loaded

   /opt/bro/share/bro/base/bif/plugins/Bro_DHCP.events.bif.bro
  /opt/bro/share/bro/base/protocols/dhcp/__load__.bro
    /opt/bro/share/bro/base/protocols/dhcp/consts.bro
    /opt/bro/share/bro/base/protocols/dhcp/main.bro
      /opt/bro/share/bro/base/protocols/dhcp/utils.bro

> Johanna


From johanna at icir.org  Fri Feb 26 09:39:56 2016
From: johanna at icir.org (Johanna Amann)
Date: Fri, 26 Feb 2016 09:39:56 -0800
Subject: [Bro] NO DHCP.log
In-Reply-To: <30069FAE-3BC2-42A2-AE72-F8E0CCE2F855@aus.edu>
References: <0295DF8E-215A-42B2-BE76-A567B2BE7C45@aus.edu>
	<20160226171835.GC60195@wifi110.sys.ICSI.Berkeley.EDU>
	<30069FAE-3BC2-42A2-AE72-F8E0CCE2F855@aus.edu>
Message-ID: <20160226173956.GA72283@wifi110.sys.ICSI.Berkeley.EDU>

Ok, with all that - I am basically out of ideas. Can you check that
local.bro does not contain anything that might prevent dhcp.log from being
written (the line would have DHCP::LOG in it). But that is very unlikely.

If that yields nothing - could you perhaps capture a tiny snippet of the
dhcp traffic with tcpdump, just run bro on the command line and see if
that generates dhcp.log? If no - could you potentially (privately) send me
a small amount of that traffic?

Johanna

On Fri, Feb 26, 2016 at 09:32:46PM +0400, Zafar Pravaiz wrote:
> 
> > On Feb 26, 2016, at 9:18 PM, Johanna Amann <johanna at icir.org> wrote:
> > 
> > Hello,
> > 
> > On Fri, Feb 26, 2016 at 10:00:25AM +0400, Zafar Pravaiz wrote:
> >> I am running SO 14.04. This is just capturing DNS and DHCP traffic on a
> >> span port. Recently i ran soup and reboot the box. After that i have
> >> noticed no DHCP log is showing up in bro log. i can see known_services
> >> shows DHCP as service but there no dhcp.log file being generate.  Any
> >> clue what went wrong? 
> > 
> > On a first glance I do not really have any idea what went wrong, but there
> > are a few things to check -
> > 
> > * just to verify, dns.log is still being written correctly?
> > 
> 
> Yes dns.log being update as expected. 
> 
> > * could you check that you see dhcp connections in conn.log? They should
> >  be tagged with dhcp in the service field.
> > 
> 
> yes i can see conn.log getting entries for DHCP 
> 
> > and
> > 
> > * could you verify that loaded_scripts.log contains
> >  scripts/base/protocols/dhcp?
> > 
> 
> These are the scripts are being loaded
> 
>    /opt/bro/share/bro/base/bif/plugins/Bro_DHCP.events.bif.bro
>   /opt/bro/share/bro/base/protocols/dhcp/__load__.bro
>     /opt/bro/share/bro/base/protocols/dhcp/consts.bro
>     /opt/bro/share/bro/base/protocols/dhcp/main.bro
>       /opt/bro/share/bro/base/protocols/dhcp/utils.bro
> 
> > Johanna
> 
> 

From aniketpsavanand at gmail.com  Fri Feb 26 22:41:19 2016
From: aniketpsavanand at gmail.com (Aniket Savanand)
Date: Fri, 26 Feb 2016 22:41:19 -0800
Subject: [Bro] Error while debugging bro in main.cc
Message-ID: <CAG2EY9jeO3io0Cju5LZRBCeeGhjgLqDjd3Q4JNJO+RFWn_CTJg@mail.gmail.com>

Hello,

I am trying to debug bro in Eclipse CDT .

At the following line in main.cc,  I am getting an error as "function init
could not be resolved"

// Must come after hash initialization.
binpac::init();

I ran  ./configure at home directory to generate makefile.
and then I have imported source folder in CDT.

Please direct my where I might be going wrong.


Thanks
Aniket Savanand
MS San Jose State University

-- 
*Regards, *
*Aniket Savanand,*
*MS Software Engineering 2016,*
*San Jose State University, CA*
*Email <aniket.savanand at sjsu.edu> **Cellphone- +1-669-226-8162*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160226/8c80af06/attachment.html 

From josh.guild at morphick.com  Mon Feb 29 09:44:14 2016
From: josh.guild at morphick.com (Josh Guild)
Date: Mon, 29 Feb 2016 12:44:14 -0500
Subject: [Bro] Bro handling of Microsoft BITS traffic
Message-ID: <CALiPqw6DYrVaA91iu9JtEzvie2MwQ7ENXpUX1GwkM0b6q3a=6A@mail.gmail.com>

Hey all,

I have a question about how Bro handles Micorsoft BITS (Background
Intelligent Transfer Service) traffic since the file is only partially
downloaded in the session it's monitoring. We've seen some traffic and it
looks like Bro just shows as an incomplete file and doesn't carve it
properly.

Is there anything we can do to mitigate this?

-- 
Josh Guild
Network Intelligence Analyst
<https://twitter.com/stay_spooky> <https://keybase.io/joshuaguild>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160229/81f466ab/attachment.html 

From cbarbaro at cert.unlp.edu.ar  Mon Feb 29 10:19:52 2016
From: cbarbaro at cert.unlp.edu.ar (Cristian Daniel Barbaro)
Date: Mon, 29 Feb 2016 15:19:52 -0300
Subject: [Bro] About Bro Cluster Configuration
Message-ID: <56D48BC8.40107@cert.unlp.edu.ar>

Hello, I have a question about Bro Cluster architecture. By default, the
cluster architecture has a frontend listening to a high-speed link;
spliting traffic to each  worker and to finally all workers information
be administered by a manager using a proxy, etc.

What we want to do is to have several workers analysing different
networks segments and that each of those workers communicate with a
manager, who will be responsible for managing all information and of
course, enabling a centralized administration of workers configuration.

Is it possible to do this?

Thanks and regards.

-- 
Cristian Daniel Barbaro
CERTUNLP
--


From vladg at illinois.edu  Mon Feb 29 11:07:52 2016
From: vladg at illinois.edu (Vlad Grigorescu)
Date: Mon, 29 Feb 2016 13:07:52 -0600
Subject: [Bro] About Bro Cluster Configuration
In-Reply-To: <56D48BC8.40107@cert.unlp.edu.ar>
References: <56D48BC8.40107@cert.unlp.edu.ar>
Message-ID: <m0y4a35o8n.fsf@vladg.net>

Yes, this should be fine. The standard architecture is meant to
provide load-balancing for monitoring points that are too large for a
single system to monitor (> 4-5 Gbps with modern, beefy hardware). As
long as each Bro worker is seeing both the upflow and downflow of each
connection it sees, the cluster doesn't care about which worker sees
which subset of the overall traffic.

  --Vlad

Cristian Daniel Barbaro <cbarbaro at cert.unlp.edu.ar> writes:

> Hello, I have a question about Bro Cluster architecture. By default, the
> cluster architecture has a frontend listening to a high-speed link;
> spliting traffic to each  worker and to finally all workers information
> be administered by a manager using a proxy, etc.
>
> What we want to do is to have several workers analysing different
> networks segments and that each of those workers communicate with a
> manager, who will be responsible for managing all information and of
> course, enabling a centralized administration of workers configuration.
>
> Is it possible to do this?
>
> Thanks and regards.
>
> -- 
> Cristian Daniel Barbaro
> CERTUNLP
> --
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 800 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160229/2c978406/attachment.bin 

From seth at icir.org  Mon Feb 29 12:51:56 2016
From: seth at icir.org (Seth Hall)
Date: Mon, 29 Feb 2016 15:51:56 -0500
Subject: [Bro] Bro handling of Microsoft BITS traffic
In-Reply-To: <CALiPqw6DYrVaA91iu9JtEzvie2MwQ7ENXpUX1GwkM0b6q3a=6A@mail.gmail.com>
References: <CALiPqw6DYrVaA91iu9JtEzvie2MwQ7ENXpUX1GwkM0b6q3a=6A@mail.gmail.com>
Message-ID: <AE613688-1ACD-4A4D-AB2A-E6AD0F49EB5F@icir.org>


> On Feb 29, 2016, at 12:44 PM, Josh Guild <josh.guild at morphick.com> wrote:
> 
> I have a question about how Bro handles Micorsoft BITS (Background Intelligent Transfer Service) traffic since the file is only partially downloaded in the session it's monitoring. We've seen some traffic and it looks like Bro just shows as an incomplete file and doesn't carve it properly. 

There is actually some support in the file analysis code to handle this type of situation.  It *probably* already works if the BITS traffic you are seeing is in a pcap file or seen by a single Bro worker.  We don't have anything in place yet to do extraction from traffic hitting multiple workers.  This is also a bit of a weird feature because none of the other network monitoring software that's around does this.

I would be interested in how you see Bro handling the traffic if you have a pcap file with the full transfer happening over multiple connections to see if Bro extracts the file correctly.  It's possible that they've changed things a bit I worked with it last.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From dnj0496 at gmail.com  Mon Feb 29 16:38:59 2016
From: dnj0496 at gmail.com (Dk Jack)
Date: Mon, 29 Feb 2016 16:38:59 -0800
Subject: [Bro] multiple interfaces using PFRing
Message-ID: <CAGATen7AMLrqhsZe87uzrSHVF9kJvmTeui+kRTKxOaf1WJ+TLg@mail.gmail.com>

Hi,
I trying to figure out how to write the node.cfg to enable pf-ring to
monitor multiple interfaces.
Is the following config correct if I am monitoring two interfaces with four
workers? Thanks.

Dj

[manager]
type=manager
host=localhost
#

[proxy-1]
type=proxy
host=localhost
#

[worker-eth0]
type=worker
host=localhost
interface=*eth0*
lb_method=pf_ring
lb_procs=*2*

[worker-eth1]
type=worker
host=localhost
interface=*eth1*
lb_method=pf_ring
lb_procs=*2*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160229/2199676f/attachment.html 

From mgill6 at student.concordia.ab.ca  Mon Feb 29 18:17:57 2016
From: mgill6 at student.concordia.ab.ca (Manmeet Gill)
Date: Mon, 29 Feb 2016 19:17:57 -0700
Subject: [Bro] Can Bro detect a traffic difference,
	according to days and time.
Message-ID: <CAMFH7YR1oUova0vz0EZnWsVitZPWmOYnjcrC6kSU37SH==mJtw@mail.gmail.com>

Hi everybody,
I will give a scenario let me know is it possible using Bro ids or not.
If there is a traffic of tcp,udp,icmp,https,smtp and dns,
80%,50%,30%,70%,80% and 60% respectively during working days(mon-fri)(from
10am-6pm) which we can say is a normal traffic. and if these traffic
differs with 10% below or above for each protocol.  then alarm should be
triggered, similarly with (off hours 7pm to 9am) if we see same amount of
traffic,  alarm should be triggered. Is it possible with Bro to make this
type of scenario detectable.

--
Manmeet Singh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160229/1493bd86/attachment.html 

From mz89924 at 126.com  Mon Feb 29 22:53:22 2016
From: mz89924 at 126.com (mz)
Date: Tue, 1 Mar 2016 14:53:22 +0800
Subject: [Bro] How use logs-to-elasticsearch.bro
Message-ID: <000d01d17387$0c4f2e30$24ed8a90$@126.com>

Dear all

I would like to use logs-to-elasticsearch.bro this script to log the Bro
Elasticsearch?

 

My Bro Version: 2.4.1

 

1?Use this script is not you do not need logstash, Bro will be sent
directly to the log Elasticsearch?

 

2?I follow the official document: https:
//www.bro.org/sphinx/components/bro-plugins/elasticsearch/README.html is
configured in /usr/local/bro/share/bro/site/local. bro added @load
bro/ElasticSearch/logs-to-elasticsearch.bro. But it was not successful, in
addition to the configuration of the document still need additional
configuration?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160301/9dbeb3a3/attachment.html