From carlopmart at gmail.com Thu Nov 1 23:50:11 2012 From: carlopmart at gmail.com (C. L. Martinez) Date: Fri, 2 Nov 2012 06:50:11 +0000 Subject: [Bro] What does it means this alert? In-Reply-To: <0F79A3E4-7BEF-4D48-8CCF-C21E52A59FC8@icir.org> References: <0F79A3E4-7BEF-4D48-8CCF-C21E52A59FC8@icir.org> Message-ID: Sorry for the delayed response. I've been out of the office until today. Yes, nice is in my path: [root at nsm01 spool]# whereis nice nice: /bin/nice /usr/share/man/man1/nice.1.gz But the problem is with "time" command, as Chuck says in another thread: http://mailman.icsi.berkeley.edu/pipermail/bro/2012-October/005962.html In my broctl-config.sh appears this: bindir="/nsm/bro/bin" time="which: no time in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin)" I have changed to: time="/usr/bin/time" So far the message has not been repeated. On Wed, Oct 31, 2012 at 6:28 PM, Seth Hall wrote: > > On Oct 31, 2012, at 2:11 PM, C. L. Martinez wrote: > >> Yes, it is under /usr/bin. > > I think the problem is the nice command. Is that in your path? > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > From eureka386 at gmail.com Sun Nov 4 04:47:07 2012 From: eureka386 at gmail.com (G-Mail) Date: Sun, 4 Nov 2012 21:47:07 +0900 Subject: [Bro] How can i find some fields? Message-ID: <9417B968-67C3-46BB-89F8-DBCF299E0469@gmail.com> first of all, please understand my poor English. Because I'm not live in English-speaking country. I'm interested in Bro and Bro scripts nowadays. It's very flexible. But, I don't know how to create a module's fields. For example, DNS module in bro has some fields of "Info". ===================== base/protocols/dns/main.bro module DNS; export { ## The DNS logging stream identifier. redef enum Log::ID += { LOG }; ## The record type which contains the column fields of the DNS log. type Info: record { ## The earliest time at which a DNS protocol message over the ## associated connection is observed. ts: time &log; ## A unique identifier of the connection over which DNS messages ## are being transferred. uid: string &log; ## The connection's 4-tuple of endpoint addresses/ports. id: conn_id &log; ## The transport layer protocol of the connection. proto: transport_proto &log; ???..last part omitted???? ===================== I have tried 'SMB' module. But I didn't know where i can get fields. here is my question. 1. Can I get protocol fields? (such as DNS, HTTP, SSL module's Info fields) 2. Where I should to find the fields? (source code?) 3. If it is not possible, how can some people write a bro script? like these https://gist.github.com/maxfeldman14/brospects/tree/4f9aee880234bd27d9fcab82a7410fa455ddaf42 https://github.com/sheharbano/scan/blob/master/scan.bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121104/fc4e8d6d/attachment.html From carlopmart at gmail.com Wed Nov 7 23:05:01 2012 From: carlopmart at gmail.com (C. L. Martinez) Date: Thu, 8 Nov 2012 07:05:01 +0000 Subject: [Bro] Question about hourly reports sended by Bro-IDS Message-ID: Hi all, Is it possible to change the hourly reports by daily reports? Thanks. From sconzo at visiblerisk.com Mon Nov 12 18:19:08 2012 From: sconzo at visiblerisk.com (Mike Sconzo) Date: Mon, 12 Nov 2012 20:19:08 -0600 Subject: [Bro] Graylog2 Message-ID: If anybody is using or looking at using Graylog2 for logging I've got some of the bro message types parsed out and would be happy to share. It's not directly related to Bro, but it could save somebody some time. -=Mike -- cat ~/.bash_history > documentation.txt -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121112/6d613afe/attachment.html From marcos.e.rodriguez at gmail.com Mon Nov 12 18:29:55 2012 From: marcos.e.rodriguez at gmail.com (Marcos Rodriguez) Date: Mon, 12 Nov 2012 21:29:55 -0500 Subject: [Bro] Graylog2 In-Reply-To: References: Message-ID: On Mon, Nov 12, 2012 at 9:19 PM, Mike Sconzo wrote: > If anybody is using or looking at using Graylog2 for logging I've got some > of the bro message types parsed out and would be happy to share. It's not > directly related to Bro, but it could save somebody some time. > > -=Mike > Hi Mike, I've been meaning to look into that. Count me in on anything you're willing to share. cheers, marcos -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121112/77260097/attachment.html From tyler.schoenke at colorado.edu Tue Nov 13 07:07:34 2012 From: tyler.schoenke at colorado.edu (Tyler T. Schoenke) Date: Tue, 13 Nov 2012 08:07:34 -0700 Subject: [Bro] Question about hourly reports sended by Bro-IDS In-Reply-To: References: Message-ID: <50A26236.3070102@colorado.edu> Short answer: For Alarm Summary, yes, for Connection Summary no. Alarm summary is modifiable if you download the git copy of Bro or wait for the Bro 2.2 release. Connection Summary emails are tied to log rotation, and happen hourly. If you don't have a lot of logs, you could switch to daily rotation to change the connection summary interval to daily. http://tracker.bro-ids.org/bro/ticket/824 Tyler -- Tyler Schoenke Network Security Manager IT Security Office University of Colorado at Boulder On 11/8/12 12:05 AM, C. L. Martinez wrote: > Hi all, > > Is it possible to change the hourly reports by daily reports? > > Thanks. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From carlopmart at gmail.com Tue Nov 13 07:33:08 2012 From: carlopmart at gmail.com (C. L. Martinez) Date: Tue, 13 Nov 2012 15:33:08 +0000 Subject: [Bro] Question about hourly reports sended by Bro-IDS In-Reply-To: <50A26236.3070102@colorado.edu> References: <50A26236.3070102@colorado.edu> Message-ID: On Tue, Nov 13, 2012 at 3:07 PM, Tyler T. Schoenke wrote: > Short answer: For Alarm Summary, yes, for Connection Summary no. > Alarm summary is modifiable if you download the git copy of Bro or wait > for the Bro 2.2 release. Connection Summary emails are tied to log > rotation, and happen hourly. If you don't have a lot of logs, you could > switch to daily rotation to change the connection summary interval to daily. > > http://tracker.bro-ids.org/bro/ticket/824 > > Tyler > > -- Many thanks Tyler ... I will try it ... From hlin33 at illinois.edu Tue Nov 13 07:55:52 2012 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Tue, 13 Nov 2012 09:55:52 -0600 Subject: [Bro] Hui Lin_Input framework to load data from files in remote machines Message-ID: Hi, >From the documentation, input framework can input data from the formatted text file which is located in the same machine with the bro instance. I am wondering whether now we can directly use input framework to input data that is located in remote machines. Best, Hui Lin -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121113/0f8ca4a8/attachment.html From bernhard at ICSI.Berkeley.EDU Tue Nov 13 08:29:04 2012 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Tue, 13 Nov 2012 08:29:04 -0800 Subject: [Bro] Hui Lin_Input framework to load data from files in remote machines In-Reply-To: References: Message-ID: <8F81DBDA-EEF6-4E5A-964D-50279FAB8182@icsi.berkeley.edu> On Nov 13, 2012, at 7:55 AM, "Hui Lin (Hugo) " wrote: > From the documentation, input framework can input data from the formatted text file which is located in the same machine with the bro instance. > > I am wondering whether now we can directly use input framework to input data that is located in remote machines. You can load data that is accessible on the file system(s) of the machine which is running the bro instance. Bernhard -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121113/dba9f36d/attachment.html From scastle at bouldercounty.org Fri Nov 16 15:40:44 2012 From: scastle at bouldercounty.org (Castle, Shane) Date: Fri, 16 Nov 2012 23:40:44 +0000 Subject: [Bro] Bro and unusual http ports Message-ID: <21DD7C64179C9843B756C6DD491634DB252BF072@Mailbox1.boco.co.boulder.co.us> I have a device inside that communicates using a weird http port (3000/tcp). I have verified that it is not malicious but it annoys me, and I'd like to be able to track what it does using Bro. Unfortunately, Bro is not recognizing its traffic as http. I've tried adding the port to likely_server_ports but to no avail. The port definitions in the base http scripts are not redef-able, and I seem to have hit my limit in tweaking Bro to make it decode this traffic. What am I missing? BTW this is Bro 2.0 (yes I know, consider me chastised) but the scripts seem to be the same in 2.1. -- Shane Castle Data Security Mgr, Boulder County IT From seth at icir.org Fri Nov 16 18:13:37 2012 From: seth at icir.org (Seth Hall) Date: Fri, 16 Nov 2012 21:13:37 -0500 Subject: [Bro] Bro and unusual http ports In-Reply-To: <21DD7C64179C9843B756C6DD491634DB252BF072@Mailbox1.boco.co.boulder.co.us> References: <21DD7C64179C9843B756C6DD491634DB252BF072@Mailbox1.boco.co.boulder.co.us> Message-ID: On Nov 16, 2012, at 6:40 PM, "Castle, Shane" wrote: > What am I missing? Could you send me a packet capture? I'm curious as to why the signature isn't matching. > BTW this is Bro 2.0 (yes I know, consider me chastised) but the scripts seem to be the same in 2.1. Hah! Yeah, not much difference between 2.0 and 2.1 with this, the change to it will be coming with 2.2. :) If you want to add port 3000/tcp as an HTTP port you can add this to a script? add dpd_config[ANALYZER_HTTP]$ports[3000/tcp]; .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From mkolkebeck at gmail.com Fri Nov 16 20:11:03 2012 From: mkolkebeck at gmail.com (Mike Kolkebeck) Date: Fri, 16 Nov 2012 22:11:03 -0600 Subject: [Bro] SSH Login Notices - Filter out internal to internal connections Message-ID: SSH::Login Notices for internal to internal connections can get fairly noisy. What is the most efficient way to filter out these notices for internal to internal without filtering for external connections? I was thinking of ignoring the SSH::Login notices altogether, but then I believe I need to add a new Notice Type and fire a new notice on event SSH::heuristic_successful_login. See example code below. Is there a more efficient way of doing this? I know editing the base ssh bro script is a big no-no. Thanks! Mike redef enum Notice::Type += { Login_Success }; # This is our list of internal addresses to exclude global ssh_ignore: set[subnet] = { 192.168.1.0/24, # internal 1 10.0.0.0/8, # internal 2 }; # Ignore SSH::Login Notice Type redef Notice::ignored_types += { SSH::Login }; # Add new Notice Type to successful login event SSH::heuristic_successful_login(c: connection) &priority=0 { if ( c$id$resp_h !in ssh_ignore ) { NOTICE([$note=Login_Success, $msg="Heuristically detected successful SSH login.", $conn=c]); } } -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121116/104aceaa/attachment.html From seth at icir.org Fri Nov 16 20:46:59 2012 From: seth at icir.org (Seth Hall) Date: Fri, 16 Nov 2012 23:46:59 -0500 Subject: [Bro] SSH Login Notices - Filter out internal to internal connections In-Reply-To: References: Message-ID: <0876E9CD-3A65-4AB8-A947-663C88AE1B1F@icir.org> On Nov 16, 2012, at 11:11 PM, Mike Kolkebeck wrote: > SSH::Login Notices for internal to internal connections can get fairly noisy. > > What is the most efficient way to filter out these notices for internal to internal without filtering for external connections? redef Notice::policy += { [$pred(n: Notice::Info) = { return ( n$note == SSH::Login && Site::is_local_addr(n$id$orig_h) && Site::is_local_addr(n$id$resp_h) ); }, $priority=10, $halt=T] }; BTW, this answer is nasty and we're working now on making this generally easier for the next release. Your approach of generating your own notice works well too. I was considering removing the SSH::Login notice anyway. It's an anachronism of an older style of scripting and isn't so relevant anymore. Does anyone have any thoughts on the removal of the SSH::Login notice? Anyone actively use it? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From carlopmart at gmail.com Mon Nov 19 02:55:59 2012 From: carlopmart at gmail.com (C. L. Martinez) Date: Mon, 19 Nov 2012 10:55:59 +0000 Subject: [Bro] Bro and unusual http ports In-Reply-To: References: <21DD7C64179C9843B756C6DD491634DB252BF072@Mailbox1.boco.co.boulder.co.us> Message-ID: On Sat, Nov 17, 2012 at 2:13 AM, Seth Hall wrote: > > On Nov 16, 2012, at 6:40 PM, "Castle, Shane" wrote: > >> What am I missing? > > Could you send me a packet capture? I'm curious as to why the signature isn't matching. > >> BTW this is Bro 2.0 (yes I know, consider me chastised) but the scripts seem to be the same in 2.1. > > > Hah! Yeah, not much difference between 2.0 and 2.1 with this, the change to it will be coming with 2.2. :) > > If you want to add port 3000/tcp as an HTTP port you can add this to a script? > > add dpd_config[ANALYZER_HTTP]$ports[3000/tcp]; > > .Seth > Interesting ... Seth, is it possible to add portranges and standalone ports too at the same time?? or is it needed to define every http port?? For example: add dpd_config[ANALYZER_HTTP]$ports[3001/tcp]; add dpd_config[ANALYZER_HTTP]$ports[3002/tcp]; add dpd_config[ANALYZER_HTTP]$ports[3003/tcp]; add dpd_config[ANALYZER_HTTP]$ports[3004/tcp]; add dpd_config[ANALYZER_HTTP]$ports[5000/tcp]; From jmellander at lbl.gov Mon Nov 19 11:18:15 2012 From: jmellander at lbl.gov (Jim Mellander) Date: Mon, 19 Nov 2012 11:18:15 -0800 Subject: [Bro] Bro & malloc implementations Message-ID: I wanted to share my experiences with bro and various malloc implementations. These are all running various versions of bro on SL 6.2 - a recompile of RedHat Enterprise Linux from Fermi Lab - https://www.scientificlinux.org/ 1. We are running a legacy Bro 1.5 installation currently supporting our old Instrumented SSHd infrastructure - http://code.google.com/p/auditing-sshd/ - we found that the standard malloc uses more memory than either of tcmalloc or jemalloc. I build bro with --enable-perftools to test tcmalloc, with a smaller memory footprint. To use jemalloc I just set LIBS=-ljemalloc - at this point we are using jemalloc, as it seems even a bit more memory thrifty than tcmalloc 2. As part of the Instrumented SSHd infrastructure, we also have a perl script (ssllogmux) that runs a select loop accepting connections from all the Instrumented SSHd clients - typically several thousand at once. This is also in the Instrumented SSHd distribution. Perl is compiled to not use its own malloc, but to use the system. However, using the system malloc, this program would freeze after a day or so of operation. By using LD_PRELOAD, we forced it to use alternate malloc libraries - under both tcmalloc and jemalloc, it runs reliably. 3. On another system, we ran Bro 2.0 (now running 2.1), and bro was (un)reliably freezing after a day or two - running but capturing no data when compiled with --enable-perftools - when compiled with --disable-perftools, it has been rock solid. I don't know enough yet about Cmake to have it use another malloc implementation easily - hopefully someone else knows how to do that - I want to test jemalloc So thats our current state of play - I would be interested in other folks experience. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121119/440efe24/attachment.html From seth at icir.org Mon Nov 19 19:07:40 2012 From: seth at icir.org (Seth Hall) Date: Mon, 19 Nov 2012 22:07:40 -0500 Subject: [Bro] Bro and unusual http ports In-Reply-To: References: <21DD7C64179C9843B756C6DD491634DB252BF072@Mailbox1.boco.co.boulder.co.us> Message-ID: <8CF3D5EE-0897-4CB9-827A-E84EFC83ABA6@icir.org> On Nov 19, 2012, at 5:55 AM, C. L. Martinez wrote: > Interesting ... Seth, is it possible to add portranges and standalone > ports too at the same time?? or is it needed to define every http > port?? For example: > > add dpd_config[ANALYZER_HTTP]$ports[3001/tcp]; You would need to add each port individually. Why would you want to add large swaths of ports though? Port number is only one of the heuristics used to find which analyzer to use on a connection. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From scastle at bouldercounty.org Tue Nov 20 10:27:00 2012 From: scastle at bouldercounty.org (Castle, Shane) Date: Tue, 20 Nov 2012 18:27:00 +0000 Subject: [Bro] Bro and unusual http ports In-Reply-To: References: <21DD7C64179C9843B756C6DD491634DB252BF072@Mailbox1.boco.co.boulder.co.us> Message-ID: <21DD7C64179C9843B756C6DD491634DB252C287A@Mailbox1.boco.co.boulder.co.us> This is giving me no joy on Bro 2.0, which barfs on seeing the "add" expression. When I try to emulate what base/protocols/http/main.bro does with redef dpd_config += { [[ANALYZER_HTTP, ANALYZER_HTTP_BINPAC]] = [$ports = set(3000/tcp)] }; I wind up having replaced the port list instead of adding to it. Also, the capture_filters var seems to need updating or replacing. Any ideas? I'm out. -- Shane Castle Data Security Mgr, Boulder County IT -----Original Message----- From: Seth Hall [mailto:seth at icir.org] Sent: Friday, November 16, 2012 19:14 To: Castle, Shane Cc: bro at bro-ids.org Subject: Re: [Bro] Bro and unusual http ports On Nov 16, 2012, at 6:40 PM, "Castle, Shane" wrote: > What am I missing? Could you send me a packet capture? I'm curious as to why the signature isn't matching. > BTW this is Bro 2.0 (yes I know, consider me chastised) but the scripts seem to be the same in 2.1. Hah! Yeah, not much difference between 2.0 and 2.1 with this, the change to it will be coming with 2.2. :) If you want to add port 3000/tcp as an HTTP port you can add this to a script... add dpd_config[ANALYZER_HTTP]$ports[3000/tcp]; .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Tue Nov 20 10:46:00 2012 From: seth at icir.org (Seth Hall) Date: Tue, 20 Nov 2012 13:46:00 -0500 Subject: [Bro] Bro and unusual http ports In-Reply-To: <21DD7C64179C9843B756C6DD491634DB252C287A@Mailbox1.boco.co.boulder.co.us> References: <21DD7C64179C9843B756C6DD491634DB252BF072@Mailbox1.boco.co.boulder.co.us> <21DD7C64179C9843B756C6DD491634DB252C287A@Mailbox1.boco.co.boulder.co.us> Message-ID: On Nov 20, 2012, at 1:27 PM, "Castle, Shane" wrote: > This is giving me no joy on Bro 2.0, which barfs on seeing the "add" expression. When I try to emulate what base/protocols/http/main.bro does with You need to make sure that add statement is outside of any event handler. Are you putting it in a bro_init event handler? (it helps if you give us the error message you got when something didn't work) :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From scastle at bouldercounty.org Tue Nov 20 12:25:19 2012 From: scastle at bouldercounty.org (Castle, Shane) Date: Tue, 20 Nov 2012 20:25:19 +0000 Subject: [Bro] Bro and unusual http ports In-Reply-To: References: <21DD7C64179C9843B756C6DD491634DB252BF072@Mailbox1.boco.co.boulder.co.us> <21DD7C64179C9843B756C6DD491634DB252C287A@Mailbox1.boco.co.boulder.co.us> Message-ID: <21DD7C64179C9843B756C6DD491634DB252C2C5C@Mailbox1.boco.co.boulder.co.us> I was putting this in site/local.bro. If I use the "add" expression, I get this sort of error message: error in /usr/local/share/bro/policy/misc/loaded-scripts.bro, line 3: syntax error, at or near "module" The name of the script is determined by whatever is "@load"ed after the occurrence of the "add". In the above, I put as first in local.bro. If I put it in last, I get error in /usr/local/share/bro/policy/frameworks/control/controllee.bro, line 15: syntax error, at or near "module" Of course, these are generated by "broctl check". -- Shane Castle Data Security Mgr, Boulder County IT -----Original Message----- From: Seth Hall [mailto:seth at icir.org] Sent: Tuesday, November 20, 2012 11:46 To: Castle, Shane Cc: bro at bro-ids.org List Subject: Re: [Bro] Bro and unusual http ports On Nov 20, 2012, at 1:27 PM, "Castle, Shane" wrote: > This is giving me no joy on Bro 2.0, which barfs on seeing the "add" expression. When I try to emulate what base/protocols/http/main.bro does with You need to make sure that add statement is outside of any event handler. Are you putting it in a bro_init event handler? (it helps if you give us the error message you got when something didn't work) :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Tue Nov 20 12:52:25 2012 From: seth at icir.org (Seth Hall) Date: Tue, 20 Nov 2012 15:52:25 -0500 Subject: [Bro] Bro and unusual http ports In-Reply-To: <21DD7C64179C9843B756C6DD491634DB252C2C5C@Mailbox1.boco.co.boulder.co.us> References: <21DD7C64179C9843B756C6DD491634DB252BF072@Mailbox1.boco.co.boulder.co.us> <21DD7C64179C9843B756C6DD491634DB252C287A@Mailbox1.boco.co.boulder.co.us> <21DD7C64179C9843B756C6DD491634DB252C2C5C@Mailbox1.boco.co.boulder.co.us> Message-ID: <2C0E42A5-6486-4A71-A00C-BF21EFEEBE35@icir.org> On Nov 20, 2012, at 3:25 PM, "Castle, Shane" wrote: > I was putting this in site/local.bro. If I use the "add" expression, I get this sort of error message: > > error in /usr/local/share/bro/policy/misc/loaded-scripts.bro, line 3: syntax error, at or near "module" Is it possible you forgot the semicolon at the end of the line? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From scastle at bouldercounty.org Tue Nov 20 12:54:41 2012 From: scastle at bouldercounty.org (Castle, Shane) Date: Tue, 20 Nov 2012 20:54:41 +0000 Subject: [Bro] Bro and unusual http ports In-Reply-To: <2C0E42A5-6486-4A71-A00C-BF21EFEEBE35@icir.org> References: <21DD7C64179C9843B756C6DD491634DB252BF072@Mailbox1.boco.co.boulder.co.us> <21DD7C64179C9843B756C6DD491634DB252C287A@Mailbox1.boco.co.boulder.co.us> <21DD7C64179C9843B756C6DD491634DB252C2C5C@Mailbox1.boco.co.boulder.co.us> <2C0E42A5-6486-4A71-A00C-BF21EFEEBE35@icir.org> Message-ID: <21DD7C64179C9843B756C6DD491634DB252C2D42@Mailbox1.boco.co.boulder.co.us> Nope, it was there. Just checked with the saved version that doesn't work. Maybe I've tickled a bug in 2.0? I've been putting off the 2.1 upgrade but maybe now I shouldn't. -- Shane Castle Data Security Mgr, Boulder County IT -----Original Message----- From: Seth Hall [mailto:seth at icir.org] Sent: Tuesday, November 20, 2012 13:52 To: Castle, Shane Cc: Seth Hall; bro at bro-ids.org List Subject: Re: [Bro] Bro and unusual http ports On Nov 20, 2012, at 3:25 PM, "Castle, Shane" wrote: > I was putting this in site/local.bro. If I use the "add" expression, I get this sort of error message: > > error in /usr/local/share/bro/policy/misc/loaded-scripts.bro, line 3: syntax error, at or near "module" Is it possible you forgot the semicolon at the end of the line? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From scastle at bouldercounty.org Tue Nov 20 15:58:13 2012 From: scastle at bouldercounty.org (Castle, Shane) Date: Tue, 20 Nov 2012 23:58:13 +0000 Subject: [Bro] Bro and unusual http ports In-Reply-To: <21DD7C64179C9843B756C6DD491634DB252C2D42@Mailbox1.boco.co.boulder.co.us> References: <21DD7C64179C9843B756C6DD491634DB252BF072@Mailbox1.boco.co.boulder.co.us> <21DD7C64179C9843B756C6DD491634DB252C287A@Mailbox1.boco.co.boulder.co.us> <21DD7C64179C9843B756C6DD491634DB252C2C5C@Mailbox1.boco.co.boulder.co.us> <2C0E42A5-6486-4A71-A00C-BF21EFEEBE35@icir.org> <21DD7C64179C9843B756C6DD491634DB252C2D42@Mailbox1.boco.co.boulder.co.us> Message-ID: <21DD7C64179C9843B756C6DD491634DB252C3081@Mailbox1.boco.co.boulder.co.us> I've made some progress here. I copied some of the components from ./base/protocols/http/main.bro and created a local script in ./site, naming it local-http-add.bro, and used a @load in local.bro for it. It's pretty short. --------------------------------------------------------------------- const ports = { 80/tcp, 81/tcp, 631/tcp, 1080/tcp, 3000/tcp, 3138/tcp, 8000/tcp, 8080/tcp, 8888/tcp, }; redef dpd_config += { [[ANALYZER_HTTP, ANALYZER_HTTP_BINPAC]] = [$ports = ports], }; redef capture_filters += { ["http"] = "tcp and port (80 or 81 or 631 or 1080 or 3000 or 3138 or 8000 or 8080 or 8888)" }; redef likely_server_ports += { 3000/tcp }; --------------------------------------------------------------------- Anyhow, this works, and does what I want. If there's a more succinct way of doing this I haven't figured it out. -- Shane Castle Data Security Mgr, Boulder County IT -----Original Message----- From: bro-bounces at bro-ids.org [mailto:bro-bounces at bro-ids.org] On Behalf Of Castle, Shane Sent: Tuesday, November 20, 2012 13:55 To: Seth Hall Cc: bro at bro-ids.org List Subject: Re: [Bro] Bro and unusual http ports Nope, it was there. Just checked with the saved version that doesn't work. Maybe I've tickled a bug in 2.0? I've been putting off the 2.1 upgrade but maybe now I shouldn't. -- Shane Castle Data Security Mgr, Boulder County IT -----Original Message----- From: Seth Hall [mailto:seth at icir.org] Sent: Tuesday, November 20, 2012 13:52 To: Castle, Shane Cc: Seth Hall; bro at bro-ids.org List Subject: Re: [Bro] Bro and unusual http ports On Nov 20, 2012, at 3:25 PM, "Castle, Shane" wrote: > I was putting this in site/local.bro. If I use the "add" expression, I get this sort of error message: > > error in /usr/local/share/bro/policy/misc/loaded-scripts.bro, line 3: syntax error, at or near "module" Is it possible you forgot the semicolon at the end of the line? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From mkolkebeck at gmail.com Wed Nov 28 19:29:13 2012 From: mkolkebeck at gmail.com (Mike Kolkebeck) Date: Wed, 28 Nov 2012 21:29:13 -0600 Subject: [Bro] Crash on SMB Analyzer - Tree Connect AndX Message-ID: Bro (2.1) crashes when I attempt to store the path of event smb_com_tree_connect_andx, which is documented as a string variable, to the Info record's smb_share, which I declared as a string variable. The stderr.log included below seems to indicate that the SMB Analyzer is interpreting the path string as a record, not sure which kind. I've attempted to escape the string, but this doesn't seem to work. Is this a known bug? Does anyone know of another event that would be better suited for identifying the share name, or is there any other easy workaround for this event? Thanks! Mike Below is a sample of the stderr.log output: --------- 1354158536.204142 fatal error in : Val::CONVERTER (record/string) ([flags=8, password=P , path=\\myhostname\IPC$, service=?????]) Below is the code snippet: ---------- event smb_com_tree_connect_andx(c: connection, hdr: smb_hdr, path: string, service: string) &priority=5 { set_session(c,hdr); local path_name = escape_string(path); c$smb$smb_share = path_name; } -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121128/40e1e2fa/attachment.html From seth at icir.org Wed Nov 28 20:35:56 2012 From: seth at icir.org (Seth Hall) Date: Wed, 28 Nov 2012 23:35:56 -0500 Subject: [Bro] Crash on SMB Analyzer - Tree Connect AndX In-Reply-To: References: Message-ID: <2C673CCC-523E-482A-99AC-690C2D9DD860@icir.org> On Nov 28, 2012, at 10:29 PM, Mike Kolkebeck wrote: > Is this a known bug? Does anyone know of another event that would be better suited for identifying the share name, or is there any other easy workaround for this event? There has been a lot of rework done on the smb analyzer that hasn't been released yet. I know that I fixed a lot of bugs existing in the existing analyzer you're working with. Unfortunately there probably isn't much of a way around the problem you're running into unless you want to try my in-progress branch. I assume you've written all of the scripts to enable the SMB analyzer and add the c$smb field? Would you be interested in putting the scripts up somewhere? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From mkolkebeck at gmail.com Wed Nov 28 21:33:53 2012 From: mkolkebeck at gmail.com (Mike Kolkebeck) Date: Wed, 28 Nov 2012 23:33:53 -0600 Subject: [Bro] Crash on SMB Analyzer - Tree Connect AndX In-Reply-To: <2C673CCC-523E-482A-99AC-690C2D9DD860@icir.org> References: <2C673CCC-523E-482A-99AC-690C2D9DD860@icir.org> Message-ID: <87A00604-E28A-4C34-8036-17F115957336@gmail.com> On Nov 28, 2012, at 10:35 PM, Seth Hall wrote: > > On Nov 28, 2012, at 10:29 PM, Mike Kolkebeck wrote: > >> Is this a known bug? Does anyone know of another event that would be better suited for identifying the share name, or is there any other easy workaround for this event? > > There has been a lot of rework done on the smb analyzer that hasn't been released yet. I know that I fixed a lot of bugs existing in the existing analyzer you're working with. Unfortunately there probably isn't much of a way around the problem you're running into unless you want to try my in-progress branch. If I use your in-progress branch, would this impact other Bro functionality, or could I isolate the update to just smb analyzer functionality? > I assume you've written all of the scripts to enable the SMB analyzer and add the c$smb field? Yes, I followed a few of the base protocol bro scripts as a guide, mostly leveraging the start of the SSH analyzer scripts. > Would you be interested in putting the scripts up somewhere? It's still a crude work in progress, but here is the full bro script that I'm currently using: ------ ##! Custom SMB analysis script. @load base/frameworks/notice @load base/utils/site @load base/utils/thresholds @load base/utils/conn-ids @load base/utils/directions-and-hosts module SMB; export { ## The SMB protocol logging stream identifier. redef enum Log::ID += { LOG }; type Info: record { ## Time when the SMB connection began. ts: time &log; ## Unique ID for the connection. uid: string &log; ## The connection's 4-tuple of endpoint addresses/ports. id: conn_id &log; ## The connection's smb_hdr variables status: count &log &optional; smb_flags: count &log &optional; smb_flags2: count &log &optional; ## The connection's tree path (at the share level) smb_share: string &log &optional; ## SMB path (past the share level) smb_path: string &log &default="\\"; }; global paths: set[string]; } const smbports = { 135/tcp, 137/tcp, 138/tcp, 139/tcp, 445/tcp }; # Configure DPD and the packet filter redef capture_filters += { ["msrpc"] = "tcp port 135", ["netbios-ns"] = "tcp port 137", ["netbios-ds"] = "tcp port 138", ["netbios"] = "tcp port 139", ["smb"] = "tcp port 445", }; redef dpd_config += { [ANALYZER_SMB] = [$ports = smbports] }; redef likely_server_ports += { 445/tcp }; redef record connection += { smb: Info &optional; }; event bro_init() &priority=5 { Log::create_stream(SMB::LOG, [$columns=Info]); } function set_session(c: connection, hdr: smb_hdr) { if ( ! c?$smb ) { c$smb = [$ts=network_time(), $id=c$id, $uid=c$uid]; c$smb$status = hdr$status; c$smb$smb_flags = hdr$flags; c$smb$smb_flags2 = hdr$flags2; } } event smb_com_tree_connect_andx(c: connection, hdr: smb_hdr, path: string, service: string) &priority=5 { set_session(c,hdr); local path_name = escape_string(path); c$smb$smb_share = path_name; } event smb_com_nt_create_andx(c: connection, hdr: smb_hdr, name: string) &priority=0 { set_session(c,hdr); c$smb$ts=network_time(); # If the path has changed, then log the new name, otherwise skip it (may need to revisit) if ( name !in paths ) { add paths[name]; c$smb$smb_path = gsub(name,/ /,"%20"); Log::write(SMB::LOG, c$smb); } } --------- Thanks! Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121128/a749d2e6/attachment.html From mcholste at gmail.com Wed Nov 28 21:46:19 2012 From: mcholste at gmail.com (Martin Holste) Date: Wed, 28 Nov 2012 23:46:19 -0600 Subject: [Bro] Crash on SMB Analyzer - Tree Connect AndX In-Reply-To: <87A00604-E28A-4C34-8036-17F115957336@gmail.com> References: <2C673CCC-523E-482A-99AC-690C2D9DD860@icir.org> <87A00604-E28A-4C34-8036-17F115957336@gmail.com> Message-ID: I like what I'm seeing on this new SMB work! On Wed, Nov 28, 2012 at 11:33 PM, Mike Kolkebeck wrote: > > On Nov 28, 2012, at 10:35 PM, Seth Hall wrote: > > > On Nov 28, 2012, at 10:29 PM, Mike Kolkebeck wrote: > > Is this a known bug? Does anyone know of another event that would be > better suited for identifying the share name, or is there any other easy > workaround for this event? > > > There has been a lot of rework done on the smb analyzer that hasn't been > released yet. I know that I fixed a lot of bugs existing in the existing > analyzer you're working with. Unfortunately there probably isn't much of a > way around the problem you're running into unless you want to try my > in-progress branch. > > > If I use your in-progress branch, would this impact other Bro > functionality, or could I isolate the update to just smb analyzer > functionality? > > I assume you've written all of the scripts to enable the SMB analyzer and > add the c$smb field? > > > Yes, I followed a few of the base protocol bro scripts as a guide, mostly > leveraging the start of the SSH analyzer scripts. > > Would you be interested in putting the scripts up somewhere? > > > It's still a crude work in progress, but here is the full bro script that > I'm currently using: > ------ > > ##! Custom SMB analysis script. > > @load base/frameworks/notice > @load base/utils/site > @load base/utils/thresholds > @load base/utils/conn-ids > @load base/utils/directions-and-hosts > > module SMB; > > export { > ## The SMB protocol logging stream identifier. > redef enum Log::ID += { LOG }; > > type Info: record { > ## Time when the SMB connection began. > ts: time &log; > ## Unique ID for the connection. > uid: string &log; > ## The connection's 4-tuple of endpoint addresses/ports. > id: conn_id &log; > ## The connection's smb_hdr variables > status: count &log &optional; > smb_flags: count &log &optional; > smb_flags2: count &log &optional; > ## The connection's tree path (at the share level) > smb_share: string &log &optional; > ## SMB path (past the share level) > smb_path: string &log &default="\\"; > }; > > global paths: set[string]; > } > > const smbports = { > 135/tcp, 137/tcp, 138/tcp, 139/tcp, 445/tcp > }; > > # Configure DPD and the packet filter > redef capture_filters += { > ["msrpc"] = "tcp port 135", > ["netbios-ns"] = "tcp port 137", > ["netbios-ds"] = "tcp port 138", > ["netbios"] = "tcp port 139", > ["smb"] = "tcp port 445", > }; > redef dpd_config += { [ANALYZER_SMB] = [$ports = smbports] }; > redef likely_server_ports += { 445/tcp }; > > redef record connection += { > smb: Info &optional; > }; > > event bro_init() &priority=5 > { > Log::create_stream(SMB::LOG, [$columns=Info]); > } > > function set_session(c: connection, hdr: smb_hdr) > { > if ( ! c?$smb ) > { > c$smb = [$ts=network_time(), $id=c$id, $uid=c$uid]; > c$smb$status = hdr$status; > c$smb$smb_flags = hdr$flags; > c$smb$smb_flags2 = hdr$flags2; > } > } > > event smb_com_tree_connect_andx(c: connection, hdr: smb_hdr, path: string, > service: string) &priority=5 > { > set_session(c,hdr); > local path_name = escape_string(path); > c$smb$smb_share = path_name; > } > > event smb_com_nt_create_andx(c: connection, hdr: smb_hdr, name: string) > &priority=0 > { > set_session(c,hdr); > > c$smb$ts=network_time(); > > # If the path has changed, then log the new name, otherwise skip it (may > need to revisit) > if ( name !in paths ) > { > add paths[name]; > c$smb$smb_path = gsub(name,/ /,"%20"); > Log::write(SMB::LOG, c$smb); > } > } > > --------- > > Thanks! > Mike > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121128/e632a503/attachment.html From seth at icir.org Thu Nov 29 05:22:08 2012 From: seth at icir.org (Seth Hall) Date: Thu, 29 Nov 2012 08:22:08 -0500 Subject: [Bro] Crash on SMB Analyzer - Tree Connect AndX In-Reply-To: <87A00604-E28A-4C34-8036-17F115957336@gmail.com> References: <2C673CCC-523E-482A-99AC-690C2D9DD860@icir.org> <87A00604-E28A-4C34-8036-17F115957336@gmail.com> Message-ID: <9FEBE83B-ED7E-4A5B-A121-B60050630698@icir.org> On Nov 29, 2012, at 12:33 AM, Mike Kolkebeck wrote: > If I use your in-progress branch, would this impact other Bro functionality, or could I isolate the update to just smb analyzer functionality? Probably, but I really need to merge the master branch into that branch so it should only be the SMB fixes in that branch that differ from master. There may be some merge conflicts, I don't really know. Actually? I did some work last night and I'll be pushing out some changes to my topic/seth/smb-smb2-work branch that fully updates it to master in a few minutes (there were a number of merge conflicts). >> Would you be interested in putting the scripts up somewhere? >> > It's still a crude work in progress, but here is the full bro script that I'm currently using: Cool, nice. :) If you -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Thu Nov 29 05:58:31 2012 From: seth at icir.org (Seth Hall) Date: Thu, 29 Nov 2012 08:58:31 -0500 Subject: [Bro] Crash on SMB Analyzer - Tree Connect AndX In-Reply-To: <9FEBE83B-ED7E-4A5B-A121-B60050630698@icir.org> References: <2C673CCC-523E-482A-99AC-690C2D9DD860@icir.org> <87A00604-E28A-4C34-8036-17F115957336@gmail.com> <9FEBE83B-ED7E-4A5B-A121-B60050630698@icir.org> Message-ID: <5E98D74B-F780-427D-8E93-20A50C1ACC0F@icir.org> On Nov 29, 2012, at 8:22 AM, Seth Hall wrote: > Actually? I did some work last night and I'll be pushing out some changes to my topic/seth/smb-smb2-work branch that fully updates it to master in a few minutes (there were a number of merge conflicts). Be a little carefully with this branch at the moment, I actually have file extraction turned on by default (yes, I went ahead and built file extraction and identification into it :) ). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From sconzo at visiblerisk.com Thu Nov 29 08:33:57 2012 From: sconzo at visiblerisk.com (Mike Sconzo) Date: Thu, 29 Nov 2012 10:33:57 -0600 Subject: [Bro] Crash on SMB Analyzer - Tree Connect AndX In-Reply-To: <5E98D74B-F780-427D-8E93-20A50C1ACC0F@icir.org> References: <2C673CCC-523E-482A-99AC-690C2D9DD860@icir.org> <87A00604-E28A-4C34-8036-17F115957336@gmail.com> <9FEBE83B-ED7E-4A5B-A121-B60050630698@icir.org> <5E98D74B-F780-427D-8E93-20A50C1ACC0F@icir.org> Message-ID: I've been seeing these too for whatever it's worth. 1352906659.976267 fatal error in : Val::CONST_ACCESSOR (record/string) ([stime=1352906659.942897, uid=V40HxsmOMT5, message=HTTP 1.1 Without Referer Client Header]) I was going to attempt to re-write the script in a different way to see if I can get it to stop tickling whatever it is. On Thu, Nov 29, 2012 at 7:58 AM, Seth Hall wrote: > > On Nov 29, 2012, at 8:22 AM, Seth Hall wrote: > > > Actually? I did some work last night and I'll be pushing out some > changes to my topic/seth/smb-smb2-work branch that fully updates it to > master in a few minutes (there were a number of merge conflicts). > > > Be a little carefully with this branch at the moment, I actually have file > extraction turned on by default (yes, I went ahead and built file > extraction and identification into it :) ). > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- cat ~/.bash_history > documentation.txt -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121129/47b8ea7e/attachment.html From sconzo at visiblerisk.com Thu Nov 29 11:03:57 2012 From: sconzo at visiblerisk.com (Mike Sconzo) Date: Thu, 29 Nov 2012 13:03:57 -0600 Subject: [Bro] Crash on SMB Analyzer - Tree Connect AndX In-Reply-To: References: <2C673CCC-523E-482A-99AC-690C2D9DD860@icir.org> <87A00604-E28A-4C34-8036-17F115957336@gmail.com> <9FEBE83B-ED7E-4A5B-A121-B60050630698@icir.org> <5E98D74B-F780-427D-8E93-20A50C1ACC0F@icir.org> Message-ID: Possible fix, try renaming some variables. I had a variable named "con" in my script and renaming that to conseen fixed the issue (or seemed to so far). I've been seeing crashing at least once an hour and now it's been > 2 hours w/o a worker crash. Have you tried renaming path_name to something else? On Thu, Nov 29, 2012 at 10:33 AM, Mike Sconzo wrote: > I've been seeing these too for whatever it's worth. > > 1352906659.976267 fatal error in : Val::CONST_ACCESSOR > (record/string) ([stime=1352906659.942897, uid=V40HxsmOMT5, message=HTTP > 1.1 Without Referer Client Header]) > > I was going to attempt to re-write the script in a different way to see if > I can get it to stop tickling whatever it is. > > > On Thu, Nov 29, 2012 at 7:58 AM, Seth Hall wrote: > >> >> On Nov 29, 2012, at 8:22 AM, Seth Hall wrote: >> >> > Actually? I did some work last night and I'll be pushing out some >> changes to my topic/seth/smb-smb2-work branch that fully updates it to >> master in a few minutes (there were a number of merge conflicts). >> >> >> Be a little carefully with this branch at the moment, I actually have >> file extraction turned on by default (yes, I went ahead and built file >> extraction and identification into it :) ). >> >> .Seth >> >> -- >> Seth Hall >> International Computer Science Institute >> (Bro) because everyone has a network >> http://www.bro-ids.org/ >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > > > > -- > cat ~/.bash_history > documentation.txt > > -- cat ~/.bash_history > documentation.txt -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121129/5f5e04ac/attachment.html From mkolkebeck at gmail.com Thu Nov 29 21:30:55 2012 From: mkolkebeck at gmail.com (Mike Kolkebeck) Date: Thu, 29 Nov 2012 23:30:55 -0600 Subject: [Bro] Crash on SMB Analyzer - Tree Connect AndX In-Reply-To: References: <2C673CCC-523E-482A-99AC-690C2D9DD860@icir.org> <87A00604-E28A-4C34-8036-17F115957336@gmail.com> <9FEBE83B-ED7E-4A5B-A121-B60050630698@icir.org> <5E98D74B-F780-427D-8E93-20A50C1ACC0F@icir.org> Message-ID: On Thu, Nov 29, 2012 at 1:03 PM, Mike Sconzo wrote: > Possible fix, try renaming some variables. I had a variable named "con" in > my script and renaming that to conseen fixed the issue (or seemed to so > far). I've been seeing crashing at least once an hour and now it's been > 2 > hours w/o a worker crash. Have you tried renaming path_name to something > else? > > Yea, I tried to do that regarding the Info record variables, which is why I prepended everything with "smb_" in my script. I also tried your suggestion of renaming path_name to something like pathfoostring, but I'm still getting the same crash when connecting to an smb file share. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20121129/95943ad0/attachment.html