From dina at ICSI.Berkeley.EDU Mon Apr 2 20:00:12 2012 From: dina at ICSI.Berkeley.EDU (Dina Hadziosmanovic) Date: Mon, 2 Apr 2012 20:00:12 -0700 Subject: [Bro] error: unrecognised characher Message-ID: <002d01cd1145$e3862f50$aa928df0$@berkeley.edu> Hi all, I found somewhere in archive in 2009 that someone had this error already. That time you guys suggested to add "use_connection_compressor=F" on the command line. This doesn?t seem to help me. Here is the error: ~/met/bin$ zcat 2009-M57-day11-21.trace.gz | ./bro -r -C - frameworks/metrics/ssl-example error in , line 1: unrecognized character - ??? error in , line 1: unrecognized character - ??? error in , line 1: unrecognized character - ??? error in , line 1: unrecognized character - ??? .............. error in , line 1: unknown identifier K, at or near "K" I had it running already for few days, so I must have done sth in the mantime:) Thanks, Dina -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120402/8708220e/attachment.html From ngojp82 at gmail.com Mon Apr 2 20:06:01 2012 From: ngojp82 at gmail.com (John Ngo) Date: Mon, 2 Apr 2012 23:06:01 -0400 Subject: [Bro] Question from a beginner Message-ID: Hi all, I've used BRO for over a year now (ver 1.5 and older). However, I'm not an advanced user. Most of my time with BRO are to access dns and http logs for analysis, and nothing fancy like writing scripts or setting up a BRO box (All of our BRO boxes were setup by someone else). Today, I decided to do something new and installed a brand new BRO 2.0 box. And let me tell you....it is day and night different from ver 1.5. Anyway, after installation, I started it up....and have no ideas where to go next... :( Here is what I'm trying to do with this setup for now: Have it detect and send email alerts on any downloads for executable/suspicious files. I remember one of our old boxes uses a script called "http-ext-identified-files.bro" for this purpose (i believe was written by Mr. Seth Hall). In the new BRO 2.0, I've looked around the /base/protocols/http folder and found something similar. How to I get this to work and have it email to a specified email address when a host performs download for these files? How to I get BRO to email me whenever something triggered? Thanks much for your time, JPN -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120402/cec78c58/attachment.html From robin at icir.org Mon Apr 2 21:36:06 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 2 Apr 2012 21:36:06 -0700 Subject: [Bro] error: unrecognised characher In-Reply-To: <002d01cd1145$e3862f50$aa928df0$@berkeley.edu> References: <002d01cd1145$e3862f50$aa928df0$@berkeley.edu> Message-ID: <20120403043606.GE49234@icir.org> On Mon, Apr 02, 2012 at 20:00 -0700, Dina Hadziosmanovic wrote: > ~/met/bin$ zcat 2009-M57-day11-21.trace.gz | ./bro -r -C - frameworks/metrics/ssl-example There's a typo in the ordering, it should be "./bro -r - -C" Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From mthompson at hexwave.com Tue Apr 3 09:36:24 2012 From: mthompson at hexwave.com (Matt Thompson) Date: Tue, 03 Apr 2012 11:36:24 -0500 Subject: [Bro] Binpac http Message-ID: <4F7B2708.9070106@hexwave.com> Hi, I'm new to binpac but really like the concept (sick of hand writing protocol decoders). I'm working on using binpac in a project to decode pcap files. I have http mostly working but came across a problem with getting the response body from responses that don't have a content-length header. RFC2616 section 4.4 5) says that the message length can be determined by the server closing the connection. I think that the server must use a Connection: close header if it's going to close the connection after the response. How can I express this in binpac? I can add another delivery mode and set that based on the presence of the Connection: close header but am a little unclear on how to get the rest of the flow. I'm setting delivery_mode_ = CLOSE if Content-Length doesn't exist and Connection: close does exist. In HTTP_Body I've added a CLOSE -> rest: bytestring &restofflow &chunked; I have: refine typeattr HTTP_Body += &let { process_message: bool = http_body($context.connection.http_conn, body); }; http_body() is never called unless there is a Content-Length. Any thoughts? Regards, Matt Thompson From mthompson at hexwave.com Tue Apr 3 09:52:36 2012 From: mthompson at hexwave.com (Matt Thompson) Date: Tue, 03 Apr 2012 11:52:36 -0500 Subject: [Bro] Binpac http In-Reply-To: <4F7B2708.9070106@hexwave.com> References: <4F7B2708.9070106@hexwave.com> Message-ID: <4F7B2AD4.9090803@hexwave.com> I also tried using &restofdata instead of &restofflow. The BinPAC user guide says &restofdata will grab the rest of the data available in the FlowBuffer. Using &restofdata results in binpac compiler returning "error: cannot handle incremental input". It just occurred to me that I'm not indicating that the flow has "ended" so how is it to know that &restofflow is satisfied? Am I missing an API call other than NewData()? Cheers, Matt Thompson On 4/3/12 11:36 AM, Matt Thompson wrote: > Hi, > > I'm new to binpac but really like the concept (sick of hand writing > protocol decoders). I'm working on using binpac in a project to decode > pcap files. I have http mostly working but came across a problem with > getting the response body from responses that don't have a > content-length header. > > RFC2616 section 4.4 5) says that the message length can be determined by > the server closing the connection. I think that the server must use a > Connection: close header if it's going to close the connection after the > response. How can I express this in binpac? I can add another delivery > mode and set that based on the presence of the Connection: close header > but am a little unclear on how to get the rest of the flow. > > I'm setting delivery_mode_ = CLOSE if Content-Length doesn't exist and > Connection: close does exist. > > In HTTP_Body I've added a CLOSE -> rest: bytestring &restofflow &chunked; > > I have: > refine typeattr HTTP_Body += &let { > process_message: bool = http_body($context.connection.http_conn, body); > }; > > http_body() is never called unless there is a Content-Length. > > Any thoughts? > > Regards, > Matt Thompson > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From sheharbano.k at gmail.com Thu Apr 5 23:41:36 2012 From: sheharbano.k at gmail.com (Sheharbano Khattak) Date: Fri, 6 Apr 2012 11:41:36 +0500 Subject: [Bro] Using create_expire and expire_func Message-ID: Dear Bro Team, I have a global table that i populate at the time of initialization (bro_init). I want to empty it every x minutes to fill it up with fresh values. This means that i also need to be notified when the table is being emptied. It seems to me that the attributes &create_expire and &expire_func will be helpful for my requirements. I wrote a quick script to check if the behavior matches my expectation. However, it doesn't seems to work as expected. Here is my script: in try.bro: ----------------------------------------------------------------------------- function inform_me(s: set[string], idx: any): interval { print "expired"; return 5secs; } global s: set[string] &create_expire=5secs &expire_func=inform_me; event bro_init() { add s["i"]; add s["am"]; add s["here"]; #s should have i,am,here print s; sleep(15); #s should be empty print s; } ---------------------------------------------------------------------------- in bro.bif ----------------------------------------------------------------------------- function sleep%(time_secs: count%): any %{ usleep(time_secs * 1000000); return 0; %} ------------------------------------------------------------------------------ OUTPUT ----------------------------------------------------------------------------- root at sheharbano-Inspiron-N5010:/home/sheharbano/Desktop# bro tryy.bro { here, i, am } { here, i, am } ----------------------------------------------------------------------------------- Why isn't 's' empty on second print? Regards, -- Sheharbano Khattak Research Assistant @ NUST, Pakistan. http://etheryell.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120406/7286097e/attachment.html From 792874862 at qq.com Fri Apr 6 04:43:33 2012 From: 792874862 at qq.com (=?gbk?B?zOzSuw==?=) Date: Fri, 6 Apr 2012 19:43:33 +0800 Subject: [Bro] (no subject) Message-ID: Hi, I uses bro-2.0 recently,and I got a question. fatal error in /usr/local/bro/share/bro/base/frameworks/packet-filter/./main.bro, line 124: Bad pcap filter 'not ip6' -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120406/bb9dc566/attachment.html From vern at icir.org Fri Apr 6 11:53:26 2012 From: vern at icir.org (Vern Paxson) Date: Fri, 06 Apr 2012 11:53:26 -0700 Subject: [Bro] Using create_expire and expire_func In-Reply-To: (Fri, 06 Apr 2012 11:41:36 +0500). Message-ID: <20120406185326.770A82C4002@rock.ICSI.Berkeley.EDU> > print s; > > sleep(15); > > #s should be empty > print s; > } > ---------------------------------------------------------------------------- > in bro.bif > ----------------------------------------------------------------------------- > function sleep%(time_secs: count%): any > %{ > usleep(time_secs * 1000000); > return 0; > %} > > ... > > Why isn't 's' empty on second print? Bro drives its evaluation of timers based on the clock advancing between events. The above code sequence doesn't include any subsequent event after bro_init(), so the expiration timers don't have a chance to run. Vern From jsiwek at illinois.edu Fri Apr 6 11:55:20 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 6 Apr 2012 18:55:20 +0000 Subject: [Bro] Using create_expire and expire_func In-Reply-To: References: Message-ID: <4D423023-7EB2-4792-B8B2-9C87F3CE7C42@illinois.edu> Hi Sheharbano, I inlined some notes: > in try.bro: > ----------------------------------------------------------------------------- > function inform_me(s: set[string], idx: any): interval > { > print "expired"; > return 5secs; > } The return value of an &expire_func indicates the amount of additional time to wait before expiring the element. So always returning "5secs" will never expire the element. Return "0secs" if you want the element removed automatically, or you could even "delete s[idx]" yourself. > global s: set[string] &create_expire=5secs &expire_func=inform_me; > > > > event bro_init() > { > > add s["i"]; > add s["am"]; > add s["here"]; > > #s should have i,am,here > print s; > > sleep(15); > > #s should be empty > print s; > } > ---------------------------------------------------------------------------- > in bro.bif > ----------------------------------------------------------------------------- > function sleep%(time_secs: count%): any > %{ > usleep(time_secs * 1000000); > return 0; > %} The sleep BIF you added doesn't look like it's enough to trigger the internal timers that Bro would use for table expiration, or at least I couldn't find a way, but reading input from a pcap file that captured traffic for longer than your expiration interval could allow you to test it. You could handle the "new_connection" event and check the contents of your global table there. I did find a bug for the case when reading input live from an interface that would prevent expiry of table values set in bro_init(), for which I committed a fix in the git fastpath branch. Also in fastpath, I made a test script you could refer to: testing/btest/language/expire_func.test. +Jon From seth at icir.org Fri Apr 6 18:29:21 2012 From: seth at icir.org (Seth Hall) Date: Fri, 6 Apr 2012 21:29:21 -0400 Subject: [Bro] Question from a beginner In-Reply-To: References: Message-ID: On Apr 2, 2012, at 11:06 PM, John Ngo wrote: > Here is what I'm trying to do with this setup for now: Have it detect and send email alerts on any downloads for executable/suspicious files. I remember one of our old boxes uses a script called "http-ext-identified-files.bro" for this purpose Yep, that same functionality is built into Bro. My -ext scripts are no longer relevant with 2.0 since they have essentially become 2.0. :) We have a shorthand method for creating a notice policy (very similar to 1.5's notice policy and documented [1]) and the new notice with the same functionality is HTTP::Incorrect_File_Type. I've included a few extra notices that you might want to be notified about as well. redef Notice::emailed_types += { HTTP::Incorrect_File_Type, HTTP::SQL_Injection_Victim, SSH::Interesting_Hostname_Login, HTTP::Malware_Hash_Registry_Match, }; 1. http://www.bro-ids.org/documentation/notice.html#processing-notices .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Fri Apr 6 19:44:15 2012 From: seth at icir.org (Seth Hall) Date: Fri, 6 Apr 2012 22:44:15 -0400 Subject: [Bro] (no subject) In-Reply-To: References: Message-ID: On Apr 6, 2012, at 7:43 AM, ?? wrote: > fatal error in /usr/local/bro/share/bro/base/frameworks/packet-filter/./main.bro, line 124: Bad pcap filter 'not ip6' What operating system did you get this on? I suppose I can see this happening if your operating system doesn't have an IPv6 stack and you have Bro compiled with IPv6 support. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Fri Apr 6 20:40:06 2012 From: seth at icir.org (Seth Hall) Date: Fri, 6 Apr 2012 23:40:06 -0400 Subject: [Bro] =?utf-8?b?5Zue5aSN77yaICBiYWQgcGNhcCBmaWx0ZXI6bm90IGlwNg==?= In-Reply-To: References: Message-ID: <7670BC7A-6D30-47B1-A2FB-7241E20C09C0@icir.org> On Apr 6, 2012, at 10:50 PM, ?? wrote: > I use VMware Fedora 10,meanwhile,the broctl cann't start. You'll have to tell me more than that. What happens when you try to run broctl? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From sheharbano.k at gmail.com Tue Apr 10 23:30:43 2012 From: sheharbano.k at gmail.com (Sheharbano Khattak) Date: Tue, 10 Apr 2012 23:30:43 -0700 Subject: [Bro] Problem with QR field in dns log Message-ID: Dear Bro Team, I was working with some DNS logs and wanted to look at total number of DNS MX queries and responses. I used the usual bro-cut/awk/sort/uniq commands and turned out that there are absolutely no DNS responses within the log. I know that this is not true, and confirmed via tshark. I looked at script /base/protocols/dns.bro. In the definition of what goes into the log, the entry says ## Whether the message is a query (F) or response (T). QR: bool &log &default=F; which sounds good. But it seems that QR has not been assigned a value anywhere in the rest of the code, therefore the default value F is displayed whether it's a query or a response. Maybe QR should become 'T' in all the dns_reply(AA/MX/....) events in the script. Regards, -- Sheharbano Khattak http://etheryell.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120410/56a71d78/attachment.html From tyler.schoenke at colorado.edu Wed Apr 11 14:29:28 2012 From: tyler.schoenke at colorado.edu (Tyler T. Schoenke) Date: Wed, 11 Apr 2012 15:29:28 -0600 Subject: [Bro] Alarms in 2.0 Message-ID: <4F85F7B8.3090608@colorado.edu> Two questions regarding Alarms in 2.0. First, I created a signature and wanted to reduce the frequency that it fires. Does anyone have sample code for SIG_ALARM_PER_ORIG or some other way to send out a single alarm per source IP? Second, I configured the MailAlarmsTo setting, but the Alarms are still going to the MailTo (bromessage@) address. I also tried setting MailAlarms = True. What am I doing wrong? Thanks, Tyler -- -- Tyler Schoenke Network Security Manager IT Security Office University of Colorado at Boulder From JAzoff at albany.edu Wed Apr 11 14:47:38 2012 From: JAzoff at albany.edu (Justin Azoff) Date: Wed, 11 Apr 2012 17:47:38 -0400 Subject: [Bro] Alarms in 2.0 In-Reply-To: <4F85F7B8.3090608@colorado.edu> References: <4F85F7B8.3090608@colorado.edu> Message-ID: <20120411214738.GC15524@datacomm.albany.edu> On Wed, Apr 11, 2012 at 03:29:28PM -0600, Tyler T. Schoenke wrote: > Two questions regarding Alarms in 2.0. > > First, I created a signature and wanted to reduce the frequency that it > fires. Does anyone have sample code for SIG_ALARM_PER_ORIG or some > other way to send out a single alarm per source IP? It looks like you are supposed to do something like redef Signatures::actions += [ ["sig_id"] = SIG_ALARM_PER_ORIG ]; -- -- Justin Azoff -- Network Security & Performance Analyst From baxterw3232 at gmail.com Thu Apr 12 07:49:51 2012 From: baxterw3232 at gmail.com (Will) Date: Thu, 12 Apr 2012 09:49:51 -0500 Subject: [Bro] Alarms in 2.0 In-Reply-To: <20120411214738.GC15524@datacomm.albany.edu> References: <4F85F7B8.3090608@colorado.edu> <20120411214738.GC15524@datacomm.albany.edu> Message-ID: On Wed, Apr 11, 2012 at 4:47 PM, Justin Azoff wrote: > On Wed, Apr 11, 2012 at 03:29:28PM -0600, Tyler T. Schoenke wrote: >> Two questions regarding Alarms in 2.0. >> >> First, I created a signature and wanted to reduce the frequency that it >> fires. ?Does anyone have sample code for SIG_ALARM_PER_ORIG or some >> other way to send out a single alarm per source IP? > > It looks like you are supposed to do something like > > redef Signatures::actions += [ ["sig_id"] = SIG_ALARM_PER_ORIG ]; > This worked once I added the "Signatures" module to the SIG_ALARM_PER_ORIG. redef Signatures::actions += [ ["sig_id"] = Signatures::SIG_ALARM_PER_ORIG ]; -will > > -- > -- Justin Azoff > -- Network Security & Performance Analyst > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From laizhiquan at gmail.com Thu Apr 12 07:51:47 2012 From: laizhiquan at gmail.com (zhiquan lai) Date: Thu, 12 Apr 2012 22:51:47 +0800 Subject: [Bro] barnyard2's error: "Unknown output plugin: "alert_bro" " (snort + barnyard2 + bro) Message-ID: Hi, I'am trying barnyard2-1.9 to send snort alerts as events to bro instance. When building barnyard2, everything looks OK. (For successfully compiling spo_alert_bro.c, I deleted the "#ifdef BROCCOLI" statement in spo_alert_bro.c, and added the dir of broccoli.h to INCLUDE) *However*, when barnyard2 started up, "alert_bro" plugin can not be found:( What's wrong with this? Any answer is appreciated. Thanks, Quan This is the details when compiling and runinig: *[root at sneat barnyard2-1.9]# make* ......(omit something) make[3]: Entering directory `/res/workspace/bro/barnyard2-1.9/src/output-plugins' gcc -DHAVE_CONFIG_H -I. -I../.. -I.. -I ../sfutil *-I/usr/local/bro/include * -g -O2 -fno-strict-aliasing -Wall -c spo_alert_bro.c gcc -DHAVE_CONFIG_H -I. -I../.. -I.. -I ../sfutil -I/usr/local/bro/include -g -O2 -fno-strict-aliasing -Wall -c spo_alert_cef.c ......(omit something) rm -f libspo.a ar cru libspo.a spo_alert_arubaaction.o *spo_alert_bro.o* spo_alert_cef.o spo_alert_csv.o spo_alert_fast.o spo_alert_full.o spo_alert_prelude.o spo_alert_syslog.o spo_alert_test.o spo_alert_unixsock.o spo_common.o spo_log_ascii.o spo_log_null.o spo_log_tcpdump.o spo_platypus.o spo_sguil.o spo_database.o ranlib libspo.a ......(omit something) /bin/sh ../libtool --tag=CC --mode=link gcc -g -O2 -fno-strict-aliasing -Wall -o *barnyard2* barnyard2.o debug.o decode.o log.o log_text.o map.o mstring.o parser.o plugbase.o spooler.o strlcatu.o strlcpyu.o util.o output-plugins/libspo.a input-plugins/libspi.a sfutil/libsfutil.a -lpcap -lnsl -lm -lm libtool: link: gcc -g -O2 -fno-strict-aliasing -Wall -o barnyard2 barnyard2.o debug.o decode.o log.o log_text.o map.o mstring.o parser.o plugbase.o spooler.o strlcatu.o strlcpyu.o util.o output-plugins/libspo.a input-plugins/libspi.a sfutil/libsfutil.a -lpcap -lnsl -lm *[root at sneat barnyard2-1.9]# ./src/barnyard2 -c /usr/local/etc/barnyard2.conf -d /var/log/snort/ -f /var/log/snort/merged.log -w /var/log/snort/snort.waldo* Running in Continuous mode --== Initializing Barnyard2 ==-- Initializing Input Plugins! Initializing Output Plugins! Parsing config file "/usr/local/etc/barnyard2.conf" Log directory = /var/log/barnyard2 *ERROR: /usr/local/etc/barnyard2.conf(204) Unknown output plugin: "alert_bro"* Fatal Error, Quitting.. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120412/002e8198/attachment.html From tyler.schoenke at colorado.edu Thu Apr 12 07:53:26 2012 From: tyler.schoenke at colorado.edu (Tyler T. Schoenke) Date: Thu, 12 Apr 2012 08:53:26 -0600 Subject: [Bro] Alarms in 2.0 In-Reply-To: References: <4F85F7B8.3090608@colorado.edu> <20120411214738.GC15524@datacomm.albany.edu> Message-ID: <4F86EC66.5070003@colorado.edu> Yes, works well when you don't misspell the sig_id. :) Thanks Justin! Any ideas on how to make the MailAlarmsTo work? Tyler -- Tyler Schoenke Network Security Manager IT Security Office University of Colorado at Boulder On 4/12/12 8:49 AM, Will wrote: > On Wed, Apr 11, 2012 at 4:47 PM, Justin Azoff wrote: >> On Wed, Apr 11, 2012 at 03:29:28PM -0600, Tyler T. Schoenke wrote: >>> Two questions regarding Alarms in 2.0. >>> >>> First, I created a signature and wanted to reduce the frequency that it >>> fires. Does anyone have sample code for SIG_ALARM_PER_ORIG or some >>> other way to send out a single alarm per source IP? >> >> It looks like you are supposed to do something like >> >> redef Signatures::actions += [ ["sig_id"] = SIG_ALARM_PER_ORIG ]; >> > > This worked once I added the "Signatures" module to the SIG_ALARM_PER_ORIG. > > redef Signatures::actions += [ ["sig_id"] = Signatures::SIG_ALARM_PER_ORIG ]; > > -will >> >> -- >> -- Justin Azoff >> -- Network Security & Performance Analyst >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From m_zubair_rafique at yahoo.com Fri Apr 13 08:13:44 2012 From: m_zubair_rafique at yahoo.com (zubair rafique) Date: Fri, 13 Apr 2012 08:13:44 -0700 (PDT) Subject: [Bro] Bro DPD (Beginner) Message-ID: <1334330024.995.YahooMailNeo@web126101.mail.ne1.yahoo.com> Hi all, I am new to Bro. I am trying to analyse the traffic trace in offline mode and detecting the outgoing http packets on non standard ports (POST request in TCP[PSH, ACK]? packet). I am using the following command line option: ?sudo /usr/local/bro/bin/bro -f tcp? -r mytrace.pcap?? /usr/local/bro/share/bro/base/frameworks/dpd/main.bro There is no dpd log file generated by bro. What I am missing here?. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120413/8ca030f0/attachment.html From seth at icir.org Fri Apr 13 08:39:21 2012 From: seth at icir.org (Seth Hall) Date: Fri, 13 Apr 2012 11:39:21 -0400 Subject: [Bro] Bro DPD (Beginner) In-Reply-To: <1334330024.995.YahooMailNeo@web126101.mail.ne1.yahoo.com> References: <1334330024.995.YahooMailNeo@web126101.mail.ne1.yahoo.com> Message-ID: On Apr 13, 2012, at 11:13 AM, zubair rafique wrote: > I am using the following command line option: > sudo /usr/local/bro/bin/bro -f tcp -r mytrace.pcap /usr/local/bro/share/bro/base/frameworks/dpd/main.bro > There is no dpd log file generated by bro. > What I am missing here?. Do you have a conn.log or http.log? conn.log will indicate which analyzer(s) successfully analyzed a connection and http.log will show the information from the log. dpd.log is mostly used for debugging when and why DPD failed. No failure, no log (failure includes the client or server not abiding the protocol). You also don't need to include "-f tcp" in your filter. Bro has a wide open filter which lets everything in by default now. You also don't need to load that script. You could condense your entire command line to "bro -r mytrace.pcap" .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From m_zubair_rafique at yahoo.com Fri Apr 13 08:48:54 2012 From: m_zubair_rafique at yahoo.com (zubair rafique) Date: Fri, 13 Apr 2012 08:48:54 -0700 (PDT) Subject: [Bro] Bro DPD (Beginner) In-Reply-To: References: <1334330024.995.YahooMailNeo@web126101.mail.ne1.yahoo.com> Message-ID: <1334332134.44595.YahooMailNeo@web126103.mail.ne1.yahoo.com> Thanks for the quick reply. One more question. How to detect/extract HTTP request in TCP payload (where TCP connection is established on the non-standard port). ________________________________ From: Seth Hall To: zubair rafique Cc: "bro at bro-ids.org" Sent: Friday, April 13, 2012 5:39 PM Subject: Re: [Bro] Bro DPD (Beginner) On Apr 13, 2012, at 11:13 AM, zubair rafique wrote: > I am using the following command line option: >? sudo /usr/local/bro/bin/bro -f tcp? -r mytrace.pcap? /usr/local/bro/share/bro/base/frameworks/dpd/main.bro > There is no dpd log file generated by bro. > What I am missing here?. Do you have a conn.log or http.log?? conn.log will indicate which analyzer(s) successfully analyzed a connection and http.log will show the information from the log.? dpd.log is mostly used for debugging when and why DPD failed.? No failure, no log (failure includes the client or server not abiding the protocol). You also don't need to include "-f tcp" in your filter.? Bro has a wide open filter which lets everything in by default now.? You also don't need to load that script.? You could condense your entire command line to "bro -r mytrace.pcap" ? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120413/76f98c04/attachment.html From seth at icir.org Fri Apr 13 09:05:00 2012 From: seth at icir.org (Seth Hall) Date: Fri, 13 Apr 2012 12:05:00 -0400 Subject: [Bro] Bro DPD (Beginner) In-Reply-To: <1334332134.44595.YahooMailNeo@web126103.mail.ne1.yahoo.com> References: <1334330024.995.YahooMailNeo@web126101.mail.ne1.yahoo.com> <1334332134.44595.YahooMailNeo@web126103.mail.ne1.yahoo.com> Message-ID: <26388B9B-ED83-4051-8419-9921B82B5C59@icir.org> On Apr 13, 2012, at 11:48 AM, zubair rafique wrote: > Thanks for the quick reply. One more question. How to detect/extract HTTP request in TCP payload (where TCP connection is established on the non-standard port). We've been having some trouble with content extraction, but if you want to try it? bro -r mytrace.pcap "HTTP::extract_file_types=/.*/" This mechanism is being heavily reworked for 2.1 right now too so that all file handling will be done through the new file analysis framework. More information and docs will come with the 2.1 release. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From jsiwek at illinois.edu Fri Apr 13 09:41:06 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 13 Apr 2012 16:41:06 +0000 Subject: [Bro] barnyard2's error: "Unknown output plugin: "alert_bro" " (snort + barnyard2 + bro) In-Reply-To: References: Message-ID: <0A31BDE9-DD84-4815-8CBB-58109B1A3FA0@illinois.edu> > > I'am trying barnyard2-1.9 to send snort alerts as events to bro instance. > > When building barnyard2, everything looks OK. > (For successfully compiling spo_alert_bro.c, I deleted the "#ifdef BROCCOLI" statement in spo_alert_bro.c, and added the dir of broccoli.h to INCLUDE) > > However, when barnyard2 started up, "alert_bro" plugin can not be found:( There should be another "#ifdef BROCCOLI" in plugbase.c that you might have missed. However, you shouldn't have to modify code directly if you use the configure options to enable the bro output plugin: ./configure --enable-bro --with-broccoli=/usr/local/bro There were also some bugs I found in the Bro output plugin that prevent it from working with the Bro 2.0 release, but I think I have then fixed and have a pending pull request here: https://github.com/firnsy/barnyard2/pull/25 So if you'd still want to play around with getting this to work, use my git version of Barnyard2 and the current git version of Bro. The other thing that's not well documented anywhere is how to configure the listening Bro node to request the right events from the Barnyard2 Broccoli client. Here's the Bro script I used when testing: @load policy/frameworks/communication/listen @load policy/integration/barnyard2 redef Communication::nodes += { ["local"] = [$host=127.0.0.1, $class="barnyard", $events=/Barnyard2::barnyard_alert/, $connect = F] }; +Jon From daltonporter at yahoo.com Fri Apr 13 10:25:30 2012 From: daltonporter at yahoo.com (Dalton Porter) Date: Fri, 13 Apr 2012 10:25:30 -0700 (PDT) Subject: [Bro] http.log reorder and skip fields, how? Message-ID: <1334337930.40769.YahooMailNeo@web120703.mail.ne1.yahoo.com> Hello All. ? It appears that the data in http.log is a listing of the Info fields which have the &log attribute.? I can see how to add fields by redefining record Info using the += syntax.? However, I want to customize the output by removing some fields and reordering others.? What is the proper way to do this?? Can the field separator be adjusted??I don't want to actually "remove" fields, I just don't want some of them displayed.??I also didn't want to parse the bro output with a shell script to reformat it, I would like to have bro write the data out the way that I need it. ? Thanks, Dalton -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120413/25d93e56/attachment.html From seth at icir.org Fri Apr 13 11:24:12 2012 From: seth at icir.org (Seth Hall) Date: Fri, 13 Apr 2012 14:24:12 -0400 Subject: [Bro] http.log reorder and skip fields, how? In-Reply-To: <1334337930.40769.YahooMailNeo@web120703.mail.ne1.yahoo.com> References: <1334337930.40769.YahooMailNeo@web120703.mail.ne1.yahoo.com> Message-ID: <4FF3C28F-ACBE-4DC6-8CB9-AE58C1B0726F@icir.org> On Apr 13, 2012, at 1:25 PM, Dalton Porter wrote: > Hello All. It appears that the data in http.log is a listing of the Info fields which have the &log attribute. I can see how to add fields by redefining record Info using the += syntax. However, I want to customize the output by removing some fields and reordering others. What is the proper way to do this? Can the field separator be adjusted? I don't want to actually "remove" fields, I just don't want some of them displayed. I also didn't want to parse the bro output with a shell script to reformat it, I would like to have bro write the data out the way that I need it. Read this blog post: http://blog.bro-ids.org/2012/02/filtering-logs-with-bro.html And this documentation: http://www.bro-ids.org/documentation/logging.html#filtering If you still have questions I'd be glad to answer. I do agree that we are missing the ability to modify flags on record fields though. I haven't been able to think of a good syntax for that though. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From daltonporter at yahoo.com Fri Apr 13 12:53:20 2012 From: daltonporter at yahoo.com (Dalton Porter) Date: Fri, 13 Apr 2012 12:53:20 -0700 (PDT) Subject: [Bro] http.log reorder and skip fields, how? In-Reply-To: <4FF3C28F-ACBE-4DC6-8CB9-AE58C1B0726F@icir.org> References: <1334337930.40769.YahooMailNeo@web120703.mail.ne1.yahoo.com> <4FF3C28F-ACBE-4DC6-8CB9-AE58C1B0726F@icir.org> Message-ID: <1334346800.61163.YahooMailNeo@web120704.mail.ne1.yahoo.com> Seth, thanks for the info. I tried this: event bro_init() &priority=5 ?{ ?Log::create_stream(HTTP::LOG, [$columns=Info, $ev=log_http]); ?local filter: Log::Filter = [$name="myfilt", $path="myfilt", $include=set("id.orig_h","id.resp_h","ts")]; ?Log::add_filter(HTTP::LOG,filter); ?} ? But in the output file, the fields are ordered ts,orig,resp.? Can I control the ordering? Is there an easy way to change field separator? Thank you. ? On Apr 13, 2012, at 1:25 PM, Dalton Porter wrote: > Hello All.? It appears that the data in http.log is a listing of the Info fields which have the &log attribute.? I can see how to add fields by redefining record Info using the += syntax.? However, I want to customize the output by removing some fields and reordering others.? What is the proper way to do this?? Can the field separator be adjusted? I don't want to actually "remove" fields, I just don't want some of them displayed.? I also didn't want to parse the bro output with a shell script to reformat it, I would like to have bro write the data out the way that I need it. Read this blog post: ??? http://blog.bro-ids.org/2012/02/filtering-logs-with-bro.html And this documentation: ??? http://www.bro-ids.org/documentation/logging.html#filtering If you still have questions I'd be glad to answer.? I do agree that we are missing the ability to modify flags on record fields though.? I haven't been able to think of a good syntax for that though. ? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120413/b7079d8a/attachment.html From seth at icir.org Fri Apr 13 14:04:20 2012 From: seth at icir.org (Seth Hall) Date: Fri, 13 Apr 2012 17:04:20 -0400 Subject: [Bro] http.log reorder and skip fields, how? In-Reply-To: <1334346800.61163.YahooMailNeo@web120704.mail.ne1.yahoo.com> References: <1334337930.40769.YahooMailNeo@web120703.mail.ne1.yahoo.com> <4FF3C28F-ACBE-4DC6-8CB9-AE58C1B0726F@icir.org> <1334346800.61163.YahooMailNeo@web120704.mail.ne1.yahoo.com> Message-ID: On Apr 13, 2012, at 3:53 PM, Dalton Porter wrote: > Seth, thanks for the info. I tried this: > event bro_init() &priority=5 > { > Log::create_stream(HTTP::LOG, [$columns=Info, $ev=log_http]); > local filter: Log::Filter = [$name="myfilt", $path="myfilt", $include=set("id.orig_h","id.resp_h","ts")]; > Log::add_filter(HTTP::LOG,filter); > } > > But in the output file, the fields are ordered ts,orig,resp. Can I control the ordering? > Is there an easy way to change field separator? > Thank you. You can't control ordering (sets aren't ordered either). If you need to change the order, you could do that by processing the logs through bro-cut like this: cat myfilt.log | bro-cut -f id.orig_h, id.resp_h, ts Also, your code above should look like this? event bro_init() { Log::add_filter(HTTP::LOG,[$name="myfilt", $path="myfilt", $include=set("id.orig_h","id.resp_h","ts")]); } You shouldn't be redefining the stream. Keep in mind that this will still create the full http log since you aren't removing the default filter. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Fri Apr 13 18:47:24 2012 From: seth at icir.org (Seth Hall) Date: Fri, 13 Apr 2012 21:47:24 -0400 Subject: [Bro] User conference? Message-ID: <1DD1F6FB-56AA-470F-89E1-2D42B3216766@icir.org> I'd like to try and collect some feedback on an idea we've been floating around for a while now. What does everyone think about a user conference this year instead of a workshop? We will probably aim for late summer or early fall. To explain myself in a bit more detail, we were thinking that a user conference would be a more heavily focused on user and developer presentations and a bit less on the teaching aspects. We would probably also cover new features Bro 2.1 during a user conference and probably give some presentations ourselves too. :) Could people get back with me privately on how you'd like to "vote"; workshop or user conference? Feel free to write to the list if you have questions. Thanks! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From asma.mtz at gmail.com Sat Apr 14 03:40:28 2012 From: asma.mtz at gmail.com (Asma Mumtaz) Date: Sat, 14 Apr 2012 15:40:28 +0500 Subject: [Bro] Reading from a text file Message-ID: Hi, I'm a beginner to Bro scripting language and started implementing an anomaly detection system using Bro. My code generates a training text file and I have to read it when I run it on the network data. Bro website's file documentation mentions that we can write a text file but not read it. Is there any way we can do it using Bro? Its a MUST part in my implementation. regards, Asma -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120414/5d60b09a/attachment.html From seth at icir.org Sun Apr 15 19:58:01 2012 From: seth at icir.org (Seth Hall) Date: Sun, 15 Apr 2012 22:58:01 -0400 Subject: [Bro] Problem with QR field in dns log In-Reply-To: References: Message-ID: On Apr 11, 2012, at 2:30 AM, Sheharbano Khattak wrote: > I looked at script /base/protocols/dns.bro. In the definition of what goes into the log, the entry says > ## Whether the message is a query (F) or response (T). > QR: bool &log &default=F; Hah! Nice catch. That QR field should have been removed before the release. The "log unit" that the DNS log represents is a query and its set of responses so the QR dns flag doesn't even make sense to be there. :) I just committed a patch to our fastpath branch which will be merged into the master branch soon. To answer your larger question, just search for MX in the qtype_name field (or 15 in the type field). Each line represents a query and the responses received to the query so most of what you're looking for should be there. Check the field documentation to understand the fields better and please ask if the documentation isn't clear. http://www.bro-ids.org/documentation/scripts/base/protocols/dns/main.html?highlight=dns%20main#type-DNS::Info Thanks! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Sun Apr 15 19:59:56 2012 From: seth at icir.org (Seth Hall) Date: Sun, 15 Apr 2012 22:59:56 -0400 Subject: [Bro] Binpac http In-Reply-To: <4F7B2708.9070106@hexwave.com> References: <4F7B2708.9070106@hexwave.com> Message-ID: <830CFF52-0433-439A-A00B-BA5877AEBE48@icir.org> On Apr 3, 2012, at 12:36 PM, Matt Thompson wrote: > I'm new to binpac but really like the concept Hi Matt. I don't want you to feel like we've been ignoring you. I've been meaning to look into your question closer and I'll make sure and get to it ASAP! > (sick of hand writing protocol decoders). I can get behind this sentiment. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Mon Apr 16 12:40:45 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 16 Apr 2012 12:40:45 -0700 Subject: [Bro] Reading from a text file In-Reply-To: References: Message-ID: <20120416194045.GW53819@icir.org> On Sat, Apr 14, 2012 at 15:40 +0500, you wrote: > Is there any way we can do it using Bro? Not yet, but there will be soon. Bernhard Amann has developed a new "input framework" that can feed external information into Bro both at startup and also continuously during run-time. Currently, it can read input from ASCII files and from PostgresSQL. There may be further interfaces to other sources coming later. This is actually pretty powerful for a range of applications, and we're currently preparing the code for merging in. Assuming all goes well, it will be part of Bro 2.1. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From mcholste at gmail.com Mon Apr 16 17:18:52 2012 From: mcholste at gmail.com (Martin Holste) Date: Mon, 16 Apr 2012 19:18:52 -0500 Subject: [Bro] Filtering PacketFilter::Dropped_Packets Message-ID: I've got this: Log::disable_stream(PacketFilter::LOG); But I'm still getting a ton of "PacketFilter::Dropped_Packets" to notice.log. What do I need to do to disable these messages? From seth at icir.org Tue Apr 17 04:34:17 2012 From: seth at icir.org (Seth Hall) Date: Tue, 17 Apr 2012 07:34:17 -0400 Subject: [Bro] Filtering PacketFilter::Dropped_Packets In-Reply-To: References: Message-ID: <12633D4A-2CC3-4C5D-B8AB-B90040F21533@icir.org> On Apr 16, 2012, at 8:18 PM, Martin Holste wrote: > But I'm still getting a ton of "PacketFilter::Dropped_Packets" to notice.log. > What do I need to do to disable these messages? Notice processing docs: http://www.bro-ids.org/documentation/notice.html You can use the notice ignore shortcut because you want to completely ignore a notice type: http://www.bro-ids.org/documentation/notice.html#id7 redef Notice::ignored_types += { PacketFilter::Dropped_Packets }; .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From baxterw3232 at gmail.com Tue Apr 17 05:51:33 2012 From: baxterw3232 at gmail.com (Will) Date: Tue, 17 Apr 2012 07:51:33 -0500 Subject: [Bro] Filtering PacketFilter::Dropped_Packets In-Reply-To: <12633D4A-2CC3-4C5D-B8AB-B90040F21533@icir.org> References: <12633D4A-2CC3-4C5D-B8AB-B90040F21533@icir.org> Message-ID: On Tue, Apr 17, 2012 at 6:34 AM, Seth Hall wrote: > > On Apr 16, 2012, at 8:18 PM, Martin Holste wrote: > >> But I'm still getting a ton of "PacketFilter::Dropped_Packets" to notice.log. >> What do I need to do to disable these messages? > > Notice processing docs: > ? ? ? ?http://www.bro-ids.org/documentation/notice.html > > You can use the notice ignore shortcut because you want to completely ignore a notice type: > ? ? ? ?http://www.bro-ids.org/documentation/notice.html#id7 > > redef Notice::ignored_types += { PacketFilter::Dropped_Packets }; > That didn't appear to completely work for me as the default action still seemed to be applied. I changed it to this: redef Notice::policy += { [$pred(n: Notice::Info) = {return n$note == PacketFilter::Dropped_Packets; }, $action = Notice::ACTION_NONE, $halt = T] }; Before adding '$halt=T', the action in the log listed both ACTION_NONE and ACTION_LOG. -will > ?.Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From mcholste at gmail.com Tue Apr 17 06:41:29 2012 From: mcholste at gmail.com (Martin Holste) Date: Tue, 17 Apr 2012 08:41:29 -0500 Subject: [Bro] Filtering PacketFilter::Dropped_Packets In-Reply-To: References: <12633D4A-2CC3-4C5D-B8AB-B90040F21533@icir.org> Message-ID: Looks like Will's method is working. Thanks much! On Tue, Apr 17, 2012 at 7:51 AM, Will wrote: > On Tue, Apr 17, 2012 at 6:34 AM, Seth Hall wrote: >> >> On Apr 16, 2012, at 8:18 PM, Martin Holste wrote: >> >>> But I'm still getting a ton of "PacketFilter::Dropped_Packets" to notice.log. >>> What do I need to do to disable these messages? >> >> Notice processing docs: >> ? ? ? ?http://www.bro-ids.org/documentation/notice.html >> >> You can use the notice ignore shortcut because you want to completely ignore a notice type: >> ? ? ? ?http://www.bro-ids.org/documentation/notice.html#id7 >> >> redef Notice::ignored_types += { PacketFilter::Dropped_Packets }; >> > > That didn't appear to completely work for me as the default action > still seemed to be applied. > > I changed it to this: > redef Notice::policy += { [$pred(n: Notice::Info) = {return n$note == > PacketFilter::Dropped_Packets; }, $action = Notice::ACTION_NONE, $halt > = T] }; > > Before adding '$halt=T', the action in the log listed both ACTION_NONE > and ACTION_LOG. > > -will > >> ?.Seth >> >> -- >> Seth Hall >> International Computer Science Institute >> (Bro) because everyone has a network >> http://www.bro-ids.org/ >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From robin at icir.org Tue Apr 17 07:40:19 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 17 Apr 2012 07:40:19 -0700 Subject: [Bro] Filtering PacketFilter::Dropped_Packets In-Reply-To: References: Message-ID: <20120417144019.GF54520@icir.org> On Mon, Apr 16, 2012 at 19:18 -0500, you wrote: > But I'm still getting a ton of "PacketFilter::Dropped_Packets" to notice.log. That are going through the notice framework and can be suppressed there with something like this: redef Notice::ignored_types += { PacketFilter::Dropped_Packets }; Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Tue Apr 17 08:02:32 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 17 Apr 2012 08:02:32 -0700 Subject: [Bro] Filtering PacketFilter::Dropped_Packets In-Reply-To: References: <12633D4A-2CC3-4C5D-B8AB-B90040F21533@icir.org> Message-ID: <20120417150232.GL54520@icir.org> On Tue, Apr 17, 2012 at 07:51 -0500, Will wrote: > That didn't appear to completely work for me as the default action > still seemed to be applied. That sounds like a bug then. Can you file a ticket please? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From baxterw3232 at gmail.com Tue Apr 17 08:18:10 2012 From: baxterw3232 at gmail.com (Will) Date: Tue, 17 Apr 2012 10:18:10 -0500 Subject: [Bro] Filtering PacketFilter::Dropped_Packets In-Reply-To: <20120417150232.GL54520@icir.org> References: <12633D4A-2CC3-4C5D-B8AB-B90040F21533@icir.org> <20120417150232.GL54520@icir.org> Message-ID: Sure thing. -will On Tue, Apr 17, 2012 at 10:02 AM, Robin Sommer wrote: > > On Tue, Apr 17, 2012 at 07:51 -0500, Will wrote: > >> That didn't appear to completely work for me as the default action >> still seemed to be applied. > > That sounds like a bug then. Can you file a ticket please? > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL ? ?* Fax ? +1 (510) 666-2956 * ? www.icir.org From seth at icir.org Tue Apr 17 09:05:04 2012 From: seth at icir.org (Seth Hall) Date: Tue, 17 Apr 2012 12:05:04 -0400 Subject: [Bro] Filtering PacketFilter::Dropped_Packets In-Reply-To: References: <12633D4A-2CC3-4C5D-B8AB-B90040F21533@icir.org> <20120417150232.GL54520@icir.org> Message-ID: <80B79D25-6218-4FE1-9C7A-821D49EF6DF4@icir.org> On Apr 17, 2012, at 11:18 AM, Will wrote: > Sure thing. Thanks, it definitely sounds like a bug. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Tue Apr 17 09:19:21 2012 From: seth at icir.org (Seth Hall) Date: Tue, 17 Apr 2012 12:19:21 -0400 Subject: [Bro] Filtering PacketFilter::Dropped_Packets In-Reply-To: References: <12633D4A-2CC3-4C5D-B8AB-B90040F21533@icir.org> Message-ID: On Apr 17, 2012, at 9:41 AM, Martin Holste wrote: > Looks like Will's method is working. Thanks much! Everything implemented internally should make this work. There is one thing I'm wondering though. In any of your scripts you're running locally, are you doing? redef Notice::policy = { ? }; Instead of? redef Notice::policy += { ? }; It's a small difference, but causes a big change because those shortcuts (like ignored_types) are basically just pre-implemented notice policy items which you are blowing away if you do full set assignment instead of adding items to the set. I'll start trying to think of way to make that more resilient to this too. This fragility is the one thing I don't like about those pre-implemented policy items. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From mcholste at gmail.com Tue Apr 17 10:53:20 2012 From: mcholste at gmail.com (Martin Holste) Date: Tue, 17 Apr 2012 12:53:20 -0500 Subject: [Bro] Filtering PacketFilter::Dropped_Packets In-Reply-To: References: <12633D4A-2CC3-4C5D-B8AB-B90040F21533@icir.org> Message-ID: Nope, not doing that. On Tue, Apr 17, 2012 at 11:19 AM, Seth Hall wrote: > > On Apr 17, 2012, at 9:41 AM, Martin Holste wrote: > >> Looks like Will's method is working. ?Thanks much! > > > Everything implemented internally should make this work. ?There is one thing I'm wondering though. ?In any of your scripts you're running locally, are you doing? > > redef Notice::policy = { ? }; > > Instead of? > > redef Notice::policy += { ? }; > > It's a small difference, but causes a big change because those shortcuts (like ignored_types) are basically just pre-implemented ?notice policy items which you are blowing away if you do full set assignment instead of adding items to the set. ?I'll start trying to think of way to make that more resilient to this too. ?This fragility is the one thing I don't like about those pre-implemented policy items. > > ?.Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > From baxterw3232 at gmail.com Tue Apr 17 14:33:25 2012 From: baxterw3232 at gmail.com (Will) Date: Tue, 17 Apr 2012 16:33:25 -0500 Subject: [Bro] Filtering PacketFilter::Dropped_Packets In-Reply-To: References: <12633D4A-2CC3-4C5D-B8AB-B90040F21533@icir.org> Message-ID: On Tue, Apr 17, 2012 at 11:19 AM, Seth Hall wrote: > > On Apr 17, 2012, at 9:41 AM, Martin Holste wrote: > >> Looks like Will's method is working. ?Thanks much! > > > Everything implemented internally should make this work. ?There is one thing I'm wondering though. ?In any of your scripts you're running locally, are you doing? > > redef Notice::policy = { ? }; > > Instead of? > > redef Notice::policy += { ? }; Yes, all my are just like the example above, "+=", so I assume I was just appending another action to the table. Maybe not a bug then? If I do a full re-assignment "=" instead, I wouldn't have multiple actions assigned to the notice? > > It's a small difference, but causes a big change because those shortcuts (like ignored_types) are basically just pre-implemented ?notice policy items which you are blowing away if you do full set assignment instead of adding items to the set. ?I'll start trying to think of way to make that more resilient to this too. ?This fragility is the one thing I don't like about those pre-implemented policy items. > > ?.Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > From oguzyarimtepe at gmail.com Thu Apr 19 04:13:24 2012 From: oguzyarimtepe at gmail.com (Oguz Yarimtepe) Date: Thu, 19 Apr 2012 14:13:24 +0300 Subject: [Bro] bro protocol detection from pcap Message-ID: <20120419141324.b8ffdb990e7cac5920d27b92@gmail.com> Hi, I installed Bro 2.0 and tried to use its dpd functionality to detect application level protocols. I uploaded two pcap files for testing: http://www.loopbacking.info/dosya/ Running bro for test-http.pcap results in http detection at the conn.log # bro -p broctl -p broctl-live -p standalone -p local -p bro -r webdav.pcap # bro-cut service < conn.log conn.log http Then i wrote a script to extract TCP flows and save them as different pcap files. The idea is to keep the packets with the same source ip, source port, destination ip, destination port (on direction) or another direction in the same pcap file. One of the pcap that has a http flow in it is 213.pcap. When i try it with bro i don't see and application level information. # bro -p broctl -p broctl-live -p standalone -p local -p bro -r 213.pcap # bro-cut service < conn.log - - But when i try it via tshark i can get the protocol information # tshark -q -z io,phs -r 213.pcap tshark: Lua: Error during loading: [string "/usr/share/wireshark/init.lua"]:45: dofile has been disabled Running as user "root" and group "root". This could be dangerous. =================================================================== Protocol Hierarchy Statistics Filter: eth frames:74 bytes:207091 ip frames:74 bytes:207091 tcp frames:74 bytes:207091 http frames:1 bytes:865 Any idea what the reason is? Am i running the right paramater for bro to detect the application level protocol (dpd)? -- Oguz Yarimtepe http://about.me/oguzy From seth at icir.org Thu Apr 19 05:55:09 2012 From: seth at icir.org (Seth Hall) Date: Thu, 19 Apr 2012 08:55:09 -0400 Subject: [Bro] bro protocol detection from pcap In-Reply-To: <20120419141324.b8ffdb990e7cac5920d27b92@gmail.com> References: <20120419141324.b8ffdb990e7cac5920d27b92@gmail.com> Message-ID: On Apr 19, 2012, at 7:13 AM, Oguz Yarimtepe wrote: > One of the pcap that has a http flow in it is 213.pcap. When i try it with bro i don't see and application level information. Your tracefile has bad checksums. Either fix the checksums or use the -C command line flag to ignore checksums. > # bro -p broctl -p broctl-live -p standalone -p local -p bro -r 213.pcap You can leave out most of this command line. This should work fine: bro -r 213.pcap BroControl runs with all of those extra args to add various functionality that you don't need to worry about when you're just looking to analyze a tracefile. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From oguzyarimtepe at gmail.com Thu Apr 19 07:01:13 2012 From: oguzyarimtepe at gmail.com (Oguz Yarimtepe) Date: Thu, 19 Apr 2012 17:01:13 +0300 Subject: [Bro] bro protocol detection from pcap In-Reply-To: References: <20120419141324.b8ffdb990e7cac5920d27b92@gmail.com> Message-ID: <20120419170113.2afa9906c9af47167a64eb53@gmail.com> Hi, On Thu, 19 Apr 2012 08:55:09 -0400 Seth Hall wrote: > Your tracefile has bad checksums. Either fix the checksums or use the -C command line flag to ignore checksums. > > > # bro -p broctl -p broctl-live -p standalone -p local -p bro -r 213.pcap > -C is what i was looking for. Thank you. > You can leave out most of this command line. This should work fine: > > bro -r 213.pcap > > BroControl runs with all of those extra args to add various functionality that you don't need to worry about when you're just looking to analyze a tracefile. -r pcap_file worked fine. -- Oguz Yarimtepe From christopher.p.crawford at gmail.com Thu Apr 19 08:13:20 2012 From: christopher.p.crawford at gmail.com (Chris Crawford) Date: Thu, 19 Apr 2012 11:13:20 -0400 Subject: [Bro] Analyzing and Visualizing Bro Logs with Splunk Message-ID: Does anybody have the slides or video from "Analyzing and Visualizing Bro Logs with Splunk" talk at Bro Workshop 2011? -Chris From seth at icir.org Thu Apr 19 09:36:41 2012 From: seth at icir.org (Seth Hall) Date: Thu, 19 Apr 2012 12:36:41 -0400 Subject: [Bro] Analyzing and Visualizing Bro Logs with Splunk In-Reply-To: References: Message-ID: <756D6E43-1D48-4DBD-84F1-FD54D44F692B@icir.org> On Apr 19, 2012, at 11:13 AM, Chris Crawford wrote: > Does anybody have the slides or video from "Analyzing and Visualizing > Bro Logs with Splunk" talk at Bro Workshop 2011? It used live data and wasn't recorded. Justin, do you have anything shareable? There was also the Security Onion app for Splunk that was recently released. Since quite a bit of it revolves around Bro data you could try working with that if you are interested in using Bro data in Splunk. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From oguzyarimtepe at gmail.com Thu Apr 19 01:14:21 2012 From: oguzyarimtepe at gmail.com (Oguz Yarimtepe) Date: Thu, 19 Apr 2012 11:14:21 +0300 Subject: [Bro] bro protocol detection from pcap Message-ID: <20120419111421.d20424299d2c5fb7d6fae8f4@gmail.com> Hi, I installed Bro 2.0 and tried to use its dpd functionality to detect application level protocols. I attacted two pcap. Running bro for test-http.pcap results in http detection at the conn.log # bro -p broctl -p broctl-live -p standalone -p local -p bro -r webdav.pcap # bro-cut service < conn.log conn.log http Then i wrote a script to extract TCP flows and save them as different pcap files. The idea is to keep the packets with the same source ip, source port, destination ip, destination port (on direction) or another direction in the same pcap file. One of the pcap that has a http flow in it is 213.pcap. When i try it with bro i don't see and application level information. # bro -p broctl -p broctl-live -p standalone -p local -p bro -r 213.pcap # bro-cut service < conn.log - - But when i try it via tshark i can get the protocol information # tshark -q -z io,phs -r 213.pcap tshark: Lua: Error during loading: [string "/usr/share/wireshark/init.lua"]:45: dofile has been disabled Running as user "root" and group "root". This could be dangerous. =================================================================== Protocol Hierarchy Statistics Filter: eth frames:74 bytes:207091 ip frames:74 bytes:207091 tcp frames:74 bytes:207091 http frames:1 bytes:865 Any idea what the reason is? Am i running the right paramater for bro to detect the application level protocol (dpd)? -- Oguz Yarimtepe http://about.me/oguzy -------------- next part -------------- A non-text attachment was scrubbed... Name: webdav.pcap Type: application/cap Size: 12382 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120419/28f34792/attachment.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: 213.pcap Type: application/cap Size: 208299 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120419/28f34792/attachment-0001.bin From JAzoff at albany.edu Thu Apr 19 08:37:10 2012 From: JAzoff at albany.edu (Justin Azoff) Date: Thu, 19 Apr 2012 11:37:10 -0400 Subject: [Bro] Analyzing and Visualizing Bro Logs with Splunk In-Reply-To: References: Message-ID: <20120419153710.GO9794@datacomm.albany.edu> On Thu, Apr 19, 2012 at 11:13:20AM -0400, Chris Crawford wrote: > Does anybody have the slides or video from "Analyzing and Visualizing > Bro Logs with Splunk" talk at Bro Workshop 2011? > > -Chris Hmm, I thought they were put on the website.. I was difficult and used the google HTML5 slideshow template :-) The presentation is attached. Let me know if you have any questions. The old metrics scripts I mention were indeed obsoleted by 2.0, but I've updated most of them: https://github.com/JustinAzoff/bro_scripts/tree/2.0/ -- -- Justin Azoff -- Network Security & Performance Analyst -------------- next part -------------- A non-text attachment was scrubbed... Name: splunk.tgz Type: application/x-gtar Size: 886351 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120419/90bf5fc1/attachment.gtar -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120419/90bf5fc1/attachment.bin From dina at ICSI.Berkeley.EDU Fri Apr 20 17:15:49 2012 From: dina at ICSI.Berkeley.EDU (Dina Hadziosmanovic) Date: Fri, 20 Apr 2012 17:15:49 -0700 Subject: [Bro] array to BifEvent Message-ID: <006d01cd1f53$e8012c30$b8038490$@berkeley.edu> Hi guys, I don't know how to pass an array to binpac function and then to BifEvent. Here is my type definition: type X(len: uint16) = record { byteCount: uint8; registers: uint16[registerCount] &length = byteCount; here is the function I want to call: function something( r: X): bool %{ // here get a pointer to registers and value to byteCount ??? if ( ::some_event ) { BifEvent::generate_some_event( connection()->bro_analyzer(), connection()->bro_analyzer()->Conn(), is_orig(),bCount,reg); } return true; %} If BifEvent passes a pointer to the array, then in bro script I can use vector type to print those values? Thanks, dina -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120420/38171767/attachment.html From srunnels at gmail.com Fri Apr 20 19:14:39 2012 From: srunnels at gmail.com (scott runnels) Date: Fri, 20 Apr 2012 22:14:39 -0400 Subject: [Bro] Learning the Bro scripting language by solving a Network Forensics Challenge Message-ID: <959C853D-E953-454A-9B97-D5294BF1A9F3@gmail.com> I've been wrestling with how to go about learning the Bro scripting language and tried my best to document the process I'm using while trying to solve a Network Forensics Challenge using Bro. http://ryesecurity.blogspot.com/2012/04/solving-network-forensic-challenges.html I'm hoping to make it a series of posts to help people new to the scripting language (myself included) who might have the same questions. I'd welcome any comments or critiques. Thanks, Scott -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120420/6a83a2a0/attachment.html From seth at icir.org Fri Apr 20 19:48:40 2012 From: seth at icir.org (Seth Hall) Date: Fri, 20 Apr 2012 22:48:40 -0400 Subject: [Bro] Learning the Bro scripting language by solving a Network Forensics Challenge In-Reply-To: <959C853D-E953-454A-9B97-D5294BF1A9F3@gmail.com> References: <959C853D-E953-454A-9B97-D5294BF1A9F3@gmail.com> Message-ID: On Apr 20, 2012, at 10:14 PM, scott runnels wrote: > I'm hoping to make it a series of posts to help people new to the scripting language (myself included) who might have the same questions. Thanks Scott! I'm looking forward to future posts. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From marcos.e.rodriguez at gmail.com Fri Apr 20 19:54:20 2012 From: marcos.e.rodriguez at gmail.com (Marcos Rodriguez) Date: Fri, 20 Apr 2012 22:54:20 -0400 Subject: [Bro] Learning the Bro scripting language by solving a Network Forensics Challenge In-Reply-To: References: <959C853D-E953-454A-9B97-D5294BF1A9F3@gmail.com> Message-ID: On Fri, Apr 20, 2012 at 10:48 PM, Seth Hall wrote: > > On Apr 20, 2012, at 10:14 PM, scott runnels wrote: > > > I'm hoping to make it a series of posts to help people new to the > scripting language (myself included) who might have the same questions. > > Thanks Scott! I'm looking forward to future posts. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > Same here! I no longer have any excuses for not learning Bro. I'm a long time Snort user, and have used Suricata since release. I've always wanted to learn Bro, but never thought to learn it in context with the packet analysis I already perform at work! *facepalm! Thanks, Scott! marcos -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120420/1f7c4ae5/attachment.html From seth at icir.org Fri Apr 20 21:37:51 2012 From: seth at icir.org (Seth Hall) Date: Sat, 21 Apr 2012 00:37:51 -0400 Subject: [Bro] array to BifEvent In-Reply-To: <006d01cd1f53$e8012c30$b8038490$@berkeley.edu> References: <006d01cd1f53$e8012c30$b8038490$@berkeley.edu> Message-ID: <622F4485-4A5A-490F-8C1F-93F53EAF82A5@icir.org> Most people should freely ignore this reply, Dina's asking about an internal detail about connecting protocol analyzers to the Bro script-land. On Apr 20, 2012, at 8:15 PM, Dina Hadziosmanovic wrote: > Here is my type definition: > > type X(len: uint16) = record { > byteCount: uint8; > registers: uint16[registerCount] &length = byteCount; I assume you have some bit of code in a &let section you haven't included that is calling your BinPAC function after the 'X' unit is parsed? > here is the function I want to call: > > function something( r: X): bool > %{ You need to convert the C array into the Bro vector type manually. Something like this... for ( unsigned int i = 0; i < ${r.registers}->size(); ++i ) { uint16 register = ${r.registers}[i]; ? build a VectorVal ? } I probably have something about this code wrong, I just wanted to show that you have to convert C/C++ values into Bro values. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From roger.larsen at hig.no Sat Apr 21 04:31:10 2012 From: roger.larsen at hig.no (=?iso-8859-1?Q?Roger_Larsen_-_H=F8gskolen_i_Gj=F8vik?=) Date: Sat, 21 Apr 2012 13:31:10 +0200 Subject: [Bro] Missing notifications in Bro Message-ID: <000f01cd1fb2$40e176d0$c2a46470$@hig.no> Dear Bro Community, I am graduate student at Gj?vik University College in Norway. I was hoping to compare the detection rate in Bro and Snort regarding some network attacks (using NMAP). I'm not so familiar with Bro and after some hours of work I still have not found any log file telling me that Bro have detected an attack... I have read a lot of the documentation on Your web page. My questions are the following; #1 - How much can Bro's default base installation tell me regarding attack events (notifications)? (I use the fresh Bro 2.0) #2 - Can I easily get more notification LOG's from attack events? Thank You! Best Regards, Roger Larsen master IS student -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120421/5624c112/attachment.html From seth at icir.org Sat Apr 21 04:55:55 2012 From: seth at icir.org (Seth Hall) Date: Sat, 21 Apr 2012 07:55:55 -0400 Subject: [Bro] Missing notifications in Bro In-Reply-To: <000f01cd1fb2$40e176d0$c2a46470$@hig.no> References: <000f01cd1fb2$40e176d0$c2a46470$@hig.no> Message-ID: <12AABCFF-7513-4E99-A3DE-E67EE2500FCF@icir.org> On Apr 21, 2012, at 7:31 AM, Roger Larsen - H?gskolen i Gj?vik wrote: > I was hoping to compare the detection rate in Bro and Snort regarding some network attacks (using NMAP). We don't focus heavily on attacks, only where it makes sense for us. Nmap being using on the network would be detected as a scan and for our 2.0 release we don't have our scan detector in place right now. It's in our contributed scripts repository and will probably return soon, but for right now it's not in the default distribution. I will say now though that comparing the detection rate between Snort and Bro is not a good thing to compare. There is a lot more to Bro than just running it and detecting a single incident of something in a tracefile. > #1 - How much can Bro's default base installation tell me regarding attack events (notifications)? (I use the fresh Bro 2.0) > #2 - Can I easily get more notification LOG's from attack events? Doing a comparison like this is heavily weighted in Snort's favor because you're looking for Bro to do what the Snort community focuses on and not what we focus on. A poor comparison in the opposite direction would be to see what activity recording logs Snort outputs for various protocols (it doesn't do much), what correlation capabilities it has (it barely has any), or what it's programming programming language can do (it doesn't have one). Please don't try to compare Bro with Snort in this way. We would love for you to write a paper involving Bro but not where the comparison is weighted against us from the beginning. Feel free to follow up if you'd like to search for a more fair comparison together. Thanks! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From patrik.lundin.swe at gmail.com Sat Apr 21 06:05:08 2012 From: patrik.lundin.swe at gmail.com (Patrik Lundin) Date: Sat, 21 Apr 2012 15:05:08 +0200 Subject: [Bro] FTP password saving Message-ID: <20120421130505.GA8006@shell.virtality.se> Hello, First of all, I am very new to bro, excuse me if I am missing something obvious! Anyway: I have been playing around with bro analyzing a pcap which among other things includes an FTP transaction. I noticed that the FTP password field was set to . I mananged to find the very nice documentation over at http://www.bro-ids.org/documentation/scripts/base/protocols/ftp/main.html which made me tweak the default_capture_password variable to "T". This however did not seem to change the password field. I then noticed that there was a specific check for known anonymous users which would make sense to not store a password for, but the user in my case, "ftpuser", was not in the list. Looking at the script it seemed to me the test case was reversed, actually changing the password to if the user was _not_ in the anonymous list so i simply changed it. This made the password visible. I then tested changing the default_capture_password variable back and forth but it didnt seem to make a difference: the password was shown either way. Based on this i grepped around somewhat in the other scripts and found that the HTTP script did a similiar thing with a default_capture_password variable. The actual use of the variable seemed to be missing from the FTP script, so i added that as well based on the HTTP example. Since i guess code says more than words, I created a git patch just to show what was done, it can be fetched here: http://dump.komsi.se/bro/0001-Fix-FTP-script-password-saving.patch Finally, i might have missed it in the docs, but what would be the preferable way to enable password capture? I'm guessing it belongs in bro/site/local.bro but i'm not sure about the syntax to describe (in this case) if the setting relates to HTTP or FTP etc. Thanks for this great framework, it sure looks very interesting! Regards, Patrik Lundin From seth at icir.org Sat Apr 21 11:38:04 2012 From: seth at icir.org (Seth Hall) Date: Sat, 21 Apr 2012 14:38:04 -0400 Subject: [Bro] FTP password saving In-Reply-To: <20120421130505.GA8006@shell.virtality.se> References: <20120421130505.GA8006@shell.virtality.se> Message-ID: On Apr 21, 2012, at 9:05 AM, Patrik Lundin wrote: > I then noticed that there was a specific check for known anonymous users > which would make sense to not store a password for, but the user in my case, > "ftpuser", was not in the list. Looking at the script it seemed to me the > test case was reversed, actually changing the password to if > the user was _not_ in the anonymous list so i simply changed it. This > made the password visible. That line of code actually works backwards from what you are thinking. The password is always captured into that field if it's seen. That line just overwrites the password before logging it if you decide that you actually don't want the password (you can inspect at runtime, but it's not logged). > I then tested changing the default_capture_password variable back and > forth but it didnt seem to make a difference: the password was shown > either way. > The actual use of the variable seemed to be missing from the FTP script, > so i added that as well based on the HTTP example. Good catch! I totally missed that. > Finally, i might have missed it in the docs, but what would be the > preferable way to enable password capture? I'm guessing it belongs in > bro/site/local.bro but i'm not sure about the syntax to describe (in > this case) if the setting relates to HTTP or FTP etc. redef FTP::default_capture_password = T; Doing it in local.bro should be fine. > Thanks for this great framework, it sure looks very interesting! Thanks for reporting the bug. I committed a slightly different fix to our fastpath branch and added "ftpuser" as another anonymous username. The fix will show up in the 2.1 release. You seem to have made the changes for yourself now to make this work at least, right? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From patrik.lundin.swe at gmail.com Sat Apr 21 17:09:58 2012 From: patrik.lundin.swe at gmail.com (Patrik Lundin) Date: Sun, 22 Apr 2012 02:09:58 +0200 Subject: [Bro] FTP password saving In-Reply-To: References: <20120421130505.GA8006@shell.virtality.se> Message-ID: <20120422000955.GA9553@shell.virtality.se> On Sat, Apr 21, 2012 at 02:38:04PM -0400, Seth Hall wrote: > > That line of code actually works backwards from what you are thinking. > The password is always captured into that field if it's seen. That > line just overwrites the password before logging it if you decide that > you actually don't want the password (you can inspect at runtime, but > it's not logged). > Ah, i guess i expected that "capture_password" included logging it, but i realize it makes sense to have it available at runtime yet keep it out of the logs. Have i grasped it correctly that the general thinking is that the "capture_password" knob is only intended to control if the password is available at runtime for analysis, but that you usually don't want to log it except for a few select users? Why have you decided that users "probably" want to log the password for anonymous/guest users? > > redef FTP::default_capture_password = T; > > Doing it in local.bro should be fine. > Not sure if i'm doing it wrong, but i just added that to the end of local.bro and it didnt't seem to do anything. > > Thanks for reporting the bug. I committed a slightly different fix to > our fastpath branch and added "ftpuser" as another anonymous username. > The fix will show up in the 2.1 release. You seem to have made the > changes for yourself now to make this work at least, right? > Thanks for looking into it and explaining stuff, i actually dont have a burning need to have "ftpuser" added, it just happened to be the user that was used in this specific pcap. Based on my misconception that capturing the password was the same as wanting to log it i thought an errenous negation had snuck in :). Lets say i wanted to actually log passwords for all users, what would be the proper way to accomplish that? Thanks for your time, Patrik Lundin From seth at icir.org Sat Apr 21 17:41:18 2012 From: seth at icir.org (Seth Hall) Date: Sat, 21 Apr 2012 20:41:18 -0400 Subject: [Bro] FTP password saving In-Reply-To: <20120422000955.GA9553@shell.virtality.se> References: <20120421130505.GA8006@shell.virtality.se> <20120422000955.GA9553@shell.virtality.se> Message-ID: On Apr 21, 2012, at 8:09 PM, Patrik Lundin wrote: > Why have you decided that users > "probably" want to log the password for anonymous/guest users? The passwords used for those accounts are considered informational. They aren't considered a secret value so we log it. :) >> redef FTP::default_capture_password = T; > Not sure if i'm doing it wrong, but i just added that to the end of > local.bro and it didnt't seem to do anything. Sorry, I forgot to be explicit that what you reported was a real bug which prevented this from working. It's fixed in our fastpath branch (for small fixed like this) and will be incorporated into our 2.1 release. > Thanks for looking into it and explaining stuff, i actually dont have a > burning need to have "ftpuser" added, it just happened to be the user > that was used in this specific pcap. That's exactly the reason I added it. We try to stick to what's actually seen in the real world and "ftpuser" seems like a reasonable name to add. > Lets say i wanted to actually log passwords for all users, what would > be the proper way to accomplish that? Considering that this feature is broken in 2.0 you can apply the changes that I did in fastpath to your repository clone: diff --git a/scripts/base/protocols/ftp/main.bro b/scripts/base/protocols/ftp/main.bro index e6c0131..aa7d824 100644 --- a/scripts/base/protocols/ftp/main.bro +++ b/scripts/base/protocols/ftp/main.bro @@ -22,7 +22,7 @@ export { const default_capture_password = F &redef; ## User IDs that can be considered "anonymous". - const guest_ids = { "anonymous", "ftp", "guest" } &redef; + const guest_ids = { "anonymous", "ftp", "ftpuser", "guest" } &redef; type Info: record { ## Time when the command was sent. @@ -160,8 +160,12 @@ function ftp_message(s: Info) # or it's a deliberately logged command. if ( |s$tags| > 0 || (s?$cmdarg && s$cmdarg$cmd in logged_commands) ) { - if ( s?$password && to_lower(s$user) !in guest_ids ) + if ( s?$password && + !s$capture_password && + to_lower(s$user) !in guest_ids ) + { s$password = ""; + } local arg = s$cmdarg$arg; if ( s$cmdarg$cmd in file_cmds ) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From patrik.lundin.swe at gmail.com Sat Apr 21 18:36:58 2012 From: patrik.lundin.swe at gmail.com (Patrik Lundin) Date: Sun, 22 Apr 2012 03:36:58 +0200 Subject: [Bro] FTP password saving In-Reply-To: References: <20120421130505.GA8006@shell.virtality.se> <20120422000955.GA9553@shell.virtality.se> Message-ID: <20120422013655.GA9787@shell.virtality.se> On Sat, Apr 21, 2012 at 08:41:18PM -0400, Seth Hall wrote: > > That's exactly the reason I added it. We try to stick to what's > actually seen in the real world and "ftpuser" seems like a reasonable > name to add. > Just to be explicit, this pcap is a "dig around these bits and find out what is bad" training/testing example. I'm not sure it is actually based on traffic caught in the wild. > > - if ( s?$password && to_lower(s$user) !in guest_ids ) > + if ( s?$password && > + !s$capture_password && > + to_lower(s$user) !in guest_ids ) > + { > s$password = ""; > + } > I'm not sure i'm mentally parsing this right... Wouldn't this change actually make the code log all passwords (as i expected in the first place) if capture_password is true? Wasn't your intention to always keep the passwords out of the logs unless specifically anonymous/guest? It's getting very late/early here, hope im not being extraordinarily slow! Regards, Patrik Lundin From seth at icir.org Sat Apr 21 18:55:46 2012 From: seth at icir.org (Seth Hall) Date: Sat, 21 Apr 2012 21:55:46 -0400 Subject: [Bro] FTP password saving In-Reply-To: <20120422013655.GA9787@shell.virtality.se> References: <20120421130505.GA8006@shell.virtality.se> <20120422000955.GA9553@shell.virtality.se> <20120422013655.GA9787@shell.virtality.se> Message-ID: <0023BA92-1ABE-4533-90F7-BAB1EA524AC1@icir.org> On Apr 21, 2012, at 9:36 PM, Patrik Lundin wrote: > I'm not sure i'm mentally parsing this right... Wouldn't this change > actually make the code log all passwords (as i expected in the first > place) if capture_password is true? Wasn't your intention to always keep > the passwords out of the logs unless specifically anonymous/guest? You've got it right. Anonymous users always have their passwords logged. You can also specify that any arbitrary FTP session should have it's password logged by setting the $capture_password field to T. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From relevantusername at gmail.com Mon Apr 23 13:00:34 2012 From: relevantusername at gmail.com (relevant username) Date: Mon, 23 Apr 2012 15:00:34 -0500 Subject: [Bro] SMB/NetBIOS Message-ID: Greetings all. I was wondering if anyone had a script (or documentation) that logs SMB traffic and activities including file names and folders being read, written, connections, etc. The only information I found regarding this is from the event.bif.bro which ships with Bro 2.0. ## .. todo:: Bro's current default configuration does not activate the protocol ## analyzer that generates this event; the corresponding script has not yet ## been ported to Bro 2.x. To still enable this event, one needs to add a ## corresponding entry to :bro:see:`dpd_config` or a DPD payload signature. Thank you! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120423/54cc7924/attachment.html From seth at icir.org Mon Apr 23 13:17:16 2012 From: seth at icir.org (Seth Hall) Date: Mon, 23 Apr 2012 16:17:16 -0400 Subject: [Bro] SMB/NetBIOS In-Reply-To: References: Message-ID: On Apr 23, 2012, at 4:00 PM, relevant username wrote: > I was wondering if anyone had a script (or documentation) that logs SMB traffic and activities including file names and folders being read, written, connections, etc. I believe that this analyzer is broken. I'm still working on the rewrite of the SMB analyzer which initially will probably focus primarily around file transfers and some of the associated data. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From mcholste at gmail.com Mon Apr 23 13:25:29 2012 From: mcholste at gmail.com (Martin Holste) Date: Mon, 23 Apr 2012 15:25:29 -0500 Subject: [Bro] SMB/NetBIOS In-Reply-To: References: Message-ID: A big +1 for this analyzer. Turning on this functionality on the server-side impacts performance significantly, so being able to do this on the network is a big win. On Mon, Apr 23, 2012 at 3:17 PM, Seth Hall wrote: > > On Apr 23, 2012, at 4:00 PM, relevant username wrote: > >> I was wondering if anyone had a script (or documentation) that logs SMB traffic and activities including file names and folders being read, written, connections, etc. > > > I believe that this analyzer is broken. ?I'm still working on the rewrite of the SMB analyzer which initially will probably focus primarily around file transfers and some of the associated data. > > ?.Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From seth at icir.org Mon Apr 23 13:41:37 2012 From: seth at icir.org (Seth Hall) Date: Mon, 23 Apr 2012 16:41:37 -0400 Subject: [Bro] SMB/NetBIOS In-Reply-To: References: Message-ID: On Apr 23, 2012, at 4:25 PM, Martin Holste wrote: > A big +1 for this analyzer. Turning on this functionality on the > server-side impacts performance significantly, so being able to do > this on the network is a big win. Do you currently monitor in locations where you see SMB traffic? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From relevantusername at gmail.com Mon Apr 23 13:43:00 2012 From: relevantusername at gmail.com (relevant username) Date: Mon, 23 Apr 2012 15:43:00 -0500 Subject: [Bro] SMB/NetBIOS In-Reply-To: References: Message-ID: I can't answer for Martin, but we do, heavily. On Mon, Apr 23, 2012 at 3:41 PM, Seth Hall wrote: > > On Apr 23, 2012, at 4:25 PM, Martin Holste wrote: > > > A big +1 for this analyzer. Turning on this functionality on the > > server-side impacts performance significantly, so being able to do > > this on the network is a big win. > > > Do you currently monitor in locations where you see SMB traffic? > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20120423/6155bbdd/attachment.html From mcholste at gmail.com Mon Apr 23 13:45:36 2012 From: mcholste at gmail.com (Martin Holste) Date: Mon, 23 Apr 2012 15:45:36 -0500 Subject: [Bro] SMB/NetBIOS In-Reply-To: References: Message-ID: Yep, the egress of a datacenter where fileservers sit is a great place to put an NSM sensor, and all clients have to pass through those gates, so all desktop-fileserver access can be audited. On Mon, Apr 23, 2012 at 3:43 PM, relevant username wrote: > I can't answer for Martin, but we do, heavily. > > > On Mon, Apr 23, 2012 at 3:41 PM, Seth Hall wrote: >> >> >> On Apr 23, 2012, at 4:25 PM, Martin Holste wrote: >> >> > A big +1 for this analyzer. ?Turning on this functionality on the >> > server-side impacts performance significantly, so being able to do >> > this on the network is a big win. >> >> >> Do you currently monitor in locations where you see SMB traffic? >> >> ?.Seth >> >> -- >> Seth Hall >> International Computer Science Institute >> (Bro) because everyone has a network >> http://www.bro-ids.org/ >> >