From Laurentiu.Cucos at compuware.com Mon Dec 1 09:40:12 2008 From: Laurentiu.Cucos at compuware.com (Cucos, Laurentiu) Date: Mon, 1 Dec 2008 12:40:12 -0500 Subject: [Bro] FW: binpac related question Message-ID: <035AF547E1FC554AA8C917834328AD3204AAFE30@nasa-dtw-ex002.nasa.cpwr.corp> Hi All, > Tying to use binpac to create a new protocol analyzer I run into a > small issue relative to byte alignment. I am thinking this must be a > known aspect however I didn't find much documentation about it. > > Padding in binpac works well inside the packet. However, sometime the > packet ends with a structure that requires padding but the packet > doesn't carry the extra bytes for the padding. As a result binpac > fires an exception and the parsing is terminated before extracting the > data. > > Here is a sample code: > type ByteArray = record{ > len : uint32; > data: uint8[len]; > pad : padding align 4; > }; > > When the last 6 bytes of a packet that ends with the above record is: > 00 00 00 02 AA BB (with: len=00 00 00 02; data ={AA, BB}) binpac > throws: > > 0.000000 binpac exception: binpac exception: out_of_bound: ByteArray: > 8 > 6 > > > Removing the padding entry from the code above results in a correct > execution, however the padding is needed for the case when ByteArray > is strictly inside the packet. > > The solution I am thinking is way too complex (pass some arguments > that will help decide if the padding is needed), but I was wondering > if there is a much more elegant approach. > > Another option will be to change the binpac compiler code > (pac_record.cc) in method: void > RecordPaddingField::GenFieldEnd(Output* out_cc, Env* env, const > DataPtr& field_begin) but I am thinking this case should be covered in > a different way. > > Any suggestions would be greatly appreciated. > > Thank you, > Laurentiu Cucos > > The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20081201/3847258e/attachment.html From edthoma at sandia.gov Mon Dec 1 12:06:27 2008 From: edthoma at sandia.gov (Eric Thomas) Date: Mon, 1 Dec 2008 12:06:27 -0800 (PST) Subject: [Bro] DPD and similar analysis Message-ID: I have a project to do DPD-like offline analysis and was looking for help and feedback. First off, I'm trying to make sure that DPD is working, so I tried to get bro to write ServerFound messages to the notice log. BTW, in all my tests I made sure my capture filter included "or (tcp or udp or icmp)". To get bro to report ALL servers found, I temporarily modified detect-protocols.bro and commented out the two sections that would prevent generating notices for "well known ports" (using dpd_config). So I would expect to see ServerFound messages for all protocols that have been detected. Here is my command line (zzz-custom just redefines capture_filters as stated above): bro -r pcapfile.pcap conn dpd irc-bot dyn-disable detect-protocols detect-protocols-http proxy http ssh zzz-custom When I run this against the pcap file that contains tons of HTTP, SSH and likely other traffic, the only ServerFound messages are for SSH. If I was getting DPD to work correctly, I would expect to find HTTP ServerFound messages. I'm looking to get bro to output all ProtocolFound and ServerFound messages, so any help to get that to happen would be appreciated. Once I figure this out, then I'll use DPD for it's intended purpose: to detect protocols on non-standard ports. However, I'm also supposed to do the inverse, that is, detect non-standard protocols on standard ports. Any thoughts on how I could do this? Thanks, Eric T Sandia National Labs From christian at whoop.org Mon Dec 1 12:20:16 2008 From: christian at whoop.org (Christian Kreibich) Date: Mon, 01 Dec 2008 12:20:16 -0800 Subject: [Bro] Problem running bro.rc --start In-Reply-To: <5174153c0811250929l1d2df394q86b1623e0b282ae5@mail.gmail.com> References: <5174153c0811240734p43fd2c8bnc93af3b402bf40d1@mail.gmail.com> <20081125074116.4086087d@mygw.lan.mynetwork.local> <5174153c0811250929l1d2df394q86b1623e0b282ae5@mail.gmail.com> Message-ID: <1228162816.18901.71.camel@strangepork> On Tue, 2008-11-25 at 18:29 +0100, M. GAD wrote: > I had so many problems with autoconf and sisters, I managed to resolve > them all but the one related to autoheader. If you're referring to issues specific to Bro's build configuration, then it'd be helpful if you could post details regarding these issues so we can look into potential fixes. > It seems that Bro is more adapted for FreeBSD installations. No, our releases should build out-of-the-box on Linux as well, including Ubuntu. -- Cheers, Christian From rpang at cs.princeton.edu Mon Dec 1 18:02:45 2008 From: rpang at cs.princeton.edu (Ruoming Pang) Date: Mon, 1 Dec 2008 21:02:45 -0500 Subject: [Bro] FW: binpac related question In-Reply-To: <035AF547E1FC554AA8C917834328AD3204AAFE30@nasa-dtw-ex002.nasa.cpwr.corp> References: <035AF547E1FC554AA8C917834328AD3204AAFE30@nasa-dtw-ex002.nasa.cpwr.corp> Message-ID: Hi Laurentiu, Is the padding always at the end? If so, you can use: padding: bytestring &restofdata; Ruoming 2008/12/1 Cucos, Laurentiu > Hi All, > > Tying to use binpac to create a new protocol analyzer I run into a small > issue relative to byte alignment. I am thinking this must be a known aspect > however I didn't find much documentation about it. > > Padding in binpac works well inside the packet. However, sometime the > packet ends with a structure that requires padding but the packet doesn't > carry the extra bytes for the padding. As a result binpac fires an exception > and the parsing is terminated before extracting the data. > > Here is a sample code: > type ByteArray = record{ > len : uint32; > data: uint8[len]; > pad : padding align 4; > }; > > When the last 6 bytes of a packet that ends with the above record is: 00 00 > 00 02 AA BB (with: len=00 00 00 02; data ={AA, BB}) binpac throws: > > 0.000000 binpac exception: binpac exception: out_of_bound: ByteArray: 8 > 6 > > Removing the padding entry from the code above results in a correct > execution, however the padding is needed for the case when ByteArray is > strictly inside the packet. > > The solution I am thinking is way too complex (pass some arguments that > will help decide if the padding is needed), but I was wondering if there is > a much more elegant approach. > > Another option will be to change the binpac compiler code (pac_record.cc) > in method: void RecordPaddingField::GenFieldEnd(Output* out_cc, Env* env, > const DataPtr& field_begin) but I am thinking this case should be covered in > a different way. > > Any suggestions would be greatly appreciated. > > Thank you, > Laurentiu Cucos > > > The contents of this e-mail are intended for the named addressee only. It > contains information that may be confidential. Unless you are the named > addressee or an authorized designee, you may not copy or use it, or disclose > it to anyone else. If you received it in error please notify us immediately > and then destroy it. > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20081201/2a774378/attachment.html From Laurentiu.Cucos at compuware.com Tue Dec 2 10:09:13 2008 From: Laurentiu.Cucos at compuware.com (Cucos, Laurentiu) Date: Tue, 2 Dec 2008 13:09:13 -0500 Subject: [Bro] FW: binpac related question In-Reply-To: References: <035AF547E1FC554AA8C917834328AD3204AAFE30@nasa-dtw-ex002.nasa.cpwr.corp> Message-ID: <035AF547E1FC554AA8C917834328AD3204AAFE37@nasa-dtw-ex002.nasa.cpwr.corp> Hi Ruoming, Thanks for the answer. First, I would like to say how impressed I am with binpac capabilities and the approach used. Once I had a test environment setup and went over the samples and documentation, I was able to develop a fairly extensive decoder for a relatively complex protocol (IIOP) in just a day or so (took longer to go over the protocol specs). Regarding the issue in question, the padding is not always at the end. Depending on some protocol data (i.e. version), the ByteArray record can be either in mid packet or at the end. The things are complicated by the fact that this record is part of other records (that ultimately depend on version) - and I would not like to duplicate records with every version. As of now I found a solution by changing: pac_record.cc: void RecordPaddingField::GenFieldEnd(Output* out_cc, Env* env, const DataPtr& field_begin) ............. case PAD_TO_NEXT_WORD: ..... // new code: disable the padding if exceeds the packet size (not sure about all implications) out_cc->println("%s = (%s == 0 || (%s + %s > %s)) ? 0 : %d - %s;", padding_var, padding_var, padding_var, field_begin.ptr_expr(), env->RValue(end_of_data), wordsize_, padding_var); // original code: // out_cc->println("%s = (%s == 0) ? 0 : %d - %s;", // padding_var, // padding_var, // wordsize_, // padding_var); Any feedback will be highly appreciated. Thank you, Laurentiu ________________________________ From: ruoming at gmail.com [mailto:ruoming at gmail.com] On Behalf Of Ruoming Pang Sent: Monday, December 01, 2008 9:03 PM To: Cucos, Laurentiu Cc: Bro at bro-ids.org Subject: Re: [Bro] FW: binpac related question Hi Laurentiu, Is the padding always at the end? If so, you can use: padding: bytestring &restofdata; Ruoming 2008/12/1 Cucos, Laurentiu Hi All, Tying to use binpac to create a new protocol analyzer I run into a small issue relative to byte alignment. I am thinking this must be a known aspect however I didn't find much documentation about it. Padding in binpac works well inside the packet. However, sometime the packet ends with a structure that requires padding but the packet doesn't carry the extra bytes for the padding. As a result binpac fires an exception and the parsing is terminated before extracting the data. Here is a sample code: type ByteArray = record{ len : uint32; data: uint8[len]; pad : padding align 4; }; When the last 6 bytes of a packet that ends with the above record is: 00 00 00 02 AA BB (with: len=00 00 00 02; data ={AA, BB}) binpac throws: 0.000000 binpac exception: binpac exception: out_of_bound: ByteArray: 8 > 6 Removing the padding entry from the code above results in a correct execution, however the padding is needed for the case when ByteArray is strictly inside the packet. The solution I am thinking is way too complex (pass some arguments that will help decide if the padding is needed), but I was wondering if there is a much more elegant approach. Another option will be to change the binpac compiler code (pac_record.cc) in method: void RecordPaddingField::GenFieldEnd(Output* out_cc, Env* env, const DataPtr& field_begin) but I am thinking this case should be covered in a different way. Any suggestions would be greatly appreciated. Thank you, Laurentiu Cucos The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20081202/f9bf9e42/attachment.html From robin at icir.org Tue Dec 2 13:30:16 2008 From: robin at icir.org (Robin Sommer) Date: Tue, 2 Dec 2008 13:30:16 -0800 Subject: [Bro] DPD and similar analysis In-Reply-To: References: Message-ID: <20081202213016.GQ58544@icir.org> On Mon, Dec 01, 2008 at 12:06 -0800, you wrote: > or icmp)". To get bro to report ALL servers found, I temporarily > modified detect-protocols.bro and commented out the two sections that > would prevent generating notices for "well known ports" (using > dpd_config). So I would expect to see ServerFound messages for all > protocols that have been detected. Hmm... I can't reproduce that. Likewise commenting out the dpd_config check in detect_protocols.bro and running with a tiny SSH trace I get: > bro -r ssh.trace ssh dpd detect-protocols 1150485521.634103 ProtocolFound XXX.XXX.XXX.XXX/49244 > XXX.XXX.XXX.XXX/ssh SSH on port 22/tcp 1150485521.634103 ServerFound XXX.XXX.XXX.XXX: SSH server on port 22/tcp > bro -r pcapfile.pcap conn dpd irc-bot dyn-disable detect-protocols > detect-protocols-http proxy http ssh zzz-custom Note that you should use http-request instead of http, and potentially also load http-{reply,body}.bro. > Once I figure this out, then I'll use DPD for it's intended purpose: to > detect protocols on non-standard ports. However, I'm also supposed to do > the inverse, that is, detect non-standard protocols on standard ports. Any > thoughts on how I could do this? dyn-disable reports ProtocolViolations when the analyzer can't parse the protocol, which is an indicator that there might be something running on that port which isn't the standard protocol. Also, if you enable dpd_conn_logs in conn.bro, the service field indicates the DPD result, including whether it could *not* parse the protocol. From CHANGES: The new script variable dpd_conn_logs (default F), if true, changes the semantics of the service field in connection logs written to conn.log, as follows (Robin Sommer). It becomes a comma-separated list of analyzers confirmed by DPD to parse the connection's payload. If no analyzer could confirm its protocol, but the connection uses a well-known port, the service is the name of the port with "?" appended (e.g., "http?"), as long as the corresponding analyzer has not declined the connection. In addition, ftp-data sessions are labeled "ftp-data" and portmapper connections are labeled with the specific method-call (just as before). dpd_conn_logs defaults to F because the change in semantics may break scripts that parse conn.logs; but it will likely change to the default in the future. With dpd_conn_logs turned off, conn logs are generated as they used to be, with a few rare exceptions (with previous versions, the service field was sometimes determined while the connection was still alive; now it's always determined at the time when the conn.log entry is written out). Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From ssmit7 at gmail.com Tue Dec 2 14:36:12 2008 From: ssmit7 at gmail.com (Stephen Smith) Date: Tue, 2 Dec 2008 17:36:12 -0500 Subject: [Bro] linux vs freebsd Message-ID: Hello all, Over time I have seen various posts about building Bro on Linux instead of FreeBSD, and recently someone said I think that it should build out of the box on both systems. My understanding has always been that performance is much better under FreeBSD due to the way bpf is implemented, but is it workable on Linux as well? Anyone had experience with a production Bro box on Linux? Thanks in advance for any input. Stephen -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20081202/66e66f61/attachment.html From vern at icir.org Wed Dec 3 12:39:26 2008 From: vern at icir.org (Vern Paxson) Date: Wed, 03 Dec 2008 12:39:26 -0800 Subject: [Bro] linux vs freebsd In-Reply-To: (Tue, 02 Dec 2008 17:36:12 EST). Message-ID: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> > My understanding has always been that performance is > much better under FreeBSD due to the way bpf is implemented Historically that's been true. > but is it > workable on Linux as well? Anyone had experience with a production Bro box > on Linux? One of our production Bro boxes is running Linux. It occasionally drops packets under a not very heavy load (it's monitoring a 100 Mbps link that's not used heavily), but so far I haven't been able to correlate these with a particular cause such as high-rate traffic spikes. That said, we continue to use FreeBSD for our very-high-performance (1-10 Gbps) systems. I don't know whether the Linux packet capture has improved to where it could also take on these loads (that would of course require that the drops seen on the 100 Mbps link aren't due simply to packet rate). Linux is supposed to have gotten quite a bit better in this regard. Vern From kgunders at teamcool.net Wed Dec 3 14:58:16 2008 From: kgunders at teamcool.net (Ken Gunderson) Date: Wed, 03 Dec 2008 15:58:16 -0700 Subject: [Bro] linux vs freebsd In-Reply-To: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> References: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> Message-ID: <1228345096.1126.79.camel@hermes> On Wed, 2008-12-03 at 12:39 -0800, Vern Paxson wrote: > > My understanding has always been that performance is > > much better under FreeBSD due to the way bpf is implemented > > Historically that's been true. > > > but is it > > workable on Linux as well? Anyone had experience with a production Bro box > > on Linux? > > One of our production Bro boxes is running Linux. It occasionally drops > packets under a not very heavy load (it's monitoring a 100 Mbps link that's > not used heavily), but so far I haven't been able to correlate these with > a particular cause such as high-rate traffic spikes. > > That said, we continue to use FreeBSD for our very-high-performance > (1-10 Gbps) systems. I don't know whether the Linux packet capture has > improved to where it could also take on these loads (that would of course > require that the drops seen on the 100 Mbps link aren't due simply to > packet rate). Linux is supposed to have gotten quite a bit better in > this regard. OpenBSD would be an attractive option but the last I checked the way back machine bro had issues on OBSD... Curious if anyone been doing anything in this regard more recently? -- Ken Gunderson From christian at whoop.org Wed Dec 3 18:15:31 2008 From: christian at whoop.org (Christian Kreibich) Date: Wed, 03 Dec 2008 18:15:31 -0800 Subject: [Bro] linux vs freebsd In-Reply-To: <1228345096.1126.79.camel@hermes> References: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> <1228345096.1126.79.camel@hermes> Message-ID: <1228356931.10810.192.camel@strangepork> On Wed, 2008-12-03 at 15:58 -0700, Ken Gunderson wrote: > OpenBSD would be an attractive option but the last I checked the way > back machine bro had issues on OBSD... Curious if anyone been doing > anything in this regard more recently? To put it mildly, I've lost track of the number of times I've cursed OpenBSD because of ongoing build headaches. We generally use it neither for development nor production, but whenever people report build problems, we try to go in and fix them. Thus, if there are OpenBSD issues with the current release, please let us know. -- Cheers, Christian From kgunders at teamcool.net Wed Dec 3 20:05:34 2008 From: kgunders at teamcool.net (Ken Gunderson) Date: Wed, 03 Dec 2008 21:05:34 -0700 Subject: [Bro] linux vs freebsd In-Reply-To: <1228356931.10810.192.camel@strangepork> References: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> <1228345096.1126.79.camel@hermes> <1228356931.10810.192.camel@strangepork> Message-ID: <1228363534.1126.124.camel@hermes> On Wed, 2008-12-03 at 18:15 -0800, Christian Kreibich wrote: > On Wed, 2008-12-03 at 15:58 -0700, Ken Gunderson wrote: > > OpenBSD would be an attractive option but the last I checked the way > > back machine bro had issues on OBSD... Curious if anyone been doing > > anything in this regard more recently? > > To put it mildly, I've lost track of the number of times I've cursed > OpenBSD because of ongoing build headaches. We generally use it neither > for development nor production, but whenever people report build > problems, we try to go in and fix them. Thus, if there are OpenBSD > issues with the current release, please let us know. > I haven't tried on OBSD in some time. Interesting comments about OpenBSD headaches. Been using both FreeBSD and OpenBSD since 2.x days. I know there have been ongoing issues w/Bro on OBSD but otherwise it's been rock solid. Wish I could say same about FreeBSD, wh/we've all but abandoned. We use OBSD in more narrowly targeted capacities, however, so perhaps not a fair comparison. -- Ken Gunderson From christian at whoop.org Thu Dec 4 11:15:37 2008 From: christian at whoop.org (Christian Kreibich) Date: Thu, 04 Dec 2008 11:15:37 -0800 Subject: [Bro] linux vs freebsd In-Reply-To: <1228363534.1126.124.camel@hermes> References: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> <1228345096.1126.79.camel@hermes> <1228356931.10810.192.camel@strangepork> <1228363534.1126.124.camel@hermes> Message-ID: <1228418137.20047.49.camel@strangepork> On Wed, 2008-12-03 at 21:05 -0700, Ken Gunderson wrote: > I haven't tried on OBSD in some time. Interesting comments about OpenBSD > headaches. Been using both FreeBSD and OpenBSD since 2.x days. I know > there have been ongoing issues w/Bro on OBSD but otherwise it's been > rock solid. Wish I could say same about FreeBSD, wh/we've all but > abandoned. We use OBSD in more narrowly targeted capacities, however, > so perhaps not a fair comparison. I don't mean to suggest doubt in OpenBSD's technical quality, it's no doubt a solid OS. It's merely the (imho) fact that once things build fine on Linux + OSX + FreeBSD, there's still a significant chance of build woes on OpenBSD, which translates into extra effort needed on our side. That said, I just downloaded 1.4 on an OpenBSD 4.2 machine, and am happy to report that the whole suite built fine. -- Cheers, Christian From kgunders at teamcool.net Thu Dec 4 11:32:26 2008 From: kgunders at teamcool.net (Ken Gunderson) Date: Thu, 04 Dec 2008 12:32:26 -0700 Subject: [Bro] linux vs freebsd In-Reply-To: <1228418137.20047.49.camel@strangepork> References: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> <1228345096.1126.79.camel@hermes> <1228356931.10810.192.camel@strangepork> <1228363534.1126.124.camel@hermes> <1228418137.20047.49.camel@strangepork> Message-ID: <1228419146.1126.185.camel@hermes> On Thu, 2008-12-04 at 11:15 -0800, Christian Kreibich wrote: > On Wed, 2008-12-03 at 21:05 -0700, Ken Gunderson wrote: > > I haven't tried on OBSD in some time. Interesting comments about OpenBSD > > headaches. Been using both FreeBSD and OpenBSD since 2.x days. I know > > there have been ongoing issues w/Bro on OBSD but otherwise it's been > > rock solid. Wish I could say same about FreeBSD, wh/we've all but > > abandoned. We use OBSD in more narrowly targeted capacities, however, > > so perhaps not a fair comparison. > > I don't mean to suggest doubt in OpenBSD's technical quality, it's no > doubt a solid OS. It's merely the (imho) fact that once things build > fine on Linux + OSX + FreeBSD, there's still a significant chance of > build woes on OpenBSD, which translates into extra effort needed on our > side. > > That said, I just downloaded 1.4 on an OpenBSD 4.2 machine, and am happy > to report that the whole suite built fine. > Thank you for the report. In next couple weeks I'll be bringing a 4.4 box online and may have time to test the build on it. Perhaps someone with a bit more time could even try to get it in the OBSD tree. I should amend my FBSD statement to note that pre 5.x releases, and 4.x in particular, served us extremely well! -- Ken Gunderson From taosecurity at gmail.com Thu Dec 4 17:05:01 2008 From: taosecurity at gmail.com (Richard Bejtlich) Date: Thu, 4 Dec 2008 20:05:01 -0500 Subject: [Bro] linux vs freebsd In-Reply-To: <1228419146.1126.185.camel@hermes> References: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> <1228345096.1126.79.camel@hermes> <1228356931.10810.192.camel@strangepork> <1228363534.1126.124.camel@hermes> <1228418137.20047.49.camel@strangepork> <1228419146.1126.185.camel@hermes> Message-ID: <120ef0530812041705w6225fa80qd1d64dd72ab4319b@mail.gmail.com> On Thu, Dec 4, 2008 at 2:32 PM, Ken Gunderson wrote: > > I should amend my FBSD statement to note that pre 5.x releases, and 4.x > in particular, served us extremely well! > > -- > Ken Gunderson > Hi Ken, It might be time to try FreeBSD again. I've been running 6.x and 7.0 (soon 7.1) in production for the last few years and have been happy. Early 5.x (5.0 through 5.2.1) weren't considered "STABLE" (that happened in 5.3), so I understand plenty of concerns with those releases. Sincerely, Richard From ssmit7 at gmail.com Thu Dec 4 19:43:48 2008 From: ssmit7 at gmail.com (Stephen Smith) Date: Thu, 4 Dec 2008 22:43:48 -0500 Subject: [Bro] linux vs freebsd In-Reply-To: <120ef0530812041705w6225fa80qd1d64dd72ab4319b@mail.gmail.com> References: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> <1228345096.1126.79.camel@hermes> <1228356931.10810.192.camel@strangepork> <1228363534.1126.124.camel@hermes> <1228418137.20047.49.camel@strangepork> <1228419146.1126.185.camel@hermes> <120ef0530812041705w6225fa80qd1d64dd72ab4319b@mail.gmail.com> Message-ID: It sounds like FreeBSD is still the way to go for production Bro boxes. I have been very pleased with the speed and stability of FreeBSD 7, and 6.x before it. My interest in running Bro on Linux was due to some issues encountered in integrating FreeBSD into the enterprise environment I deal with. Something like RHEL is accepted a bit more by the C&A auditors since there are more defined "guidelines" for "securing" it. If that makes any sense at all. Stephen On Thu, Dec 4, 2008 at 8:05 PM, Richard Bejtlich wrote: > On Thu, Dec 4, 2008 at 2:32 PM, Ken Gunderson > wrote: > > > > I should amend my FBSD statement to note that pre 5.x releases, and 4.x > > in particular, served us extremely well! > > > > -- > > Ken Gunderson > > > > Hi Ken, > > It might be time to try FreeBSD again. I've been running 6.x and 7.0 > (soon 7.1) in production for the last few years and have been happy. > Early 5.x (5.0 through 5.2.1) weren't considered "STABLE" (that > happened in 5.3), so I understand plenty of concerns with those > releases. > > Sincerely, > > Richard > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20081204/465ee8e7/attachment.html From kgunders at teamcool.net Thu Dec 4 21:00:00 2008 From: kgunders at teamcool.net (Ken Gunderson) Date: Thu, 04 Dec 2008 22:00:00 -0700 Subject: [Bro] linux vs freebsd In-Reply-To: References: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> <1228345096.1126.79.camel@hermes> <1228356931.10810.192.camel@strangepork> <1228363534.1126.124.camel@hermes> <1228418137.20047.49.camel@strangepork> <1228419146.1126.185.camel@hermes> <120ef0530812041705w6225fa80qd1d64dd72ab4319b@mail.gmail.com> Message-ID: <1228453200.1126.301.camel@hermes> Lest you misinterpret, I could tell a LOT more horror stories about Linux, particularly RH, than FBSD. But with RHEL you get someone else to blame. That's the big one for corp liability types. On Thu, 2008-12-04 at 22:43 -0500, Stephen Smith wrote: > It sounds like FreeBSD is still the way to go for production Bro > boxes. I have been very pleased with the speed and stability of > FreeBSD 7, and 6.x before it. My interest in running Bro on Linux was > due to some issues encountered in integrating FreeBSD into the > enterprise environment I deal with. Something like RHEL is accepted a > bit more by the C&A auditors since there are more defined "guidelines" > for "securing" it. If that makes any sense at all. > > > Stephen > > > On Thu, Dec 4, 2008 at 8:05 PM, Richard Bejtlich > wrote: > On Thu, Dec 4, 2008 at 2:32 PM, Ken Gunderson > wrote: > > > > I should amend my FBSD statement to note that pre 5.x > releases, and 4.x > > in particular, served us extremely well! > > > > -- > > Ken Gunderson > > > > > Hi Ken, > > It might be time to try FreeBSD again. I've been running 6.x > and 7.0 > (soon 7.1) in production for the last few years and have been > happy. > Early 5.x (5.0 through 5.2.1) weren't considered > "STABLE" (that > happened in 5.3), so I understand plenty of concerns with > those > releases. > > Sincerely, > > Richard > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Ken Gunderson From vallentin at ICSI.Berkeley.EDU Fri Dec 5 10:31:42 2008 From: vallentin at ICSI.Berkeley.EDU (Matthias Vallentin) Date: Fri, 5 Dec 2008 10:31:42 -0800 Subject: [Bro] Determine potential events with a specific set of *.bro scripts Message-ID: <20081205183142.GE9473@icsi.berkeley.edu> Hi, I would like to determine which events could potentially be generated after loading a specific set of scripts. Has anyone tried this before? In this context, I remember seeing a script some time ago, that created a dependency/include graph of *.bro policy files. Unfortunately, I lost track of it. Matthias -- Matthias Vallentin vallentin at icsi.berkeley.edu http://matthias.vallentin.cc From vanepp at sfu.ca Fri Dec 5 10:58:31 2008 From: vanepp at sfu.ca (Peter Van Epp) Date: Fri, 5 Dec 2008 10:58:31 -0800 Subject: [Bro] linux vs freebsd In-Reply-To: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> References: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> Message-ID: <20081205185831.GA15971@sfu.ca> On Wed, Dec 03, 2008 at 12:39:26PM -0800, Vern Paxson wrote: > > My understanding has always been that performance is > > much better under FreeBSD due to the way bpf is implemented > > Historically that's been true. > > > but is it > > workable on Linux as well? Anyone had experience with a production Bro box > > on Linux? > > One of our production Bro boxes is running Linux. It occasionally drops > packets under a not very heavy load (it's monitoring a 100 Mbps link that's > not used heavily), but so far I haven't been able to correlate these with > a particular cause such as high-rate traffic spikes. > > That said, we continue to use FreeBSD for our very-high-performance > (1-10 Gbps) systems. I don't know whether the Linux packet capture has > improved to where it could also take on these loads (that would of course > require that the drops seen on the 100 Mbps link aren't due simply to > packet rate). Linux is supposed to have gotten quite a bit better in > this regard. > > Vern > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro If you haven't already, you may want to try Phil Wood's mmapped pcap library from http://public.lanl.gov/cpw/. While I haven't beat on this one at high volumes I have had argus losing more than %50 of the traffic on a loaded (jumbo frame) gig link and reduced that to close to 0 loss with the pf-ring mmapped linux code. Pf-ring (from www.ntop.org) is hard to get in and then somewhat unstable (at least in my experience but then we have web100 in the same kernel which may not be helping :-)). Phil Wood's code needs no kernel mods just the libpcap library rebuilt and an environment variable set to cause the program to use the mmap functions. Peter Van Epp / Operations and Technical Support Simon Fraser University, Burnaby, B.C. Canada From rreitz at fnal.gov Fri Dec 5 13:43:46 2008 From: rreitz at fnal.gov (Randolph Reitz) Date: Fri, 05 Dec 2008 15:43:46 -0600 Subject: [Bro] Trying to drop addresses... Message-ID: <205A6321-B5A9-4704-9FFF-45BAF9085AE5@fnal.gov> I am running... [brother at dtmb ~/work]$ svn info Path: . URL: http://svn.icir.org/bro/branches/robin/work Repository Root: http://svn.icir.org/bro Repository UUID: 040645db-9414-0410-b69e-f32faa466a09 Revision: 6442 Node Kind: directory Schedule: normal Last Changed Author: robin Last Changed Rev: 6440 Last Changed Date: 2008-12-03 12:42:35 -0600 (Wed, 03 Dec 2008) on [brother at dtmb ~]$ uname -a FreeBSD dtmb.fnal.gov 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #5: Thu Dec 4 10:49:18 CST 2008 rreitz at dtmb.fnal.gov:/usr/obj/usr/src/sys/ DTMB i386 I want to enable catch-and-release. So I modified 'policy/local/ local.bro' with ... # use catch-and-release redef Drop::use_catch_release = T; redef Drop::can_drop_connectivity = T; ... redef notice_action_filters += { ... [Drop::AddressDropIgnored] = tally_notice_type_and_ignore, [Drop::AddressDropped] = drop_source, ... }; I installed with 'cluster install' followed by 'cluster restart'. I see lots of 'PortScan' and 'AddressScan' in the logs, but nothing is dropping. [brother at dtmb ~]$ cluster print Drop::drop_info bro Drop::drop_info = { } I noticed that the notice_policy does not contain a 'NOTICE_DROP' action... [brother at dtmb ~/work]$ cluster print notice_policy bro notice_policy = { [result=NOTICE_FILE, pred=anonymous-function { if (Scan::n$note == AddressRestored && Scan::n?$src && Scan::n$src in Scan::shut_down_thresh_reached) Scan::shut_down_thresh_reached[Scan::n$src] = F; return (F); }, priority=1], [result=NOTICE_ALARM_ALWAYS, pred=anonymous-function { return (T); }, priority=0] } Do I need to modify the notice_policy? I don't see any NOTICE_DROP examples in notice-policy.bro. Thanks, Randy From vern at icir.org Fri Dec 5 14:09:22 2008 From: vern at icir.org (Vern Paxson) Date: Fri, 05 Dec 2008 14:09:22 -0800 Subject: [Bro] Determine potential events with a specific set of *.bro scripts In-Reply-To: <20081205183142.GE9473@icsi.berkeley.edu> (Fri, 05 Dec 2008 10:31:42 PST). Message-ID: <200812052209.mB5M9Q7Z009496@pork.ICSI.Berkeley.EDU> > I would like to determine which events could potentially be generated > after loading a specific set of scripts. Has anyone tried this before? Internal to Bro there's an analysis of which declared events do not have any means of being generated (this is the reporting enabled by redef check_for_unused_event_handlers = T). It should be straightforward to tweak that to report on which events *can* be generated. > In this context, I remember seeing a script some time ago, that created > a dependency/include graph of *.bro policy files. Unfortunately, I lost > track of it. (I don't recall this either.) Vern From christian at whoop.org Fri Dec 5 17:47:26 2008 From: christian at whoop.org (Christian Kreibich) Date: Fri, 05 Dec 2008 17:47:26 -0800 Subject: [Bro] Determine potential events with a specific set of *.bro scripts In-Reply-To: <200812052209.mB5M9Q7Z009496@pork.ICSI.Berkeley.EDU> References: <200812052209.mB5M9Q7Z009496@pork.ICSI.Berkeley.EDU> Message-ID: <1228528046.20047.258.camel@strangepork> On Fri, 2008-12-05 at 14:09 -0800, Vern Paxson wrote: > > In this context, I remember seeing a script some time ago, that created > > a dependency/include graph of *.bro policy files. Unfortunately, I lost > > track of it. > > (I don't recall this either.) I think you're referring to the plot I made a long time ago of the @load structure of the policy files: http://www.icir.org/christian/bro/brodep.pdf Alas, I can't seem to find any of the scripts I used. It only took a few minutes to create though, using GraphViz. -- Cheers, Christian From vallentin at ICSI.Berkeley.EDU Sun Dec 7 10:48:18 2008 From: vallentin at ICSI.Berkeley.EDU (Matthias Vallentin) Date: Sun, 7 Dec 2008 10:48:18 -0800 Subject: [Bro] Determine potential events with a specific set of *.bro scripts In-Reply-To: <200812052209.mB5M9Q7Z009496@pork.ICSI.Berkeley.EDU> References: <20081205183142.GE9473@icsi.berkeley.edu> <200812052209.mB5M9Q7Z009496@pork.ICSI.Berkeley.EDU> Message-ID: <20081207184818.GP9473@icsi.berkeley.edu> On Fri, Dec 05, 2008 at 02:09:22PM -0800, Vern Paxson wrote: > > I would like to determine which events could potentially be generated > > after loading a specific set of scripts. Has anyone tried this before? > > Internal to Bro there's an analysis of which declared events do not > have any means of being generated (this is the reporting enabled by > redef check_for_unused_event_handlers = T). It should be straightforward > to tweak that to report on which events *can* be generated. Thanks for the pointer, I attached a little patch that does exactly that. Matthias -- Matthias Vallentin vallentin at icir.org http://matthias.vallentin.cc -------------- next part -------------- Index: src/main.cc =================================================================== --- src/main.cc (revision 6147) +++ src/main.cc (working copy) @@ -1014,6 +1014,18 @@ delete dead_handlers; + EventRegistry::string_list* alive_handlers = + event_registry->UsedHandlers(); + + if ( alive_handlers->length() > 0 && dump_used_event_handlers ) + { + message("invoked event handlers:"); + for ( int i = 0; i < alive_handlers->length(); ++i ) + message((*alive_handlers)[i]); + } + + delete alive_handlers; + if ( do_notice_analysis ) notice_analysis(); Index: src/EventRegistry.cc =================================================================== --- src/EventRegistry.cc (revision 6147) +++ src/EventRegistry.cc (working copy) @@ -54,6 +54,25 @@ return names; } +EventRegistry::string_list* EventRegistry::UsedHandlers() + { + string_list* names = new string_list; + + IterCookie* c = handlers.InitForIteration(); + + HashKey* k; + EventHandler* v; + while ( (v = handlers.NextEntry(k, c)) ) + { + if ( v->LocalHandler() && v->Used() ) + names->append(v->Name()); + + delete k; + } + + return names; + } + void EventRegistry::PrintDebug() { IterCookie* c = handlers.InitForIteration(); Index: src/EventRegistry.h =================================================================== --- src/EventRegistry.h (revision 6147) +++ src/EventRegistry.h (working copy) @@ -35,6 +35,7 @@ void EnableGroup(const char* group, bool enable); string_list* UnusedHandlers(); + string_list* UsedHandlers(); void PrintDebug(); private: Index: src/NetVar.h =================================================================== --- src/NetVar.h (revision 6147) +++ src/NetVar.h (working copy) @@ -267,6 +267,7 @@ extern int time_machine_profiling; extern int check_for_unused_event_handlers; +extern int dump_used_event_handlers; extern int suppress_local_output; Index: src/NetVar.cc =================================================================== --- src/NetVar.cc (revision 6147) +++ src/NetVar.cc (working copy) @@ -263,6 +263,7 @@ int time_machine_profiling; int check_for_unused_event_handlers; +int dump_used_event_handlers; StringVal* trace_output_file; @@ -314,6 +315,8 @@ check_for_unused_event_handlers = opt_internal_int("check_for_unused_event_handlers"); + dump_used_event_handlers = + opt_internal_int("dump_used_event_handlers"); suppress_local_output = opt_internal_int("suppress_local_output"); Index: policy/bro.init =================================================================== --- policy/bro.init (revision 6147) +++ policy/bro.init (working copy) @@ -1376,6 +1376,9 @@ # If true, warns about unused event handlers at startup. const check_for_unused_event_handlers = T &redef; +# If true, dumps all invoked event handlers at startup. +const dump_used_event_handlers = F &redef; + # If true, we suppress prints to local files if we have a receiver for # print_hook events. Ignored for files with a &disable_print_hook attribute. const suppress_local_output = F &redef; From vallentin at ICSI.Berkeley.EDU Sun Dec 7 10:53:49 2008 From: vallentin at ICSI.Berkeley.EDU (Matthias Vallentin) Date: Sun, 7 Dec 2008 10:53:49 -0800 Subject: [Bro] Determine potential events with a specific set of *.bro scripts In-Reply-To: <1228528046.20047.258.camel@strangepork> References: <200812052209.mB5M9Q7Z009496@pork.ICSI.Berkeley.EDU> <1228528046.20047.258.camel@strangepork> Message-ID: <20081207185349.GQ9473@icsi.berkeley.edu> On Fri, Dec 05, 2008 at 05:47:26PM -0800, Christian Kreibich wrote: > On Fri, 2008-12-05 at 14:09 -0800, Vern Paxson wrote: > I think you're referring to the plot I made a long time ago of the @load > structure of the policy files: > > http://www.icir.org/christian/bro/brodep.pdf Yes, that's the labyrinthine picture I had in my mind. Crazy... Matthias -- Matthias Vallentin vallentin at icir.org http://matthias.vallentin.cc From jean-philippe.luiggi at didconcept.com Sun Dec 7 17:00:52 2008 From: jean-philippe.luiggi at didconcept.com (jean-philippe luiggi) Date: Sun, 7 Dec 2008 20:00:52 -0500 Subject: [Bro] linux vs freebsd In-Reply-To: <1228345096.1126.79.camel@hermes> References: <200812032039.mB3KdUfI002626@pork.ICSI.Berkeley.EDU> <1228345096.1126.79.camel@hermes> Message-ID: <20081207200052.01038822@mygw.lan.mynetwork.local> Hello, On Wed, 03 Dec 2008 15:58:16 -0700 Ken Gunderson wrote: > OpenBSD would be an attractive option but the last I checked the way > back machine bro had issues on OBSD... Curious if anyone been doing > anything in this regard more recently? Bro builds well and works fine on it, i used "Bro" 1.3.x then 1.4 on various recent OpenBSD's distribution (i mean versions) without problems. As i follow OpenBSD's development (i run "-CURRENT"), i try Bro 1.4 on all new releases. Be sure i'll post some news in case of problems . With regards, Jean-Philippe. From suroot at gmail.com Mon Dec 8 12:09:15 2008 From: suroot at gmail.com (0100) Date: Mon, 8 Dec 2008 12:09:15 -0800 Subject: [Bro] ignoring ContentGap and friends Message-ID: Hi. New to bro so this is probably a dumb question. I'm following the instructions here ( http://www.bro-ids.org/wiki/index.php/User_Manual:_Customizing_Bro) on how to squelch the ContentGap and various other messages. Here's my config file: @load bittorrent @load bt-tracker redef notice_action_filters += { WeirdActivity, ContentGap = ignore_notice, }; When I run this, I get: Line 6: error: unknown identifier WeirdActivity, at or near "WeirdActivity" I couldn't figure out if these have been renamed or what? Thanks for the help... 0100 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20081208/b116719b/attachment.html From rmkml at free.fr Mon Dec 8 10:36:10 2008 From: rmkml at free.fr (rmkml) Date: Mon, 8 Dec 2008 19:36:10 +0100 (CET) Subject: [Bro] ignoring ContentGap and friends In-Reply-To: References: Message-ID: Hi, maybe add `Weird::`, example: Weird::WeirdActivity, Weird::ContentGap = ignore_notice, Regards Rmkml Crusoe-Researches.com On Mon, 8 Dec 2008, 0100 wrote: > Date: Mon, 8 Dec 2008 12:09:15 -0800 > From: 0100 > To: bro at ICSI.Berkeley.EDU > Subject: [Bro] ignoring ContentGap and friends > > Hi. > > New to bro so this is probably a dumb question. > > I'm following the instructions here (http://www.bro-ids.org/wiki/index.php/User_Manual:_Customizing_Bro) on how to squelch the ContentGap and various other > messages. Here's my config file: > > @load bittorrent > @load bt-tracker > > redef notice_action_filters += { > ???? WeirdActivity, ContentGap = ignore_notice, > }; > > When I run this, I get: > > Line 6: error: unknown identifier WeirdActivity, at or near "WeirdActivity" > > I couldn't figure out if these have been renamed or what? > > Thanks for the help... > > 0100 > > From robin at icir.org Mon Dec 8 12:56:32 2008 From: robin at icir.org (Robin Sommer) Date: Mon, 8 Dec 2008 12:56:32 -0800 Subject: [Bro] ignoring ContentGap and friends In-Reply-To: References: Message-ID: <20081208205632.GA76037@icir.org> On Mon, Dec 08, 2008 at 12:09 -0800, 0100 wrote: > http://www.bro-ids.org/wiki/index.php/User_Manual:_Customizing_Bro) on how You're likely using a newer version than this was written for (as it seems you already guessed). The variables from weird.bro are now in a namespace Weird so you should write Weird::WeirdActivity etc. (and also @load weird). Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From suroot at gmail.com Mon Dec 8 13:24:41 2008 From: suroot at gmail.com (0100) Date: Mon, 8 Dec 2008 13:24:41 -0800 Subject: [Bro] ignoring ContentGap and friends In-Reply-To: <20081208205632.GA76037@icir.org> References: <20081208205632.GA76037@icir.org> Message-ID: Thanks all. So the good news is that it seems to have found the referenced variables, but they are creating a type error in the notice_action_filters table: notice.bro, line 6 and /usr/local/bro/policy/notice.bro, lines 74-75 (enum and table[enum] of function(n:record { note:enum; msg:string; sub:string; conn:record { id:record { orig_h:addr; orig_p:port; resp_h:addr; resp_p:port; }; orig:record { size:count; state:count; }; resp:record { size:count; state:count; }; start_time:time; duration:interval; service:set[string]; addl:string; hot:count; history:string; }; iconn:record { orig_h:addr; resp_h:addr; itype:count; icode:count; len:count; }; id:record { orig_h:addr; orig_p:port; resp_h:addr; resp_p:port; }; src:addr; dst:addr; p:port; user:string; filename:string; method:string; URL:string; n:count; src_peer:record { id:count; host:addr; p:port; is_local:bool; descr:string; class:string; }; tag:string; dropped:bool; captured:string; }; a:enum;) : enum): error, type clash in table initializer Thoughts? Does this enum need to be updated somewhere? 0100 On Mon, Dec 8, 2008 at 12:56 PM, Robin Sommer wrote: > > On Mon, Dec 08, 2008 at 12:09 -0800, 0100 wrote: > > > > http://www.bro-ids.org/wiki/index.php/User_Manual:_Customizing_Bro) on > how > > You're likely using a newer version than this was written for (as it > seems you already guessed). The variables from weird.bro are now in > a namespace Weird so you should write Weird::WeirdActivity etc. (and > also @load weird). > > Robin > > -- > Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20081208/e9f22a9b/attachment.html From vern at icir.org Mon Dec 8 13:38:10 2008 From: vern at icir.org (Vern Paxson) Date: Mon, 08 Dec 2008 13:38:10 -0800 Subject: [Bro] ignoring ContentGap and friends In-Reply-To: (Mon, 08 Dec 2008 12:09:15 PST). Message-ID: <200812082138.mB8LcDgN018659@pork.ICSI.Berkeley.EDU> > redef notice_action_filters += { > WeirdActivity, ContentGap = ignore_notice, > }; Along with the scoping issues that others have discussed, there's a syntax problem here (which explains your follow-on problem). The above needs to be: redef notice_action_filters += { [[Weird::WeirdActivity, Weird::ContentGap]] = ignore_notice, }; - Vern From suroot at gmail.com Mon Dec 8 13:44:50 2008 From: suroot at gmail.com (0100) Date: Mon, 8 Dec 2008 13:44:50 -0800 Subject: [Bro] ignoring ContentGap and friends In-Reply-To: <200812082138.mB8LcDgN018659@pork.ICSI.Berkeley.EDU> References: <200812082138.mB8LcDgN018659@pork.ICSI.Berkeley.EDU> Message-ID: Thanks! That seems to work. I will request a wiki account and update the docs if that's okay. 0100 On Mon, Dec 8, 2008 at 1:38 PM, Vern Paxson wrote: > > redef notice_action_filters += { > > WeirdActivity, ContentGap = ignore_notice, > > }; > > Along with the scoping issues that others have discussed, there's > a syntax problem here (which explains your follow-on problem). The > above needs to be: > > redef notice_action_filters += { > [[Weird::WeirdActivity, Weird::ContentGap]] = ignore_notice, > }; > > - Vern > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20081208/cb73a425/attachment.html From robin at icir.org Mon Dec 8 14:08:05 2008 From: robin at icir.org (Robin Sommer) Date: Mon, 8 Dec 2008 14:08:05 -0800 Subject: [Bro] [Announcement] Bro Workshop 2009 Message-ID: <20081208220805.GT24821@icir.org> Bro Workshop 2009 ================= The Bro team and the Lawrence Berkeley National Lab are pleased to announce the "Bro Workshop 2009", a 2.5-day Bro training event that will take place in Berkeley, CA, on Feb 10-12 2009. The workshop is primarily targeted at site security personnel wishing to learn more about how Bro works, how to use its scripting language and how to generally customize the system based on a site's local policy. Similar to the 2007 workshop, the agenda will be an informal mix of tutorial-style presentations and hands-on lab sessions. No prior knowledge about using Bro is assumed though attendees should be familiar with Unix shell usage as well as with typical networking tools like tcpdump and Wireshark. All participants are expected to bring a Unix-based (Linux, Mac OS X, FreeBSD) laptop with a working Bro configuration. We will provide sample trace files to work with. The 2009 workshop will be hosted by the Lawrence Berkeley National Lab. We will soon provide a web site with more detailed registration and location information. To facilitate a productive lab environment, the number of attendees will be limited to 30 people. A nominal registration fee of $50 will be charged. We also expect to have time for 2-3 case-study presentations from people using Bro in their environments. If you have something interesting to talk about, please send me a mail. Looking forward to a great workshop, Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Mon Dec 8 18:21:07 2008 From: robin at icir.org (Robin Sommer) Date: Mon, 8 Dec 2008 18:21:07 -0800 Subject: [Bro] New Bro issue tracker Message-ID: <20081209022106.GD77388@icir.org> Hello all, we're happy to announce our new Bro Issue Tracker at http://tracker.icir.org/bro Feel free to submit your favorite bug there (and the other one too!) Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From edthoma at sandia.gov Tue Dec 9 11:21:29 2008 From: edthoma at sandia.gov (Thomas, Eric D) Date: Tue, 9 Dec 2008 12:21:29 -0700 Subject: [Bro] DPD and similar analysis In-Reply-To: <20081202213016.GQ58544@icir.org> References: <20081202213016.GQ58544@icir.org> Message-ID: <2BF0A7935DCC2448AF7A835AADC644B379EE395A17@ES03SNLNT.srn.sandia.gov> Hi Robin, When I do the run, I have no trouble getting the SSH ServerFound and ProtocolFound messages either. But I'm not getting the expected ServerFound and ProtocolFound notices for HTTP. I re-ran my test using your suggestion of loading http-request, http-reply and http-body, and I still didn't get those notices for HTTP. So I'd like to know, what does it take to see ServerFound and ProtocolFound notices for HTTP on standard HTTP ports (e.g. 80)? Thanks, Eric -----Original Message----- From: Robin Sommer [mailto:robin at icir.org] Sent: Tuesday, December 02, 2008 1:30 PM To: bro at bro-ids.org; Thomas, Eric D Subject: Re: [Bro] DPD and similar analysis On Mon, Dec 01, 2008 at 12:06 -0800, you wrote: > or icmp)". To get bro to report ALL servers found, I temporarily > modified detect-protocols.bro and commented out the two sections that > would prevent generating notices for "well known ports" (using > dpd_config). So I would expect to see ServerFound messages for all > protocols that have been detected. Hmm... I can't reproduce that. Likewise commenting out the dpd_config check in detect_protocols.bro and running with a tiny SSH trace I get: > bro -r ssh.trace ssh dpd detect-protocols 1150485521.634103 ProtocolFound XXX.XXX.XXX.XXX/49244 > XXX.XXX.XXX.XXX/ssh SSH on port 22/tcp 1150485521.634103 ServerFound XXX.XXX.XXX.XXX: SSH server on port 22/tcp > bro -r pcapfile.pcap conn dpd irc-bot dyn-disable detect-protocols > detect-protocols-http proxy http ssh zzz-custom Note that you should use http-request instead of http, and potentially also load http-{reply,body}.bro. > Once I figure this out, then I'll use DPD for it's intended purpose: > to detect protocols on non-standard ports. However, I'm also supposed > to do the inverse, that is, detect non-standard protocols on standard > ports. Any thoughts on how I could do this? dyn-disable reports ProtocolViolations when the analyzer can't parse the protocol, which is an indicator that there might be something running on that port which isn't the standard protocol. Also, if you enable dpd_conn_logs in conn.bro, the service field indicates the DPD result, including whether it could *not* parse the protocol. From CHANGES: The new script variable dpd_conn_logs (default F), if true, changes the semantics of the service field in connection logs written to conn.log, as follows (Robin Sommer). It becomes a comma-separated list of analyzers confirmed by DPD to parse the connection's payload. If no analyzer could confirm its protocol, but the connection uses a well-known port, the service is the name of the port with "?" appended (e.g., "http?"), as long as the corresponding analyzer has not declined the connection. In addition, ftp-data sessions are labeled "ftp-data" and portmapper connections are labeled with the specific method-call (just as before). dpd_conn_logs defaults to F because the change in semantics may break scripts that parse conn.logs; but it will likely change to the default in the future. With dpd_conn_logs turned off, conn logs are generated as they used to be, with a few rare exceptions (with previous versions, the service field was sometimes determined while the connection was still alive; now it's always determined at the time when the conn.log entry is written out). Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Tue Dec 9 20:52:20 2008 From: robin at icir.org (Robin Sommer) Date: Tue, 9 Dec 2008 20:52:20 -0800 Subject: [Bro] Trying to drop addresses... In-Reply-To: <205A6321-B5A9-4704-9FFF-45BAF9085AE5@fnal.gov> References: <205A6321-B5A9-4704-9FFF-45BAF9085AE5@fnal.gov> Message-ID: <20081210045220.GC2646@icir.org> On Fri, Dec 05, 2008 at 15:43 -0600, you wrote: > [Drop::AddressDropped] = drop_source, You nedd to map drop_source to the notice which you want to trigger the drop, e.g., PortScan or AddressScan. AddressDropped is generated when an address has been dropped *already* (which won't happen if you don't assign drop_source to anything else). > Do I need to modify the notice_policy? I don't see any NOTICE_DROP > examples in notice-policy.bro. That would be the alternative. You can either return NOTICE_DROP in the notice_policy, or assign drop_source in notice_action_filters. Have you seen this posting about the notice framework? http://blog.icir.org/2008/03/telling-bro-what-important.html Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From adayadil.thomas at gmail.com Thu Dec 11 11:34:33 2008 From: adayadil.thomas at gmail.com (Adayadil Thomas) Date: Thu, 11 Dec 2008 14:34:33 -0500 Subject: [Bro] Timers in Bro Message-ID: Hello, Is the timer implementation in bro based on the packet timestamps ? I am trying to understand how bro keeps separate timers for, say, 1000 simultaneous tcp sessions, for example. Any clarification is appreciated. Thanks for any info ! From robin at icir.org Thu Dec 11 13:14:45 2008 From: robin at icir.org (Robin Sommer) Date: Thu, 11 Dec 2008 13:14:45 -0800 Subject: [Bro] Timers in Bro In-Reply-To: References: Message-ID: <20081211211445.GA60285@icir.org> On Thu, Dec 11, 2008 at 14:34 -0500, Adayadil Thomas wrote: > Is the timer implementation in bro based on the packet timestamps ? Yes, right. (That's generally true for Bro's notion of time; the "network time" is advanced as packets are processed). Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From kevlo at kevlo.org Tue Dec 16 22:20:06 2008 From: kevlo at kevlo.org (Kevin Lo) Date: Wed, 17 Dec 2008 14:20:06 +0800 Subject: [Bro] linux vs freebsd In-Reply-To: <1229493575.6572.5.camel@nsl> References: <1229493575.6572.5.camel@nsl> Message-ID: <1229494806.13638.11.camel@srg.kevlo.org> Christian Kreibich wrote: > On Wed, 2008-12-03 at 15:58 -0700, Ken Gunderson wrote: > > OpenBSD would be an attractive option but the last I checked the way > > back machine bro had issues on OBSD... Curious if anyone been doing > > anything in this regard more recently? > > To put it mildly, I've lost track of the number of times I've cursed > OpenBSD because of ongoing build headaches. We generally use it neither > for development nor production, but whenever people report build > problems, we try to go in and fix them. Thus, if there are OpenBSD > issues with the current release, please let us know. I made an OpenBSD port of bro-1.4, which included in the ports tree: http://www.openbsd.org/cgi-bin/cvsweb/ports/net/bro/ > -- > Cheers, > Christian Kevin From akkala at colorado.edu Thu Dec 18 06:26:12 2008 From: akkala at colorado.edu (Sirisha Akkala) Date: Thu, 18 Dec 2008 07:26:12 -0700 (MST) Subject: [Bro] Bro - a general question Message-ID: <20081218072612.AGF95164@riddler.int.colorado.edu> Is Bro 1.4 the first attempt to make a parallelizable IDS? Would anyone know, offhand, what is the optimum number of worker nodes? and i think that might change with network speed? I will be going through the paper, shortly. http://www.icir.org/robin/papers/raid07.pdf but if anyone can reply with quick answer that would be helpful. thanks,sirisha. From rmkml at free.fr Thu Dec 18 06:08:11 2008 From: rmkml at free.fr (rmkml) Date: Thu, 18 Dec 2008 15:08:11 +0100 (CET) Subject: [Bro] Bro v1.4.0 smtp_port Erreur de segmentation Message-ID: Hi, When I start `./bro140ipv6 bro.init mt` I have this error: policy/snort.bro, line 28 and policy/smtp.bro, line 115 (smtp_ports): error, already defined policy/smtp.bro, line 116 ($ports=smtp_ports): run-time error, uninitialized list value Erreur de segmentation policy/mt.bro: @load alarm @load dns-lookup @load snort @load smtp policy/snort.bro: 28: const smtp_ports = { 25/tcp }; policy/smtp.bro: (original, not modified) 115: global smtp_ports = { 25/tcp, 587/tcp } &redef; Im understand than why bro error, but fix segmentation error please ? Regards Rmkml Crusoe-Researches.com From hall.692 at osu.edu Thu Dec 18 09:44:00 2008 From: hall.692 at osu.edu (Seth Hall) Date: Thu, 18 Dec 2008 12:44:00 -0500 Subject: [Bro] Bro v1.4.0 smtp_port Erreur de segmentation In-Reply-To: References: Message-ID: <3D57923F-9A4E-4B8F-9F93-03E264C54BAF@osu.edu> On Dec 18, 2008, at 9:08 AM, rmkml wrote: > When I start `./bro140ipv6 bro.init mt` I have this error: > policy/snort.bro, line 28 and snort.bro in the 1.4 release only has 21 lines. > Erreur de segmentation I don't see this behavior but because your error mentions line 28 in the snort.bro script I believe you may have some other problem going on. > Im understand than why bro error, but fix segmentation error please ? It would be helpful if you provided a backtrace with your report. Thanks, .Seth --- Seth Hall Network Security - Office of the CIO The Ohio State University Phone: 614-292-9721 From rmkml at free.fr Thu Dec 18 07:50:03 2008 From: rmkml at free.fr (rmkml) Date: Thu, 18 Dec 2008 16:50:03 +0100 (CET) Subject: [Bro] Bro v1.4.0 smtp_port Erreur de segmentation In-Reply-To: <3D57923F-9A4E-4B8F-9F93-03E264C54BAF@osu.edu> References: <3D57923F-9A4E-4B8F-9F93-03E264C54BAF@osu.edu> Message-ID: Hi Seth, I have restored snort.bro and added this line: 22: const smtp_ports = { 25/tcp }; same pb, how create backtrace please ? Regards Rmkml Crusoe-Researches.com On Thu, 18 Dec 2008, Seth Hall wrote: > Date: Thu, 18 Dec 2008 12:44:00 -0500 > From: Seth Hall > To: rmkml > Cc: bro at bro-ids.org > Subject: Re: [Bro] Bro v1.4.0 smtp_port Erreur de segmentation > > On Dec 18, 2008, at 9:08 AM, rmkml wrote: > >> When I start `./bro140ipv6 bro.init mt` I have this error: >> policy/snort.bro, line 28 and > > snort.bro in the 1.4 release only has 21 lines. > >> Erreur de segmentation > > I don't see this behavior but because your error mentions line 28 in the > snort.bro script I believe you may have some other problem going on. > >> Im understand than why bro error, but fix segmentation error please ? > > It would be helpful if you provided a backtrace with your report. > > Thanks, > .Seth > > --- > Seth Hall > Network Security - Office of the CIO > The Ohio State University > Phone: 614-292-9721 > From hall.692 at osu.edu Thu Dec 18 12:20:49 2008 From: hall.692 at osu.edu (Seth Hall) Date: Thu, 18 Dec 2008 15:20:49 -0500 Subject: [Bro] Bro - a general question In-Reply-To: <20081218072612.AGF95164@riddler.int.colorado.edu> References: <20081218072612.AGF95164@riddler.int.colorado.edu> Message-ID: On Dec 18, 2008, at 9:26 AM, Sirisha Akkala wrote: > Is Bro 1.4 the first attempt to make a parallelizable IDS? > Would anyone know, offhand, what is the optimum number of worker > nodes? and i think that might change with network speed? I think that the optimum number of worker nodes (the hosts actually sniffing traffic) is extremely site specific. Things that can make a big difference... Number of packets per second, total bps bandwidth, the hardware your worker nodes are using, the analysis you're choosing to do, etc. Those are just the few things I could think of the top of my head right now. I'll describe my environment as an example. During the day we typically see upwards of 1.4 Gbps at just over 200K packets per second. We do full traffic analysis, our bpf filter is just "ip". We run the DPD (dynamic port detection) code, so we can identify protocols on any port. We run most of the "http-" suite of scripts (including my own custom scripts), causing the HTTP analyzer to be enabled which seems to be the most intense analyzer that Bro has. With all of this in mind, we don't drop packets for all intents and purposes. We are currently running 6 2.4Ghz Core2 quad workers and another 8 or so 2.8Ghz pentium 4 workers. I consider this fairly reasonable because we purchased all of the 1U quad core hosts new at just under $650 each and the pentium 4s were free (but they can't process that much traffic because they have slow memory buses). Currently, our plan for the future is to continue purchasing more hosts as the need (packet loss) arises. I think we should be able to scale reasonably well for quite some time with that strategy. .Seth --- Seth Hall Network Security - Office of the CIO The Ohio State University Phone: 614-292-9721 From hall.692 at osu.edu Thu Dec 18 13:53:54 2008 From: hall.692 at osu.edu (Seth Hall) Date: Thu, 18 Dec 2008 16:53:54 -0500 Subject: [Bro] Bro v1.4.0 smtp_port Erreur de segmentation In-Reply-To: References: <3D57923F-9A4E-4B8F-9F93-03E264C54BAF@osu.edu> Message-ID: On Dec 18, 2008, at 10:50 AM, rmkml wrote: > Hi Seth, > I have restored snort.bro and added this line: > 22: const smtp_ports = { 25/tcp }; Ok, I replicated your crash and added it to the ticket tracker. Here's the most minimal script I could make that replicated the segfault... global smtp_ports; const blah = [$ports=smtp_ports]; I think there are two problems occurring here. Causing the problem between the smtp.bro and snort.bro scripts are these two lines... const smtp_ports = { 25/tcp }; global smtp_ports = { 25/tcp, 587/tcp }; It causes the error message "(smtp_ports): error, already defined". I don't think that execution should continue after the attempted globalizing of the smtp_ports variable. It seems like that should be a terminating error because it should always be caught at initialization time and wouldn't just crop up at arbitrary points during runtime. Anyway, I filed a ticket for this too. Here are the tickets: http://tracker.icir.org/bro/ticket/34 http://tracker.icir.org/bro/ticket/35 > same pb, how create backtrace please ? Use gdb. $ gdb ./src/bro (gbd) run test-crash.bro (gdb) bt .Seth --- Seth Hall Network Security - Office of the CIO The Ohio State University Phone: 614-292-9721 From vern at icir.org Fri Dec 19 18:29:03 2008 From: vern at icir.org (Vern Paxson) Date: Fri, 19 Dec 2008 18:29:03 -0800 Subject: [Bro] Bro Workshop 2009 -- registration open Message-ID: <200812200229.mBK2T8IR006344@pork.ICSI.Berkeley.EDU> The Bro Team, LBNL, and UCB are pleased to announce the "Bro Workshop 2009", a 2.5-day Bro training event that will take place in Berkeley, CA, on Feb 10-12, 2009. The workshop is described below. (Note that this workshop is in structure a repeat of the last one we held, rather than a follow-on, so if you attended the last one, this may not be what you're looking for.) If you'd like to attend, please register at www.regonline.com/BRO2009 soon as the workshop is expected to fill up quickly. To facilitate a productive lab environment, the number of attendees will be limited to 30. A nominal registration fee of $50 will be charged. Please contact Robin Sommer (robin at icir.org) or me if you have questions or need further information. Vern ========================== Bro Workshop Description: The workshop is primarily targeted at site security personnel wishing to learn more about how Bro works, how to use its scripting language and how to generally customize the system based on a site's local policy. The agenda will be an informal mix of tutorial-style presentations and hands-on lab sessions. No prior knowledge about using Bro is assumed, though attendees should be familiar with Unix shell usage as well as with typical networking tools like tcpdump and Wireshark. All participants are expected to bring a Unix-based (Linux, Mac OS X, FreeBSD) laptop with a working Bro configuration (http://bro-ids.org). We will provide sample trace files to work with. We also expect to have 2-3 case-study presentations from people using Bro in their environments. If you have something interesting to talk about, please contact Robin Sommer (robin at icir.org).