From surya.t at gmail.com Tue Apr 1 03:57:17 2008 From: surya.t at gmail.com (surya) Date: Tue, 1 Apr 2008 16:27:17 +0530 Subject: [Bro] Bro state implementation Message-ID: Hello, What is the behaviour of Bro when "suspend_processing" is called. What i have observed is Bro is buffering packets in suspended state. Can any one help me out how to avoid buffering the packets in suspended state. Thanks, Surya -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080401/66bde8e4/attachment.html From ssmit7 at gmail.com Tue Apr 1 05:39:55 2008 From: ssmit7 at gmail.com (Stephen G. Smith) Date: Tue, 01 Apr 2008 08:39:55 -0400 Subject: [Bro] scan.bro error In-Reply-To: <20080401043000.GA2045@icir.org> References: <20080401043000.GA2045@icir.org> Message-ID: <47F22D1B.8080609@gmail.com> That is extremely possible. After building I did a make update to update my existing installation and I did notice that it didn't install a new bro binary. I just didn't think anything of it at the time. Do I need to just back up my site folder, do make install, then copy it back in? Thanks, Stephen Robin Sommer wrote: > On Mon, Mar 31, 2008 at 15:11 -0400, you wrote: > > >> I checked out and built the latest version from stable svn tree, and now >> startup is failing on an error from the scan.bro policy. >> > > Any chance that you're mixing the bro binary with policy scripts > from a different Bro versions? > > Robin > > > From seth at net.ohio-state.edu Tue Apr 1 05:41:41 2008 From: seth at net.ohio-state.edu (Seth Hall) Date: Tue, 1 Apr 2008 08:41:41 -0400 Subject: [Bro] URL and datastructures..... In-Reply-To: <244449.85120.qm@web46316.mail.sp1.yahoo.com> References: <244449.85120.qm@web46316.mail.sp1.yahoo.com> Message-ID: <9221A81B-7252-4516-B67A-FA2226F33EA6@net.ohio-state.edu> On Apr 1, 2008, at 1:37 AM, Navdeep Singh wrote: > Hello Mr.Seth the code you have provided is untested ....it not > working and its not giving URL's....plz review it and send the exact > code...i will be very thankful to you.... After I sent the email, I realized that it had a small bug. The bug is fixed now (I forgot to set the "r" variable). The mailing list is messing up the formatting of the script too, you just need to fix that. The script is attached as a file now too because the mailing list screwed up the formatting. -------------- next part -------------- A non-text attachment was scrubbed... Name: http-url-print.bro Type: application/octet-stream Size: 457 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080401/227d023e/attachment.obj -------------- next part -------------- .Seth From robin at icir.org Tue Apr 1 11:36:11 2008 From: robin at icir.org (Robin Sommer) Date: Tue, 1 Apr 2008 11:36:11 -0700 Subject: [Bro] scan.bro error In-Reply-To: <47F22D1B.8080609@gmail.com> References: <20080401043000.GA2045@icir.org> <47F22D1B.8080609@gmail.com> Message-ID: <20080401183611.GC15846@icir.org> On Tue, Apr 01, 2008 at 08:39 -0400, you wrote: > just back up my site folder, do make install, then copy it back in? Yes, should work (it shouldn't touch the site policies but backing them up is certainly recommended ...) Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Tue Apr 1 11:40:41 2008 From: robin at icir.org (Robin Sommer) Date: Tue, 1 Apr 2008 11:40:41 -0700 Subject: [Bro] Bro state implementation In-Reply-To: References: Message-ID: <20080401184041.GD15846@icir.org> On Tue, Apr 01, 2008 at 16:27 +0530, you wrote: > What is the behaviour of Bro when "suspend_processing" is called. What i > have observed is Bro is buffering packets in suspended state. No, it's not buffering packets, it just stops processing them (i.e., the kernel will drop packets once its internal buffers are exhausted). What are you using suspend_processing() for? Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From joel.ebrahimi at gmail.com Tue Apr 1 18:28:33 2008 From: joel.ebrahimi at gmail.com (Joel Ebrahimi) Date: Tue, 1 Apr 2008 18:28:33 -0700 Subject: [Bro] Select Loop Message-ID: <46ee7b1c0804011828s1df4ba63u2cfa3b240630b949@mail.gmail.com> What is the behavior when you disable the select loop? The documentation is pretty vague other than to say there is a new structure of the event loop. I looked at the code and noticed this will put pcap into blocking mode, but Im not sure what other impact this may have. Thanks, // Joel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080401/030d8228/attachment.html From rpang at cs.princeton.edu Wed Apr 2 07:58:21 2008 From: rpang at cs.princeton.edu (Ruoming Pang) Date: Wed, 2 Apr 2008 10:58:21 -0400 Subject: [Bro] http-protocol.pac parsing error on HTTP 1.1 folded headers In-Reply-To: References: Message-ID: Hi Kelvin, Sorry for the late reply (I missed the email in my inbox). 2008/2/8 Kelvin Edmison : > > > I've found an interesting binpac parse error when parsing http headers from > www.golfsmith.com using http-protocol.pac. The problem is that the > golfsmith server is replying with a header that http-protocol.pac is > interpreting as corrupt. > > Here's an example of the golfsmith.com headers > HTTP/1.1 200 OK > Date: Fri, 01 Feb 2008 17:10:30 GMT > Server: Apache/2.2.6 (Unix) mod_ssl/2.2.6 > DAV/2 PHP/5.2.5 > X-Powered-By: PHP/5.2.5 > Content-Type: text/html > > Note the line DAV/2 that is started with a space. That's where the parsing > error occurs. However, it seems like this may actually be legal according > to the standards. > > RFC2616 section 2.2 indicates that > "HTTP/1.1 header field values can be folded onto multiple lines if the > continuation line begins with a space or horizontal tab. All linear white > space, including folding, has the same semantics as SP. A recipient MAY > replace any linear white space with a single SP before interpreting the > field value or forwarding the message downstream." Yes, you are right. This is legal and the original Bro HTTP analyzer handles it. > According to this section, the www.golfsmith.com header "Server:" is broken > across the two lines, and it's value is actually "Apache/2.2.6 (Unix) > mod_ssl/2.2.6DAV/2 PHP/5.2.5" > > Does anyone have ideas on how http-protocol.pac should be modified to handle > this situation? I think the current http-protocol.pac should handle it, too, because HTTP_HEADER_NAME may match an empty string: type HTTP_HEADER_NAME = RE/|([^: \t]+:)/; type HTTP_Header = record { name: HTTP_HEADER_NAME &transient; : HTTP_WS; value: bytestring &restofdata &transient; } &oneline; Could you please verify that the above snippet is the same in your Bro tree? If so, could you send me a trace snippet for me to debug it? Thanks! Ruoming > Thanks, > Kelvin Edmison > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > From robin at icir.org Wed Apr 2 11:41:21 2008 From: robin at icir.org (Robin Sommer) Date: Wed, 2 Apr 2008 11:41:21 -0700 Subject: [Bro] Select Loop In-Reply-To: <46ee7b1c0804011828s1df4ba63u2cfa3b240630b949@mail.gmail.com> References: <46ee7b1c0804011828s1df4ba63u2cfa3b240630b949@mail.gmail.com> Message-ID: <20080402184121.GC27863@icir.org> On Tue, Apr 01, 2008 at 18:28 -0700, you wrote: > of the event loop. I looked at the code and noticed this will put pcap into > blocking mode, but Im not sure what other impact this may have. That's right, disabling the select loop will let Bro use blocking pcap descriptors. That's the only difference and it is only noticable when using other input than packets as well, in particular when doing communication with other Bro nodes. If pcap blocks, communication can stall. Historically, Bro used to use only blocking descriptors, which we changed when introducing the communication. Initially, the non-blocking select loop was optional and needed to be turned on explicitly. These days however it is the default and I don't really know any reason to turn it off. The option to go back to blocking is just a relict from the old times. Any particular reason why you're interested in this? Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From joel.ebrahimi at gmail.com Wed Apr 2 12:20:58 2008 From: joel.ebrahimi at gmail.com (Joel Ebrahimi) Date: Wed, 2 Apr 2008 13:20:58 -0600 Subject: [Bro] Select Loop In-Reply-To: <20080402184121.GC27863@icir.org> References: <46ee7b1c0804011828s1df4ba63u2cfa3b240630b949@mail.gmail.com> <20080402184121.GC27863@icir.org> Message-ID: <46ee7b1c0804021220o2fdc1a86ya654ff3742f57083@mail.gmail.com> I have been working on Bro on both an Intell and Bivio platform. On the Bivio system there is some strange occurence where if I my capture_filters are excluding traffic it will cause the application to periodically stall and completley destroy performance. Profiling did not really turn up much. I coded in some timers into the code and noticed the stalls were happening in post pcap_next() and prior to calling for the next packet. I never got to the root of the problem but by disabling the select loop my performance sky rocketed! This build will not to communicate with other nodes so this sounds like a perfect solution. Thanks, // Joel On Wed, Apr 2, 2008 at 12:41 PM, Robin Sommer wrote: > > On Tue, Apr 01, 2008 at 18:28 -0700, you wrote: > > > of the event loop. I looked at the code and noticed this will put pcap > into > > blocking mode, but Im not sure what other impact this may have. > > That's right, disabling the select loop will let Bro use blocking > pcap descriptors. That's the only difference and it is only > noticable when using other input than packets as well, in particular > when doing communication with other Bro nodes. If pcap blocks, > communication can stall. > > Historically, Bro used to use only blocking descriptors, which we > changed when introducing the communication. Initially, the > non-blocking select loop was optional and needed to be turned on > explicitly. These days however it is the default and I don't really > know any reason to turn it off. The option to go back to blocking is > just a relict from the old times. > > Any particular reason why you're interested in this? > > Robin > > -- > Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080402/de0e4a6b/attachment.html From robin at icir.org Wed Apr 2 13:43:17 2008 From: robin at icir.org (Robin Sommer) Date: Wed, 2 Apr 2008 13:43:17 -0700 Subject: [Bro] Select Loop In-Reply-To: <46ee7b1c0804021220o2fdc1a86ya654ff3742f57083@mail.gmail.com> References: <46ee7b1c0804011828s1df4ba63u2cfa3b240630b949@mail.gmail.com> <20080402184121.GC27863@icir.org> <46ee7b1c0804021220o2fdc1a86ya654ff3742f57083@mail.gmail.com> Message-ID: <20080402204317.GA42956@icir.org> On Wed, Apr 02, 2008 at 13:20 -0600, Joel Ebrahimi wrote: > I have been working on Bro on both an Intell and Bivio platform. On the > Bivio system there is some strange occurence where if I my capture_filters > are excluding traffic it will cause the application to periodically stall > and completley destroy performance. What OS is the Bivio platform running? We've recently been seeing stalls with FreeBSD 7 which sound pretty similar to what you describe. My guess is that the FB7 problems are related to non-blocking pcap as well because Bro seems to be the only application which triggers them. > This build will not to communicate with other nodes so this sounds like a > perfect solution. (There's actually a bit more functionality which in principle could stall, such as async DNS lookups (not a problem) or the new NetFlow analyzer (which isn't in trunk yet)). But in any case, as long as you have a steady stream of packets on the wire, pcap will always have something to pass back to Bro and thus not block anyway.) Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From joel.ebrahimi at gmail.com Wed Apr 2 14:12:36 2008 From: joel.ebrahimi at gmail.com (Joel Ebrahimi) Date: Wed, 2 Apr 2008 14:12:36 -0700 Subject: [Bro] Select Loop In-Reply-To: <20080402204317.GA42956@icir.org> References: <46ee7b1c0804011828s1df4ba63u2cfa3b240630b949@mail.gmail.com> <20080402184121.GC27863@icir.org> <46ee7b1c0804021220o2fdc1a86ya654ff3742f57083@mail.gmail.com> <20080402204317.GA42956@icir.org> Message-ID: <46ee7b1c0804021412j6bf9486ald0f3a4594e40efd7@mail.gmail.com> Its based of Suse on a powerpc architecture. The stalls only seemed to happen when there are filtered packets. If I had redefined my capture_filters to accept all traffic the stalls do not happen. // Joel On Wed, Apr 2, 2008 at 1:43 PM, Robin Sommer wrote: > > On Wed, Apr 02, 2008 at 13:20 -0600, Joel Ebrahimi wrote: > > > I have been working on Bro on both an Intell and Bivio platform. On the > > Bivio system there is some strange occurence where if I my > capture_filters > > are excluding traffic it will cause the application to periodically > stall > > and completley destroy performance. > > What OS is the Bivio platform running? We've recently been seeing > stalls with FreeBSD 7 which sound pretty similar to what you > describe. My guess is that the FB7 problems are related to > non-blocking pcap as well because Bro seems to be the only > application which triggers them. > > > This build will not to communicate with other nodes so this sounds like > a > > perfect solution. > > (There's actually a bit more functionality which in principle could > stall, such as async DNS lookups (not a problem) or the new NetFlow > analyzer (which isn't in trunk yet)). But in any case, as long as > you have a steady stream of packets on the wire, pcap will always > have something to pass back to Bro and thus not block anyway.) > > Robin > > -- > Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080402/d2453bc4/attachment.html From surya.t at gmail.com Wed Apr 2 21:21:46 2008 From: surya.t at gmail.com (surya) Date: Thu, 3 Apr 2008 09:51:46 +0530 Subject: [Bro] Bro state implementation In-Reply-To: <20080401184041.GD15846@icir.org> References: <20080401184041.GD15846@icir.org> Message-ID: >No, it's not buffering packets, it just stops processing them (i.e., >the kernel will drop packets once its internal buffers are >exhausted). Once its resumed will the buffered packets processed by Bro ?. In our case we doesn't want this. Is it possible to stop capturing the packets at libpcap level and later resume capturing the packets with out libpcap initialization. Thanks, Surya -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080403/d72e4ce9/attachment.html From robin at icir.org Thu Apr 3 11:22:14 2008 From: robin at icir.org (Robin Sommer) Date: Thu, 3 Apr 2008 11:22:14 -0700 Subject: [Bro] Bro state implementation In-Reply-To: References: <20080401184041.GD15846@icir.org> Message-ID: <20080403182214.GI87226@icir.org> On Thu, Apr 03, 2008 at 09:51 +0530, surya wrote: > Is it possible to stop capturing the packets at libpcap level and > later resume capturing the packets with out libpcap initialization. As far as I know, there's no way to tell pcap to clear all internal buffers. What one could do is close the interface and reopen it. Or one could just eat all old packets without actually processing them after calling continue_processing(). However, Bro does not support either at the moment. Robn -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From renaud.luca at gmail.com Sat Apr 5 14:43:50 2008 From: renaud.luca at gmail.com (Luca Renaud) Date: Sat, 5 Apr 2008 22:43:50 +0100 Subject: [Bro] Basic questions about the use of Bro. Message-ID: <628233b10804051443j1676501bh187cd94ed5c7cd08@mail.gmail.com> How can I get the output of Bro in normal time and not UNIX time,using cf. for example,processing a tcpdump capture file: /usr/local/bro-1.2/bin/bro -r tcpdumpfile ,I get a list of weird events in UNIX time,and I prefer normal time. I did not do a complete installation of Bro,I use Bro to analyze my home ADSL connections right after the end of the session,so Bro does not report to log files in the logs directory,it reports to standard output. When I analyze dump files: /usr/local/bro-1.2/bin/bro -r tcpdumpcapturefile so far I get a list of weird events: ...weird: spontaneous_FIN ... ...weird: spontaneous_RST ... ...weird: window_recision ... ...weird: unsolicited_SYN_response etc.,which by itself is not specially troublesome.My question is: if bro ever needs to report more troublesome events,does it follow the same terminology(name) used for the diverse files in the logs directory? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080405/ffa92353/attachment.html From vern at icir.org Sat Apr 5 17:20:38 2008 From: vern at icir.org (Vern Paxson) Date: Sat, 05 Apr 2008 17:20:38 -0700 Subject: [Bro] Basic questions about the use of Bro. In-Reply-To: <628233b10804051443j1676501bh187cd94ed5c7cd08@mail.gmail.com> (Sat, 05 Apr 2008 22:43:50 BST). Message-ID: <200804060020.m360KgXX016353@pork.ICSI.Berkeley.EDU> > How can I get the output of Bro in normal time and not UNIX time,using cf. > for example,processing a tcpdump capture file: There's no general option for this. For any particular value you want to print from a script, you can use fmt()'s %D or %T format. > I did not do a complete installation of Bro,I use Bro to analyze my home > ADSL connections right after the end of the session,so Bro does not report > to log files in the logs directory,it reports to standard output. You can "@load weird" to get the "weird" output into a file instead. > if bro ever needs to report more troublesome events,does it follow > the same terminology(name) used for the diverse files in the logs > directory? I'm not quite sure what you mean, but it will write alarms to stdout if you haven't done "@load alarm", and the name used is the same in either case. Vern From ssmit7 at gmail.com Mon Apr 7 11:16:21 2008 From: ssmit7 at gmail.com (Stephen Smith) Date: Mon, 7 Apr 2008 14:16:21 -0400 Subject: [Bro] bad tag in Val::CONVERTER Message-ID: Hello, Another error with trying to run the current svn version, 1.3.25. Doing a proper make install fixed the scan.bro error, but now when I start it fails immediately with the following error. lobo# etc/bro.rc start bro.rc: Running as non-root user bro bro.rc: Starting .bro.rc: Failed to start Bro pcap bufsize = 32768 listening on fxp1 1207590206.353962 (45:1C:A7:1D:CB:BE:82:EC:52:83:01:09:CC:9E:13:ED:D6:9E:7E:5E:C7:A1:8A:77:B3:5E:C3:B0:BB:2F:16:C1): bad tag in Val::CONVERTER (string/table) ............ FAILED I mucked about with gdb for a bit but figured I would just ask. I saw on the wiki that there was a known issue somehwat similar to this, or is it just that I am overlooking something. Let me know if you need other debug logs, etc. Thanks, Stephen -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080407/9ca94013/attachment.html From robin at icir.org Mon Apr 7 16:31:15 2008 From: robin at icir.org (Robin Sommer) Date: Mon, 7 Apr 2008 16:31:15 -0700 Subject: [Bro] bad tag in Val::CONVERTER In-Reply-To: References: Message-ID: <20080407233115.GE87476@icir.org> On Mon, Apr 07, 2008 at 14:16 -0400, you wrote: > (45:1C:A7:1D:CB:BE:82:EC:52:83:01:09:CC:9E:13:ED:D6:9E:7E:5E:C7:A1:8A:77:B3:5E:C3:B0:BB:2F:16:C1): Iirc, this is a problem with the SSL analyzer. Assuming it is enabled, can you try not loading ssl.bro? Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From vern at icir.org Mon Apr 7 16:39:21 2008 From: vern at icir.org (Vern Paxson) Date: Mon, 07 Apr 2008 16:39:21 -0700 Subject: [Bro] Bro mailing list maintenance/outage Message-ID: <200804072339.m37NdQK0023567@pork.ICSI.Berkeley.EDU> FYI, we're in the process of shifting the configuration for the Bro mailing list. Mail may stop working briefly. I'll send a note when it's again working. Vern From vern at icir.org Mon Apr 7 17:53:47 2008 From: vern at icir.org (Vern Paxson) Date: Mon, 07 Apr 2008 17:53:47 -0700 Subject: [Bro] testing new config Message-ID: <200804080053.m380rpAE024860@pork.ICSI.Berkeley.EDU> From vern at icir.org Mon Apr 7 18:09:28 2008 From: vern at icir.org (Vern Paxson) Date: Mon, 07 Apr 2008 18:09:28 -0700 Subject: [Bro] mailing list maintenance complete Message-ID: <200804080109.m3819XIG025142@pork.ICSI.Berkeley.EDU> It looks like it's all working again now. Vern From ssmit7 at gmail.com Wed Apr 9 16:50:44 2008 From: ssmit7 at gmail.com (Stephen G. Smith) Date: Wed, 09 Apr 2008 19:50:44 -0400 Subject: [Bro] bad tag in Val::CONVERTER In-Reply-To: <20080407233115.GE87476@icir.org> References: <20080407233115.GE87476@icir.org> Message-ID: <47FD5654.30401@gmail.com> That took care of it. Incidentally it looks like the libGeoIP function is missing the ntohl() on the address argument again. I'm getting results for inverted addresses. Thanks, Stephen Robin Sommer wrote: > On Mon, Apr 07, 2008 at 14:16 -0400, you wrote: > > >> (45:1C:A7:1D:CB:BE:82:EC:52:83:01:09:CC:9E:13:ED:D6:9E:7E:5E:C7:A1:8A:77:B3:5E:C3:B0:BB:2F:16:C1): >> > > Iirc, this is a problem with the SSL analyzer. Assuming it is > enabled, can you try not loading ssl.bro? > > Robin > > From pauls at utdallas.edu Tue Apr 15 14:20:00 2008 From: pauls at utdallas.edu (Paul Schmehl) Date: Tue, 15 Apr 2008 16:20:00 -0500 Subject: [Bro] Problems compiling with --enable-int64 Message-ID: I'm working on updating the FreeBSD port of bro (I'm the port maintainer), and I've been testing various combinations of options. I've run into a problem compiling with --enable-int64. Val.cc: In member function 'virtual Val* Val::SizeVal() const': Val.cc:567: error: call of overloaded 'abs(const bro_int_t&)' is ambiguous /usr/include/stdlib.h:84: note: candidates are: int abs(int) /usr/include/c++/4.2/cstdlib:143: note: long int std::abs(long int) *** Error code 1 Is this a bug in the code? Or do I need to point to a library that's not found? Configure works fine, but make fails. Compiler is gcc 4.2.1. OS version is 7.0 STABLE. -- Paul Schmehl (pauls at utdallas.edu) Senior Information Security Analyst The University of Texas at Dallas http://www.utdallas.edu/ir/security/ From vern at icir.org Tue Apr 15 15:11:48 2008 From: vern at icir.org (Vern Paxson) Date: Tue, 15 Apr 2008 15:11:48 -0700 Subject: [Bro] Problems compiling with --enable-int64 In-Reply-To: (Tue, 15 Apr 2008 16:20:00 CDT). Message-ID: <200804152211.m3FMBpJK011650@pork.ICSI.Berkeley.EDU> > Val.cc: In member function 'virtual Val* Val::SizeVal() const': > Val.cc:567: error: call of overloaded 'abs(const bro_int_t&)' is ambiguous I'm guessing that the line you're looking at is return new Val(abs(val.int_val), TYPE_COUNT); (it's at a different location in the latest snapshot). If so, try return new Val(bro_int_t(abs(val.int_val)), TYPE_COUNT); and see if that fixes the problem. Vern From pauls at utdallas.edu Tue Apr 15 15:42:51 2008 From: pauls at utdallas.edu (Paul Schmehl) Date: Tue, 15 Apr 2008 17:42:51 -0500 Subject: [Bro] Problems compiling with --enable-int64 In-Reply-To: <200804152211.m3FMBpJK011650@pork.ICSI.Berkeley.EDU> References: <200804152211.m3FMBpJK011650@pork.ICSI.Berkeley.EDU> Message-ID: --On Tuesday, April 15, 2008 15:11:48 -0700 Vern Paxson wrote: >> Val.cc: In member function 'virtual Val* Val::SizeVal() const': >> Val.cc:567: error: call of overloaded 'abs(const bro_int_t&)' is ambiguous > > I'm guessing that the line you're looking at is > > return new Val(abs(val.int_val), TYPE_COUNT); > > (it's at a different location in the latest snapshot). If so, try > > return new Val(bro_int_t(abs(val.int_val)), TYPE_COUNT); > > and see if that fixes the problem. > That did not fix the problem. Val.cc: In member function 'virtual Val* Val::SizeVal() const': Val.cc:567: error: call of overloaded 'abs(const bro_int_t&)' is ambiguous /usr/include/stdlib.h:84: note: candidates are: int abs(int) /usr/include/c++/4.2/cstdlib:143: note: long int std::abs(long int) -- Paul Schmehl (pauls at utdallas.edu) Senior Information Security Analyst The University of Texas at Dallas http://www.utdallas.edu/ir/security/ From vern at icir.org Tue Apr 15 16:02:14 2008 From: vern at icir.org (Vern Paxson) Date: Tue, 15 Apr 2008 16:02:14 -0700 Subject: [Bro] Problems compiling with --enable-int64 In-Reply-To: (Tue, 15 Apr 2008 17:42:51 CDT). Message-ID: <200804152302.m3FN2HAl012265@pork.ICSI.Berkeley.EDU> > That did not fix the problem. Rats. Well, this doesn't repreoduce for me (w/ g++ 4.0.1), so I'm reduced to guessing. The next thing I'd try is return new Val(abs(bro_int_t(val.int_val)), TYPE_COUNT); but if that doesn't work, I'm not sure what next to try. Vern From pauls at utdallas.edu Tue Apr 15 18:30:27 2008 From: pauls at utdallas.edu (Paul Schmehl) Date: Tue, 15 Apr 2008 20:30:27 -0500 Subject: [Bro] Problems compiling with --enable-int64 In-Reply-To: <200804152302.m3FN2HAl012265@pork.ICSI.Berkeley.EDU> References: <200804152302.m3FN2HAl012265@pork.ICSI.Berkeley.EDU> Message-ID: <03452E86950AD04A71CABA8B@Macintosh.local> --On April 15, 2008 4:02:14 PM -0700 Vern Paxson wrote: >> That did not fix the problem. > > Rats. Well, this doesn't repreoduce for me (w/ g++ 4.0.1), so I'm > reduced to guessing. The next thing I'd try is > > return new Val(abs(bro_int_t(val.int_val)), TYPE_COUNT); > > but if that doesn't work, I'm not sure what next to try. > > Vern Sadly, that did not work either. I'm afraid I'm not a programmer, so all I can do is follow your lead. For now I'll make that option is non-working and continue working on the update to the port. If anyone on the list has the same version of gcc and can test this and find a resolution, please post it to the list. I'll include a patch in the port to fix it. Paul Schmehl (pauls at utdallas.edu) Senior Information Security Analyst The University of Texas at Dallas http://www.utdallas.edu/ir/security/ From joel.ebrahimi at gmail.com Tue Apr 15 18:54:44 2008 From: joel.ebrahimi at gmail.com (Joel Ebrahimi) Date: Tue, 15 Apr 2008 19:54:44 -0600 Subject: [Bro] Problems compiling with --enable-int64 In-Reply-To: <03452E86950AD04A71CABA8B@Macintosh.local> References: <200804152302.m3FN2HAl012265@pork.ICSI.Berkeley.EDU> <03452E86950AD04A71CABA8B@Macintosh.local> Message-ID: <46ee7b1c0804151854r166aaadby83f8bf0462342570@mail.gmail.com> When compiling with the --enable-int64 flag, the variable val.int_val is a 'long long int'. It appears the abs function on your system is defined for 'int' and 'long int'. You could probalby typecast the variable so that the compiler uses the 'long int' version. Though depending of the size of 'long int' on your system this could have an undesired effect. If 'long int' on your system is 64bits wide though you should be ok. You can try: return new Val(abs((long int)val.int_val), TYPE_COUNT); The other concern here is that this is a one function fix and your going to run into this issue everytime you pass a 'long long int' to a function that has defines for just 'int' and 'long int'. If 'long int' is 64bits wide on your system, you might be better off editing the bro header file util.h so that you redefine int64. typedef long long int int64; would become typedef long int int64; // Joel On Tue, Apr 15, 2008 at 7:30 PM, Paul Schmehl wrote: > --On April 15, 2008 4:02:14 PM -0700 Vern Paxson wrote: > > >> That did not fix the problem. > > > > Rats. Well, this doesn't repreoduce for me (w/ g++ 4.0.1), so I'm > > reduced to guessing. The next thing I'd try is > > > > return new Val(abs(bro_int_t(val.int_val)), TYPE_COUNT); > > > > but if that doesn't work, I'm not sure what next to try. > > > > Vern > > Sadly, that did not work either. I'm afraid I'm not a programmer, so all > I can do is follow your lead. For now I'll make that option is > non-working and continue working on the update to the port. > > If anyone on the list has the same version of gcc and can test this and > find a resolution, please post it to the list. I'll include a patch in > the port to fix it. > > Paul Schmehl (pauls at utdallas.edu) > Senior Information Security Analyst > The University of Texas at Dallas > http://www.utdallas.edu/ir/security/ > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080415/49fea286/attachment.html From pauls at utdallas.edu Tue Apr 15 19:07:28 2008 From: pauls at utdallas.edu (Paul Schmehl) Date: Tue, 15 Apr 2008 21:07:28 -0500 Subject: [Bro] Problems compiling with --enable-int64 In-Reply-To: <46ee7b1c0804151854r166aaadby83f8bf0462342570@mail.gmail.com> References: <200804152302.m3FN2HAl012265@pork.ICSI.Berkeley.EDU> <03452E86950AD04A71CABA8B@Macintosh.local> <46ee7b1c0804151854r166aaadby83f8bf0462342570@mail.gmail.com> Message-ID: <4A0187A29638E4ADC28B90FE@Macintosh.local> --On April 15, 2008 7:54:44 PM -0600 Joel Ebrahimi wrote: > > When compiling with the --enable-int64 flag, the variable val.int_val is > a 'long long int'. It appears the abs function on your system is defined > for 'int' and 'long int'. You could probalby typecast the variable so > that the compiler uses the 'long int' version. Though depending of the > size of 'long int' on your system this could have an undesired effect. > If 'long int' on your system is 64bits wide though you should be ok. > > You can try: > > return new Val(abs((long int)val.int_val), TYPE_COUNT); > > The other concern here is that this is a one function fix and your going > to run into this issue everytime you pass a 'long long int' to a > function that has defines for just 'int' and 'long int'. > > If 'long int' is 64bits wide on your system, you might be better off > editing the bro header file util.h so that you redefine int64. > > > typedef long long int int64; > > would become > > typedef long int int64; > > Thanks, Joel. This seems to point to a bigger problem. This software could be installed on any FreeBSD system, from an older OS to a brand new one, from i386 to amd to solaris. So, it's seems this is a situation that would call for an #ifdef, would it not? I don't know the size of a long int on my system, but I'm almost certain that it won't be the same on *every* system. Paul Schmehl (pauls at utdallas.edu) Senior Information Security Analyst The University of Texas at Dallas http://www.utdallas.edu/ir/security/ From pauls at utdallas.edu Tue Apr 15 19:42:33 2008 From: pauls at utdallas.edu (Paul Schmehl) Date: Tue, 15 Apr 2008 21:42:33 -0500 Subject: [Bro] Problems compiling with --enable-int64 In-Reply-To: <4A0187A29638E4ADC28B90FE@Macintosh.local> References: <200804152302.m3FN2HAl012265@pork.ICSI.Berkeley.EDU> <03452E86950AD04A71CABA8B@Macintosh.local> <46ee7b1c0804151854r166aaadby83f8bf0462342570@mail.gmail.com> <4A0187A29638E4ADC28B90FE@Macintosh.local> Message-ID: <840F380EC634D59157D4AABA@Macintosh.local> --On April 15, 2008 9:07:28 PM -0500 Paul Schmehl wrote: > > I don't know the size of a long int on my system, but I'm almost certain > that it won't be the same on *every* system. > Here's what I have in my includes: /usr/src/sys/i386/include/_types.h:typedef long long __int64_t; /usr/src/sys/i386/include/_types.h:typedef unsigned long long __uint64_t; /usr/src/sys/i386/include/_types.h:typedef long long __int64_t; /usr/src/sys/i386/include/_types.h:typedef unsigned long long __uint64_t; Fixing the function per your instructions allows the software to make successfully. I'm concerned, however, about your second point. If I change the define in util.h, will it break on some FreeBSD systems? I don't know. According to the cvs repository, _inttypes.h hasn't been changed in almost six years on i386. It's been almost 5 years on amd64 and 5 years on ia64. I guess I'm alright to redefine in util.h. Paul Schmehl (pauls at utdallas.edu) Senior Information Security Analyst The University of Texas at Dallas http://www.utdallas.edu/ir/security/ From pauls at utdallas.edu Tue Apr 15 19:59:12 2008 From: pauls at utdallas.edu (Paul Schmehl) Date: Tue, 15 Apr 2008 21:59:12 -0500 Subject: [Bro] Problems compiling with --enable-int64 In-Reply-To: <840F380EC634D59157D4AABA@Macintosh.local> References: <200804152302.m3FN2HAl012265@pork.ICSI.Berkeley.EDU> <03452E86950AD04A71CABA8B@Macintosh.local> <46ee7b1c0804151854r166aaadby83f8bf0462342570@mail.gmail.com> <4A0187A29638E4ADC28B90FE@Macintosh.local> <840F380EC634D59157D4AABA@Macintosh.local> Message-ID: --On April 15, 2008 9:42:33 PM -0500 Paul Schmehl wrote: > > According to the cvs repository, _inttypes.h hasn't been changed in > almost six years on i386. > > It's been almost 5 years on amd64 and 5 years on ia64. > > I guess I'm alright to redefine in util.h. > That didn't work. I'll have to patch Val.cc. Paul Schmehl (pauls at utdallas.edu) Senior Information Security Analyst The University of Texas at Dallas http://www.utdallas.edu/ir/security/ From joel.ebrahimi at gmail.com Tue Apr 15 20:10:04 2008 From: joel.ebrahimi at gmail.com (Joel Ebrahimi) Date: Tue, 15 Apr 2008 21:10:04 -0600 Subject: [Bro] Problems compiling with --enable-int64 In-Reply-To: <840F380EC634D59157D4AABA@Macintosh.local> References: <200804152302.m3FN2HAl012265@pork.ICSI.Berkeley.EDU> <03452E86950AD04A71CABA8B@Macintosh.local> <46ee7b1c0804151854r166aaadby83f8bf0462342570@mail.gmail.com> <4A0187A29638E4ADC28B90FE@Macintosh.local> <840F380EC634D59157D4AABA@Macintosh.local> Message-ID: <46ee7b1c0804152010w79d3539eybda49df6aef94da6@mail.gmail.com> I didnt think about the port aspect(not much experience). Definately not the way to maintain a build of somethign that would work across all OS's And in most cases there probably be a difference in bytes (4 vs 8) on most OS's anyways. An #ifdef would be the way to go, but my said typecasting would not be good for a port. Looks like there are different functions for calculating absolute zero on different OS's. Check your headers for llabs. This might be the way to go for portability try swapping: return new Val(abs(val.int_val), TYPE_COUNT); for: #ifdef USE_INT64 return new Val(llabs(val.int_val), TYPE_COUNT); #else return new Val(abs(val.int_val), TYPE_COUNT); #endif It may be better to define this function in a header but you can see if this works. // Joel On Tue, Apr 15, 2008 at 8:42 PM, Paul Schmehl wrote: > --On April 15, 2008 9:07:28 PM -0500 Paul Schmehl > wrote: > > > > I don't know the size of a long int on my system, but I'm almost certain > > that it won't be the same on *every* system. > > > > Here's what I have in my includes: > > /usr/src/sys/i386/include/_types.h:typedef long long > __int64_t; > /usr/src/sys/i386/include/_types.h:typedef unsigned long long > __uint64_t; > /usr/src/sys/i386/include/_types.h:typedef long long > __int64_t; > /usr/src/sys/i386/include/_types.h:typedef unsigned long long > __uint64_t; > > Fixing the function per your instructions allows the software to make > successfully. I'm concerned, however, about your second point. If I > change the define in util.h, will it break on some FreeBSD systems? I > don't know. > > According to the cvs repository, _inttypes.h hasn't been changed in almost > six years on i386. > > It's been almost 5 years on amd64 and 5 years on ia64. > > I guess I'm alright to redefine in util.h. > > Paul Schmehl (pauls at utdallas.edu) > Senior Information Security Analyst > The University of Texas at Dallas > http://www.utdallas.edu/ir/security/ > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080415/675d98ea/attachment.html From kevlo at kevlo.org Wed Apr 16 02:21:40 2008 From: kevlo at kevlo.org (Kevin Lo) Date: Wed, 16 Apr 2008 17:21:40 +0800 Subject: [Bro] [PATCH] Add the correct setup of the devfs.rules(5) file for FreeBSD Message-ID: <4805C524.4080105@kevlo.org> Hi, Here is a patch that fixes devfs(8) rules on FreeBSD 5.x and later. Kevin -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: patch-scripts-bro-config.in Url: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080416/5f62fd70/attachment.ksh From kevlo at kevlo.org Wed Apr 16 21:55:10 2008 From: kevlo at kevlo.org (Kevin Lo) Date: Thu, 17 Apr 2008 12:55:10 +0800 Subject: [Bro] [PATCH] Add the correct setup of the devfs.rules(5) file for FreeBSD In-Reply-To: <4805C524.4080105@kevlo.org> References: <4805C524.4080105@kevlo.org> Message-ID: <1208408110.5946.30.camel@monet> Kevin Lo wrote: > Hi, > > Here is a patch that fixes devfs(8) rules on FreeBSD 5.x and later. Since FreeBSD 4.x end of life, attached is a revised patch against development branch that fixes devfs(8) rules on FreeBSD 5.x and later. Kevin -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-scripts-bro-config.in Type: text/x-patch Size: 3838 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080417/736771b8/attachment.bin From kevlo at kevlo.org Wed Apr 16 21:55:23 2008 From: kevlo at kevlo.org (Kevin Lo) Date: Thu, 17 Apr 2008 12:55:23 +0800 Subject: [Bro] [PATCH] Fix deprecated conversion warnings Message-ID: <1208408123.5946.32.camel@monet> Hi, This patch against development branch is to get rid of deprecated conversion warnings like the one below: ../src/builtin-func.y:545: warning: deprecated conversion from string constant to 'char*' I know it's just a warning and it should compile fine, but I'd like to get rid of all the warnings :-) Kevin -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-bro Type: text/x-patch Size: 2518 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080417/6fcedc15/attachment.bin From robin at icir.org Thu Apr 17 10:09:43 2008 From: robin at icir.org (Robin Sommer) Date: Thu, 17 Apr 2008 10:09:43 -0700 Subject: [Bro] [PATCH] Fix deprecated conversion warnings In-Reply-To: <1208408123.5946.32.camel@monet> References: <1208408123.5946.32.camel@monet> Message-ID: <20080417170943.GA78855@icir.org> On Thu, Apr 17, 2008 at 12:55 +0800, you wrote: > This patch against development branch is to get rid of deprecated > conversion warnings like the one below: Thanks, I've applied it to my development branch. > but I'd like to get rid of all the warnings :-) Me too! Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Thu Apr 17 10:17:07 2008 From: robin at icir.org (Robin Sommer) Date: Thu, 17 Apr 2008 10:17:07 -0700 Subject: [Bro] [PATCH] Add the correct setup of the devfs.rules(5) file for FreeBSD In-Reply-To: <1208408110.5946.30.camel@monet> References: <4805C524.4080105@kevlo.org> <1208408110.5946.30.camel@monet> Message-ID: <20080417171707.GB78855@icir.org> On Thu, Apr 17, 2008 at 12:55 +0800, you wrote: > Since FreeBSD 4.x end of life, attached is a revised patch against Hmmm... Not sure we should drop 4.x as long as it's easy to keep the support for it. So I'd prefer to go with your first patch. However, perhaps we can simplify that one a bit more by making one BRO_SYS_TYPE for all of >= 6.0 ? Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From pauls at utdallas.edu Thu Apr 17 10:37:37 2008 From: pauls at utdallas.edu (Paul Schmehl) Date: Thu, 17 Apr 2008 12:37:37 -0500 Subject: [Bro] [PATCH] Add the correct setup of the devfs.rules(5) file for FreeBSD In-Reply-To: <20080417171707.GB78855@icir.org> References: <4805C524.4080105@kevlo.org> <1208408110.5946.30.camel@monet> <20080417171707.GB78855@icir.org> Message-ID: --On Thursday, April 17, 2008 10:17:07 -0700 Robin Sommer wrote: > > On Thu, Apr 17, 2008 at 12:55 +0800, you wrote: > >> Since FreeBSD 4.x end of life, attached is a revised patch against > > Hmmm... Not sure we should drop 4.x as long as it's easy to keep the > support for it. So I'd prefer to go with your first patch. However, > perhaps we can simplify that one a bit more by making one > BRO_SYS_TYPE for all of >= 6.0 ? That's what I was thinking. And if the previous versions all use the same convention, perhaps a simple: if [ BRO_SYS_TYPE == FREEBSD ]; then if [ FREEBSD_OS_VERS >= 6.0 ]; then blah else blah fi fi would suffice? -- Paul Schmehl (pauls at utdallas.edu) Senior Information Security Analyst The University of Texas at Dallas http://www.utdallas.edu/ir/security/ From kevlo at kevlo.org Thu Apr 17 18:38:43 2008 From: kevlo at kevlo.org (Kevin Lo) Date: Fri, 18 Apr 2008 09:38:43 +0800 Subject: [Bro] [PATCH] Add the correct setup of the devfs.rules(5) file for FreeBSD In-Reply-To: <20080417171707.GB78855@icir.org> References: <4805C524.4080105@kevlo.org> <1208408110.5946.30.camel@monet> <20080417171707.GB78855@icir.org> Message-ID: <1208482723.5910.9.camel@monet> Robin Sommer wrote: > On Thu, Apr 17, 2008 at 12:55 +0800, you wrote: > > > Since FreeBSD 4.x end of life, attached is a revised patch against > > Hmmm... Not sure we should drop 4.x as long as it's easy to keep the > support for it. So I'd prefer to go with your first patch. However, > perhaps we can simplify that one a bit more by making one > BRO_SYS_TYPE for all of >= 6.0 ? I'm ok. Is there any reason to run bro on an unmaintained OS? Note that FreeBSD 5.x will reached end of line soon, see: http://lists.freebsd.org/pipermail/freebsd-stable/2008-April/041676.html > Robin Kevin From vern at icir.org Thu Apr 17 19:18:43 2008 From: vern at icir.org (Vern Paxson) Date: Thu, 17 Apr 2008 19:18:43 -0700 Subject: [Bro] [PATCH] Add the correct setup of the devfs.rules(5) file for FreeBSD In-Reply-To: <1208482723.5910.9.camel@monet> (Fri, 18 Apr 2008 09:38:43 +0800). Message-ID: <200804180218.m3I2IjqD013424@pork.ICSI.Berkeley.EDU> > I'm ok. Is there any reason to run bro on an unmaintained OS? Yep - we still run FreeBSD 4.x internally and sometimes for development. Vern From d.nechay at gmail.com Mon Apr 21 11:14:44 2008 From: d.nechay at gmail.com (Danny Nechay) Date: Mon, 21 Apr 2008 14:14:44 -0400 Subject: [Bro] If Statement Syntax Message-ID: Hi, I'm trying to find out what is in the variable c$id$resp_p (which is a port number). I tried using the following syntax if ( c$id$resp_p == "443" ) { ... } On 443 I try it with and without quotations around it but I always get the following error: line 293 (c$id$resp_p == 443): error, type clash in comparison Does anyone know the proper syntax? Also what is the proper syntax for else if in Bro? Is it else if () or elsif ()? Thanks. Daniel. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080421/5d17b209/attachment.html From vern at icir.org Mon Apr 21 11:41:15 2008 From: vern at icir.org (Vern Paxson) Date: Mon, 21 Apr 2008 11:41:15 -0700 Subject: [Bro] If Statement Syntax In-Reply-To: (Mon, 21 Apr 2008 14:14:44 EDT). Message-ID: <200804211841.m3LIfHGP008912@pork.ICSI.Berkeley.EDU> > line 293 (c$id$resp_p == 443): error, type clash in comparison > Does anyone know the proper syntax? Port constants are /, so you need 443/tcp if you mean a TCP port. > Also what is the proper syntax for else if in Bro? Is it else if () or elsif > ()? These sorts of questions are easy to investigate by looking in the scripts provided in bro/policy/*.bro. Vern From robin at icir.org Mon Apr 21 12:11:56 2008 From: robin at icir.org (Robin Sommer) Date: Mon, 21 Apr 2008 12:11:56 -0700 Subject: [Bro] If Statement Syntax In-Reply-To: References: Message-ID: <20080421191156.GL31117@icir.org> On Mon, Apr 21, 2008 at 14:14 -0400, Danny Nechay wrote: > if ( c$id$resp_p == "443" ) { Bro has a dedicated port type. Try c$id$resp_p == 443/tcp . > Also what is the proper syntax for else if in Bro? Is it else if () or elsif else if. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From d.nechay at gmail.com Mon Apr 21 15:33:55 2008 From: d.nechay at gmail.com (Danny Nechay) Date: Mon, 21 Apr 2008 18:33:55 -0400 Subject: [Bro] Partial tcpdump traces Message-ID: Hi, I have a trickier question than last time. I am inputting into Bro partial tcpdump traces (by using the -s option in tcpdump - I am now getting only the first 100 bytes of a packet instead of the full packet). The problem though is that in Bro it seems to discard the entire payload. Is there any way to force Bro to keep the payload? This is a problem as since it is discarding the payload any signature that does a payload match does not work anymore. For a tcpdump trace that has the entire payload this does not occur as it keeps the payload then and finds the proper signature. As an example, in the following snippet of code, the event only prints the payload when a full tcpdump trace is given. # http://osdir.com/ml/security.detection.bro/2004-07/msg00013.html @load site @load snort @load weird @load alarm redef signature_files += "sigs/test.sig"; event signature_match(state: signature_state, msg: string, data: string) { # Note: data is the payload. Example: print fmt("%s", data); print fmt("Print payload:\n%s", data); } Any suggestions? Thanks. Daniel. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080421/5bd9fa63/attachment.html From joel.ebrahimi at gmail.com Mon Apr 21 16:21:52 2008 From: joel.ebrahimi at gmail.com (Joel Ebrahimi) Date: Mon, 21 Apr 2008 16:21:52 -0700 Subject: [Bro] Bro 1.3.x / FC5 / Linking Issue Message-ID: <46ee7b1c0804211621h5d6442e1ub2b0b3901971b2a6@mail.gmail.com> I was trying to build Bro 1.3.2 on FC5. Im running into linking issues and I wanted to see if there is a known fix before I start racking my brain on this. It appears to be having problems with externally defined functions. These problem do not exist for 1.2.x on the same system. Here is a sample of the linking and errors: g++ -g -O2 -o bro DNS-binpac.o HTTP-binpac.o main.o net_util.o util.o parse.o scan.o re-parse.o re-scan.o rule-parse.o rule-scan.o Active.o Analyzer.o Anon.o ARP.o Attr.o BackDoor.o Base64.o BPF_Program.o BroString.o CCL.o ChunkedIO.o CompHash.o Conn.o ConnCompressor.o ContentLine.o DCE_RPC.o DFA.o DNS.o DNS_Mgr.o DbgBreakpoint.o DbgHelp.o DbgWatch.o Debug.o DebugCmds.o DebugLogger.o Desc.o Dict.o Discard.o DPM.o EquivClass.o Event.o EventHandler.o EventLauncher.o EventRegistry.o Expr.o FTP.o File.o FileAnalyzer.o Finger.o Frag.o Frame.o Func.o Gnutella.o HTTP.o Hash.o ICMP.o ID.o Ident.o IntSet.o InterConn.o IOSource.o IRC.o List.o Logger.o Login.o MIME.o NCP.o NFA.o NFS.o NTP.o NVT.o Net.o NetVar.o NetbiosSSN.o Obj.o OSFinger.o PacketFilter.o PacketSort.o PersistenceSerializer.o PktDagSrc.o PktSrc.o PIA.o PolicyFile.o POP3.o Portmap.o PrefixTable.o PriorityQueue.o Queue.o RE.o RPC.o Reassem.o RemoteSerializer.o Rlogin.o RSH.o Rule.o RuleAction.o RuleCondition.o RuleMatcher.o ScriptAnaly.o SmithWaterman.o SMB.o SMTP.o SSH.o Scope.o SerializationFormat.o SerialObj.o Serializer.o Sessions.o StateAccess.o Stats.o SteppingStone.o Stmt.o TCP.o TCP_Endpoint.o TCP_Reassembler.o TCP_Rewriter.o Telnet.o Timer.o Traverse.o Trigger.o TwoWise.o Type.o UDP.o Val.o Var.o XDR.o ZIP.o bsd-getopt-long.o cq.o md5.o patricia.o setsignal.o version.o UDP_Rewriter.o DNS_Rewriter.o PacketDumper.o Rewriter.o strsep.o nb_dns.o X509.o SSLCiphers.o SSLInterpreter.o SSLProxy.o SSLv2.o SSLv3.o SSLv3Automaton.o binpac-lib_pac.o binpac_bro-lib_pac.o dce_rpc_pac.o dce_rpc_simple_pac.o dns_pac.o dns_tcp_pac.o http_pac.o ncp_pac.o rpc_pac.o smb_pac.o -Llibedit -ledit -lmagic -lz -lpcap -lpcap -lssl -lcrypto -lpcap /usr/lib/libresolv.a -ltermcap -ltermcap -lm -L../aux/binpac/lib -lbinpac -lmagic -lz -lpcap -lpcap -lssl -lcrypto -lpcap /usr/lib/libresolv.a -ltermcap -ltermcap main.o: In function `bro_new_handler':/root/BUILD/Bro/bro-1.3.2/src/main.cc:373: undefined reference to `out_of_memory(char const*)' main.o: In function `usage()':/root/BUILD/Bro/bro-1.3.2/src/main.cc:201: undefined reference to `bro_path()' :/root/BUILD/Bro/bro-1.3.2/src/main.cc:202: undefined reference to `bro_prefixes()' main.o: In function `termination_signal()':/root/BUILD/Bro/bro-1.3.2/src/main.cc:279: undefined reference to `message(char const*)' main.o: In function `sig_handler(int)':/root/BUILD/Bro/bro-1.3.2/src/main.cc:298: undefined reference to `internal_error(char const*, ...)' main.o: In function `main':/root/BUILD/Bro/bro-1.3.2/src/main.cc:382: undefined reference to `copy_string(char const*)' :/root/BUILD/Bro/bro-1.3.2/src/main.cc:691: undefined reference to `current_time(bool)' :/root/BUILD/Bro/bro-1.3.2/src/main.cc:693: undefined reference to `init_random_seed(unsigned int, char const*, char const*)' :/root/BUILD/Bro/bro-1.3.2/src/main.cc:553: undefined reference to `streq(char const*, char const*)' :/root/BUILD/Bro/bro-1.3.2/src/main.cc:604: undefined reference to `hash_md5(unsigned int, unsigned char const*, unsigned char*)' Thanks, // Joel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080421/add90db7/attachment.html From vern at cs.berkeley.edu Mon Apr 21 19:25:00 2008 From: vern at cs.berkeley.edu (vern at cs.berkeley.edu) Date: Mon, 21 Apr 2008 19:25:00 -0700 Subject: [Bro] Partial tcpdump traces In-Reply-To: (Mon, 21 Apr 2008 18:33:55 EDT). Message-ID: <200804220225.m3M2P2I8016174@pork.ICSI.Berkeley.EDU> > I have a trickier question than last time. I am inputting into Bro partial > tcpdump traces (by using the -s option in tcpdump - I am now getting only > the first 100 bytes of a packet instead of the full packet). You can force more analysis by running with bro -C to disable checksum validation. However, you'll only get very limited analysis out of the system, since it's designed to operate on full payloads. Vern From vern at icir.org Mon Apr 21 19:27:52 2008 From: vern at icir.org (Vern Paxson) Date: Mon, 21 Apr 2008 19:27:52 -0700 Subject: [Bro] Bro 1.3.x / FC5 / Linking Issue In-Reply-To: <46ee7b1c0804211621h5d6442e1ub2b0b3901971b2a6@mail.gmail.com> (Mon, 21 Apr 2008 16:21:52 PDT). Message-ID: <200804220227.m3M2RsTk016219@pork.ICSI.Berkeley.EDU> > I was trying to build Bro 1.3.2 on FC5. Im running into linking issues and I > wanted to see if there is a known fix before I start racking my brain on > this. Not a known problem to my knowledge. > `bro_new_handler':/root/BUILD/Bro/bro-1.3.2/src/main.cc:373: undefined > reference to `out_of_memory(char const*)' > ... All of the missing functions are in util.o, so the problem is either that util.o didn't build correctly (and is an empty object) or your linker is somehow not loading it like it's supposed to. Vern From sanmeetkbhatia at gmail.com Tue Apr 22 00:25:44 2008 From: sanmeetkbhatia at gmail.com (Sanmeet Bhatia) Date: Tue, 22 Apr 2008 12:55:44 +0530 Subject: [Bro] Regarding set_record_packets Message-ID: <9b5f0ba60804220025y245618f1s5e295f681b3a4117@mail.gmail.com> Hi, Can u please tell me where the packets traced by set_record_packets() are stored in BRO? Please help. Regards, Sanmeet -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080422/e49698ca/attachment.html From robin at icir.org Tue Apr 22 08:50:28 2008 From: robin at icir.org (Robin Sommer) Date: Tue, 22 Apr 2008 08:50:28 -0700 Subject: [Bro] Regarding set_record_packets In-Reply-To: <9b5f0ba60804220025y245618f1s5e295f681b3a4117@mail.gmail.com> References: <9b5f0ba60804220025y245618f1s5e295f681b3a4117@mail.gmail.com> Message-ID: <20080422155028.GE81034@icir.org> On Tue, Apr 22, 2008 at 12:55 +0530, Sanmeet Bhatia wrote: > Can u please tell me where the packets traced by set_record_packets() are > stored in BRO? They go into the trace file specified with -W. Note that when using -W, capturing packets is the default for most connections so you usually use this function to turn capturing *off*. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Tue Apr 22 11:34:32 2008 From: robin at icir.org (Robin Sommer) Date: Tue, 22 Apr 2008 11:34:32 -0700 Subject: [Bro] Regarding set_record_packets In-Reply-To: <20080422155028.GE81034@icir.org> References: <9b5f0ba60804220025y245618f1s5e295f681b3a4117@mail.gmail.com> <20080422155028.GE81034@icir.org> Message-ID: <20080422183432.GA91470@icir.org> On Tue, Apr 22, 2008 at 08:50 -0700, I wrote: > They go into the trace file specified with -W. Note that when using > -W, capturing packets is the default for most connections so you > usually use this function to turn capturing *off*. Oops, -W was supposed to be -w. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From joel.ebrahimi at gmail.com Tue Apr 22 12:42:38 2008 From: joel.ebrahimi at gmail.com (Joel Ebrahimi) Date: Tue, 22 Apr 2008 12:42:38 -0700 Subject: [Bro] Bro 1.3.x / FC5 / Linking Issue In-Reply-To: <200804220227.m3M2RsTk016219@pork.ICSI.Berkeley.EDU> References: <46ee7b1c0804211621h5d6442e1ub2b0b3901971b2a6@mail.gmail.com> <200804220227.m3M2RsTk016219@pork.ICSI.Berkeley.EDU> Message-ID: <46ee7b1c0804221242i43b801e9q632f2ad3bbc224fc@mail.gmail.com> The util object file seems to have built fine and it is present in the linking. I will take a look at the loader to see what I can determine. // Joel On Mon, Apr 21, 2008 at 7:27 PM, Vern Paxson wrote: > > I was trying to build Bro 1.3.2 on FC5. Im running into linking issues > and I > > wanted to see if there is a known fix before I start racking my brain on > > this. > > Not a known problem to my knowledge. > > > `bro_new_handler':/root/BUILD/Bro/bro-1.3.2/src/main.cc:373: undefined > > reference to `out_of_memory(char const*)' > > ... > > All of the missing functions are in util.o, so the problem is either that > util.o didn't build correctly (and is an empty object) or your linker is > somehow not loading it like it's supposed to. > > Vern > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080422/ea7f6d15/attachment.html From shoeyfighter at gmail.com Tue Apr 22 23:41:12 2008 From: shoeyfighter at gmail.com (Shoey Fighter) Date: Tue, 22 Apr 2008 23:41:12 -0700 Subject: [Bro] Connection Events related to scan.bro Message-ID: Hello. I am trying to understand the scanning algorithm, and am having some slight problems understanding when certain events are generated. Below I have included a list of the events I am interested in and my best understanding: connection_established A TCP handshake has been completed successfully. partial_connection ? connection_attempt A TCP SYN packet has been sent. connection_rejected A TCP RST was seen in response to a TCP SYN. connection_pending I am not too sure about this one. Can this only happen if the analyzer is shut down in the middle of a connection? connection_half_finished Is this when one side of a connection attempts to close a non-existant connection? Also, slightly unrelated, I noticed in the udp-common.bro, the code relating to "use_TRW_algorithm" is commented out... Is there a special reason for this? Thanks, Cameron Hertel From kevlo at kevlo.org Wed Apr 23 00:32:17 2008 From: kevlo at kevlo.org (Kevin Lo) Date: Wed, 23 Apr 2008 15:32:17 +0800 Subject: [Bro] [PATCH] Fix signedness issue in src/nb_dns.c Message-ID: <1208935937.5920.10.camel@monet> Hi, This patch - uses socklet_t where appropriate - fixes signedness issue Kevin -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-nb_dns.c Type: text/x-patch Size: 934 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080423/704c2365/attachment.bin From seth at net.ohio-state.edu Wed Apr 23 00:34:07 2008 From: seth at net.ohio-state.edu (Seth Hall) Date: Wed, 23 Apr 2008 03:34:07 -0400 Subject: [Bro] Connection Events related to scan.bro In-Reply-To: References: Message-ID: On Apr 23, 2008, at 2:41 AM, Shoey Fighter wrote: > Below I have included a list of the events I am interested in and > my best > understanding: Here's the documentation for those events. http://www.bro-ids.org/wiki/index.php/Reference_Manual:_Analyzers_and_Events#Generic_TCP_connection_events .Seth --- Seth Hall Network Security - Office of the CIO The Ohio State University Phone: 614-292-9721 From shaw_yxf at 163.com Wed Apr 23 02:58:11 2008 From: shaw_yxf at 163.com (shaw_yxf) Date: Wed, 23 Apr 2008 17:58:11 +0800 Subject: [Bro] I can't download current vesion of Bro-ids Message-ID: <200804231758081258055@163.com> Hello: There is always a dialog-box poped up with a connection problem every time i tried to download the current version of Bro. Can any nice guy take the trouble to send me a copy via email. Thanks! Shaw shaw_yxf 2008-04-23 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080423/2b907f83/attachment.html From kjk at eecs.berkeley.edu Wed Apr 23 06:29:30 2008 From: kjk at eecs.berkeley.edu (Jayanth Kannan) Date: Wed, 23 Apr 2008 06:29:30 -0700 Subject: [Bro] help regarding using bro on application-level byte stream Message-ID: <5ea2307f0804230629k67b2a0b5k70bb050adbc8fc1d@mail.gmail.com> Hi, I have a question regarding running Bro on a application-level TCP byte stream, and was wondering which implementation option to choose. Any help is much appreciated! Details below. I have access to a application-level byte stream (eg: say, a http session consisting of http put and get packets) that I would like to run Bro on it in an online fashion (I specifically plan to use its trace anonymization capabilities). I do not have access to the corresponding TCP byte stream / IP byte stream, but I do have the TCP state information required (source/dest addr, source/dest port). I am wondering how to have Bro process these packets. I can think of the following ways by reading the various docs, but am not sure whether there is anything else I have missed. 1. Cook up fake link-layer, TCP,IP headers, and feed Bro via a FIFO. 2. Use Brocolli to send really low-level events (events being "so and so bytes seen on so and so conn"). These events have to be low-level because I am trying to minimize any application-specific parsing before sending to Bro. 3. Use the Bro source code directly, and somehow instantiate an analyzer directly on the byte-stream. Any state needed (such as connection endpoints) have to be cooked up. After reading the source code and various docs, I am tending towards (3), since it won't have the performance hit of a FIFO/broccoli, but am wondering whether the state is seperable enough for me to do this. Thanks in advance, and if anything is not clear, please let me know, Jayanth -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080423/1b350578/attachment.html From d.nechay at gmail.com Tue Apr 22 23:18:23 2008 From: d.nechay at gmail.com (Danny Nechay) Date: Wed, 23 Apr 2008 02:18:23 -0400 Subject: [Bro] Partial tcpdump traces In-Reply-To: <20080422220210.GB98168@icir.org> References: <200804220631.m3M6Vtxl019471@pork.ICSI.Berkeley.EDU> <20080422153604.GA81034@icir.org> <20080422220210.GB98168@icir.org> Message-ID: Hi, could you possibly point me towards which files or functions I should look at to get rid of these sanity checks? I know I'm not exactly using Bro for its proper use - I just need it to provide a ground truth for all flows inside of a trace. So far I've had no problems with full tcpdump traces, but if I could just find a way for it to handle partial tcpdump traces then it would suit my needs perfectly. Thanks. Daniel. On Tue, Apr 22, 2008 at 6:02 PM, Robin Sommer wrote: > > On Tue, Apr 22, 2008 at 08:36 -0700, I wrote: > > > I think yes, it should. My guess would have also been that it's the > > checksum check which prevents Bro from doing the matching. I'll try > > it later to see what I can find. > > So I looked briefly into this: there are more sanity checks inside > the TCP analyzer which prevent the payload from reaching the > signature engine. Nothing we'd really want to change though I think. > > Robin > > -- > Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080423/44bc69bf/attachment.html From robin at icir.org Wed Apr 23 12:19:39 2008 From: robin at icir.org (Robin Sommer) Date: Wed, 23 Apr 2008 12:19:39 -0700 Subject: [Bro] [PATCH] Fix signedness issue in src/nb_dns.c In-Reply-To: <1208935937.5920.10.camel@monet> References: <1208935937.5920.10.camel@monet> Message-ID: <20080423191939.GB38223@icir.org> Thanks for the patch! On Wed, Apr 23, 2008 at 15:32 +0800, you wrote: > - hp = (HEADER *)msg; > + hp = (HEADER *)(u_char *)msg; I'm not sure I understand this change? Isn't the second cast neutralized by the first? Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Wed Apr 23 12:23:21 2008 From: robin at icir.org (Robin Sommer) Date: Wed, 23 Apr 2008 12:23:21 -0700 Subject: [Bro] Connection Events related to scan.bro In-Reply-To: References: Message-ID: <20080423192321.GC38223@icir.org> On Tue, Apr 22, 2008 at 23:41 -0700, you wrote: > Also, slightly unrelated, I noticed in the udp-common.bro, the code > relating to "use_TRW_algorithm" is commented out... Is there a special > reason for this? There is some problem with the TRW implementation when used with UDP traffic but I don't recall what exactly it was. The SVN log says: r3297 [...] Also, don't analyze UDP traffic, which currently is misaccounted. Vern, do you remember what this was about? Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Wed Apr 23 12:27:34 2008 From: robin at icir.org (Robin Sommer) Date: Wed, 23 Apr 2008 12:27:34 -0700 Subject: [Bro] Partial tcpdump traces In-Reply-To: References: <200804220631.m3M6Vtxl019471@pork.ICSI.Berkeley.EDU> <20080422153604.GA81034@icir.org> <20080422220210.GB98168@icir.org> Message-ID: <20080423192734.GD38223@icir.org> On Wed, Apr 23, 2008 at 02:18 -0400, you wrote: > could you possibly point me towards which files or functions I should look > at to get rid of these sanity checks? The one I found is this snippet in TCP.cc: if ( len > 0 && (caplen >= len || packet_children.size()) && ! flags.RST() && ! Skipping() ) need_contents = DeliverData(t, data, len, caplen, ip, tp, endpoint, base_seq, is_orig, flags); The condition "caplen >= len" prevents Bro from passing the payload on. However, just removing this still doesn't get the data to the signature engine so there must be more such checks at other places (to check whether the sig engine sees the data, you can configure with --enable-debug and then run Bro with "-B rules"; that outputs some debugging info into debug.log; your payload should show up in there). Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Wed Apr 23 12:52:02 2008 From: robin at icir.org (Robin Sommer) Date: Wed, 23 Apr 2008 12:52:02 -0700 Subject: [Bro] help regarding using bro on application-level byte stream In-Reply-To: <5ea2307f0804230629k67b2a0b5k70bb050adbc8fc1d@mail.gmail.com> References: <5ea2307f0804230629k67b2a0b5k70bb050adbc8fc1d@mail.gmail.com> Message-ID: <20080423195202.GF38223@icir.org> On Wed, Apr 23, 2008 at 06:29 -0700, you wrote: > capabilities). I do not have access to the corresponding TCP byte stream / > IP byte stream, but I do have the TCP state information required > (source/dest addr, source/dest port). I am wondering how to have Bro process > these packets. Uh, that's a tricky situation! > 1. Cook up fake link-layer, TCP,IP headers, and feed Bro via a FIFO. That seems to be the easiest option for an implementation as you wouldn't need to dive into Bro but could write the conversion completely externally. Also, with tools like tcpdump etc. you could quickly see if things look like they're supposed to. However, I'm not sure I fully understand in which format your input is in exactly, so not sure how easy it would be to turn it into fake packets (e.g., is it already reassembeled or still packetized?). > 2. Use Brocolli to send really low-level events (events being "so and so > bytes seen on so and so conn"). Won't really work because Bro doesn't have any events which are so low-level. All its events are coming out of the packet/payload analysis, they aren't any which provide input for it. (You could add some of your on to feed your data into Bro protocol processing via Broccoli but that wouldn't be too different from faking packets as in (1).) > 3. Use the Bro source code directly, and somehow instantiate an analyzer > directly on the byte-stream. Any state needed (such as connection endpoints) > have to be cooked up. That's an interesting thought. I don't have an immediate opinion on how difficult this would be. My guess is that you'd quickly be running into lots of subtle problems with lacking the state you need to keep the analysis going and which is hard to cook up. That said, if you're game to dive into Bro's internals for such a solution, you could just give it a try. However, I wouldn't spend too much time on it if it turns out to get problematic (and again at lot of this depends on how *exactly* your input looks like). One other thought: which applications are you interested in? If it's only a few and there happen to be binpac analyzers for them, you could write a standalone program feeding your data into these binpac analyzers. Final note: you mention that you want to rewrite the content: I'm not very familiar with that part of Bro but I'm guessing it also has quite a few dependencies on having packets as input. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From kjk at eecs.berkeley.edu Wed Apr 23 13:56:38 2008 From: kjk at eecs.berkeley.edu (Jayanth Kannan) Date: Wed, 23 Apr 2008 13:56:38 -0700 Subject: [Bro] help regarding using bro on application-level byte stream In-Reply-To: <20080423195202.GF38223@icir.org> References: <5ea2307f0804230629k67b2a0b5k70bb050adbc8fc1d@mail.gmail.com> <20080423195202.GF38223@icir.org> Message-ID: <5ea2307f0804231356r44764defy4c721e7a5516c955@mail.gmail.com> Hi Robin, Thanks for the quick reply! Just for more context: What I have is a application-level byte-stream in both directions which is already re-assembled and sequenced. I would like to use the trace anonymization (by Ruoming Pang et. al.) which strips out user-sensitive information from a given trace according to a user-provided script. I also need to do this in an online fashion. > > 1. Cook up fake link-layer, TCP,IP headers, and feed Bro via a FIFO. > > That seems to be the easiest option for an implementation as you > wouldn't need to dive into Bro but could write the conversion > completely externally. Also, with tools like tcpdump etc. you could > quickly see if things look like they're supposed to. However, I'm > not sure I fully understand in which format your input is in > exactly, so not sure how easy it would be to turn it into fake > packets (e.g., is it already reassembeled or still packetized?). > Well, actually cooking up the fake headers should be simple, since my data stream is already reassembled, and only needs to wrapped up in the appropriate TCP and IP headers, along with some fake SYNs, SYNACKs, and FINs. I didn't really like the idea of cooking up fake stuff, since I don't really want Bro to do analysis on these fake headers. But, as you say, this is probably the simplest option for me. > > > 2. Use Brocolli to send really low-level events (events being "so and so > > bytes seen on so and so conn"). > > Won't really work because Bro doesn't have any events which are so > low-level. All its events are coming out of the packet/payload > analysis, they aren't any which provide input for it. (You could add > some of your on to feed your data into Bro protocol processing via > Broccoli but that wouldn't be too different from faking packets as > in (1).) Oh, I see. > > 3. Use the Bro source code directly, and somehow instantiate an analyzer > > directly on the byte-stream. Any state needed (such as connection > endpoints) > > have to be cooked up. > > > That's an interesting thought. I don't have an immediate opinion on > how difficult this would be. My guess is that you'd quickly be > running into lots of subtle problems with lacking the state you need > to keep the analysis going and which is hard to cook up. That said, > if you're game to dive into Bro's internals for such a solution, you > could just give it a try. However, I wouldn't spend too much time on > it if it turns out to get problematic (and again at lot of this > depends on how *exactly* your input looks like). > Oh, I see. I have been nosing around the source code to figure this out, and the new DPD framework seems fairly subtle to get right. As you say, I will probably do this for some more time, and then go to the fake header option. > One other thought: which applications are you interested in? If it's > only a few and there happen to be binpac analyzers for them, you > could write a standalone program feeding your data into these binpac > analyzers. Well, I would like it to be as general as possible (since the application-level stream is coming from a decrypted SSL connection, which may be in use by any application), which is why I thought of leveraging Bro's broad support rather than BinPac support. Also, the anonymization script (by Pang et al) relies on the event processing of Bro, and so again, I need to run the trace through Bro to get those events. > Final note: you mention that you want to rewrite the content: I'm > not very familiar with that part of Bro but I'm guessing it also has > quite a few dependencies on having packets as input. Yes, Pang's scripts maintain a lot of application-level state in doing the anonymization, which is why I need to run them through Bro. Once again, thanks for the quick reply. Thanks, Jayanth -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080423/cc985b66/attachment.html From vern at icir.org Wed Apr 23 16:29:20 2008 From: vern at icir.org (Vern Paxson) Date: Wed, 23 Apr 2008 16:29:20 -0700 Subject: [Bro] Connection Events related to scan.bro In-Reply-To: <20080423192321.GC38223@icir.org> (Wed, 23 Apr 2008 12:23:21 PDT). Message-ID: <200804232329.m3NNTMRW006431@pork.ICSI.Berkeley.EDU> > There is some problem with the TRW implementation when used with UDP > traffic but I don't recall what exactly it was. The SVN log says: > > r3297 [...] Also, don't analyze UDP traffic, which currently is misaccounted. > > Vern, do you remember what this was about? The current structure of the code treats all UDP "connections" as failures, which breaks TRW. Vern From kevlo at kevlo.org Wed Apr 23 18:34:55 2008 From: kevlo at kevlo.org (Kevin Lo) Date: Thu, 24 Apr 2008 09:34:55 +0800 Subject: [Bro] [PATCH] Fix signedness issue in src/nb_dns.c In-Reply-To: <20080423191939.GB38223@icir.org> References: <1208935937.5920.10.camel@monet> <20080423191939.GB38223@icir.org> Message-ID: <1209000895.6120.28.camel@monet> Robin Sommer wrote: > Thanks for the patch! > > On Wed, Apr 23, 2008 at 15:32 +0800, you wrote: > > > - hp = (HEADER *)msg; > > + hp = (HEADER *)(u_char *)msg; > > I'm not sure I understand this change? Isn't the second cast > neutralized by the first? Oops, I sent the wrong diff :-( Here's the correct patch. Sorry for the confusion! > Robin Kevin -------------- next part -------------- A non-text attachment was scrubbed... Name: patch-src-nb_dns.c Type: text/x-patch Size: 765 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080424/27287a78/attachment.bin From sphinxman at gmail.com Mon Apr 28 04:54:10 2008 From: sphinxman at gmail.com (Dr. Aaron J. Ferguson) Date: Mon, 28 Apr 2008 07:54:10 -0400 Subject: [Bro] Unicode Parser?? Message-ID: <55061d630804280454j4e60168ci7f33910c17247e79@mail.gmail.com> Can Bro be configured to look for Unicode code points in network traffic then execute event-oriented analyzers that compare the activity with patterns known bad traffic? I saw a reference language called BINPac that may be able to do this. Thoughts? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080428/c9ba44d6/attachment.html From robin at icir.org Mon Apr 28 09:57:07 2008 From: robin at icir.org (Robin Sommer) Date: Mon, 28 Apr 2008 09:57:07 -0700 Subject: [Bro] Unicode Parser?? In-Reply-To: <55061d630804280454j4e60168ci7f33910c17247e79@mail.gmail.com> References: <55061d630804280454j4e60168ci7f33910c17247e79@mail.gmail.com> Message-ID: <20080428165707.GC69614@icir.org> On Mon, Apr 28, 2008 at 07:54 -0400, Dr. Aaron J. Ferguson wrote: > Can Bro be configured to look for Unicode code points in network traffic > then execute event-oriented analyzers that compare the activity with > patterns known bad traffic? One could write a signature to detect Unicode. The signature match would raise an event which can then be further analyzed for whatever indicators the known patterns rely on. Not sure if this is what you looking for. Perhaps you could give us a bit more context? > I saw a reference language called BINPac that may be able to do > this. Thoughts? Binpac is high-level language to write parsers for application-layer protocols. A Binpac parser wouldn't look for unicode itself; it could however further analyze a specific application which uses Unicode. See http://www.icir.org/robin/papers/imc06.pdf for more information. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Mon Apr 28 13:22:03 2008 From: robin at icir.org (Robin Sommer) Date: Mon, 28 Apr 2008 13:22:03 -0700 Subject: [Bro] [PATCH] Fix signedness issue in src/nb_dns.c In-Reply-To: <1209000895.6120.28.camel@monet> References: <1208935937.5920.10.camel@monet> <20080423191939.GB38223@icir.org> <1209000895.6120.28.camel@monet> Message-ID: <20080428202203.GB14365@icir.org> On Thu, Apr 24, 2008 at 09:34 +0800, you wrote: > Here's the correct patch. Sorry for the confusion! Thanks, integrated into my branch to be later merged into trunk. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From tkho at eecs.berkeley.edu Mon Apr 28 13:43:50 2008 From: tkho at eecs.berkeley.edu (Thomas Kho) Date: Mon, 28 Apr 2008 13:43:50 -0700 Subject: [Bro] Bug analyzing trace with payload stripped Message-ID: <363ff490804281343j4d34df29g331e016a39445ed6@mail.gmail.com> Hi, I'm trying to run Bro on traces with packet payloads removed and ran across a problem where an analyzer seems to be trying to do a large allocation due to payload that isn't in the trace. I haven't made any modifications to Bro or its policy scripts. I ran across the problem using Bro 1.3.2: (gdb) run -r mt Starting program: /n/banquet/db/tkho/bin/bro -r mt bro: out of memory in new. Program received signal SIGABRT, Aborted. 0x004437a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 (gdb) bt #0 0x004437a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 #1 0x00484815 in raise () from /lib/tls/libc.so.6 #2 0x00486279 in abort () from /lib/tls/libc.so.6 #3 0x08053220 in out_of_memory () #4 0x0804f730 in bro_new_handler () at main.cc:373 #5 0x0091944a in operator new () from /usr/lib/libstdc++.so.6 #6 0x081a5243 in std::vector >::reserve (this=0xa045e78, __n=892614210) at /usr/lib/gcc/i386-redhat-linux/3.4.6/../../../../include/c++/3.4.6/ext/new_allocator.h:81 #7 0x081bc674 in binpac::SunRPC::RPC_Opaque::Parse (this=0xa045e68, t_begin_of_data=0xa02ed60 "546B07", t_end_of_data=0xa02ed7c "", t_byteorder=0) at rpc_pac.cc:577 #8 0x081bcf42 in binpac::SunRPC::RPC_OpaqueAuth::Parse (this=0xa045b28, t_begin_of_data=0xa02ed5c "BNPI546B07", t_end_of_data=0xa02ed7c "", t_byteorder=0) at rpc_pac.cc:654 #9 0x081bd10a in binpac::SunRPC::RPC_Call::Parse (this=0xa045c68, t_begin_of_data=0xa02ed4c "", t_end_of_data=0xa02ed7c "", t_context=0xa040bf0, t_byteorder=0) at rpc_pac.cc:191 #10 0x081be1a9 in binpac::SunRPC::RPC_Message::Parse (this=0xa046688, t_begin_of_data=0xa02ed44 "", t_end_of_data=0xa02ed7c "", t_context=0xa040bf0) at ../src/rpc_pac.h:155 #11 0x081be421 in binpac::SunRPC::RPC_Flow::NewData (this=0xa047910, t_begin_of_data=0xa02ed44 "", t_end_of_data=0xa02ed7c "") at rpc_pac.cc:1062 #12 0x0813139e in RPC_UDP_Analyzer_binpac::DeliverPacket (this=0xa0409b8, len=56, data=0xa02ed44 "", orig=true, seq=-1, ip=0xbff786c0, caplen=8) at RPC.cc:608 #13 0x0806e496 in Analyzer::ForwardPacket (this=0xa0467d8, len=56, data=0xa02ed44 "", is_orig=false, seq=-1, ip=0xbff786c0, caplen=8) at Analyzer.cc:363 #14 0x0817f6b4 in UDP_Analyzer::DeliverPacket (this=0xa0467d8, len=56, data=0xa02ed44 "", is_orig=true, seq=-1, ip=0xbff786c0, caplen=8) at UDP.cc:179 #15 0x0807c2b3 in Connection::NextPacket (this=0xa046d9c, t=1138500525.8319471, is_orig=1, ip=0xbff786c0, len=64, caplen=8, data=@0x0, record_packet=@0xbff78638, record_content=@0xbff7863c, hdr=0xa02e728, pkt=0xa02ed1a "", hdr_size=14) at Conn.cc:241 #16 0x08153270 in NetSessions::DoNextPacket (this=0xa03edc8, t=1138500525.8319471, hdr=0xa02e728, ip_hdr=0xbff786c0, pkt=0xa02ed1a "", hdr_size=14) at Sessions.cc:584 #17 0x081537d1 in NetSessions::NextPacket (this=0xa03edc8, t=1138500525.8319471, hdr=0xa02e728, pkt=0xa02ed1a "", hdr_size=14, pkt_elem=0x0) at Sessions.cc:294 #18 0x0811960e in net_packet_dispatch (t=1138500525.8319471, hdr=0xa02e728, pkt=0xa02ed1a "", hdr_size=14, src_ps=0xa02e6f0, pkt_elem=0x0) at Net.cc:402 #19 0x081198b2 in net_packet_arrival (t=1138500525.8319471, hdr=0xa02e728, pkt=0xa02ed1a "", hdr_size=14, src_ps=0xa02e6f0) at Net.cc:464 #20 0x08126036 in PktSrc::Process (this=0xa02e6f0) at PktSrc.cc:216 #21 0x08119d21 in net_run () at Net.cc:491 #22 0x080508ee in main (argc=4, argv=0xbff78c64) at main.cc:1009 (gdb) and it also occurs in Bro 1.3.27 (rev 5632) I just pulled from SVN: (gdb) run -r mt Starting program: /n/banquet/db/tkho/bin/bro -r mt terminate called after throwing an instance of 'std::bad_alloc' what(): St9bad_alloc Program received signal SIGABRT, Aborted. 0x004437a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 (gdb) bt #0 0x004437a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 #1 0x00484815 in raise () from /lib/tls/libc.so.6 #2 0x00486279 in abort () from /lib/tls/libc.so.6 #3 0x0091b1bb in __gnu_cxx::__verbose_terminate_handler () from /usr/lib/libstdc++.so.6 #4 0x00918ed1 in ?? () from /usr/lib/libstdc++.so.6 #5 0x00918f06 in std::terminate () from /usr/lib/libstdc++.so.6 #6 0x0091904f in __cxa_throw () from /usr/lib/libstdc++.so.6 #7 0x0091949c in operator new () from /usr/lib/libstdc++.so.6 #8 0x081aef57 in std::vector >::reserve (this=0x9454400, __n=892614210) at /usr/lib/gcc/i386-redhat-linux/3.4.6/../../../../include/c++/3.4.6/ext/new_allocator.h:81 #9 0x081cb75c in binpac::SunRPC::RPC_Opaque::Parse (this=0x9455d48, t_begin_of_data=0x943a900 "546B07", t_end_of_data=0x943a91c "", t_byteorder=0) at rpc_pac.cc:577 #10 0x081cc02e in binpac::SunRPC::RPC_OpaqueAuth::Parse (this=0x9454620, t_begin_of_data=0x943a8fc "BNPI546B07", t_end_of_data=0x943a91c "", t_byteorder=0) at rpc_pac.cc:654 #11 0x081cc1f6 in binpac::SunRPC::RPC_Call::Parse (this=0x944e728, t_begin_of_data=0x943a8ec "", t_end_of_data=0x943a91c "", t_context=0x94557c0, t_byteorder=0) at rpc_pac.cc:191 #12 0x081cd295 in binpac::SunRPC::RPC_Message::Parse (this=0x94543d0, t_begin_of_data=0x943a8e4 "", t_end_of_data=0x943a91c "", t_context=0x94557c0) at ../src/rpc_pac.h:155 #13 0x081cd50d in binpac::SunRPC::RPC_Flow::NewData (this=0x944e8d0, t_begin_of_data=0x943a8e4 "", t_end_of_data=0x943a91c "") at rpc_pac.cc:1062 #14 0x08136b42 in RPC_UDP_Analyzer_binpac::DeliverPacket (this=0x944e190, len=56, data=0x943a8e4 "", orig=true, seq=-1, ip=0xbff8b2a0, caplen=8) at RPC.cc:610 #15 0x0806bfa2 in Analyzer::ForwardPacket (this=0x944e480, len=56, data=0x943a8e4 "", is_orig=false, seq=-1, ip=0xbff8b2a0, caplen=8) at Analyzer.cc:371 #16 0x0818826b in UDP_Analyzer::DeliverPacket (this=0x944e480, len=56, data=0x943a8e4 "", is_orig=true, seq=-1, ip=0xbff8b2a0, caplen=8) at UDP.cc:181 #17 0x0807a163 in Connection::NextPacket (this=0x944d7ec, t=1138500525.8319471, is_orig=1, ip=0xbff8b2a0, len=64, caplen=8, data=@0x0, record_packet=@0xbff8b218, record_content=@0xbff8b21c, hdr=0x943a2c8, pkt=0x943a8ba "", hdr_size=14) at Conn.cc:263 #18 0x08159f60 in NetSessions::DoNextPacket (this=0x944a968, t=1138500525.8319471, hdr=0x943a2c8, ip_hdr=0xbff8b2a0, pkt=0x943a8ba "", hdr_size=14) at Sessions.cc:675 #19 0x0815a50d in NetSessions::NextPacket (this=0x944a968, t=1138500525.8319471, hdr=0x943a2c8, pkt=0x943a8ba "", hdr_size=14, pkt_elem=0x0) at Sessions.cc:319 #20 0x0811e8ce in net_packet_dispatch (t=1138500525.8319471, hdr=0x943a2c8, pkt=0x943a8ba "", hdr_size=14, src_ps=0x943a290, pkt_elem=0x0) at Net.cc:417 #21 0x0811eb5a in net_packet_arrival (t=1138500525.8319471, hdr=0x943a2c8, pkt=0x943a8ba "", hdr_size=14, src_ps=0x943a290) at Net.cc:479 #22 0x0812b61a in PktSrc::Process (this=0x943a290) at PktSrc.cc:216 #23 0x0811efce in net_run () at Net.cc:508 #24 0x0805004a in main (argc=4, argv=0xbff8b854) at main.cc:965 (gdb) -Tom From sanmeetkbhatia at gmail.com Wed Apr 30 02:02:45 2008 From: sanmeetkbhatia at gmail.com (Sanmeet Bhatia) Date: Wed, 30 Apr 2008 14:32:45 +0530 Subject: [Bro] Reporting problem in http_header event Message-ID: <9b5f0ba60804300202l1981c420x9030dc35345d0bd0@mail.gmail.com> Dear sir, I have found a bug in the event called http_header. The value : string it returns has a space at the beginning. Like "www.yahoo.com" will be returned as " www.yahoo.com". I am posting You a script where I found it @load weird @load alarm @load http global path: string; global urls: set[string] ={"www.yahoo.com","mail.google.com","www.ieee.org ","www.youtube.com","www.bro-ids.org"} ; global shanz_log = open_log_file("http") &redef; redef ignore_checksums = T; event http_request(c: connection, method: string, original_URI: string, unescaped_URI: string, version: string) { path = original_URI; } event http_header(c: connection, is_orig: bool, name: string, value: string) { if(name == "HOST" ) { local v = edit(value," "); if( v in urls) { print shanz_log, fmt("%s:%s->%s:%s",c$id$orig_h,c$id$orig_p,c$id$resp_h,c$id$resp_p); } } } If I simply compare the value it doesn't match. or even if I print the value its printing with one whitespace prefixed at the beginning. Regards, Sanmeet -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20080430/3cd99b7d/attachment.html