From E.H.Beekman at amc.nl Fri Aug 6 06:14:38 2010 From: E.H.Beekman at amc.nl (Ewald Beekman) Date: Fri, 6 Aug 2010 15:14:38 +0200 Subject: [Bro] filtering types from http-ext-outbound.log Message-ID: <20100806131438.GI23884@core4.amc.nl> My http ext outbound logfile is getting a little to large to work with and contains information about downloaded jpg's, gif's, etc. which i am not interested in anyway. How can i filter out those url's so they don't end up into this logfile, or if that's complicated, how can i limit logging into this file to only contain "application/x-dosexec" downloads? thanks in advance, Ewald... From seth at remor.com Fri Aug 6 06:35:36 2010 From: seth at remor.com (Seth Hall) Date: Fri, 6 Aug 2010 09:35:36 -0400 Subject: [Bro] filtering types from http-ext-outbound.log In-Reply-To: <20100806131438.GI23884@core4.amc.nl> References: <20100806131438.GI23884@core4.amc.nl> Message-ID: On Aug 6, 2010, at 9:14 AM, Ewald Beekman wrote: > How can i filter out those url's so they don't end up into this > logfile, or if that's complicated, how can i limit logging into > this file to only contain "application/x-dosexec" downloads? You have two choices, you can handle the http_ext event yourself and do logging however you want (check out logging.http-ext.bro for an example), or you can do the following after you load the logging.http-ext.bro script. redef HTTP::logging = None; # Other options are Inbound, Outbound, and the default All It still logs requests matching file types you want logged because the http-ext-identified-files.bro script forces identified files to be logged. All of the options for HTTP logging through the http-ext.bro script are documented at the top of the logging.http-ext.bro script. Options for identifying files you want to log can be found at the top of the http-ext-identified-files.bro script. I hadn't considered doing a negative filter for logs, but that is certainly something I could add to my logging framework. My initial thought is that it would just be a regular expression for matching the full log line and if the regex matches the line, it wouldn't be logged. .Seth From E.H.Beekman at amc.nl Fri Aug 6 07:17:21 2010 From: E.H.Beekman at amc.nl (Ewald Beekman) Date: Fri, 6 Aug 2010 16:17:21 +0200 Subject: [Bro] filtering types from http-ext-outbound.log In-Reply-To: References: <20100806131438.GI23884@core4.amc.nl> Message-ID: <20100806141721.GJ23884@core4.amc.nl> Hi Seth, Thanks for the quick reply. If i use the redef in my local.bro config: @load all-ext.bro redef HTTP::logging = None; http-ext-outbound.log stays empty, even when i download some executables myself. Next i removed the redef from my local.bro config and tried to change ignored_urls from the file http-ext-identified-files.bro from: const ignored_urls = /^http:\/\/(au\.|www\.)?download\.windowsupdate\.com\/msdownload\/update/ &redef; in: const ignored_urls = /^http:\/\/.*\.(jpg|png|html|gif|htm)$/ &redef; But that doesn't stop those from being logged :-( thanks in advance, Ewald... On Fri, Aug 06, 2010 at 03:35:36PM CEST, Seth Hall wrote: > > On Aug 6, 2010, at 9:14 AM, Ewald Beekman wrote: > > > How can i filter out those url's so they don't end up into this > > logfile, or if that's complicated, how can i limit logging into > > this file to only contain "application/x-dosexec" downloads? > > You have two choices, you can handle the http_ext event yourself and do logging however you want (check out logging.http-ext.bro for an example), or you can do the following after you load the logging.http-ext.bro script. > > redef HTTP::logging = None; # Other options are Inbound, Outbound, and the default All > > It still logs requests matching file types you want logged because the http-ext-identified-files.bro script forces identified files to be logged. All of the options for HTTP logging through the http-ext.bro script are documented at the top of the logging.http-ext.bro script. Options for identifying files you want to log can be found at the top of the http-ext-identified-files.bro script. > > I hadn't considered doing a negative filter for logs, but that is certainly something I could add to my logging framework. My initial thought is that it would just be a regular expression for matching the full log line and if the regex matches the line, it wouldn't be logged. > > .Seth > -- Ewald Beekman, CISSP. Academic Medical Center, NL From seth at remor.com Fri Aug 6 08:57:57 2010 From: seth at remor.com (Seth Hall) Date: Fri, 6 Aug 2010 11:57:57 -0400 Subject: [Bro] filtering types from http-ext-outbound.log In-Reply-To: <20100806141721.GJ23884@core4.amc.nl> References: <20100806131438.GI23884@core4.amc.nl> <20100806141721.GJ23884@core4.amc.nl> Message-ID: On Aug 6, 2010, at 10:17 AM, Ewald Beekman wrote: > @load all-ext.bro > redef HTTP::logging = None; Sorry about that, I can't even configure my own script correctly. :) redef HTTP::logging=Neither; Then check for your requests in your http-ext-identified-files-outbound.log. When you disable logging for the main http-ext file, it only disables it for that file, but the tagged files (with "identified-files" added) are still logged. .Seth From redlamb19 at gmail.com Sat Aug 7 18:10:03 2010 From: redlamb19 at gmail.com (Peter Erickson) Date: Sat, 7 Aug 2010 20:10:03 -0500 Subject: [Bro] Reassembling packets during inspection Message-ID: <20100807201003.kgmnci9ggkw4w0c4@imp.redlamb.net> Is it possible to reassemble TCP and UDP streams while Bro inspects a captured tracefile from a different machine? I have several pcap files that contain approx 6 hrs worth of traffic. I would like to have Bro analyze the data, but I also need the streams (both tcp and udp) reassembled and stored on the hard drive for use with custom python scripts. I've noticed that the contents.bro script will reassemble TCP streams, but it doesn't appear to assemble UDP as well. Any help with this would be greatly appreciated. I have read through the quick start, wiki, and list archives with no luck. I am new to Bro so sorry if this is a basic question. Thanks in advance... I'm running Bro 1.5 -- Peter Erickson redlamb19 _at_ gmail _dot_ com From vern at icir.org Sat Aug 7 18:20:03 2010 From: vern at icir.org (Vern Paxson) Date: Sat, 07 Aug 2010 18:20:03 -0700 Subject: [Bro] Reassembling packets during inspection In-Reply-To: <20100807201003.kgmnci9ggkw4w0c4@imp.redlamb.net> (Sat, 07 Aug 2010 20:10:03 CDT). Message-ID: <201008080120.o781K3vj010450@pork.ICSI.Berkeley.EDU> > Is it possible to reassemble TCP and UDP streams while Bro inspects a What do you mean by reassembling a UDP stream? These don't have a particular reassembly ordering associated with them. If you just want to extract the contents of a given UDP flow, you can do so using tcpdump directly. Vern From redlamb19 at gmail.com Sat Aug 7 18:33:33 2010 From: redlamb19 at gmail.com (Peter Erickson) Date: Sat, 7 Aug 2010 20:33:33 -0500 Subject: [Bro] Reassembling packets during inspection In-Reply-To: <201008080120.o781K3vj010450@pork.ICSI.Berkeley.EDU> References: <201008080120.o781K3vj010450@pork.ICSI.Berkeley.EDU> Message-ID: <20100807203333.jr00joxv404kc8w0@imp.redlamb.net> >> Is it possible to reassemble TCP and UDP streams while Bro inspects a > > What do you mean by reassembling a UDP stream? These don't have a particular > reassembly ordering associated with them. If you just want to extract the > contents of a given UDP flow, you can do so using tcpdump directly. > Thanks for the quick response. I realize that UDP doesn't have sequence numbers, etc, but I was hoping that Bro would be able to assemble the flow into something that could be externally processed. As a crude example (I realize that Bro has DNS analyzers), but I wanted a file that contained the raw dns request and raw dns reply. I work with malware a lot and I have scripts that look for custom protocols. -- Peter Erickson redlamb19 _at_ gmail _dot_ com From trueblue7711 at gmail.com Tue Aug 10 10:08:47 2010 From: trueblue7711 at gmail.com (trueblue7711) Date: Tue, 10 Aug 2010 13:08:47 -0400 Subject: [Bro] Problem with Bro Reports Message-ID: I have Bro 1.4 configured and running fine, however, it does not appear to be generating any summary reports. I ran bro_config and edited bro.cfg with settings for my setup, but I am not receiving any summary report at the address specified in bro.cfg, nor is it generating any summary report file. Any ideas on what could be causing the problem? Also, is the summary report written to a file in /usr/local/bro/reports or is it only emailed? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100810/8bef9d89/attachment.html From seth at remor.com Tue Aug 10 10:27:37 2010 From: seth at remor.com (Seth Hall) Date: Tue, 10 Aug 2010 13:27:37 -0400 Subject: [Bro] Problem with Bro Reports In-Reply-To: References: Message-ID: <71358336-47CE-4475-AA90-605F345C1089@remor.com> On Aug 10, 2010, at 1:08 PM, trueblue7711 wrote: > I have Bro 1.4 configured and running fine Please update to Bro 1.5, the 1.4 release is no longer supported and it looks like you are using the Brolite configuration framework which is currently deprecated and due to be completely stripped from future releases. .Seth From trueblue7711 at gmail.com Tue Aug 10 19:02:57 2010 From: trueblue7711 at gmail.com (trueblue7711) Date: Tue, 10 Aug 2010 22:02:57 -0400 Subject: [Bro] Problem with Bro Reports In-Reply-To: <71358336-47CE-4475-AA90-605F345C1089@remor.com> References: <71358336-47CE-4475-AA90-605F345C1089@remor.com> Message-ID: I would, but there appears to be a compatibility issue with Bro 1.5 on my system -- it does not complete compiling and returns errors. So for now, I have been using Bro 1.4. On Tue, Aug 10, 2010 at 1:27 PM, Seth Hall wrote: > > On Aug 10, 2010, at 1:08 PM, trueblue7711 wrote: > > > I have Bro 1.4 configured and running fine > > Please update to Bro 1.5, the 1.4 release is no longer supported and it > looks like you are using the Brolite configuration framework which is > currently deprecated and due to be completely stripped from future releases. > > .Seth -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100810/b4e50381/attachment.html From seth at remor.com Tue Aug 10 19:47:13 2010 From: seth at remor.com (Seth Hall) Date: Tue, 10 Aug 2010 22:47:13 -0400 Subject: [Bro] Problem with Bro Reports In-Reply-To: References: <71358336-47CE-4475-AA90-605F345C1089@remor.com> Message-ID: <5B4FEF6C-B4EF-40FF-9114-B239F71FEA07@remor.com> On Aug 10, 2010, at 10:02 PM, trueblue7711 wrote: > I would, but there appears to be a compatibility issue with Bro 1.5 on my system -- it does not complete compiling and returns errors. What's the error? .Seth From trueblue7711 at gmail.com Tue Aug 10 23:35:41 2010 From: trueblue7711 at gmail.com (trueblue7711) Date: Wed, 11 Aug 2010 02:35:41 -0400 Subject: [Bro] Problem with Bro Reports In-Reply-To: <5B4FEF6C-B4EF-40FF-9114-B239F71FEA07@remor.com> References: <71358336-47CE-4475-AA90-605F345C1089@remor.com> <5B4FEF6C-B4EF-40FF-9114-B239F71FEA07@remor.com> Message-ID: Running ./configure works fine. Then after doing a 'make', it compiles for a bit and then stops and prints these errors: capstats.cc:14:26: net/ethernet.h: No such file or directory. (About 6 more capstats.cc errors) capstyats.cc:423:error (Each undeclared identifier is reported only once for each function it appears in.) *** Error code 1 Stop in /home/trueblue/bro-1.5.1/aux/capstats (line 104 of /usr/share/mk/ sys.mk). *** Error code 1 Stop in /home/trueblue/bro-1.5.1/aux/capstats (line 120 of Makefile). *** Error code 1 Stop in /home/trueblue/bro-1.5.1/aux/broctl (line 65 of Makefile). *** Error code 1 Stop in /home/trueblue/bro-1.5.1/aux (line 181 of Makefile). *** Error code 1 Stop in /home/trueblue/bro-1.5.1/aux (line 148 of Makefile). *** Error code 1 Stop in /home/trueblue/bro-1.5.1 (line 198 of Makefile). *** Error code 1 Stop in /home/trueblue/bro-1.5.1 (line 137 of Makefile). On Tue, Aug 10, 2010 at 10:47 PM, Seth Hall wrote: > > On Aug 10, 2010, at 10:02 PM, trueblue7711 wrote: > > > I would, but there appears to be a compatibility issue with Bro 1.5 on my > system -- it does not complete compiling and returns errors. > > What's the error? > > .Seth > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100811/63f1b25c/attachment.html From Tyler.Schoenke at colorado.edu Wed Aug 11 07:28:54 2010 From: Tyler.Schoenke at colorado.edu (Tyler T. Schoenke) Date: Wed, 11 Aug 2010 08:28:54 -0600 Subject: [Bro] Problem with Bro Reports In-Reply-To: References: <71358336-47CE-4475-AA90-605F345C1089@remor.com> <5B4FEF6C-B4EF-40FF-9114-B239F71FEA07@remor.com> Message-ID: <4C62B3A6.8070600@colorado.edu> It looks like that error might be related to tracker ticket #213 http://tracker.icir.org/bro/ticket/213 -- Tyler Schoenke Network Security Analyst IT Security Office University of Colorado - Boulder On 08/11/2010 12:35 AM, trueblue7711 wrote: > Running ./configure works fine. Then after doing a 'make', it compiles > for a bit and then stops and prints these errors: > > capstats.cc:14:26: net/ethernet.h: No such file or directory. > From hsbedi at memphis.edu Wed Aug 11 20:18:36 2010 From: hsbedi at memphis.edu (Harkeerat Bedi) Date: Wed, 11 Aug 2010 22:18:36 -0500 Subject: [Bro] Using BRO for measuring TCP flow bandwidth Message-ID: Hello, I am a beginner to BRO IDS and am currently using it for monitoring one interface of a FreeBSD machine over an experiment network. Part of my project now requires to also capture the network bandwidth being utilized by a flow that passes thorough the BRO monitored interface. By flow we mean, a source-destination IP pair. Is this kind of measurement possible in BRO? If not, is there any add-on which can be used to accomplish the same task using BRO? Kindly suggest and thanks in advance. Regards, Harkeerat Bedi -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100811/4cf6124c/attachment.html From JAzoff at uamail.albany.edu Thu Aug 12 06:13:16 2010 From: JAzoff at uamail.albany.edu (Justin Azoff) Date: Thu, 12 Aug 2010 09:13:16 -0400 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: References: Message-ID: <20100812131315.GF24971@datacomm.albany.edu> On Wed, Aug 11, 2010 at 11:18:36PM -0400, Harkeerat Bedi wrote: > Part of my project now requires to also capture the network bandwidth > being utilized by a flow that passes thorough the BRO monitored > interface. By flow we mean, a source-destination IP pair. capstats(included with bro) can do this: capstats -I 5 -i eth0 -f 'host a.b.c.d and host e.f.g.h' -- -- Justin Azoff -- Network Security & Performance Analyst From hsbedi at memphis.edu Thu Aug 12 09:51:03 2010 From: hsbedi at memphis.edu (Harkeerat Bedi) Date: Thu, 12 Aug 2010 11:51:03 -0500 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: <20100812131315.GF24971@datacomm.albany.edu> References: <20100812131315.GF24971@datacomm.albany.edu> Message-ID: Thank you for your prompt assistance Justin. I will look into this. Regards, Harkeerat Bedi On Thu, Aug 12, 2010 at 8:13 AM, Justin Azoff wrote: > On Wed, Aug 11, 2010 at 11:18:36PM -0400, Harkeerat Bedi wrote: > > Part of my project now requires to also capture the network bandwidth > > being utilized by a flow that passes thorough the BRO monitored > > interface. By flow we mean, a source-destination IP pair. > > capstats(included with bro) can do this: > > capstats -I 5 -i eth0 -f 'host a.b.c.d and host e.f.g.h' > > -- > -- Justin Azoff > -- Network Security & Performance Analyst > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100812/7a934f75/attachment.html From sridhar.basam at gmail.com Thu Aug 12 10:19:14 2010 From: sridhar.basam at gmail.com (sridhar basam) Date: Thu, 12 Aug 2010 13:19:14 -0400 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: References: Message-ID: If you are looking to get averages over the tcp session, look at the conn.bro file. It records enough information for you to derive the average throughput in either direction over the life of the connection. You can change the routine "record_connection" to calculate the avg. throughput in each direction. sridhar On Wed, Aug 11, 2010 at 11:18 PM, Harkeerat Bedi wrote: > Hello, > > I am a beginner to BRO IDS and am currently using it for monitoring one > interface of a FreeBSD machine over an experiment network. > > Part of my project now requires to also capture the network bandwidth being > utilized by a flow that passes thorough the BRO monitored interface. By > flow we mean, a source-destination IP pair. > > Is this kind of measurement possible in BRO? If not, is there any add-on > which can be used to accomplish the same task using BRO? > > Kindly suggest and thanks in advance. > > Regards, > Harkeerat Bedi > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Sridhar -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100812/72d6e221/attachment.html From hsbedi at memphis.edu Thu Aug 12 16:03:28 2010 From: hsbedi at memphis.edu (Harkeerat Bedi) Date: Thu, 12 Aug 2010 18:03:28 -0500 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: References: Message-ID: Thank you Sridhar. I think you what you mentioned is kind of what I am trying to do. Allow me to look into the conn.bro file and I will update here accordingly. Thank you once again. Regards, Harkeerat Bedi On Thu, Aug 12, 2010 at 12:19 PM, sridhar basam wrote: > If you are looking to get averages over the tcp session, look at the > conn.bro file. It records enough information for you to derive the average > throughput in either direction over the life of the connection. You can > change the routine "record_connection" to calculate the avg. throughput in > each direction. > > sridhar > > > On Wed, Aug 11, 2010 at 11:18 PM, Harkeerat Bedi wrote: > >> Hello, >> >> I am a beginner to BRO IDS and am currently using it for monitoring one >> interface of a FreeBSD machine over an experiment network. >> >> Part of my project now requires to also capture the network bandwidth >> being utilized by a flow that passes thorough the BRO monitored interface. >> By flow we mean, a source-destination IP pair. >> >> Is this kind of measurement possible in BRO? If not, is there any add-on >> which can be used to accomplish the same task using BRO? >> >> Kindly suggest and thanks in advance. >> >> Regards, >> Harkeerat Bedi >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > > > > -- > Sridhar > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100812/a0ad8653/attachment.html From vern at icir.org Thu Aug 12 23:26:59 2010 From: vern at icir.org (Vern Paxson) Date: Thu, 12 Aug 2010 23:26:59 -0700 Subject: [Bro] Reassembling packets during inspection In-Reply-To: <20100807203333.jr00joxv404kc8w0@imp.redlamb.net> (Sat, 07 Aug 2010 20:33:33 CDT). Message-ID: <201008130626.o7D6QxLL015689@pork.ICSI.Berkeley.EDU> > I realize that UDP doesn't have > sequence numbers, etc, but I was hoping that Bro would be able to > assemble the flow into something that could be externally processed. Yeah, I see your point, but no, it doesn't have that in it. It wouldn't be hard to add in simplistic terms (i.e., append the payload of every new datagram in a flow to the end of the content file). Doing it in a semantic- aware fashion would of course be considerably trickier. Vern From pgoodman at ucar.edu Wed Aug 18 10:38:36 2010 From: pgoodman at ucar.edu (Paul Goodman) Date: Wed, 18 Aug 2010 11:38:36 -0600 Subject: [Bro] Bro sending sporadic email Message-ID: <4C6C1A9C.1000102@ucar.edu> Just last weekend, our Bro cluster started randomly skipping emailed reports. No change has been made to the Postfix configuration or the Bro configuration. There seems to be no pattern to the missed reports, which are supposed to be sent out every 2 hours. Sometimes they are sent, sometimes not. The logs show nothing at all, not even an attempted send for the missing reports, the report simply isn't there. From nour.g.h at hotmail.com Wed Aug 18 11:56:15 2010 From: nour.g.h at hotmail.com (nour ghandour) Date: Wed, 18 Aug 2010 18:56:15 +0000 Subject: [Bro] installing Bro on fedora 5 Message-ID: Hello, I installed Linux Fedora 5 as Virtual PC on my laptop (main operating system is Windows vista). I tried to install Bro according to instructions: 1- ./configure 2- make 3- make install but this didn't work :( I don't want to work on live network, I just want to read a pcap file and then to verify if the alarm is generated So can somebody help me?? Nour -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100818/c1911316/attachment.html From robin at icir.org Wed Aug 18 14:42:41 2010 From: robin at icir.org (Robin Sommer) Date: Wed, 18 Aug 2010 14:42:41 -0700 Subject: [Bro] Bro sending sporadic email In-Reply-To: <4C6C1A9C.1000102@ucar.edu> References: <4C6C1A9C.1000102@ucar.edu> Message-ID: <20100818214241.GC70941@icir.org> On Wed, Aug 18, 2010 at 11:38 -0600, you wrote: > Just last weekend, our Bro cluster started randomly skipping emailed > reports. Sorry, no immediate idea. Does this apply to both the alarm mail and the traffic summaries? Are Bro's log files rotated and archived correctly? There's no logging for the reporting, so it's hard to say what's going on. You could add some debugging output to the postprocessor/{mail-log,summarize-connections} scripts to see if they get executed. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From rmkml at free.fr Wed Aug 18 23:44:14 2010 From: rmkml at free.fr (rmkml) Date: Thu, 19 Aug 2010 08:44:14 +0200 (CEST) Subject: [Bro] installing Bro on fedora 5 In-Reply-To: References: Message-ID: Hi nour, It's not easy to find what is your pb without more info, do you have a error when starting bro in live capture please? could you check if you have installed libpcap + libpcap-devel please? Regards Rmkml On Wed, 18 Aug 2010, nour ghandour wrote: > Hello, > I installed Linux Fedora 5 as Virtual PC on my laptop (main operating system is Windows vista). > I tried to install Bro according to instructions: > 1- ./configure > 2- make > 3- make install > but this didn't work :( > I don't?want to work on live network, I just want to read a pcap file and then to verify if the alarm is generated > So can somebody help me?? > Nour From nour.g.h at hotmail.com Thu Aug 19 06:15:20 2010 From: nour.g.h at hotmail.com (nour ghandour) Date: Thu, 19 Aug 2010 13:15:20 +0000 Subject: [Bro] installing Bro on fedora 5 In-Reply-To: References: , Message-ID: Hi, I installed only libpcap but no libpcap-devel. I cheked if bro is working as noted in the instructions by the command "bro.rc start" and it didn't work. If there is another way to check please tell me.. In fact I have no experience with softwares under UNIX, i learned some commands from Internet. Thanks a lot, waiting for you help :) Nour > Date: Thu, 19 Aug 2010 08:44:14 +0200 > From: rmkml at free.fr > To: nour.g.h at hotmail.com > CC: bro at ICSI.Berkeley.EDU; rmkml at free.fr > Subject: Re: [Bro] installing Bro on fedora 5 > > Hi nour, > It's not easy to find what is your pb without more info, > do you have a error when starting bro in live capture please? > could you check if you have installed libpcap + libpcap-devel please? > Regards > Rmkml > > > On Wed, 18 Aug 2010, nour ghandour wrote: > > > Hello, > > I installed Linux Fedora 5 as Virtual PC on my laptop (main operating system is Windows vista). > > I tried to install Bro according to instructions: > > 1- ./configure > > 2- make > > 3- make install > > but this didn't work :( > > I don't want to work on live network, I just want to read a pcap file and then to verify if the alarm is generated > > So can somebody help me?? > > Nour -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100819/a343de1b/attachment.html From Tyler.Schoenke at colorado.edu Thu Aug 19 07:49:43 2010 From: Tyler.Schoenke at colorado.edu (Tyler T. Schoenke) Date: Thu, 19 Aug 2010 08:49:43 -0600 Subject: [Bro] installing Bro on fedora 5 In-Reply-To: References: , Message-ID: <4C6D4487.809@colorado.edu> Make sure you install all of the dependencies listed on the wiki. http://www.bro-ids.org/wiki/index.php/Bro_OS_Dependencies It might be easiest to follow the cluster install instructions for Bro version 1.5. You will want to install the cluster in stand-alone mode. http://svn.icir.org/bro/releases/release_1_5/bro/aux/broctl/README.html Tyler -- Tyler Schoenke Network Security Analyst IT Security Office University of Colorado - Boulder On 08/19/2010 07:15 AM, nour ghandour wrote: > Hi, > I installed only libpcap but no libpcap-devel. > I cheked if bro is working as noted in the instructions by the command > "bro.rc start" and it didn't work. > If there is another way to check please tell me.. > In fact I have no experience with softwares under UNIX, i learned some > commands from Internet. > Thanks a lot, > waiting for you help :) > Nour > > > Date: Thu, 19 Aug 2010 08:44:14 +0200 > > From: rmkml at free.fr > > To: nour.g.h at hotmail.com > > CC: bro at ICSI.Berkeley.EDU; rmkml at free.fr > > Subject: Re: [Bro] installing Bro on fedora 5 > > > > Hi nour, > > It's not easy to find what is your pb without more info, > > do you have a error when starting bro in live capture please? > > could you check if you have installed libpcap + libpcap-devel please? > > Regards > > Rmkml > > > > > > On Wed, 18 Aug 2010, nour ghandour wrote: > > > > > Hello, > > > I installed Linux Fedora 5 as Virtual PC on my laptop (main > operating system is Windows vista). > > > I tried to install Bro according to instructions: > > > 1- ./configure > > > 2- make > > > 3- make install > > > but this didn't work :( > > > I don't want to work on live network, I just want to read a pcap > file and then to verify if the alarm is generated > > > So can somebody help me?? > > > Nour -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100819/21e001d7/attachment.html From mattern at caltech.edu Thu Aug 19 09:54:39 2010 From: mattern at caltech.edu (Blake Mattern) Date: Thu, 19 Aug 2010 09:54:39 -0700 Subject: [Bro] Compile error (Broccoli?) Message-ID: <20100819165439.GB2326@atreides> Greetings All, I'm having an issue with compiling Bro with IPv6 and Cluster support. The base OS is FreeBSD 8.1 i368. Here is the error I'm getting: building '_broccoli_intern' extension creating /usr/home/user/bro/bro-1.5.1/aux/broctl/.python-build/temp.freebsd-8.1-RELEASE-i386-2.6 cc -DNDEBUG -O2 -pipe -D__wchar_t=wchar_t -DTHREAD_STACK_SIZE=0x100000 -fno-strict-aliasing -I/usr/home/user/bro/bro-1.5.1/aux/broccoli/src -fPIC -I../../src -I/usr/local/include/python2.6 -c broccoli_intern_wrap.c -o /usr/home/user/bro/bro-1.5.1/aux/broctl/.python-build/temp.freebsd-8.1-RELEASE-i386-2.6/broccoli_intern_wrap.o In file included from broccoli_intern_wrap.c:118: /usr/local/include/python2.6/Python.h:168:17: error: pth.h: No such file or directory broccoli_intern_wrap.c: In function '_wrap_bro_event_add_val': broccoli_intern_wrap.c:5585: warning: assignment discards qualifiers from pointer target type broccoli_intern_wrap.c: In function '_wrap_bro_event_set_val': broccoli_intern_wrap.c:5655: warning: assignment discards qualifiers from pointer target type broccoli_intern_wrap.c: In function '_wrap_bro_record_add_val': broccoli_intern_wrap.c:6810: warning: assignment discards qualifiers from pointer target type broccoli_intern_wrap.c: In function '_wrap_bro_record_set_nth_val': broccoli_intern_wrap.c:6996: warning: assignment discards qualifiers from pointer target type broccoli_intern_wrap.c: In function '_wrap_bro_record_set_named_val': broccoli_intern_wrap.c:7067: warning: assignment discards qualifiers from pointer target type error: command 'cc' failed with exit status 1 *** Error code 1 Stop in /usr/home/user/bro/bro-1.5.1/aux/broctl. *** Error code 1 The problem looks to be in the compiling of broccoli. It states that pth.h is not in /usr/local/include/python2.6/ Although I checked and its there. Any ideas. Thanks, =-=Blake From hsbedi at memphis.edu Thu Aug 19 15:26:01 2010 From: hsbedi at memphis.edu (Harkeerat Bedi) Date: Thu, 19 Aug 2010 17:26:01 -0500 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: References: Message-ID: Hello, Thank you once again for your suggestions. I have been going through the Reference Manual, the conn.bro, and the methods in that file. I also went through some examples from the Bro 2007 workshop. I am able to obtain the flow duration and amount of data transferred as I want. However I am facing one issue which is explained below. Following is what I have done, kindly suggest. 1. I have created one policy file called "ex2e.bro" and rewritten the connection_established method: event connection_established(c: connection) { local id = c$id; local log_msg = fmt("%.6f %.6f %s %s %d %d %d %d ", c$start_time, c$duration, id$orig_h, id$resp_h, id$orig_p, id$resp_p, c$orig$size, c$resp$size); print log_msg; schedule 5 sec { connection_established(c) }; } In the above policy, I call same method every 5 seconds and the connection values are printed. 2. I have one tcpdump which contains one tcp flow from 10.1.1.3 to 10.1.2.3. BRO and TCPDUMP run on an intermediate node which is analyzing this flow. This flow was started after BRO was started and was ended after BRO was ended. 3. I use my created policy file "ex2e.bro" on that tcpdump using the command: # sudo /.../bro -r testCapture4.dump ex2e.bro weird alarm | /.../cf Aug 19 13:23:21 0.001304 10.1.1.3 10.1.2.3 50191 5001 0 0 Aug 19 13:23:21 4.986504 10.1.1.3 10.1.2.3 50191 5001 593704 0 Aug 19 13:23:21 10.001823 10.1.1.3 10.1.2.3 50191 5001 1193176 0 Aug 19 13:23:21 14.993030 10.1.1.3 10.1.2.3 50191 5001 1789752 0 Aug 19 13:23:21 20.016351 10.1.1.3 10.1.2.3 50191 5001 2389224 0 Aug 19 13:23:21 25.007562 10.1.1.3 10.1.2.3 50191 5001 2985800 0 Aug 19 13:23:21 29.998899 10.1.1.3 10.1.2.3 50191 5001 3582376 0 Aug 19 13:23:21 35.014104 10.1.1.3 10.1.2.3 50191 5001 4181848 0 Aug 19 13:23:21 40.005321 10.1.1.3 10.1.2.3 50191 5001 4778424 0 Aug 19 13:23:21 45.020655 10.1.1.3 10.1.2.3 50191 5001 5377896 0 Aug 19 13:23:21 50.012500 10.1.1.3 10.1.2.3 50191 5001 5974472 0 Aug 19 13:23:21 55.027839 10.1.1.3 10.1.2.3 50191 5001 6573944 0 Aug 19 13:23:21 58.371315 10.1.1.3 10.1.2.3 50191 5001 6973592 0 As we can see, the duration of the connection is updated every 5 seconds (as the method is called every 5 seconds.) Also, the amount of originator's bytes sent are incremented accordingly . This is what I wanted. 4. However, when I run the same command on actual network traffic, that is: $ sudo /.../bro -i em2 ex2e.bro weird alarm I do not see similar kind of output. Following is what I observe: pcap bufsize = 32768 listening on em2 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 As we can see, both the duration and originator's bytes sent are not incremented. Shouldn't the duration and the originator's bytes sent increment the same way as it did on the tcpdump because I am using the same commands? Am I missing something? Also is this approach of modifying the connection_established() correct? I went with this approach as it worked on the tcpdump. I am interested in obtaining the duration of an ongoing tcp flow, and the amount of bytes transferred over an actual network so far in a periodic manner before the connection is closed. Kindly provide your suggestions. Thank you, Regards, Harkeerat Bedi On Thu, Aug 12, 2010 at 6:03 PM, Harkeerat Bedi wrote: > Thank you Sridhar. I think you what you mentioned is kind of what I am > trying to do. Allow me to look into the conn.bro file and I will update here > accordingly. > > Thank you once again. > > Regards, > Harkeerat Bedi > > > > On Thu, Aug 12, 2010 at 12:19 PM, sridhar basam wrote: > >> If you are looking to get averages over the tcp session, look at the >> conn.bro file. It records enough information for you to derive the average >> throughput in either direction over the life of the connection. You can >> change the routine "record_connection" to calculate the avg. throughput in >> each direction. >> >> sridhar >> >> >> On Wed, Aug 11, 2010 at 11:18 PM, Harkeerat Bedi wrote: >> >>> Hello, >>> >>> I am a beginner to BRO IDS and am currently using it for monitoring one >>> interface of a FreeBSD machine over an experiment network. >>> >>> Part of my project now requires to also capture the network bandwidth >>> being utilized by a flow that passes thorough the BRO monitored interface. >>> By flow we mean, a source-destination IP pair. >>> >>> Is this kind of measurement possible in BRO? If not, is there any add-on >>> which can be used to accomplish the same task using BRO? >>> >>> Kindly suggest and thanks in advance. >>> >>> Regards, >>> Harkeerat Bedi >>> >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> >> >> >> >> -- >> Sridhar >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100819/2b08d854/attachment.html From robin at icir.org Fri Aug 20 08:46:17 2010 From: robin at icir.org (Robin Sommer) Date: Fri, 20 Aug 2010 08:46:17 -0700 Subject: [Bro] Compile error (Broccoli?) In-Reply-To: <20100819165439.GB2326@atreides> References: <20100819165439.GB2326@atreides> Message-ID: <20100820154617.GA32153@icir.org> On Thu, Aug 19, 2010 at 09:54 -0700, you wrote: > /usr/local/include/python2.6/Python.h:168:17: error: pth.h: No such file or directory Not sure what's going on but my guess is that it's not a Bro/Broccoli problem but something with the Python installation. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From mattern at caltech.edu Fri Aug 20 08:53:46 2010 From: mattern at caltech.edu (Blake Mattern) Date: Fri, 20 Aug 2010 08:53:46 -0700 Subject: [Bro] Compile error (Broccoli?) In-Reply-To: <20100820154617.GA32153@icir.org> References: <20100819165439.GB2326@atreides> <20100820154617.GA32153@icir.org> Message-ID: <4C6EA50A.5000206@caltech.edu> After some digging yesterday, I think your right. The version of FreeBSD I was using on the first box was 8.0. I installed Bro on a new box with OS version 8.1. My prior setup worked just fine. (Well LibGeoIP was giving me grief) Thanks, =-=Blake On 08/20/2010 08:46 AM, Robin Sommer wrote: > > On Thu, Aug 19, 2010 at 09:54 -0700, you wrote: > >> /usr/local/include/python2.6/Python.h:168:17: error: pth.h: No such file or directory > > Not sure what's going on but my guess is that it's not a > Bro/Broccoli problem but something with the Python installation. > > Robin > From hsbedi at memphis.edu Sat Aug 21 12:36:28 2010 From: hsbedi at memphis.edu (Harkeerat Bedi) Date: Sat, 21 Aug 2010 14:36:28 -0500 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: References: Message-ID: Anyone? My question is why does BRO appear to behave differently when reading from a tcpdump or an interface. Kindly advice. Thanks, Harkeerat Bedi On Thu, Aug 19, 2010 at 5:26 PM, Harkeerat Bedi wrote: > Hello, > > Thank you once again for your suggestions. I have been going through the > Reference Manual, the conn.bro, and the methods in that file. I also went > through some examples from the Bro 2007 workshop. > > I am able to obtain the flow duration and amount of data transferred as I > want. However I am facing one issue which is explained below. Following is > what I have done, kindly suggest. > > 1. I have created one policy file called "ex2e.bro" and rewritten the > connection_established method: > > event connection_established(c: connection) > { > local id = c$id; > local log_msg = > fmt("%.6f %.6f %s %s %d %d %d %d ", > c$start_time, c$duration, id$orig_h, id$resp_h, > id$orig_p, id$resp_p, c$orig$size, c$resp$size); > print log_msg; > schedule 5 sec { connection_established(c) }; > } > > In the above policy, I call same method every 5 seconds and the connection > values are printed. > > 2. I have one tcpdump which contains one tcp flow from 10.1.1.3 to > 10.1.2.3. BRO and TCPDUMP run on an intermediate node which is analyzing > this flow. This flow was started after BRO was started and was ended after > BRO was ended. > > 3. I use my created policy file "ex2e.bro" on that tcpdump using the > command: > # sudo /.../bro -r testCapture4.dump ex2e.bro weird alarm | /.../cf > Aug 19 13:23:21 0.001304 10.1.1.3 10.1.2.3 50191 5001 0 0 > Aug 19 13:23:21 4.986504 10.1.1.3 10.1.2.3 50191 5001 593704 0 > Aug 19 13:23:21 10.001823 10.1.1.3 10.1.2.3 50191 5001 1193176 0 > Aug 19 13:23:21 14.993030 10.1.1.3 10.1.2.3 50191 5001 1789752 0 > Aug 19 13:23:21 20.016351 10.1.1.3 10.1.2.3 50191 5001 2389224 0 > Aug 19 13:23:21 25.007562 10.1.1.3 10.1.2.3 50191 5001 2985800 0 > Aug 19 13:23:21 29.998899 10.1.1.3 10.1.2.3 50191 5001 3582376 0 > Aug 19 13:23:21 35.014104 10.1.1.3 10.1.2.3 50191 5001 4181848 0 > Aug 19 13:23:21 40.005321 10.1.1.3 10.1.2.3 50191 5001 4778424 0 > Aug 19 13:23:21 45.020655 10.1.1.3 10.1.2.3 50191 5001 5377896 0 > Aug 19 13:23:21 50.012500 10.1.1.3 10.1.2.3 50191 5001 5974472 0 > Aug 19 13:23:21 55.027839 10.1.1.3 10.1.2.3 50191 5001 6573944 0 > Aug 19 13:23:21 58.371315 10.1.1.3 10.1.2.3 50191 5001 6973592 0 > > As we can see, the duration of the connection is updated every 5 seconds > (as the method is called every 5 seconds.) > Also, the amount of originator's bytes sent are incremented accordingly . > This is what I wanted. > > 4. However, when I run the same command on actual network traffic, that is: > $ sudo /.../bro -i em2 ex2e.bro weird alarm > > I do not see similar kind of output. Following is what I observe: > pcap bufsize = 32768 > listening on em2 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > 1282249401.443512 0.001360 10.1.1.3 10.1.2.3 50191 5001 0 0 > > As we can see, both the duration and originator's bytes sent are not > incremented. > > Shouldn't the duration and the originator's bytes sent increment the same > way as it did on the tcpdump because I am using the same commands? Am I > missing something? > > Also is this approach of modifying the connection_established() correct? I > went with this approach as it worked on the tcpdump. I am interested in > obtaining the duration of an ongoing tcp flow, and the amount of bytes > transferred over an actual network so far in a periodic manner before the > connection is closed. > > Kindly provide your suggestions. > > Thank you, > > Regards, > Harkeerat Bedi > > > > On Thu, Aug 12, 2010 at 6:03 PM, Harkeerat Bedi wrote: > >> Thank you Sridhar. I think you what you mentioned is kind of what I am >> trying to do. Allow me to look into the conn.bro file and I will update here >> accordingly. >> >> Thank you once again. >> >> Regards, >> Harkeerat Bedi >> >> >> >> On Thu, Aug 12, 2010 at 12:19 PM, sridhar basam wrote: >> >>> If you are looking to get averages over the tcp session, look at the >>> conn.bro file. It records enough information for you to derive the average >>> throughput in either direction over the life of the connection. You can >>> change the routine "record_connection" to calculate the avg. throughput in >>> each direction. >>> >>> sridhar >>> >>> >>> On Wed, Aug 11, 2010 at 11:18 PM, Harkeerat Bedi wrote: >>> >>>> Hello, >>>> >>>> I am a beginner to BRO IDS and am currently using it for monitoring one >>>> interface of a FreeBSD machine over an experiment network. >>>> >>>> Part of my project now requires to also capture the network bandwidth >>>> being utilized by a flow that passes thorough the BRO monitored interface. >>>> By flow we mean, a source-destination IP pair. >>>> >>>> Is this kind of measurement possible in BRO? If not, is there any add-on >>>> which can be used to accomplish the same task using BRO? >>>> >>>> Kindly suggest and thanks in advance. >>>> >>>> Regards, >>>> Harkeerat Bedi >>>> >>>> _______________________________________________ >>>> Bro mailing list >>>> bro at bro-ids.org >>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>> >>> >>> >>> >>> -- >>> Sridhar >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100821/6bb73335/attachment.html From vern at icir.org Sun Aug 22 15:38:07 2010 From: vern at icir.org (Vern Paxson) Date: Sun, 22 Aug 2010 15:38:07 -0700 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: (Sat, 21 Aug 2010 14:36:28 CDT). Message-ID: <201008222238.o7MMc7S0005838@pork.ICSI.Berkeley.EDU> > My question is why does BRO appear to behave differently when reading from a > tcpdump or an interface. Kindly advice. It's not clear to me just why you're seeing the difference. The symptoms suggest that the live run is using a different packet filter (in particular, the default SYN/FIN/RST-only filter), and thus after the connection is established, there's no input to update things further. However, if so then you should have that same effect running on the trace. You could test for this by running with a filter "-f tcp", which will capture all TCP packets. Note that your script misuses the connection_established event. It's not meant to be generated at the script-level, and the semantics of executing it again 5 seconds in the future are undefined. (Also, timing for executing such scheduled events is actually driven by the arrival of traffic, so that would be another potential difference between the live execution vs. the trace one. But again I don't offhand see why it would lead to different results.) Vern From sridhar.basam at gmail.com Mon Aug 23 03:02:32 2010 From: sridhar.basam at gmail.com (sridhar basam) Date: Mon, 23 Aug 2010 06:02:32 -0400 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: <201008222238.o7MMc7S0005838@pork.ICSI.Berkeley.EDU> References: <201008222238.o7MMc7S0005838@pork.ICSI.Berkeley.EDU> Message-ID: Sorry, i don't know why you are seeing a difference between the live and offline trace. Here is something i have used in the past to look at similar things you are trying to get at. I expanded the connection structure to include a last recorded timestamp. Then use the new_packet or tcp_packet event to compare network_time to the last recorded time (or connection close) to do the printing vs the timer event. Sridhar On Sun, Aug 22, 2010 at 6:38 PM, Vern Paxson wrote: > > My question is why does BRO appear to behave differently when reading > from a > > tcpdump or an interface. Kindly advice. > > It's not clear to me just why you're seeing the difference. The symptoms > suggest that the live run is using a different packet filter (in > particular, > the default SYN/FIN/RST-only filter), and thus after the connection is > established, there's no input to update things further. However, if so > then you should have that same effect running on the trace. > > You could test for this by running with a filter "-f tcp", which will > capture all TCP packets. > > Note that your script misuses the connection_established event. It's not > meant to be generated at the script-level, and the semantics of executing > it again 5 seconds in the future are undefined. (Also, timing for > executing > such scheduled events is actually driven by the arrival of traffic, so > that would be another potential difference between the live execution vs. > the trace one. But again I don't offhand see why it would lead to > different > results.) > > Vern > -- Sridhar -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100823/efe2d046/attachment.html From robin at icir.org Tue Aug 24 16:53:06 2010 From: robin at icir.org (Robin Sommer) Date: Tue, 24 Aug 2010 16:53:06 -0700 Subject: [Bro] [Announcement] Major NSF funding for Bro development Message-ID: <20100824235306.GA41593@icir.org> Some of you folks may remember that back in January we did a survey soliciting input on operational Bro deployments in the hope of attracting funding for Bro. Today, the Bro team is jazzed to announce that the National Science Foundation has awarded a grant of almost $3M to the International Computer Science Institute (ICSI) and the National Center for Supercomputing Applications (NCSA) for extensive Bro development. The funded project aims specifically at addressing much of the feedback that we have received from Bro users over the years. It will enable us to refine many of the rough edges that the system has accumulated over time[*], improve Bro's performance significantly, and also make it much easier for the community to contribute to the project. For further information, see the joint ICSI/NCSA press release at: http://www.ncsa.illinois.edu/News/10/0824NSFawards.html While we are still in process of planning our next steps, we'd already like to encourage you folks to take an active role in shaping the course of Bro's future development. In response to our earlier survey, many of you have already sent in ideas on what kind of improvements and new functionality you would like to see. If you have further thoughts, feel free to send them either to the list or to Robin personally. Now is also the time to file your favorite Bro quirk with our tracker at http://tracker.icir.org/bro ... Thanks to everybody who helped make this happen! The Bro Team [*] Yes, that includes documentation! -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From epsalon135 at yahoo.com Thu Aug 26 15:11:38 2010 From: epsalon135 at yahoo.com (epsalon135) Date: Thu, 26 Aug 2010 15:11:38 -0700 (PDT) Subject: [Bro] Bro Cluster Issue Message-ID: <346554.53867.qm@web55302.mail.re4.yahoo.com> Greetings group, ?I'm having an issue with Bro and clustering. I keep getting this error: [BroControl] > install removing old policies in /usr/local/bro/share/bro/.site ... done. creating policy directories ... done. installing site policies ... done. generating broctl-layout.bro ... done. generating analysis-policy.bro ... done. generating local-networks.bro ... done. updating nodes ... Permission denied (publickey,keyboard-interactive). warning: connection to 10.10.2.13 broke Permission denied (publickey,keyboard-interactive). warning: connection to 10.10.2.14 broke warning: connection to 10.10.2.13 broke warning: connection to 10.10.2.14 broke warning: connection to 10.10.2.13 broke warning: connection to 10.10.2.14 broke warning: connection to 10.10.2.13 broke warning: connection to 10.10.2.14 broke warning: cannot create directory /usr/local/bro/spool/tmp on worker-1 warning: cannot create directory /usr/local/bro/spool/tmp on worker-2 warning: cannot create directory /usr/local/bro/spool/tmp on worker-1 warning: cannot create directory /usr/local/bro/spool/tmp on worker-2 warning: cannot create directory /usr/local/bro/spool/tmp on worker-1 warning: cannot create directory /usr/local/bro/spool/tmp on worker-2 warning: cannot create directory /usr/local/bro/spool/tmp on worker-1 warning: cannot create directory /usr/local/bro/spool/tmp on worker-2 warning: error rsyncing to 10.10.2.13: ['Permission denied (publickey,keyboard-interactive).\r', 'rsync: connection unexpectedly closed (0 bytes received so far) [sender]', 'rsync error: unexplained error (code 255) at io.c(600) [sender=3.0.6]'] warning: error rsyncing to 10.10.2.14: ['Permission denied (publickey,keyboard-interactive).\r', 'rsync: connection unexpectedly closed (0 bytes received so far) [sender]', 'rsync error: unexplained error (code 255) at io.c(600) [sender=3.0.6]'] done. I thought I had all my permissions correct but I guess not. I know rsync works and ssh (with out a password). Any thoughts? Thanks, --John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100826/f8bd7c82/attachment.html From robin at icir.org Thu Aug 26 22:03:58 2010 From: robin at icir.org (Robin Sommer) Date: Thu, 26 Aug 2010 22:03:58 -0700 Subject: [Bro] Bro Cluster Issue In-Reply-To: <346554.53867.qm@web55302.mail.re4.yahoo.com> References: <346554.53867.qm@web55302.mail.re4.yahoo.com> Message-ID: <20100827050358.GL49514@icir.org> On Thu, Aug 26, 2010 at 15:11 -0700, you wrote: > warning: error rsyncing to 10.10.2.13: ['Permission denied > (publickey,keyboard-interactive).\r', 'rsync: connection > unexpectedly closed (0 bytes received so far) [sender]', 'rsync > error: unexplained error (code 255) at io.c(600) [sender=3.0.6]'] While broctl could fail a bit nicer, this certainly looks like the SSH login isn't working. > rsync works and ssh (with out a password). Could there be different accounts involved in some form, some for which it works and others for which it doesn't? Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From mfw113 at psu.edu Sat Aug 28 08:59:38 2010 From: mfw113 at psu.edu (Mike Waite) Date: Sat, 28 Aug 2010 11:59:38 -0400 Subject: [Bro] Worker nodes logs Message-ID: <4C79326A.7040902@psu.edu> I am not seeing any of the logs from the worker nodes on the manger node in the Log dir. Is there some setting that I am missing? Using broctl I can see the worker nodes are working, and if I ssh to the worker nodes I can see the log in the spool dir, they just never seem to make it back to the manager. This is a new setup from SVN running on RHEL 5.5 64 bit and my first attempt at Bro, so please be gentle. -- Michael Waite Enterprise Security Analyst Enterprise Information Privacy and Security Services (EIPSs) Security Operations and Services (SOS) Information Technology Services (ITS) The Pennsylvania State University (PSU) Direct Telephone: 814-865-2297 ITS-SOS Telephone: 814-863-9533 ITS-SOS E-Mail: security at psu.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 551 bytes Desc: OpenPGP digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100828/ef8c1089/attachment.bin From robin at icir.org Sat Aug 28 14:59:56 2010 From: robin at icir.org (Robin Sommer) Date: Sat, 28 Aug 2010 14:59:56 -0700 Subject: [Bro] Worker nodes logs In-Reply-To: <4C79326A.7040902@psu.edu> References: <4C79326A.7040902@psu.edu> Message-ID: <20100828215956.GB42200@icir.org> On Sat, Aug 28, 2010 at 11:59 -0400, you wrote: > I am not seeing any of the logs from the worker nodes on the manger > node in the Log dir. That's intentional, the workers' logs are discarded upon rotation. What information in there would you like to see archived? Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From hsbedi at memphis.edu Mon Aug 30 02:02:14 2010 From: hsbedi at memphis.edu (Harkeerat Bedi) Date: Mon, 30 Aug 2010 04:02:14 -0500 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: <201008222238.o7MMc7S0005838@pork.ICSI.Berkeley.EDU> References: <201008222238.o7MMc7S0005838@pork.ICSI.Berkeley.EDU> Message-ID: Thank you Vern for your feedback. I am still working on this problem. I tested your suggestion of using "-f tcp", but I could not see any difference. This post is divided in three parts: 1. I have one basic question regarding the way BRO works. My confusion is as follows: Which is the main event handler in BRO that "usually" updates the c$duration, c$orig$size and c$resp$size variables of the connection object? In my case I believe, the event handler for updating the duration and sizes for the connection when a new packet is observed is not being called when BRO is reading from live traffic. However, it is being called with BRO reads from TCPDUMP. 2. Regarding your suggestion on my use and invocation of the connection_established event, I have made some changes to my policy file and attached the same to this mail. Can you kindly provide your feedback on this. I do not schedule the connection_established event anymore, I have defined another event (conn_bitrate_updater), which I schedule from the connection_established event. I have also tried to define the semantics of future execution of this event. 3. Regarding your last comment: "Also, timing for executing such scheduled events is actually driven by the arrival of traffic, so that would be another potential difference between the live execution vs. the trace one" Can you suggest how can I address this scenario? Kindly provide your feedback regarding the above. Thank you, Regards, Harkeerat Bedi On Sun, Aug 22, 2010 at 5:38 PM, Vern Paxson wrote: > > My question is why does BRO appear to behave differently when reading > from a > > tcpdump or an interface. Kindly advice. > > It's not clear to me just why you're seeing the difference. The symptoms > suggest that the live run is using a different packet filter (in > particular, > the default SYN/FIN/RST-only filter), and thus after the connection is > established, there's no input to update things further. However, if so > then you should have that same effect running on the trace. > You could test for this by running with a filter "-f tcp", which will > capture all TCP packets. > > Note that your script misuses the connection_established event. It's not > meant to be generated at the script-level, and the semantics of executing > it again 5 seconds in the future are undefined. (Also, timing for > executing > such scheduled events is actually driven by the arrival of traffic, so > that would be another potential difference between the live execution vs. > the trace one. But again I don't offhand see why it would lead to > different > results.) > > Vern > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100830/54fa9270/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: ex2e.bro Type: application/octet-stream Size: 1196 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100830/54fa9270/attachment.obj From hsbedi at memphis.edu Mon Aug 30 02:20:55 2010 From: hsbedi at memphis.edu (Harkeerat Bedi) Date: Mon, 30 Aug 2010 04:20:55 -0500 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: References: <201008222238.o7MMc7S0005838@pork.ICSI.Berkeley.EDU> Message-ID: Thank you Sridhar for your feedback and time. I am still working on this problem. To recall, in my case BRO seemed to be behaving differently when reading from a live interface or a TCPDUMP. My previous experiment setup was as follows. Setup1: Node1 (Client) <------> Node2 (running BRO) < ------ > Node3 (Server) Node1 (the client) was sending TCP traffic to Node3 (the server) via Node2 (which is running BRO). And the output of BRO on this live traffic was not as expected. However, if I ran BRO on the TCPDUMP of the same traffic the output was as expected. (As shown in my previous email). Now, as I was experimenting, I noticed that if I rearrange the experiment as follows: Setup2: Node1 (Client) <------> Node2 (Server + running BRO) I obtain the results as I wanted even on live traffic. I am able to obtain the periodically updated duration and sizes which then I use to calculate the bitrate. I was wondering, is the difference in behavior observed by BRO related to its location in the network? (In Setup1 it was an intermediate node, where as in Setup2 it is the terminal node.) Kindly provide your feedback if this makes any sense of gives any clues. Regarding your suggestion, I understand what you are implying, however I am not sure how to do it. Is it possible for you to provide me with a snippet of code so that I can follow it? Thank you in advance. Regards, Harkeerat Bedi On Mon, Aug 23, 2010 at 5:02 AM, sridhar basam wrote: > Sorry, i don't know why you are seeing a difference between the live and > offline trace. > > Here is something i have used in the past to look at similar things you are > trying to get at. I expanded the connection structure to include a last > recorded timestamp. Then use the new_packet or tcp_packet event to compare > network_time to the last recorded time (or connection close) to do the > printing vs the timer event. > > Sridhar > > On Sun, Aug 22, 2010 at 6:38 PM, Vern Paxson wrote: > >> > My question is why does BRO appear to behave differently when reading >> from a >> > tcpdump or an interface. Kindly advice. >> >> It's not clear to me just why you're seeing the difference. The symptoms >> suggest that the live run is using a different packet filter (in >> particular, >> the default SYN/FIN/RST-only filter), and thus after the connection is >> established, there's no input to update things further. However, if so >> then you should have that same effect running on the trace. >> >> You could test for this by running with a filter "-f tcp", which will >> capture all TCP packets. >> >> Note that your script misuses the connection_established event. It's not >> meant to be generated at the script-level, and the semantics of executing >> it again 5 seconds in the future are undefined. (Also, timing for >> executing >> such scheduled events is actually driven by the arrival of traffic, so >> that would be another potential difference between the live execution vs. >> the trace one. But again I don't offhand see why it would lead to >> different >> results.) >> >> Vern >> > > > > -- > Sridhar > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100830/83dc96ca/attachment.html From sridhar.basam at gmail.com Mon Aug 30 13:08:42 2010 From: sridhar.basam at gmail.com (sridhar basam) Date: Mon, 30 Aug 2010 16:08:42 -0400 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: References: <201008222238.o7MMc7S0005838@pork.ICSI.Berkeley.EDU> Message-ID: Below is the gist of what i was suggesting. You need to clean it up so that you catch the pre connection established phase and connection close case etc but you get the idea. global conn_print: table[conn_id] of time; global print_delay = 1 sec; event connection_established(c: connection) { conn_print[c$id] = network_time(); } event new_packet (c: connection,p: pkt_hdr) { if ( network_time() >= print_delay + conn_print[c$id] ) { print network_time(), c$id, c$orig$size, c$resp$size; conn_print[c$id] = network_time(); } } Sridhar On Mon, Aug 30, 2010 at 5:20 AM, Harkeerat Bedi wrote: > Thank you Sridhar for your feedback and time. I am still working on this > problem. To recall, in my case BRO seemed to be behaving differently when > reading from a live interface or a TCPDUMP. > > My previous experiment setup was as follows. > > Setup1: > Node1 (Client) <------> Node2 (running BRO) < ------ > Node3 (Server) > > Node1 (the client) was sending TCP traffic to Node3 (the server) via Node2 > (which is running BRO). And the output of BRO on this live traffic was not > as expected. However, if I ran BRO on the TCPDUMP of the same traffic the > output was as expected. (As shown in my previous email). > > Now, as I was experimenting, I noticed that if > I rearrange the experiment as follows: > > Setup2: > Node1 (Client) <------> Node2 (Server + running BRO) > > I obtain the results as I wanted even on live traffic. I am able to obtain > the periodically updated duration and sizes which then I use to calculate > the bitrate. > > I was wondering, is the difference in behavior observed by BRO related to > its location in the network? > (In Setup1 it was an intermediate node, where as in Setup2 it is the > terminal node.) > Kindly provide your feedback if this makes any sense of gives any clues. > > Regarding your suggestion, I understand what you are implying, however I am > not sure how to do it. Is it possible for you to provide me with a snippet > of code so that I can follow it? > > Thank you in advance. > > Regards, > Harkeerat Bedi > > > > On Mon, Aug 23, 2010 at 5:02 AM, sridhar basam wrote: > >> Sorry, i don't know why you are seeing a difference between the live and >> offline trace. >> >> Here is something i have used in the past to look at similar things you >> are trying to get at. I expanded the connection structure to include a last >> recorded timestamp. Then use the new_packet or tcp_packet event to compare >> network_time to the last recorded time (or connection close) to do the >> printing vs the timer event. >> >> Sridhar >> >> On Sun, Aug 22, 2010 at 6:38 PM, Vern Paxson wrote: >> >>> > My question is why does BRO appear to behave differently when reading >>> from a >>> > tcpdump or an interface. Kindly advice. >>> >>> It's not clear to me just why you're seeing the difference. The symptoms >>> suggest that the live run is using a different packet filter (in >>> particular, >>> the default SYN/FIN/RST-only filter), and thus after the connection is >>> established, there's no input to update things further. However, if so >>> then you should have that same effect running on the trace. >>> >>> You could test for this by running with a filter "-f tcp", which will >>> capture all TCP packets. >>> >>> Note that your script misuses the connection_established event. It's not >>> meant to be generated at the script-level, and the semantics of executing >>> it again 5 seconds in the future are undefined. (Also, timing for >>> executing >>> such scheduled events is actually driven by the arrival of traffic, so >>> that would be another potential difference between the live execution vs. >>> the trace one. But again I don't offhand see why it would lead to >>> different >>> results.) >>> >>> Vern >>> >> >> >> >> -- >> Sridhar >> > > -- Sridhar -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100830/9b694fd9/attachment.html From vern at icir.org Mon Aug 30 15:42:16 2010 From: vern at icir.org (Vern Paxson) Date: Mon, 30 Aug 2010 15:42:16 -0700 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: (Mon, 30 Aug 2010 04:20:55 CDT). Message-ID: <201008302242.o7UMgGTM017834@pork.ICSI.Berkeley.EDU> > My previous experiment setup was as follows. > > Setup1: > Node1 (Client) <------> Node2 (running BRO) < ------ > Node3 (Server) If on Node2 instead of running Bro you capture packets with tcpdump, does Bro run correctly on the resulting trace? (Perhaps this is how you're already capturing the traffic that it works correctly on, but I thought of asking because on some systems packet capture for local traffic is incomplete, and in particular lacks locally sent packets.) What OS's are the Nodes running? Vern From vern at icir.org Mon Aug 30 15:52:01 2010 From: vern at icir.org (Vern Paxson) Date: Mon, 30 Aug 2010 15:52:01 -0700 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: (Mon, 30 Aug 2010 04:02:14 CDT). Message-ID: <201008302252.o7UMq14T017948@pork.ICSI.Berkeley.EDU> > Which is the main event handler in BRO that "usually" updates the > c$duration, c$orig$size and c$resp$size variables of the connection object? It does so on any connection_* event that it generates. However, in between those events, the variables are *not* updated. (That is, their updates are driven by the generation of the events.) Looking at the code, it appears that the new_packet event will also spur an update, so capturing that should suffice. In addition, there's a connection_status_update(c: connection) event that you can turn on by defining a handler for it, and by setting connection_status_update_interval to a positive time interval (e.g., "1 sec"). That will then be generated periodically at the given interval. > 2. Regarding your suggestion on my use and invocation of the > connection_established event, I have made some changes to my policy file and > attached the same to this mail. Can you kindly provide your feedback on > this. The way you structured it now looks good, modulo the consideration above of when the variables actually get updated. That said, just using connection_status_update directly would be simpler. Vern From hsbedi at memphis.edu Mon Aug 30 22:48:23 2010 From: hsbedi at memphis.edu (Harkeerat Bedi) Date: Tue, 31 Aug 2010 00:48:23 -0500 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: <201008302242.o7UMgGTM017834@pork.ICSI.Berkeley.EDU> References: <201008302242.o7UMgGTM017834@pork.ICSI.Berkeley.EDU> Message-ID: On Mon, Aug 30, 2010 at 5:42 PM, Vern Paxson wrote: > > My previous experiment setup was as follows. > > > > Setup1: > > Node1 (Client) <------> Node2 (running BRO) < ------ > Node3 (Server) > > If on Node2 instead of running Bro you capture packets with tcpdump, does > Bro run correctly on the resulting trace? 1. Yes. The command that I use is: $ sudo /usr/local/.../bin/bro -r ../testCapture6.dump ex2e.bro Task: I am ftp'ing one file from Node 1 to Node3. Snippet of output: 10.1.2.3 10.1.1.3 20 57713 bitrate: 117337.34, duration: 3.011079, size: 0 353312 10.1.1.3 10.1.2.3 43580 21 bitrate: 64.74, duration: 6.889347, size: 105 446 10.1.2.3 10.1.1.3 20 57713 bitrate: 117722.29, duration: 4.022144, size: 0 473496 10.1.1.3 10.1.2.3 43580 21 bitrate: 64.74, duration: 6.889347, size: 105 446 10.1.2.3 10.1.1.3 20 57713 bitrate: 117969.47, duration: 5.020214, size: 0 592232 10.1.1.3 10.1.2.3 43580 21 bitrate: 64.74, duration: 6.889347, size: 105 446 10.1.2.3 10.1.1.3 20 57713 bitrate: 118139.81, duration: 6.030279, size: 0 712416 Notice the increase in size and duration every one second. This is as expected. 2. When I run the following command (that is reading from an interface "em2"): $ sudo /usr/local/.../bin/bro -i em2 ex2e.bro Task: Same as before (I am ftp'ing one file from Node 1 to Node3). Snippet of output observed: 10.1.2.3 10.1.1.3 20 47271 bitrate: 0.00, duration: 0.003685, size: 0 0 10.1.1.3 10.1.2.3 36270 21 bitrate: 0.00, duration: 0.001420, size: 0 0 10.1.2.3 10.1.1.3 20 47271 bitrate: 0.00, duration: 0.003685, size: 0 0 10.1.1.3 10.1.2.3 36270 21 bitrate: 0.00, duration: 0.001420, size: 0 0 10.1.2.3 10.1.1.3 20 47271 bitrate: 0.00, duration: 0.003685, size: 0 0 10.1.1.3 10.1.2.3 36270 21 bitrate: 0.00, duration: 0.001420, size: 0 0 10.1.2.3 10.1.1.3 20 47271 bitrate: 0.00, duration: 0.003685, size: 0 0 10.1.1.3 10.1.2.3 36270 21 bitrate: 0.00, duration: 0.001420, size: 0 0 1283232336.474291 8.932614 10.1.2.3 10.1.1.3 20 47271 0 1052696 - TCP_CLOSED Notice that size and duration do not increase every second. But when I stop the file transfer, I see updated values. 3. One more thing I noticed is that: When I run my policy file along with TCP and FTP analyzers on the live interface using below command. Command: $ sudo /usr/local/.../bin/bro -i em2 ex2e.bro tcp ftp Task: Same as before (I am ftp'ing one file from Node 1 to Node3). I see the following output: Snippet: 1283232724.432981 0.001834 10.1.1.3 10.1.2.3 53747 21 0 0 10.1.1.3 10.1.2.3 53747 21 bitrate: 0.00, duration: 0.001834, size: 0 0 10.1.1.3 10.1.2.3 53747 21 bitrate: 10608.46, duration: 0.007824, size: 0 83 10.1.1.3 10.1.2.3 53747 21 bitrate: 10608.46, duration: 0.007824, size: 0 83 10.1.1.3 10.1.2.3 53747 21 bitrate: 63.43, duration: 2.222891, size: 16 141 10.1.1.3 10.1.2.3 53747 21 bitrate: 72.37, duration: 3.150419, size: 29 228 10.1.1.3 10.1.2.3 53747 21 bitrate: 72.37, duration: 3.150419, size: 29 228 10.1.1.3 10.1.2.3 53747 21 bitrate: 41.28, duration: 6.007643, size: 37 248 1283232730.452595 0.003124 10.1.2.3 10.1.1.3 20 40035 0 0 10.1.2.3 10.1.1.3 20 40035 bitrate: 0.00, duration: 0.003124, size: 0 0 10.1.1.3 10.1.2.3 53747 21 bitrate: 56.27, duration: 6.024612, size: 76 339 10.1.2.3 10.1.1.3 20 40035 bitrate: 0.00, duration: 0.003124, size: 0 0 10.1.1.3 10.1.2.3 53747 21 bitrate: 56.27, duration: 6.024612, size: 76 339 10.1.2.3 10.1.1.3 20 40035 bitrate: 0.00, duration: 0.003124, size: 0 0 10.1.1.3 10.1.2.3 53747 21 bitrate: 56.27, duration: 6.024612, size: 76 339 10.1.2.3 10.1.1.3 20 40035 bitrate: 0.00, duration: 0.003124, size: 0 0 10.1.1.3 10.1.2.3 53747 21 bitrate: 56.27, duration: 6.024612, size: 76 339 10.1.2.3 10.1.1.3 20 40035 bitrate: 0.00, duration: 0.003124, size: 0 0 10.1.1.3 10.1.2.3 53747 21 bitrate: 56.27, duration: 6.024612, size: 76 339 10.1.2.3 10.1.1.3 20 40035 bitrate: 0.00, duration: 0.003124, size: 0 0 10.1.1.3 10.1.2.3 53747 21 bitrate: 56.27, duration: 6.024612, size: 76 339 10.1.2.3 10.1.1.3 20 40035 bitrate: 0.00, duration: 0.003124, size: 0 0 10.1.1.3 10.1.2.3 53747 21 bitrate: 31.48, duration: 12.549360, size: 76 395 1283232730.452595 6.530739 10.1.2.3 10.1.1.3 20 40035 0 770336 - TCP_CLOSED 10.1.1.3 10.1.2.3 53747 21 bitrate: 31.48, duration: 12.549360, size: 76 395 10.1.1.3 10.1.2.3 53747 21 bitrate: 31.48, duration: 12.549360, size: 76 395 1283232724.432981 15.582105 10.1.1.3 10.1.2.3 53747 21 82 409 - TCP_CLOSED Notice the duration and size variables of the control connection (port 21) update every time I enter a new ftp command. (e.g. ls - to list the files in that remote directory). This did not happen earlier - when I did not use TCP and FTP analyzers. And when I stop the transfer and close the connection, I see the total duration and size. I think that I am missing some event handlers but I cannot figure out which ones. I even tried running BRO with "brolite" (which loads many of the standard analyzers) along with my policy file, but in vain. (Perhaps this is how you're > already capturing the traffic that it works correctly on, but I thought > of asking because on some systems packet capture for local traffic is > incomplete, and in particular lacks locally sent packets.) > > What OS's are the Nodes running? > Node 2 and 3 are FreeBSD 7.2 Node 1 is Ubuntu 10.04 > > Vern > Thank you. Harkeerat Bedi -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100831/b44bf452/attachment.html From hsbedi at memphis.edu Mon Aug 30 22:49:44 2010 From: hsbedi at memphis.edu (Harkeerat Bedi) Date: Tue, 31 Aug 2010 00:49:44 -0500 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: References: <201008222238.o7MMc7S0005838@pork.ICSI.Berkeley.EDU> Message-ID: Thank you for this Sridhar. Allow me to work on this. Thanks, Harkeerat Bedi On Mon, Aug 30, 2010 at 3:08 PM, sridhar basam wrote: > Below is the gist of what i was suggesting. You need to clean it up so that > you catch the pre connection established phase and connection close case etc > but you get the idea. > > global conn_print: table[conn_id] of time; > global print_delay = 1 sec; > > > event connection_established(c: connection) > { > conn_print[c$id] = network_time(); > } > > event new_packet (c: connection,p: pkt_hdr) > { > if ( network_time() >= print_delay + conn_print[c$id] ) { > print network_time(), c$id, c$orig$size, c$resp$size; > conn_print[c$id] = network_time(); > } > } > > Sridhar > > > > > On Mon, Aug 30, 2010 at 5:20 AM, Harkeerat Bedi wrote: > >> Thank you Sridhar for your feedback and time. I am still working on this >> problem. To recall, in my case BRO seemed to be behaving differently when >> reading from a live interface or a TCPDUMP. >> >> My previous experiment setup was as follows. >> >> Setup1: >> Node1 (Client) <------> Node2 (running BRO) < ------ > Node3 (Server) >> >> Node1 (the client) was sending TCP traffic to Node3 (the server) via Node2 >> (which is running BRO). And the output of BRO on this live traffic was not >> as expected. However, if I ran BRO on the TCPDUMP of the same traffic the >> output was as expected. (As shown in my previous email). >> >> Now, as I was experimenting, I noticed that if >> I rearrange the experiment as follows: >> >> Setup2: >> Node1 (Client) <------> Node2 (Server + running BRO) >> >> I obtain the results as I wanted even on live traffic. I am able to obtain >> the periodically updated duration and sizes which then I use to calculate >> the bitrate. >> >> I was wondering, is the difference in behavior observed by BRO related to >> its location in the network? >> (In Setup1 it was an intermediate node, where as in Setup2 it is the >> terminal node.) >> Kindly provide your feedback if this makes any sense of gives any clues. >> >> Regarding your suggestion, I understand what you are implying, however I >> am not sure how to do it. Is it possible for you to provide me with a >> snippet of code so that I can follow it? >> >> Thank you in advance. >> >> Regards, >> Harkeerat Bedi >> >> >> >> On Mon, Aug 23, 2010 at 5:02 AM, sridhar basam wrote: >> >>> Sorry, i don't know why you are seeing a difference between the live and >>> offline trace. >>> >>> Here is something i have used in the past to look at similar things you >>> are trying to get at. I expanded the connection structure to include a last >>> recorded timestamp. Then use the new_packet or tcp_packet event to compare >>> network_time to the last recorded time (or connection close) to do the >>> printing vs the timer event. >>> >>> Sridhar >>> >>> On Sun, Aug 22, 2010 at 6:38 PM, Vern Paxson wrote: >>> >>>> > My question is why does BRO appear to behave differently when reading >>>> from a >>>> > tcpdump or an interface. Kindly advice. >>>> >>>> It's not clear to me just why you're seeing the difference. The >>>> symptoms >>>> suggest that the live run is using a different packet filter (in >>>> particular, >>>> the default SYN/FIN/RST-only filter), and thus after the connection is >>>> established, there's no input to update things further. However, if so >>>> then you should have that same effect running on the trace. >>>> >>>> You could test for this by running with a filter "-f tcp", which will >>>> capture all TCP packets. >>>> >>>> Note that your script misuses the connection_established event. It's >>>> not >>>> meant to be generated at the script-level, and the semantics of >>>> executing >>>> it again 5 seconds in the future are undefined. (Also, timing for >>>> executing >>>> such scheduled events is actually driven by the arrival of traffic, so >>>> that would be another potential difference between the live execution >>>> vs. >>>> the trace one. But again I don't offhand see why it would lead to >>>> different >>>> results.) >>>> >>>> Vern >>>> >>> >>> >>> >>> -- >>> Sridhar >>> >> >> > > > -- > Sridhar > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100831/223e4ea4/attachment.html From JAzoff at uamail.albany.edu Tue Aug 31 06:28:40 2010 From: JAzoff at uamail.albany.edu (Justin Azoff) Date: Tue, 31 Aug 2010 09:28:40 -0400 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: References: <201008302242.o7UMgGTM017834@pork.ICSI.Berkeley.EDU> Message-ID: <20100831132840.GG10259@datacomm.albany.edu> On Tue, Aug 31, 2010 at 01:48:23AM -0400, Harkeerat Bedi wrote: > $ sudo /usr/local/.../bin/bro -i em2 ex2e.bro have you tried sudo /usr/local/.../bin/bro -i em2 ex2e.bro -f ip yet? I believe it was suggested a couple of messages back. -- -- Justin Azoff -- Network Security & Performance Analyst