From mfw113 at psu.edu Wed Sep 1 18:25:08 2010 From: mfw113 at psu.edu (Mike Waite) Date: Wed, 01 Sep 2010 21:25:08 -0400 Subject: [Bro] Worker nodes logs In-Reply-To: <20100828215956.GB42200@icir.org> References: <4C79326A.7040902@psu.edu> <20100828215956.GB42200@icir.org> Message-ID: <4C7EFCF4.1050507@psu.edu> This was do a dumb mistake on my proxy's firewall. I have allowed ports 47760 and 47761 thru and I am now seeing the results I was expecting. Now to get the other 12 worker nodes set up... Is there any documentation related to the Cflow headend and capstats working together as eluded to some of the config files? Thanks again Michael Waite Enterprise Security Analyst Enterprise Information Privacy and Security Services (EIPSs) Security Operations and Services (SOS) Information Technology Services (ITS) The Pennsylvania State University (PSU) Direct Telephone: 814-865-2297 ITS-SOS Telephone: 814-863-9533 ITS-SOS E-Mail: security at psu.edu On 8/28/10 5:59 PM, Robin Sommer wrote: > > On Sat, Aug 28, 2010 at 11:59 -0400, you wrote: > >> I am not seeing any of the logs from the worker nodes on the manger >> node in the Log dir. > > That's intentional, the workers' logs are discarded upon rotation. > What information in there would you like to see archived? > > Robin > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 551 bytes Desc: OpenPGP digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100901/047ee9d9/attachment.bin From robin at icir.org Fri Sep 3 08:30:04 2010 From: robin at icir.org (Robin Sommer) Date: Fri, 3 Sep 2010 08:30:04 -0700 Subject: [Bro] Worker nodes logs In-Reply-To: <4C7EFCF4.1050507@psu.edu> References: <4C79326A.7040902@psu.edu> <20100828215956.GB42200@icir.org> <4C7EFCF4.1050507@psu.edu> Message-ID: <20100903153004.GB17369@icir.org> On Wed, Sep 01, 2010 at 21:25 -0400, you wrote: > Is there any documentation related to the Cflow headend and capstats working together as eluded to some of the config files? No, sorry. broctl can talk to the cFlow to get traffic stats and will print and record them in parallel with what capstats reports. You'll need cPacket's command line tool for that, as broctl just calls it for doing the communication with the device. I can send you a config example if you have the tool installed. Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From hsbedi at memphis.edu Fri Sep 3 14:21:14 2010 From: hsbedi at memphis.edu (Harkeerat Bedi) Date: Fri, 3 Sep 2010 16:21:14 -0500 Subject: [Bro] Using BRO for measuring TCP flow bandwidth In-Reply-To: References: <201008222238.o7MMc7S0005838@pork.ICSI.Berkeley.EDU> Message-ID: Thank you Sridhar, Vern and Justin for your time and support. It is highly appreciated. I was finally able to get the output I required - that is, scheduled updates on the size and duration variables of the connection object on live traffic. I had tried -f filter option for TCP (-f tcp) earlier when it was suggested, but it did not help then. The trick was to use the -f filter option for either IP or TCP along with the definition of an event handler like new_packet or tcp_packet in the policy file. Using just the event handler without the filter, or just the filter without the event handler did not work. As Vern mentioned, the updates on the variables are driven by the generation of the events, therefore even by defining an empty new_packet or tcp_packet event handler, the connection object was getting updated and I was able to get the desired output. Thank you all once again, as without your support it would not have been possible. Thanks, Harkeerat Bedi On Mon, Aug 30, 2010 at 3:08 PM, sridhar basam wrote: > Below is the gist of what i was suggesting. You need to clean it up so that > you catch the pre connection established phase and connection close case etc > but you get the idea. > > global conn_print: table[conn_id] of time; > global print_delay = 1 sec; > > > event connection_established(c: connection) > { > conn_print[c$id] = network_time(); > } > > event new_packet (c: connection,p: pkt_hdr) > { > if ( network_time() >= print_delay + conn_print[c$id] ) { > print network_time(), c$id, c$orig$size, c$resp$size; > conn_print[c$id] = network_time(); > } > } > > Sridhar > > > > > On Mon, Aug 30, 2010 at 5:20 AM, Harkeerat Bedi wrote: > >> Thank you Sridhar for your feedback and time. I am still working on this >> problem. To recall, in my case BRO seemed to be behaving differently when >> reading from a live interface or a TCPDUMP. >> >> My previous experiment setup was as follows. >> >> Setup1: >> Node1 (Client) <------> Node2 (running BRO) < ------ > Node3 (Server) >> >> Node1 (the client) was sending TCP traffic to Node3 (the server) via Node2 >> (which is running BRO). And the output of BRO on this live traffic was not >> as expected. However, if I ran BRO on the TCPDUMP of the same traffic the >> output was as expected. (As shown in my previous email). >> >> Now, as I was experimenting, I noticed that if >> I rearrange the experiment as follows: >> >> Setup2: >> Node1 (Client) <------> Node2 (Server + running BRO) >> >> I obtain the results as I wanted even on live traffic. I am able to obtain >> the periodically updated duration and sizes which then I use to calculate >> the bitrate. >> >> I was wondering, is the difference in behavior observed by BRO related to >> its location in the network? >> (In Setup1 it was an intermediate node, where as in Setup2 it is the >> terminal node.) >> Kindly provide your feedback if this makes any sense of gives any clues. >> >> Regarding your suggestion, I understand what you are implying, however I >> am not sure how to do it. Is it possible for you to provide me with a >> snippet of code so that I can follow it? >> >> Thank you in advance. >> >> Regards, >> Harkeerat Bedi >> >> >> >> On Mon, Aug 23, 2010 at 5:02 AM, sridhar basam wrote: >> >>> Sorry, i don't know why you are seeing a difference between the live and >>> offline trace. >>> >>> Here is something i have used in the past to look at similar things you >>> are trying to get at. I expanded the connection structure to include a last >>> recorded timestamp. Then use the new_packet or tcp_packet event to compare >>> network_time to the last recorded time (or connection close) to do the >>> printing vs the timer event. >>> >>> Sridhar >>> >>> On Sun, Aug 22, 2010 at 6:38 PM, Vern Paxson wrote: >>> >>>> > My question is why does BRO appear to behave differently when reading >>>> from a >>>> > tcpdump or an interface. Kindly advice. >>>> >>>> It's not clear to me just why you're seeing the difference. The >>>> symptoms >>>> suggest that the live run is using a different packet filter (in >>>> particular, >>>> the default SYN/FIN/RST-only filter), and thus after the connection is >>>> established, there's no input to update things further. However, if so >>>> then you should have that same effect running on the trace. >>>> >>>> You could test for this by running with a filter "-f tcp", which will >>>> capture all TCP packets. >>>> >>>> Note that your script misuses the connection_established event. It's >>>> not >>>> meant to be generated at the script-level, and the semantics of >>>> executing >>>> it again 5 seconds in the future are undefined. (Also, timing for >>>> executing >>>> such scheduled events is actually driven by the arrival of traffic, so >>>> that would be another potential difference between the live execution >>>> vs. >>>> the trace one. But again I don't offhand see why it would lead to >>>> different >>>> results.) >>>> >>>> Vern >>>> >>> >>> >>> >>> -- >>> Sridhar >>> >> >> > > > -- > Sridhar > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100903/96f6c2b4/attachment.html From mfw113 at psu.edu Tue Sep 7 14:55:04 2010 From: mfw113 at psu.edu (Mike Waite) Date: Tue, 07 Sep 2010 17:55:04 -0400 Subject: [Bro] cFlow 10G headend Message-ID: <4C86B4B8.9030203@psu.edu> I am looking for someone that has successfully deployed the cFlow 10G as a headend in their cluster. The box looks pretty simple to setup, not a whole bunch of settings in there to really mess with, but I am not seeing the results that I was expecting. After setting up the MAC addresses on the worker nodes NICs in the appliance I am still seeing all the traffic on all interface on all the worker nodes, not just the traffic destined to mac address of the workers NIC that entered in the maccfg page. Is there some other simple setting that I am missing? Thanks -- Michael Waite Enterprise Security Analyst Enterprise Information Privacy and Security Services (EIPSs) Security Operations and Services (SOS) Information Technology Services (ITS) The Pennsylvania State University (PSU) Direct Telephone: 814-865-2297 ITS-SOS Telephone: 814-863-9533 ITS-SOS E-Mail: security at psu.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 553 bytes Desc: OpenPGP digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100907/3277aead/attachment.bin From mfw113 at psu.edu Tue Sep 7 16:46:35 2010 From: mfw113 at psu.edu (Michael Waite) Date: Tue, 7 Sep 2010 19:46:35 -0400 Subject: [Bro] cFlow 10G headend In-Reply-To: <4C86B4B8.9030203@psu.edu> References: <4C86B4B8.9030203@psu.edu> Message-ID: <6A2DFC25-73E9-4286-AFEB-2EE7A12D1913@psu.edu> Nevermind. It was a simple network issue. The cFlow is now working as I expected it too Thanks Mike Waite Please excuse any typo's as this was sent from my iPhone On Sep 7, 2010, at 17:55, Mike Waite wrote: > I am looking for someone that has successfully deployed the cFlow 10G as a headend in their cluster. > > The box looks pretty simple to setup, not a whole bunch of settings in there to really mess with, but I am not seeing the results that I was expecting. After setting up the MAC addresses on the worker nodes NICs in the appliance I am still seeing all the traffic on all interface on all the worker nodes, not just the traffic destined to mac address of the workers NIC that entered in the maccfg page. Is there some other simple setting that I am missing? > > Thanks > -- > Michael Waite > Enterprise Security Analyst > Enterprise Information Privacy and Security Services (EIPSs) > Security Operations and Services (SOS) > Information Technology Services (ITS) > The Pennsylvania State University (PSU) > > Direct Telephone: 814-865-2297 > ITS-SOS Telephone: 814-863-9533 > ITS-SOS E-Mail: security at psu.edu > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From robin at icir.org Wed Sep 8 09:49:53 2010 From: robin at icir.org (Robin Sommer) Date: Wed, 8 Sep 2010 09:49:53 -0700 Subject: [Bro] [Announcement] Major NSF funding for Bro development In-Reply-To: <20100824235306.GA41593@icir.org> References: <20100824235306.GA41593@icir.org> Message-ID: <20100908164953.GL22585@icir.org> > The funded project aims specifically at addressing much of the > feedback that we have received from Bro users over the years. Following up on my earlier mail, I'm also happy to announce that Seth Hall has joined us over here at ICSI to work with us on moving Bro forward. Many of you folks already know Seth very well, as he has been contributing to Bro and the Bro community for a long time. It's great to have him on the core team now. Welcome, Seth! Robin -- Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From adayadil.thomas at gmail.com Wed Sep 8 10:12:42 2010 From: adayadil.thomas at gmail.com (Adayadil Thomas) Date: Wed, 8 Sep 2010 13:12:42 -0400 Subject: [Bro] Question: Handling a SYN segment when the TCP session is in the ESTABLISHED state Message-ID: Hello, The question may be obvious from the subject line itself. How does Bro handle the case when a SYN is received for a session that is in the established state. (we are referring to the a SYN that has the same src/dst ip/port) - IDS analyzes the TWH between client and server - IDS sees/analyzes the data transfer .. - Now IDS sees a SYN from client to server (same tuple) One case how this may happen is if client machine crashes (Page 33/RFC 793) and restarts and connects using same tuple. In this case the server responds with an ACK, client tears down the session and a new one starts. Other than this case are there any case when the client/server needs to send SYN/SYN-ACK after a session is established? For e.g. to renegotiate window scaling ? Thanks for the reply. From seth at icir.org Wed Sep 8 10:45:09 2010 From: seth at icir.org (Seth Hall) Date: Wed, 8 Sep 2010 13:45:09 -0400 Subject: [Bro] [Announcement] Major NSF funding for Bro development In-Reply-To: <20100908164953.GL22585@icir.org> References: <20100824235306.GA41593@icir.org> <20100908164953.GL22585@icir.org> Message-ID: On Sep 8, 2010, at 12:49 PM, Robin Sommer wrote: >> The funded project aims specifically at addressing much of the >> feedback that we have received from Bro users over the years. > > Following up on my earlier mail, I'm also happy to announce that > Seth Hall has joined us over here at ICSI to work with us on moving > Bro forward. I wanted to point out too, that I'll try and frequently be in the #bro-ids channel on the Freenode IRC network in case anyone wants to talk or has questions. Thanks! I'm looking forward to making Bro much more prevalent in the network monitoring world. :) Seth From pspda5id at gmail.com Thu Sep 9 13:41:28 2010 From: pspda5id at gmail.com (da5id psp) Date: Thu, 9 Sep 2010 16:41:28 -0400 Subject: [Bro] Fwd: binpac: variable length In-Reply-To: References: Message-ID: Hi, I'm using binpac as a standalone parser for RTMP ( http://www.adobe.com/devnet/rtmp/) RTMP is a binary protocol that does not delimit frames/chunks like other fixed frame protocols. eg. An RTMP chunk can be of type 0-3, the only way to find the length is to first read the whole packet and then derive the length depending on the chunk type. My question is how can I do this in binpac since the enclosing record expects a length but it can only be provided after parsing the child records, eg. type Chunk0 = record { msgLen:uint32; }; type Chunk0 = record { chunkId:uint32; } &let { msgLen = # calculated from chunkId }; type Chunk = record { basicHeader:uint8; chunk: case basicHeader of { 0 -> chunk0:Chunk0; 1 -> chunk0:Chunk1; 2 -> chunk0:Chunk2; 3 -> chunk0:Chunk3; } data: bytestring &restofdata; } &length = ???; Any help will be appreciated. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20100909/eaf00217/attachment.html From robin at icir.org Mon Sep 13 15:17:56 2010 From: robin at icir.org (Robin Sommer) Date: Mon, 13 Sep 2010 15:17:56 -0700 Subject: [Bro] Postdoc opening Message-ID: <20100913221756.GB84261@icir.org> ICSI's networking group has an opening for a Postdoctoral Fellow with compiler expertise, to work on high-performace network monitoring including potentially Bro-related projects: http://www.icsi.berkeley.edu/about/netjob.html Please feel free to forward further. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From estrada.veronica at gmail.com Tue Sep 14 05:19:23 2010 From: estrada.veronica at gmail.com (Veronica Estrada) Date: Tue, 14 Sep 2010 14:19:23 +0200 Subject: [Bro] differences between p and sp/dp port numbers inside the alert log Message-ID: Hello. I need to understand why the field port (p) is used when 'PortScan' alert is logged instead of using the field source port (sp). For example, in the connection summaries, I've got 51 udp connections with flag S0, from one host to another host using 51 different destination ports and the same originator port=43210. In the alerts log, the same host appears to have scanned 50 ports but instead of identifying the same originator port number, p=29638/udp is recorded. All my 'PortScan' alerts records have no coincidence between the originator port written in the connection summaries and the port looged in the alerts. Thanks in advance, Veronica Estrada From vern at icir.org Tue Sep 14 07:33:28 2010 From: vern at icir.org (Vern Paxson) Date: Tue, 14 Sep 2010 07:33:28 -0700 Subject: [Bro] differences between p and sp/dp port numbers inside the alert log In-Reply-To: (Tue, 14 Sep 2010 14:19:23 +0200). Message-ID: <20100914143328.2D01236A416@taffy.ICSI.Berkeley.EDU> > I need to understand why the field port (p) is used when 'PortScan' > alert is logged instead of using the field source port (sp). The main reconnaissance information gathered by a scan is whether the destination has a listener on the given port. So the source port isn't relevant to the semantics of the scan. (Bro will however try to determine when it's observing TCP backscatter, in which case the apparent source becomes relevant. That's not the case here.) > In the alerts log, the same host appears to have scanned 50 ports but > instead of identifying the same originator port number, p=29638/udp is > recorded. In the notice, what's included is the port of the most recent activity (the activity that triggered generation of the notice). Often for routine scanning this readily identifies the attacker's intent. In your case, however, it doesn't. (Indeed, I imagine what you're seeing isn't a scan at all.) Vern From seth at icir.org Wed Sep 15 14:07:06 2010 From: seth at icir.org (Seth Hall) Date: Wed, 15 Sep 2010 17:07:06 -0400 Subject: [Bro] Patch for Barnyard2 to output alerts to Bro Message-ID: <969E2BCB-EB98-42B1-BBDA-1BB16F9E6300@icir.org> The patch which has been submitted to the Barnyard2 developers can be found here: http://gist.github.com/581458 The accompanying Bro script can be found here (this is still very early but does logging very familiar to Snort users): http://github.com/sethhall/bro_scripts/blob/master/testing/barnyard2-alerts.bro If there are any thoughts or questions, let me know! Testing would be greatly appreciated! Thanks, .Seth (don't forget about the #bro-ids channel on Freenode if you have questions too) From james.swaro at gmail.com Wed Sep 22 10:13:53 2010 From: james.swaro at gmail.com (James Swaro) Date: Wed, 22 Sep 2010 13:13:53 -0400 Subject: [Bro] Adding Events to Bro Message-ID: <4C9A3951.8040300@gmail.com> To the Bro-IDS team, My name is James Swaro and I am a graduate student at Ohio University. I am performing research on the retransmission timeout mechanism of TCP and I am using Bro to do this. Bro provides a very good base for my research and I would like to modify the system as needed to create the events and policy files necessary. The documentation that is available on your wiki is extensive and has been very helpful to understanding the general structure of the system. Mark Allman and Katrina were generous to share the RTT branch that they were working on. I need to add events to the systems to generate specific information when congestion control states have possibly been triggered. I've attempted to create an event in the source code by editing event.bif and TCP.cc, but it does not seem to recognize the event and crashes. Either that, or I've misunderstood the way that the data from the event is created. Am I incorrect with the process of adding a new event? 1. Add the event in event.bif. (Ex. event test_something...) 2. Add the event in the intended location to be called by Bro as it parses the file. (Ex. Add Event(test_something, vl); to some file). 3. recompile and test. I am still learning the framework and I appreciate any help. Thank you for your time. Sincerely, James Swaro From vern at icir.org Wed Sep 22 10:23:13 2010 From: vern at icir.org (Vern Paxson) Date: Wed, 22 Sep 2010 10:23:13 -0700 Subject: [Bro] Adding Events to Bro In-Reply-To: <4C9A3951.8040300@gmail.com> (Wed, 22 Sep 2010 13:13:53 EDT). Message-ID: <20100922172313.070C236A416@taffy.ICSI.Berkeley.EDU> > Am I incorrect with the process of adding a new event? > 1. Add the event in event.bif. (Ex. event test_something...) > 2. Add the event in the intended location to be called by Bro as it > parses the file. (Ex. Add Event(test_something, vl); to some file). > 3. recompile and test. In general, yes, that's the procedure. Note that test_something will be nil if you run using a script that doesn't include a handler for it. (But Connection::Event checks for this, so that shouldn't cause a crash.) A common problem is observing that the event doesn't get generated (as opposed to a crash). That often comes about because you're running with the default pcap filter, or in any case with a filter that doesn't include the traffic you're analyzing. So you might need to run with e.g. "-f tcp" to capture all TCP packets. Vern From jmellander at lbl.gov Thu Sep 23 15:28:47 2010 From: jmellander at lbl.gov (Jim Mellander) Date: Thu, 23 Sep 2010 15:28:47 -0700 Subject: [Bro] Surprising behavior when reading packets from file vs interface. Message-ID: In the course of testing a broccoli-enabled bro policy, I discovered a surprising behavior, and a workaround. My broccoli app runs in an event loop, waiting for bro events, and sending back events of its own to bro. When running bro listening to an interface, it binds to the broccoli port, and the broccoli app communicates with bro as expected. Running 'bro -r' with a tracefile, however, causes bro to not bind to the broccoli port, thus not communicating with the app - verified w/netstat. This snippet from RemoteSerialier.cc appears to be the issue: if ( reading_traces && ! pseudo_realtime ) { using_communication = 0; return; } Accordingly, I used the --pseudo-realtime flag to bro, which caused it to bind to the port as expected, when reading tracefiles. However, I'm not sure this behavior is optimal - there are a number of bro applications now that may not need to listen to an interface (or to a file, for that matter), but are strictly broccoli event driven, for instance those which are strictly driven by syslog events. Perhaps there could be a flag to force bro to bind to the broccoli ports. Are there other options? I would appreciate any opinions/comments. From robin at icir.org Fri Sep 24 11:09:20 2010 From: robin at icir.org (Robin Sommer) Date: Fri, 24 Sep 2010 11:09:20 -0700 Subject: [Bro] Surprising behavior when reading packets from file vs interface. In-Reply-To: References: Message-ID: <20100924180920.GP90579@icir.org> On Thu, Sep 23, 2010 at 15:28 -0700, you wrote: > Running 'bro -r' with a tracefile, however, causes bro to not bind to > the broccoli port, thus not communicating with the app - verified > w/netstat. This is actually intentional (because the timing can't be kept in sync between the trace and the (real-time) communication), and pseudo-realtime is indeed the recommended solution. > However, I'm not sure this behavior is optimal - there are a number of > bro applications now that may not need to listen to an interface (or > to a file, for that matter), but are strictly broccoli event driven, That's right, but when Bro is started without -r, it should actually work as you'd expect. Communication is only disabled when you give Bro a trace to read from . Are you seeing something different? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From jmellander at lbl.gov Fri Sep 24 11:41:46 2010 From: jmellander at lbl.gov (Jim Mellander) Date: Fri, 24 Sep 2010 11:41:46 -0700 Subject: [Bro] Surprising behavior when reading packets from file vs interface. In-Reply-To: <20100924180920.GP90579@icir.org> References: <20100924180920.GP90579@icir.org> Message-ID: On Fri, Sep 24, 2010 at 11:09 AM, Robin Sommer wrote: > > On Thu, Sep 23, 2010 at 15:28 -0700, you wrote: > >> Running 'bro -r' with a tracefile, however, causes bro to not bind to >> the broccoli port, thus not communicating with the app - verified >> w/netstat. > > This is actually intentional (because the timing can't be kept in > sync between the trace and the (real-time) communication), and > pseudo-realtime is indeed the recommended solution. I gathered that from the code, however pseudo-realtime has its own problems with timing. I, for instance, want to measure the real propagation delay and end-to-end processing time between bro and the broccoli client in a controlled environment by passing a tracefile thru bro to be processed in part via the broccoli client. So bro sends current_time() to the broccoli client, which then takes a timestamp, and can then determine the propagation delay - but current_time() is altered by the pseudo-realtime value.. If you want to go thru the tracefile as rapidly as possible, you will set pseudo-realtime to >1, which messes up current_time(), and thus the measurements. Thus, I have a need to: 1. run thru a tracefile as rapidly as possible for testing, 2. keep the notion of current_time() accurate to wall time, and 3. run broccoli Seems to me a few options can fulfill this: 1. Remove the test I mentioned 2. Have an explicit remote communications flag - probably having remote communications be the default, and allow disabling via flag if desired. Anything I'm missing? > >> However, I'm not sure this behavior is optimal - there are a number of >> bro applications now that may not need to listen to an interface (or >> to a file, for that matter), but are strictly broccoli event driven, > > That's right, but when Bro is started without -r, it should actually > work as you'd expect. Communication is only disabled when you give > Bro a trace to read from . Are you seeing something different? I've heard apocryphal stories of needing to have an interface open, but that may be due to to old event-loop requiring packets. Will ask around and gather more data. > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL ? ?* Fax ? +1 (510) 666-2956 * ? www.icir.org > From jmellander at lbl.gov Sun Sep 26 19:24:53 2010 From: jmellander at lbl.gov (Jim Mellander) Date: Sun, 26 Sep 2010 19:24:53 -0700 Subject: [Bro] Surprising behavior when reading packets from file vs interface. In-Reply-To: References: <20100924180920.GP90579@icir.org> Message-ID: BTW: I commented out the test I mentioned, and Bro performs as desired. Thanks for your assistance. On Fri, Sep 24, 2010 at 11:41 AM, Jim Mellander wrote: > On Fri, Sep 24, 2010 at 11:09 AM, Robin Sommer wrote: >> >> On Thu, Sep 23, 2010 at 15:28 -0700, you wrote: >> >>> Running 'bro -r' with a tracefile, however, causes bro to not bind to >>> the broccoli port, thus not communicating with the app - verified >>> w/netstat. >> >> This is actually intentional (because the timing can't be kept in >> sync between the trace and the (real-time) communication), and >> pseudo-realtime is indeed the recommended solution. > > I gathered that from the code, however pseudo-realtime has its own > problems with timing. ?I, for instance, want to measure the real > propagation delay and end-to-end processing time between bro and the > broccoli client in a controlled environment by passing a tracefile > thru bro to be processed in part via the broccoli client. > > So bro sends current_time() to the broccoli client, which then takes a > timestamp, and can then determine the propagation delay - but > current_time() is altered by the pseudo-realtime value.. ?If you want > to go thru the tracefile as rapidly as possible, you will set > pseudo-realtime to >1, which messes up current_time(), and thus the > measurements. ?Thus, I have a need to: > > 1. run thru a tracefile as rapidly as possible for testing, > 2. keep the notion of current_time() accurate to wall time, and > 3. run broccoli > > Seems to me a few options can fulfill this: > > 1. Remove the test I mentioned > 2. Have an explicit remote communications flag - probably having > remote communications be the default, and allow disabling via flag if > desired. > > Anything I'm missing? > > >> >>> However, I'm not sure this behavior is optimal - there are a number of >>> bro applications now that may not need to listen to an interface (or >>> to a file, for that matter), but are strictly broccoli event driven, >> >> That's right, but when Bro is started without -r, it should actually >> work as you'd expect. Communication is only disabled when you give >> Bro a trace to read from . Are you seeing something different? > > I've heard apocryphal stories of needing to have an interface open, > but that may be due to to old event-loop requiring packets. ?Will ask > around and gather more data. > > >> >> Robin >> >> -- >> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org >> ICSI/LBNL ? ?* Fax ? +1 (510) 666-2956 * ? www.icir.org >> > From mayank at niksun.com Mon Sep 27 08:46:22 2010 From: mayank at niksun.com (Mayank Jain) Date: Mon, 27 Sep 2010 11:46:22 -0400 Subject: [Bro] SIP Analyzer and pat file binpac Message-ID: <4CA0BC4E.8000806@niksun.com> Hi, I believe there is SIP analyzer available for BRO using binpac, but I can't find that in the source code. Is it publicly available? Can somebody please point me to its location. -- Thank You, Regards, Mayank Jain From robin at icir.org Mon Sep 27 09:23:55 2010 From: robin at icir.org (Robin Sommer) Date: Mon, 27 Sep 2010 09:23:55 -0700 Subject: [Bro] Surprising behavior when reading packets from file vs interface. In-Reply-To: References: <20100924180920.GP90579@icir.org> Message-ID: <20100927162355.GJ16365@icir.org> On Fri, Sep 24, 2010 at 11:41 -0700, you wrote: > 1. run thru a tracefile as rapidly as possible for testing, > 2. keep the notion of current_time() accurate to wall time, and > 3. run broccoli Even without the problematic test, I'm not sure this will give you realistic results. If Bro reads the trace as quickly as possible, that can delay processing remote events more than in a live setting where it will likely have more slack. Here's another idea: it seems the only problem with pseudo-realtime is that current_time() doesn't return wall time but the adjusted time, right? Internally, current_time() actually takes a second parameter indicating which of the two the caller wants. That option is not exposed to the script layer, but you could add a new function wall_time() or so that does that (see src/bro.bif, current_time()). One more thought: Bro already comes with a script remote-ping.bro that does periodic pings between two Bro's and measures the delay for those. Depending on what exactly you want to measure, that might already be sufficient? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org