From robin at icir.org Mon Sep 27 12:26:43 2010 From: robin at icir.org (Robin Sommer) Date: Mon, 27 Sep 2010 12:26:43 -0700 Subject: [Bro-Dev] test Message-ID: <20100927192643.GO16365@icir.org> -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Mon Sep 27 12:43:57 2010 From: seth at icir.org (Seth Hall) Date: Mon, 27 Sep 2010 15:43:57 -0400 Subject: [Bro-Dev] test Message-ID: <3A0ACF8F-5341-4E15-94C7-631237128CC1@icir.org> test From bro-dev at bro-ids.org Tue Sep 28 19:20:27 2010 From: bro-dev at bro-ids.org (Bro Tracker) Date: Wed, 29 Sep 2010 02:20:27 -0000 Subject: [Bro-Dev] #283: Bro should support mixed MPLS and non-MPLS traffic Message-ID: <044.24649684e17527d36edb517ec058a51a@bro-ids.org> #283: Bro should support mixed MPLS and non-MPLS traffic ----------------------------+----------------------------------------------- Reporter: robin | Owner: Type: Feature Request | Status: new Priority: Normal | Component: Bro Version: 1.5.1 | Keywords: ----------------------------+----------------------------------------------- Shouldn't be too hard to add based on an older patch for MPLS decapsulation. I have a trace for testing the mixed case, but I can't add that here for privacy reasons. For my own reference: msg-id for trace 4C743FE2.3030900 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro-dev at bro-ids.org Wed Sep 29 11:02:46 2010 From: bro-dev at bro-ids.org (Bro Tracker) Date: Wed, 29 Sep 2010 18:02:46 -0000 Subject: [Bro-Dev] #284: Making email_notice_to customizable Message-ID: <044.c221231e947578bb3ce0938f4c8d9cb5@bro-ids.org> #284: Making email_notice_to customizable ------------------------------+--------------------------------------------- Reporter: robin | Owner: vern Type: Patch | Status: new Priority: Normal | Component: Bro Version: 1.5.2-devel (svn) | Keywords: ------------------------------+--------------------------------------------- Making email_notice_to customizable -- Ticket URL: Bro Tracker Bro Issue Tracker From bro-dev at bro-ids.org Wed Sep 29 12:25:57 2010 From: bro-dev at bro-ids.org (Bro Tracker) Date: Wed, 29 Sep 2010 19:25:57 -0000 Subject: [Bro-Dev] #285: unsolicited SYN|ACK leads to Event(connection_established) in TCP.cc Message-ID: <057.076fcf083c93333f2382dae2683885b0@bro-ids.org> #285: unsolicited SYN|ACK leads to Event(connection_established) in TCP.cc --------------------------------+------------------------------------------- Reporter: hartley.87@? | Type: Problem Status: new | Priority: Normal Component: Bro | Version: 1.5.1 Keywords: | --------------------------------+------------------------------------------- In TCP_Analyzer::UpdateInactiveState, in line 674 (of TCP.cc, bro 1.5.1), Weird() is called with "unsolicited_SYN_response" but then the following code is run, creating a connection_established event in bro. endpoint->SetState(TCP_ENDPOINT_ESTABLISHED); if ( peer->state != TCP_ENDPOINT_PARTIAL ) { Event(connection_established); Conn()->EnableStatusUpdateTimer(); } } I manually anonymized our IP in the packet, otherwise it is as it was on the wire. This came up because there is a bro script we use (attr. Seth Hall) that fires on connection_established -- otherwise I'd never have noticed. It still consumes a modest amount of memory for the session timeout.. This packet has SYN|ACK, was unsolicited and has seq = ack = 0 but I don't think that's a component of the issue. I'll forge one differently, try it and update the ticket when I get a chance. Perhaps the StatusUpdateTimer() is meant to delete this event but fails to do so because of another feature of the packet. Like I said, I'm new to this code. I was thinking of just returning after the Weird() but it's clear that's not the right thing to do since other flags may go unprocessed. I spoke with Seth Hall about this but I wanted to create a ticket as well. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro-dev at bro-ids.org Wed Sep 29 12:44:30 2010 From: bro-dev at bro-ids.org (Bro Tracker) Date: Wed, 29 Sep 2010 19:44:30 -0000 Subject: [Bro-Dev] #285: unsolicited SYN|ACK leads to Event(connection_established) in TCP.cc In-Reply-To: <057.076fcf083c93333f2382dae2683885b0@bro-ids.org> References: <057.076fcf083c93333f2382dae2683885b0@bro-ids.org> Message-ID: <066.24bfcb69944f1976fbecc7124d89241d@bro-ids.org> #285: unsolicited SYN|ACK leads to Event(connection_established) in TCP.cc --------------------------------+------------------------------------------- Reporter: hartley.87@? | Type: Problem Status: new | Priority: Normal Component: Bro | Version: 1.5.1 Keywords: | --------------------------------+------------------------------------------- Comment(by hartley.87@?): The sequence numbers are not 0, I was too quick in looking at that. It seems reasonable to make this a half-open connection, hoping for a late-arriving (because of load-balancing, whatever) reverse (corresponding) SYN, and that's probably what's happening... -- Ticket URL: Bro Tracker Bro Issue Tracker From marlitt at ucalgary.ca Wed Sep 29 14:43:25 2010 From: marlitt at ucalgary.ca (marlitt at ucalgary.ca) Date: Wed, 29 Sep 2010 15:43:25 -0600 (MDT) Subject: [Bro-Dev] DataSeries for Bro (was: Major NSF funding for Bro development) In-Reply-To: <20100929211256.GA18993@icir.org> References: <20100824235306.GA41593@icir.org> <4C750EBB.6060605@ucalgary.ca> <20100825143639.GB50273@icir.org> <4C752D81.4000408@ucalgary.ca> <20100825155558.GI50273@icir.org> <4C753E43.1090300@ucalgary.ca> <20100825160611.GN50273@icir.org> <4C755103.30402@ucalgary.ca> <20100827041452.GH49514@icir.org> <4C77B100.1000609@ucalgary.ca> <20100929211256.GA18993@icir.org> Message-ID: <5e936a2be758e596bc77b51bbd70a550.squirrel@webmail.ucalgary.ca> hi Robin my intern has finished his project, and has returned to his university. We are still wrapping up the documentation. I'll see him next week at OSDI, so he and I can discuss this in person. If you, Vern or any of the developers happen to be at OSDI, we could have an informal discussion then. We can also plan for a more formal presentation a bit later. thanks Martin > > Hi Martin, > > as we're ramping up with our new project, we had a meeting over here > recently where everybody agreed that switching Bro to a binary > logging format makes a lot of sense, and generally folks also liked > what DataSeries offers. So I was wondering whether you might already > have an update on how the work on integrating DataSeries went? If > you (or your intern) could give us a summary of your experiences, > and perhaps even point us to some code, that would be very helpful. > > If you don't mind, please cc the Bro development list at > bro-dev at bro-ids.org in your reply; the folks who will work on this > are listening there. :) > > Thanks, > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > > > From robin at icir.org Wed Sep 29 14:58:53 2010 From: robin at icir.org (Robin Sommer) Date: Wed, 29 Sep 2010 14:58:53 -0700 Subject: [Bro-Dev] DataSeries for Bro (was: Major NSF funding for Bro development) In-Reply-To: <5e936a2be758e596bc77b51bbd70a550.squirrel@webmail.ucalgary.ca> References: <20100825143639.GB50273@icir.org> <4C752D81.4000408@ucalgary.ca> <20100825155558.GI50273@icir.org> <4C753E43.1090300@ucalgary.ca> <20100825160611.GN50273@icir.org> <4C755103.30402@ucalgary.ca> <20100827041452.GH49514@icir.org> <4C77B100.1000609@ucalgary.ca> <20100929211256.GA18993@icir.org> <5e936a2be758e596bc77b51bbd70a550.squirrel@webmail.ucalgary.ca> Message-ID: <20100929215853.GD17368@icir.org> Sounds good, thanks, Martin. I don't think anybody from us will be at OSDI unfortunately but we're looking forward to hear more (doesn't need to be formal in any way, just something like an email summary will already be helpful). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Wed Sep 29 15:23:57 2010 From: robin at icir.org (Robin Sommer) Date: Wed, 29 Sep 2010 15:23:57 -0700 Subject: [Bro-Dev] first try at a skeleton script In-Reply-To: <790F4138-BAD8-483B-BE02-0B4F3D373FCF@icir.org> References: <790F4138-BAD8-483B-BE02-0B4F3D373FCF@icir.org> Message-ID: <20100929222357.GB20718@icir.org> > http://github.com/sethhall/bro_scripts/blob/master/skeleton.bro Sorry for taking a bit to get back on this, but I'm still mulling over this. Generally, this looks good. Here are some additional (unsorted) thoughts: - The explanatory text is very helpful when writing a script, however we should make it clear that once a script is done, this should be removed, as otherwise we'll end up with many scripts having the same boilerplate in them. So, perhaps it makes sense to start anything that's just explaining something with an explicit marker, like # REMOVE-ME: The export section contains the external ... # REMOVE-ME: ...... - Also, some elements have a documentation string (i.e., something that will end up in the auto-generated documentation later) while others don't. I think that generally at least everything in the export section should have documentation. - Taking the two previous points together, it doesn't yet become clear what is documentation for a script's *users*, vs. what is help for a script *writer*. - As an alternative, we could keep the writer help inside a script to a minimum, but have an additional external document giving more details. Not sure which approach is better. - The Notice enums should be documented as well. - Another thought, considering our goal of building "configuration wizards": it would be cool if the such a wizard could extract from the script everything that it needs for providing the user with a corresponding configuration dialog. Actually I don't think there's much additional stuff needed to facilitate that once the documentation strings are in place, but it's worth thinking about what else could be helpful here. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Wed Sep 29 15:27:05 2010 From: robin at icir.org (Robin Sommer) Date: Wed, 29 Sep 2010 15:27:05 -0700 Subject: [Bro-Dev] first try at a skeleton script In-Reply-To: <20100929222357.GB20718@icir.org> References: <790F4138-BAD8-483B-BE02-0B4F3D373FCF@icir.org> <20100929222357.GB20718@icir.org> Message-ID: <20100929222705.GA21337@icir.org> Regarding the TODO: "Figure out a style for Sphinx-type documentation", here are my thoughts: - As a general rule, I guess a comment block should always be associated with the next language element. - The script itslef should start with a block describing the overall purpose of the script and perhaps potential caveats/things to keep in mind etc. - Each indidivual doc section should start with a single short sentence summarizing the documented element, followed by further text going into more detail. - Generally, all doc text is formatted in reST. However, one thing I always hate when writing documentation strings is adding all kinds of markers for things like parameters, return values, etc. (e.g., "@param", or ":param:"). What I've played with for HILTI is reducing this stuff to a minimum and extracting it from the structure of the text. Example: # Function for checking That. # # This function checks That and also does This, and # sometimes even More. # # id: The connection for doing That. # # Returns: True if That was successful. global check_that: function(id: conn_id): bool; We can parse everything out here that we need: - anything that's of the form "x: texttextext" is a parameter (with type information inferred from the prototype, not typed again). - "Returns: texttextext" is the magic to mark the description of the return value. This can be implemented via a pretty simple Sphinx plugin. Thoughts? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Wed Sep 29 16:37:17 2010 From: robin at icir.org (Robin Sommer) Date: Wed, 29 Sep 2010 16:37:17 -0700 Subject: [Bro-Dev] [Fwd] Re: DataSeries for Bro (was: Major NSF funding for Bro development) Message-ID: <20100929233717.GA23617@icir.org> (Did that ever appear on the list? I had approved it, but don't think I saw it coming through?) Robin ----- Forwarded message from marlitt at ucalgary.ca ----- Date: Wed, 29 Sep 2010 15:43:25 -0600 (MDT) From: marlitt at ucalgary.ca To: Robin Sommer Cc: Martin Arlitt , Vern Paxson , bro-dev at bro-ids.org Subject: Re: DataSeries for Bro (was: Major NSF funding for Bro development) Message-ID: <5e936a2be758e596bc77b51bbd70a550.squirrel at webmail.ucalgary.ca> hi Robin my intern has finished his project, and has returned to his university. We are still wrapping up the documentation. I'll see him next week at OSDI, so he and I can discuss this in person. If you, Vern or any of the developers happen to be at OSDI, we could have an informal discussion then. We can also plan for a more formal presentation a bit later. thanks Martin > > Hi Martin, > > as we're ramping up with our new project, we had a meeting over here > recently where everybody agreed that switching Bro to a binary > logging format makes a lot of sense, and generally folks also liked > what DataSeries offers. So I was wondering whether you might already > have an update on how the work on integrating DataSeries went? If > you (or your intern) could give us a summary of your experiences, > and perhaps even point us to some code, that would be very helpful. > > If you don't mind, please cc the Bro development list at > bro-dev at bro-ids.org in your reply; the folks who will work on this > are listening there. :) > > Thanks, > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > > > ----- End forwarded message ----- -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From vern at icir.org Wed Sep 29 18:21:00 2010 From: vern at icir.org (Vern Paxson) Date: Wed, 29 Sep 2010 18:21:00 -0700 Subject: [Bro-Dev] [Fwd] Re: DataSeries for Bro (was: Major NSF funding for Bro development) In-Reply-To: <20100929233717.GA23617@icir.org> (Wed, 29 Sep 2010 16:37:17 PDT). Message-ID: <20100930012100.9F9EB36A413@taffy.ICSI.Berkeley.EDU> > (Did that ever appear on the list? I had approved it, but don't > think I saw it coming through?) I still haven't seen it. Vern From robin at icir.org Wed Sep 29 20:09:27 2010 From: robin at icir.org (Robin Sommer) Date: Wed, 29 Sep 2010 20:09:27 -0700 Subject: [Bro-Dev] [Fwd] Re: DataSeries for Bro (was: Major NSF funding for Bro development) In-Reply-To: <20100930012100.9F9EB36A413@taffy.ICSI.Berkeley.EDU> References: <20100929233717.GA23617@icir.org> <20100930012100.9F9EB36A413@taffy.ICSI.Berkeley.EDU> Message-ID: <20100930030927.GC29136@icir.org> On Wed, Sep 29, 2010 at 18:21 -0700, you wrote: > I still haven't seen it. Matthias did interestingly. Not sure what happened, hope we don't have issues with the lists ... Let's keep an eye on it. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Wed Sep 29 20:32:34 2010 From: seth at icir.org (Seth Hall) Date: Wed, 29 Sep 2010 23:32:34 -0400 Subject: [Bro-Dev] [Fwd] Re: DataSeries for Bro (was: Major NSF funding for Bro development) In-Reply-To: <20100930030927.GC29136@icir.org> References: <20100929233717.GA23617@icir.org> <20100930012100.9F9EB36A413@taffy.ICSI.Berkeley.EDU> <20100930030927.GC29136@icir.org> Message-ID: On Sep 29, 2010, at 11:09 PM, Robin Sommer wrote: > Matthias did interestingly. Not sure what happened, hope we don't > have issues with the lists ... Let's keep an eye on it. I saw it too. Lots of email lately, I actually had to search for the email to make sure I had it. :) .Seth From bro-dev at bro-ids.org Wed Sep 29 22:53:05 2010 From: bro-dev at bro-ids.org (Bro Tracker) Date: Thu, 30 Sep 2010 05:53:05 -0000 Subject: [Bro-Dev] #285: unsolicited SYN|ACK leads to Event(connection_established) in TCP.cc In-Reply-To: <057.076fcf083c93333f2382dae2683885b0@bro-ids.org> References: <057.076fcf083c93333f2382dae2683885b0@bro-ids.org> Message-ID: <066.79ab9754858de8cccab789b677821082@bro-ids.org> #285: unsolicited SYN|ACK leads to Event(connection_established) in TCP.cc --------------------------------+------------------------------------------- Reporter: hartley.87@? | Type: Problem Status: new | Priority: Normal Component: Bro | Version: 1.5.1 Keywords: | --------------------------------+------------------------------------------- Comment(by hartley.87@?): I bet scheduling an event which checks for the existence of the connection a short time after connection_established would "fix" this -- maybe not an error in TCP.cc, but rather in the bro script which acts on it? But http://www.bro-ids.org/wiki/index.php/User_Manual:_Customizing_Bro in "Writing New Policy" refutes that idea pretty well. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro-dev at bro-ids.org Wed Sep 29 23:12:06 2010 From: bro-dev at bro-ids.org (Bro Tracker) Date: Thu, 30 Sep 2010 06:12:06 -0000 Subject: [Bro-Dev] #246: (IRC::active_channels[IRC::my_c]): run-time error, no such index In-Reply-To: <053.89c1530ecd614cfd34ed6a87597d93a4@bro-ids.org> References: <053.89c1530ecd614cfd34ed6a87597d93a4@bro-ids.org> Message-ID: <062.74240744248f4426853ba82a63369419@bro-ids.org> #246: (IRC::active_channels[IRC::my_c]): run-time error, no such index ----------------------------+----------------------------------------------- Reporter: Tyler.Schoenke | Type: defect Status: new | Priority: Normal Component: Bro | Version: 1.5.2-devel (svn) Keywords: | ----------------------------+----------------------------------------------- Comment(by hartley.87@?): I've seen this -- I'm not looking at it just now, but IIRC those lines perform a "delete" without checking the "in"ness of the index. It looks like the connection is removed from active connections before this code is called, so it could be modified to do an "in"ness check before deleting to remove the error ... or, it could be safely ignored, whichever makes people happiest. I could easily be mistaken... -- Ticket URL: Bro Tracker Bro Issue Tracker From bro-dev at bro-ids.org Thu Sep 30 08:35:37 2010 From: bro-dev at bro-ids.org (Bro Tracker) Date: Thu, 30 Sep 2010 15:35:37 -0000 Subject: [Bro-Dev] #246: (IRC::active_channels[IRC::my_c]): run-time error, no such index In-Reply-To: <053.89c1530ecd614cfd34ed6a87597d93a4@bro-ids.org> References: <053.89c1530ecd614cfd34ed6a87597d93a4@bro-ids.org> Message-ID: <062.3273ce2aaf6740b86170a888ffca0674@bro-ids.org> #246: (IRC::active_channels[IRC::my_c]): run-time error, no such index ----------------------------+----------------------------------------------- Reporter: Tyler.Schoenke | Type: defect Status: new | Priority: Normal Component: Bro | Version: 1.5.2-devel (svn) Keywords: | ----------------------------+----------------------------------------------- Comment(by Tyler.Schoenke): Thanks. I'll ignore the message for now. If someone wants to fix the code to check for the existence of the index, that would be great. I see this error fairly often in the crash logs, so one less error would help in debugging the actual source of a crash. -- Ticket URL: Bro Tracker Bro Issue Tracker