From christian at icir.org Wed Mar 2 16:08:37 2011 From: christian at icir.org (Christian Kreibich) Date: Wed, 02 Mar 2011 16:08:37 -0800 Subject: [Bro-Dev] Tracker not mailing out password reminders? Message-ID: <1299110917.28296.345.camel@strangepork> I'd like to update a ticket I own, but no longer know the password for the tracker. I've requested a few reminders, but seem to be getting no email. Could the notifications be broken, or am I missing something? -- Cheers, Christian From bro at tracker.icir.org Wed Mar 2 17:55:16 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Thu, 03 Mar 2011 01:55:16 -0000 Subject: [Bro-Dev] #397: double quoted broccoli config values broken due to misordering of lex rules In-Reply-To: <044.d55f158ee8058347ec1d2b758ae5f404@tracker.icir.org> References: <044.d55f158ee8058347ec1d2b758ae5f404@tracker.icir.org> Message-ID: <059.fdd4a7c79a2fc65839a769b81f55f941@tracker.icir.org> #397: double quoted broccoli config values broken due to misordering of lex rules -----------------------+----------------------- Reporter: leres | Owner: kreibich Type: Problem | Status: closed Priority: Normal | Milestone: Component: Broccoli | Version: git/topic Resolution: Solved | Keywords: -----------------------+----------------------- Changes (by kreibich): * status: new => closed * resolution: => Solved Comment: Thanks Craig, applied! -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Thu Mar 3 06:34:42 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Thu, 03 Mar 2011 14:34:42 -0000 Subject: [Bro-Dev] #406: Tracker not sending password reset emails Message-ID: <043.e9b2c3898aa071f83f30948383bb5289@tracker.icir.org> #406: Tracker not sending password reset emails ---------------------------+------------------ Reporter: seth | Owner: seth Type: Problem | Status: new Priority: Normal | Milestone: Component: TicketTracker | Version: Keywords: | ---------------------------+------------------ like the summary says. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Fri Mar 4 10:26:46 2011 From: robin at icir.org (Robin Sommer) Date: Fri, 4 Mar 2011 10:26:46 -0800 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/policy-scripts: Moved events for filling in connection service field to dpd.bro (1a327cd) In-Reply-To: References: <201103021732.p22HWKoJ012051@envoy.icir.org> <4D6E9853.9050100@icir.org> Message-ID: <20110304182646.GJ81375@icir.org> On Wed, Mar 02, 2011 at 21:44 -0500, you wrote: > field in connection records. We had talked about DPD being enabled by > default for the next release anyway so I was basically pushing for the > service field being filled out for all analyzed protocols by default > without needing to load the conn.bro script. Yeah, I think that makes sense. I'm really looking forward to have this all on by default to avoid the regular confusion of what "running with DPD" means. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.icir.org Fri Mar 4 13:25:50 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Fri, 04 Mar 2011 21:25:50 -0000 Subject: [Bro-Dev] #326: HTTP Analyzer overflow on content-lengths > 2GB In-Reply-To: <045.82bf4b86731e1736daaef4caa8c63d05@tracker.icir.org> References: <045.82bf4b86731e1736daaef4caa8c63d05@tracker.icir.org> Message-ID: <060.1463e2835b2bdf062d3f31f647d29bfe@tracker.icir.org> #326: HTTP Analyzer overflow on content-lengths > 2GB ---------------------+----------------------------- Reporter: gregor | Owner: robin Type: Patch | Status: accepted Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: git/master Resolution: | Keywords: inttypes,sprint ---------------------+----------------------------- Comment (by seth): Has this been integrated yet? I vaguely feel like it is. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Fri Mar 4 13:31:01 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Fri, 04 Mar 2011 21:31:01 -0000 Subject: [Bro-Dev] #326: HTTP Analyzer overflow on content-lengths > 2GB In-Reply-To: <045.82bf4b86731e1736daaef4caa8c63d05@tracker.icir.org> References: <045.82bf4b86731e1736daaef4caa8c63d05@tracker.icir.org> Message-ID: <060.b04634b2f078c007821318a79035fe48@tracker.icir.org> #326: HTTP Analyzer overflow on content-lengths > 2GB ---------------------+----------------------------- Reporter: gregor | Owner: robin Type: Patch | Status: accepted Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: git/master Resolution: | Keywords: inttypes,sprint ---------------------+----------------------------- Comment (by gregor): No it hasn't. There was another, unrelated 2GB problem in the TCP Reassembler that has been integrated. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Fri Mar 4 14:52:21 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Fri, 04 Mar 2011 22:52:21 -0000 Subject: [Bro-Dev] #407: broctl/scripts/send-mail hardwires hostnames, subject prefixes, etc. Message-ID: <044.8a181b0ceb9e69592ab1a784320e9316@tracker.icir.org> #407: broctl/scripts/send-mail hardwires hostnames, subject prefixes, etc. ------------------------+------------------- Reporter: leres | Owner: robin Type: Problem | Status: new Priority: Normal | Milestone: Component: BroControl | Version: 1.5.1 Keywords: | ------------------------+------------------- If I build bro on one machine and copy the resulting files to another (perhaps using a FreeBSD package), the broctl send-mail script sends mail from bro at the build host's hostname. In addition, send-mail does not honor the MailSubjectPrefix setting in broctl.cfg: {{{ from="Big Brother " replyto="" subject="[Bro] $1" if [ $# = 2 ]; then to=$2 else to="root at localhost" fi }}} This script should pick up MailFrom and MailSubjectPrefix from broctl.cfg. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Fri Mar 4 14:56:22 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Fri, 04 Mar 2011 22:56:22 -0000 Subject: [Bro-Dev] #407: broctl/scripts/send-mail hardwires hostnames, subject prefixes, etc. In-Reply-To: <044.8a181b0ceb9e69592ab1a784320e9316@tracker.icir.org> References: <044.8a181b0ceb9e69592ab1a784320e9316@tracker.icir.org> Message-ID: <059.0080e5ed62088a8d1a71061a205ab61e@tracker.icir.org> #407: broctl/scripts/send-mail hardwires hostnames, subject prefixes, etc. -------------------------+------------------- Reporter: leres | Owner: robin Type: Problem | Status: new Priority: Normal | Milestone: Component: BroControl | Version: 1.5.1 Resolution: | Keywords: -------------------------+------------------- Comment (by leres): Also, why do "MailFrom" and "MailSubjectPrefix" in my posts have q-marks after them?? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Fri Mar 4 15:41:05 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Fri, 04 Mar 2011 23:41:05 -0000 Subject: [Bro-Dev] #407: broctl/scripts/send-mail hardwires hostnames, subject prefixes, etc. In-Reply-To: <044.8a181b0ceb9e69592ab1a784320e9316@tracker.icir.org> References: <044.8a181b0ceb9e69592ab1a784320e9316@tracker.icir.org> Message-ID: <059.7d6f1f28ae722fed7a635d67012dabc8@tracker.icir.org> #407: broctl/scripts/send-mail hardwires hostnames, subject prefixes, etc. -------------------------+------------------- Reporter: leres | Owner: robin Type: Problem | Status: new Priority: Normal | Milestone: Component: BroControl | Version: 1.5.1 Resolution: | Keywords: -------------------------+------------------- Comment (by leres): Oops, I didn't realize send-mail was modified on "broctl install" -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Fri Mar 4 16:05:07 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Sat, 05 Mar 2011 00:05:07 -0000 Subject: [Bro-Dev] #407: broctl/scripts/send-mail hardwires hostnames, subject prefixes, etc. In-Reply-To: <044.8a181b0ceb9e69592ab1a784320e9316@tracker.icir.org> References: <044.8a181b0ceb9e69592ab1a784320e9316@tracker.icir.org> Message-ID: <059.9879b29247b1ece9fc0ab049fc10921d@tracker.icir.org> #407: broctl/scripts/send-mail hardwires hostnames, subject prefixes, etc. -------------------------+------------------- Reporter: leres | Owner: robin Type: Problem | Status: new Priority: Normal | Milestone: Component: BroControl | Version: 1.5.1 Resolution: | Keywords: -------------------------+------------------- Comment (by robin): On Fri, Mar 04, 2011 at 23:41 -0000, you wrote: > Oops, I didn't realize send-mail was modified on "broctl install" Right. This has actually changed in git: now all the values are read from a single configuration file and the scripts are static. -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Fri Mar 4 21:07:30 2011 From: seth at icir.org (Seth Hall) Date: Sat, 5 Mar 2011 00:07:30 -0500 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: <4D67DE1A.90409@icir.org> References: <4D67DE1A.90409@icir.org> Message-ID: On Feb 25, 2011, at 11:51 AM, Gregor Maier wrote: > the analyzer to count bytes and packets as seen on the wire per > connection (endpoint) is now in devel. If enabled the counters are part > of the connection record (actually the endpoint records) and can thus be > access by any event that gets a connection as argument. Thanks for doing the work on this, I've been wanting this functionality built into Bro for a long time. Is there any plan for getting this integrated into master? I see that there isn't a merge request yet, are you waiting for more testing? It just came up for me because I'm rewriting the conn.bro script and I want to include that data if the analyzer is enabled as a replacement for the normal c$orig$size and c$resp$size. Thanks, .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Fri Mar 4 21:18:52 2011 From: robin at icir.org (Robin Sommer) Date: Fri, 4 Mar 2011 21:18:52 -0800 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: References: <4D67DE1A.90409@icir.org> Message-ID: <20110305051852.GB30572@icir.org> On Sat, Mar 05, 2011 at 00:07 -0500, you wrote: > Thanks for doing the work on this, I've been wanting this > functionality built into Bro for a long time. Is there any plan for > getting this integrated into master? It's merged into devel and I going to give a bit of testing there before merging it into master. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From gregor at icir.org Sat Mar 5 10:28:49 2011 From: gregor at icir.org (Gregor Maier) Date: Sat, 05 Mar 2011 10:28:49 -0800 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: References: <4D67DE1A.90409@icir.org> Message-ID: <4D7280E1.30008@icir.org> On 3/4/11 21:07 , Seth Hall wrote: > > On Feb 25, 2011, at 11:51 AM, Gregor Maier wrote: > >> the analyzer to count bytes and packets as seen on the wire per >> connection (endpoint) is now in devel. If enabled the counters are part >> of the connection record (actually the endpoint records) and can thus be >> access by any event that gets a connection as argument. > > > Thanks for doing the work on this, I've been wanting this functionality built into Bro for a long time. Is there any plan for getting this integrated into master? I see that there isn't a merge request yet, are you waiting for more testing? see Robin's mail > It just came up for me because I'm rewriting the conn.bro script and I want to include that data if the analyzer is enabled as a replacement for the normal c$orig$size and c$resp$size. I've actually added the counters as additional columns in conn.bro (there's one flag to enable the analyzer and another one to enable logging in conn.log) You might also want to consider that osize and rsize try to count the logical number of payload bytes whereas my analyzer counts the number of IP bytes so I don't know whether replacing the osize and rsize makes sense (since it breaks behavior and given a conn.log file, it's hard to know which format it's in). OTOH, I think that people new to bro often get confused as to what osize and rsize are (e.g., that can be heavily inflated and need sanity checking). (I don't have a preference to do it one way or the other, btw) cu gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From seth at icir.org Mon Mar 7 06:54:49 2011 From: seth at icir.org (Seth Hall) Date: Mon, 7 Mar 2011 09:54:49 -0500 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: <4D7280E1.30008@icir.org> References: <4D67DE1A.90409@icir.org> <4D7280E1.30008@icir.org> Message-ID: On Mar 5, 2011, at 1:28 PM, Gregor Maier wrote: >> It just came up for me because I'm rewriting the conn.bro script and I want to include that data if the analyzer is enabled as a replacement for the normal c$orig$size and c$resp$size. > > I've actually added the counters as additional columns in conn.bro > (there's one flag to enable the analyzer and another one to enable > logging in conn.log) I'm reworking a lot of the analyzer companion scripts now (including conn.bro) and changing them to use the logging framework. I may make it an option in the new conn.bro script too. It should be pretty straightforward, especially since we've been working on giving the logging framework the specific capability for doing record extension to do this sort of thing. > You might also want to consider that osize and rsize try to count the > logical number of payload bytes whereas my analyzer counts the number of > IP bytes Oh, good point. That's definitely something to think about. > OTOH, I think that people new to bro often get confused as to what osize > and rsize are (e.g., that can be heavily inflated and need sanity > checking). I was confused by that for a while. It was weird for a while since I came from netflow analysis where you can pretty reliably trust the byte counts once you put all of the related flows back together. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Mon Mar 7 08:43:17 2011 From: robin at icir.org (Robin Sommer) Date: Mon, 7 Mar 2011 08:43:17 -0800 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: References: <4D67DE1A.90409@icir.org> <4D7280E1.30008@icir.org> Message-ID: <20110307164317.GB9654@icir.org> On Mon, Mar 07, 2011 at 09:54 -0500, you wrote: > > You might also want to consider that osize and rsize try to count the > > logical number of payload bytes whereas my analyzer counts the number of > > IP bytes > > Oh, good point. That's definitely something to think about. Yeah, it's actually an important distinction. I think I'd like the default to remain as it it: count TCP payload, and have an option to optionally add (but not replace with) the raw bytes. One selling point here is that the TCP values are actually something that NetFloow can not offer. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Mon Mar 7 10:05:25 2011 From: seth at icir.org (Seth Hall) Date: Mon, 7 Mar 2011 13:05:25 -0500 Subject: [Bro-Dev] $tag in notice_info Message-ID: <6763C4BA-10A6-4408-849A-CD5F615267E9@icir.org> Is there anyone around that can explain the purpose of the $tag field in the notice_info type? .Seth From seth at icir.org Mon Mar 7 13:02:32 2011 From: seth at icir.org (Seth Hall) Date: Mon, 7 Mar 2011 16:02:32 -0500 Subject: [Bro-Dev] capitalization standardization Message-ID: <61A16165-5B99-4348-9604-ED432251E988@icir.org> Now that I'm *finally* digging into these scripts, I've noticed that there isn't a lot of standardization across scripts with regard to capitalization. I want to propose the following style for scripts (keep in mind this won't be enforced by the language, just a common practice). Module names: Camel cased, no underscores (all uppercase for abbreviations) Examples: FTP, SSH, Notice, Remote, Signatures Types: Camel cased, no underscores Examples: Log, Connection, EntropyTestResult, GeoLocation, Packet Enum values: Upper-case with underscores Examples: KNOWN_HOSTS, SSH, NFS3_REG Variables: Lower-case with underscores Examples: example_variable, ftp_sessions This should follow the established conventions of most scripts (unfortunately not all). Did I leave anything out? The enum values doesn't quite work because Notice enum values have been defined as Camel-cased with underscores mostly. I'm sort of inclined to leave this as it is because it pinpoints when a notice type is being defined. I suppose that would be a subcategory of enum values that is just done specially. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Mon Mar 7 13:37:13 2011 From: robin at icir.org (Robin Sommer) Date: Mon, 7 Mar 2011 13:37:13 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <6763C4BA-10A6-4408-849A-CD5F615267E9@icir.org> References: <6763C4BA-10A6-4408-849A-CD5F615267E9@icir.org> Message-ID: <20110307213713.GB54540@icir.org> On Mon, Mar 07, 2011 at 13:05 -0500, you wrote: > Is there anyone around that can explain the purpose of the $tag field in the notice_info type? It uniquely identifies the NOTICE and can then be used at other locations to refer to it. The only use of it I recall right now is in conn.log: the relevant connection shows the tag in the addl field. I'm actually not sure how helpful having the tag is, I don't think I've ever used the tag but always grep for the 4-tuple right away. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Mon Mar 7 13:47:56 2011 From: seth at icir.org (Seth Hall) Date: Mon, 7 Mar 2011 16:47:56 -0500 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110307213713.GB54540@icir.org> References: <6763C4BA-10A6-4408-849A-CD5F615267E9@icir.org> <20110307213713.GB54540@icir.org> Message-ID: <65AD8578-8E10-48FF-8A22-687AE4D5AF80@icir.org> On Mar 7, 2011, at 4:37 PM, Robin Sommer wrote: > It uniquely identifies the NOTICE and can then be used at other > locations to refer to it. The only use of it I recall right now is in > conn.log: the relevant connection shows the tag in the addl field. Ah, ok. > I'm actually not sure how helpful having the tag is, I don't think > I've ever used the tag but always grep for the 4-tuple right away. That's sort of what I tend towards as well. My first inclination was to remove it since it's not used much. It should be possible to extend the connection logging with the logging framework to modularly add it back in later if someone wants to use it. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From gregor at icir.org Mon Mar 7 14:58:30 2011 From: gregor at icir.org (Gregor Maier) Date: Mon, 07 Mar 2011 14:58:30 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <65AD8578-8E10-48FF-8A22-687AE4D5AF80@icir.org> References: <6763C4BA-10A6-4408-849A-CD5F615267E9@icir.org> <20110307213713.GB54540@icir.org> <65AD8578-8E10-48FF-8A22-687AE4D5AF80@icir.org> Message-ID: <4D756316.2020507@icir.org> ... hmm. This actually reminds me about our discussion about having unique connection IDs (e.g., 64bit ints) in bro, that can then be used to locate a connection across log files. It seems we never filed a ticket for that. Do you think it's worth it? I might then summarize the the mail thread and plug it into a ticket. cu gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From robin at icir.org Mon Mar 7 15:36:50 2011 From: robin at icir.org (Robin Sommer) Date: Mon, 7 Mar 2011 15:36:50 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <4D756316.2020507@icir.org> References: <6763C4BA-10A6-4408-849A-CD5F615267E9@icir.org> <20110307213713.GB54540@icir.org> <65AD8578-8E10-48FF-8A22-687AE4D5AF80@icir.org> <4D756316.2020507@icir.org> Message-ID: <20110307233650.GF56177@icir.org> On Mon, Mar 07, 2011 at 14:58 -0800, you wrote: > It seems we never filed a ticket for that. Do you think it's worth it? Yes, definitly worth recording. If we go for it is another question, but I actually already don't remember the outcome of the email thread. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.icir.org Mon Mar 7 15:53:21 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 07 Mar 2011 23:53:21 -0000 Subject: [Bro-Dev] #408: Unique connection ID for bro Message-ID: <045.d1dcc001adbef43f9bd34f06ae252340@tracker.icir.org> #408: Unique connection ID for bro -----------------------------+------------------------ Reporter: gregor | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Keywords: | -----------------------------+------------------------ {{{ #!rst This is a summary of a mail thread on bro-dev and be worth considering as part of the work on the new logging framework. I was wondering whether it would make sense to assign each connection an ID that's unique for this bro run. This ID can just be a 64-bit counter that gets incremented on every new connection. Why: If we add this ID to log outputs, it would be much easier to correlate activity across logs (e.g., find the connection in http.log, alarm.log, and conn.log, without having to match 5-tuples and timestamps). Other use cases: * I want to count the number of HTTP request per connection * I do per connection stats (e.g., number of packets, number of bytes, retransmissions, RTTs), store them in their own log files and then want to correlate with the conn.log or the http.log * Easier debugging / analysis: I can just grep for the connectionID, instead of having to map between different connection formattings (e.g., notices have origIP:origPort -> respIP:respPort but when I want to grep for them in conn.log, I have to do some awk to get there) I think this would be a rather nice (and very easy to implement) feature. Ideally these ID should be **unique across bro runs**. Otherwise crunching information from a big log archive woulnd't be much better than it is today. But that might mean we'd need to go beyond 64-bit integers, perhaps to a string prefixed with something likely to be unique. Ideas on how to achieve uniqueness across Bro runs: We can probably keep a 64 bit counter internally and also add a bro_instance_ID, that's globally unique across Bro runs. For logging, we can then log the 64 bit counter and the instance_ID, or concatenate the two (I would guess that the instance_ID will be handy in other situations too). Doesn't the cluster already have/need something like that? In order to generate such an instance_ID, we could: * make sure it's truly globally unique, e.g., by using a cryptographically secure, long (128 bit, maybe even 160 or more) random number. Possibly from an entropy pool (can we use OpenSSL for that?) * the user supplies a "hostID", we can then add time and PID and hash all that together to get the instance ID, e.g., md5(hostID + PID + gettimeofday()) (this should probably be fairly tolerant even if the hostID gets reused across machines). Alternatively, to save memory, we could hash the run_ID and the connection counter into a single 64bit number. It might be nice to be able to keep the run_ID part and the counter separate. E.g., assign 32 bit to the run_ID and the other 32 bit to the connection counter and then concatenate them to make up the 64 bit unique connection ID. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From gregor at icir.org Mon Mar 7 15:54:19 2011 From: gregor at icir.org (Gregor Maier) Date: Mon, 07 Mar 2011 15:54:19 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110307233650.GF56177@icir.org> References: <6763C4BA-10A6-4408-849A-CD5F615267E9@icir.org> <20110307213713.GB54540@icir.org> <65AD8578-8E10-48FF-8A22-687AE4D5AF80@icir.org> <4D756316.2020507@icir.org> <20110307233650.GF56177@icir.org> Message-ID: <4D75702B.3030503@icir.org> Just filed a ticket for that. The thread basically just "fell-asleep" after discussing particular ideas on how to assign such an ID and how to make it unique across bro runs. cu gregor On 3/7/11 15:36 , Robin Sommer wrote: > > On Mon, Mar 07, 2011 at 14:58 -0800, you wrote: > >> It seems we never filed a ticket for that. Do you think it's worth it? > > Yes, definitly worth recording. If we go for it is another question, > but I actually already don't remember the outcome of the email thread. > > Robin > -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From seth at icir.org Mon Mar 7 17:27:44 2011 From: seth at icir.org (Seth Hall) Date: Mon, 7 Mar 2011 20:27:44 -0500 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <4D756316.2020507@icir.org> References: <6763C4BA-10A6-4408-849A-CD5F615267E9@icir.org> <20110307213713.GB54540@icir.org> <65AD8578-8E10-48FF-8A22-687AE4D5AF80@icir.org> <4D756316.2020507@icir.org> Message-ID: On Mar 7, 2011, at 5:58 PM, Gregor Maier wrote: > ... hmm. This actually reminds me about our discussion about having > unique connection IDs (e.g., 64bit ints) in bro, that can then be used > to locate a connection across log files. Oh yeah. What's your thought on this? Would you like to have that value print out along with the IP addresses and ports with the connection log and other logs? I think we may be able to work something out with the logging framework that makes it a little easier to work with. I can imagine choosing to output that value instead of the 4-tuple for database logging since it should be easy to do the join to tie data back together. As I think about it, I'm liking that idea more and more. Especially if we can pull it off cleanly. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From gregor at icir.org Mon Mar 7 18:00:43 2011 From: gregor at icir.org (Gregor Maier) Date: Mon, 07 Mar 2011 18:00:43 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: References: <6763C4BA-10A6-4408-849A-CD5F615267E9@icir.org> <20110307213713.GB54540@icir.org> <65AD8578-8E10-48FF-8A22-687AE4D5AF80@icir.org> <4D756316.2020507@icir.org> Message-ID: <4D758DCB.6020405@icir.org> On 3/7/11 17:27 , Seth Hall wrote: > > On Mar 7, 2011, at 5:58 PM, Gregor Maier wrote: > >> ... hmm. This actually reminds me about our discussion about having >> unique connection IDs (e.g., 64bit ints) in bro, that can then be used >> to locate a connection across log files. > > > Oh yeah. What's your thought on this? Would you like to have that value print out along with the IP addresses and ports with the connection log and other logs? I do! My thinking is that I find somethind interesting in one of the logfiles (e.g., http.log, alarm.log, conn.log, whatever) and now I want to look up the connection responsible for that log-entry in other log files. Using such an ID I could just grep for it (assuming text based logs, but it should apply similarly to binary logs). cu gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From seth at icir.org Mon Mar 7 22:07:33 2011 From: seth at icir.org (Seth Hall) Date: Tue, 8 Mar 2011 01:07:33 -0500 Subject: [Bro-Dev] removing trace rewriting code Message-ID: Do I have the go ahead to remove trace rewriting code from the policy scripts? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Mon Mar 7 22:34:54 2011 From: robin at icir.org (Robin Sommer) Date: Mon, 7 Mar 2011 22:34:54 -0800 Subject: [Bro-Dev] removing trace rewriting code In-Reply-To: References: Message-ID: <20110308063454.GA64800@icir.org> On Tue, Mar 08, 2011 at 01:07 -0500, you wrote: > Do I have the go ahead to remove trace rewriting code from the policy scripts? Yes. Per earlier discussion, we'll have to prepare a patch for readding rewriting back in, but I'll do that when I remove the internal code (based on the old scripts). No point keeping the rewriting when working on the new scripts. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From vern at icir.org Tue Mar 8 00:30:46 2011 From: vern at icir.org (Vern Paxson) Date: Tue, 08 Mar 2011 00:30:46 -0800 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: <4D7280E1.30008@icir.org> (Sat, 05 Mar 2011 10:28:49 PST). Message-ID: <20110308083046.7871A36A41F@taffy.ICSI.Berkeley.EDU> > You might also want to consider that osize and rsize try to count the > logical number of payload bytes whereas my analyzer counts the number of I'm confused. Are you referring to [or]{lower,upper}? I'm not recalling [or]size ... Vern From vern at icir.org Tue Mar 8 00:33:15 2011 From: vern at icir.org (Vern Paxson) Date: Tue, 08 Mar 2011 00:33:15 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110307213713.GB54540@icir.org> (Mon, 07 Mar 2011 13:37:13 PST). Message-ID: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> > I'm actually not sure how helpful having the tag is, I don't think > I've ever used the tag but always grep for the 4-tuple right away. Hmmm, I think I not-uncommonly grep on the tag to map from a notice.log entry to a conn.log entry, unless I'm missing some context here. That said, having a more general connection identifier, as subsequently discussed, would work for this, too. Vern From vern at icir.org Tue Mar 8 00:33:22 2011 From: vern at icir.org (Vern Paxson) Date: Tue, 08 Mar 2011 00:33:22 -0800 Subject: [Bro-Dev] capitalization standardization In-Reply-To: <61A16165-5B99-4348-9604-ED432251E988@icir.org> (Mon, 07 Mar 2011 16:02:32 EST). Message-ID: <20110308083322.4E9EC36A41F@taffy.ICSI.Berkeley.EDU> > I want to propose the following style for scripts What you sketch is fine by me. Vern From seth at icir.org Tue Mar 8 06:04:01 2011 From: seth at icir.org (Seth Hall) Date: Tue, 8 Mar 2011 09:04:01 -0500 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> References: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> Message-ID: <06D06C32-8293-4C7F-9E77-B5B88749E562@icir.org> On Mar 8, 2011, at 3:33 AM, Vern Paxson wrote: > That said, having a more general connection identifier, as subsequently > discussed, would work for this, too. That's what I was going to aim for. Centralizing on a connection identifier makes it possible to remove some other code. There is a sort-of-tag notion in ftp.bro which I'm removing now that goes the other direction by identifying an FTP session. .Seth From seth at icir.org Tue Mar 8 06:07:40 2011 From: seth at icir.org (Seth Hall) Date: Tue, 8 Mar 2011 09:07:40 -0500 Subject: [Bro-Dev] removing trace rewriting code In-Reply-To: <20110308063454.GA64800@icir.org> References: <20110308063454.GA64800@icir.org> Message-ID: On Mar 8, 2011, at 1:34 AM, Robin Sommer wrote: > On Tue, Mar 08, 2011 at 01:07 -0500, you wrote: > >> Do I have the go ahead to remove trace rewriting code from the policy scripts? > > Yes. Consider it done then. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From gregor at icir.org Tue Mar 8 07:57:28 2011 From: gregor at icir.org (Gregor Maier) Date: Tue, 08 Mar 2011 07:57:28 -0800 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: <20110308083046.7871A36A41F@taffy.ICSI.Berkeley.EDU> References: <20110308083046.7871A36A41F@taffy.ICSI.Berkeley.EDU> Message-ID: <4D7651E8.7080509@icir.org> On 3/8/11 0:30 , Vern Paxson wrote: >> You might also want to consider that osize and rsize try to count the >> logical number of payload bytes whereas my analyzer counts the number of > > I'm confused. Are you referring to [or]{lower,upper}? I'm not recalling > [or]size ... I'm confused. I was referring the connection size as reported in conn.log (the size that's tracked in the endpoint record of a connection). hth Gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From gregor at icir.org Tue Mar 8 08:01:53 2011 From: gregor at icir.org (Gregor Maier) Date: Tue, 08 Mar 2011 08:01:53 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> References: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> Message-ID: <4D7652F1.9060000@icir.org> On 3/8/11 0:33 , Vern Paxson wrote: >> I'm actually not sure how helpful having the tag is, I don't think >> I've ever used the tag but always grep for the 4-tuple right away. > > Hmmm, I think I not-uncommonly grep on the tag to map from a notice.log > entry to a conn.log entry, unless I'm missing some context here. For notices that's true. I would like to have the same / a similar mechanism for other log files (e.g., http.log) as well. cu gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From robin at icir.org Tue Mar 8 08:07:41 2011 From: robin at icir.org (Robin Sommer) Date: Tue, 8 Mar 2011 08:07:41 -0800 Subject: [Bro-Dev] removing trace rewriting code In-Reply-To: References: <20110308063454.GA64800@icir.org> Message-ID: <20110308160741.GB88108@icir.org> On Tue, Mar 08, 2011 at 09:07 -0500, you wrote: > Consider it done then. :) Cool. :) Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Tue Mar 8 08:11:15 2011 From: seth at icir.org (Seth Hall) Date: Tue, 8 Mar 2011 11:11:15 -0500 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: <20110308083046.7871A36A41F@taffy.ICSI.Berkeley.EDU> References: <20110308083046.7871A36A41F@taffy.ICSI.Berkeley.EDU> Message-ID: On Mar 8, 2011, at 3:30 AM, Vern Paxson wrote: > I'm confused. Are you referring to [or]{lower,upper}? I'm not recalling > [or]size ... [or]size is in the endpoint record. I think you may be thinking of the large-conns.bro script that calculates an upper and lower bound for the connection size. .Seth From seth at icir.org Tue Mar 8 08:14:00 2011 From: seth at icir.org (Seth Hall) Date: Tue, 8 Mar 2011 11:14:00 -0500 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <4D7652F1.9060000@icir.org> References: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> <4D7652F1.9060000@icir.org> Message-ID: On Mar 8, 2011, at 11:01 AM, Gregor Maier wrote: > For notices that's true. > I would like to have the same / a similar mechanism for other log files > (e.g., http.log) as well. That's what I'm working towards. I'm not too concerned about disk space so I was thinking of just including the identifier alongside the connection 4-tuple in every log. It would actually be kind of nice. If someone is particularly concerned about it disk space issues in their environment, they'd be able to reconfigure the logging framework locally to either not include the 4-tuple or not include the connection identifier (or include neither if they're crazy). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Tue Mar 8 08:37:13 2011 From: robin at icir.org (Robin Sommer) Date: Tue, 8 Mar 2011 08:37:13 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: References: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> <4D7652F1.9060000@icir.org> Message-ID: <20110308163713.GH88108@icir.org> I like switching from notice tags to a generic conn id used consistently across logs. My only request is that we make sure we can identify a connection uniqule even across Bro runs. Then one can just scan a whole log archive for a specific connection without needing to worry about when Bro started etc. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From vern at icir.org Tue Mar 8 09:13:35 2011 From: vern at icir.org (Vern Paxson) Date: Tue, 08 Mar 2011 09:13:35 -0800 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: (Tue, 08 Mar 2011 11:11:15 EST). Message-ID: <20110308171335.AC14736A016@taffy.ICSI.Berkeley.EDU> > [or]size is in the endpoint record. I think you may be thinking of the > large-conns.bro script that calculates an upper and lower bound for the > connection size. Yep, exactly. I grepped around for osize/rsize and didn't find it. In fact, I'm still not finding it. So is this an informal term, or am I not looking in the right place? Vern From seth at icir.org Tue Mar 8 09:30:00 2011 From: seth at icir.org (Seth Hall) Date: Tue, 8 Mar 2011 12:30:00 -0500 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: <20110308171335.AC14736A016@taffy.ICSI.Berkeley.EDU> References: <20110308171335.AC14736A016@taffy.ICSI.Berkeley.EDU> Message-ID: <2BF1FFCE-BEA3-42D3-8FB3-0825488C83F6@icir.org> On Mar 8, 2011, at 12:13 PM, Vern Paxson wrote: > Yep, exactly. I grepped around for osize/rsize and didn't find it. In > fact, I'm still not finding it. So is this an informal term, or am I not > looking in the right place? c$orig$size and c$resp$size type endpoint: record { size: count; state: count; }; type connection: record { id: conn_id; orig: endpoint; resp: endpoint; start_time: time; duration: interval; service: string_set; # if empty, service hasn't been determined addl: string; hot: count; # how hot; 0 = don't know or not hot history: string; }; .Seth From vern at icir.org Tue Mar 8 09:32:49 2011 From: vern at icir.org (Vern Paxson) Date: Tue, 08 Mar 2011 09:32:49 -0800 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: <2BF1FFCE-BEA3-42D3-8FB3-0825488C83F6@icir.org> (Tue, 08 Mar 2011 12:30:00 EST). Message-ID: <20110308173249.D144C36A016@taffy.ICSI.Berkeley.EDU> > c$orig$size and c$resp$size Ah, sure. I thought the thread was literally referring to "osize" and "rsize" (given their similarity to olower/oupper, etc.). Vern From gregor at icir.org Tue Mar 8 09:33:27 2011 From: gregor at icir.org (Gregor Maier) Date: Tue, 08 Mar 2011 09:33:27 -0800 Subject: [Bro-Dev] Bro byte and packet counting in devel In-Reply-To: <20110308171335.AC14736A016@taffy.ICSI.Berkeley.EDU> References: <20110308171335.AC14736A016@taffy.ICSI.Berkeley.EDU> Message-ID: <4D766867.50501@icir.org> On 3/8/11 9:13 , Vern Paxson wrote: >> [or]size is in the endpoint record. I think you may be thinking of the >> large-conns.bro script that calculates an upper and lower bound for the >> connection size. > > Yep, exactly. I grepped around for osize/rsize and didn't find it. In > fact, I'm still not finding it. So is this an informal term, or am I not > looking in the right place? I guess it's indeed an informal term then. I guess I've been using it so frequently in my scripts, etc. that I've assumed it's a common term. cu gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From seth at icir.org Tue Mar 8 10:08:59 2011 From: seth at icir.org (Seth Hall) Date: Tue, 8 Mar 2011 13:08:59 -0500 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110308163713.GH88108@icir.org> References: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> <4D7652F1.9060000@icir.org> <20110308163713.GH88108@icir.org> Message-ID: On Mar 8, 2011, at 11:37 AM, Robin Sommer wrote: > I like switching from notice tags to a generic conn id used > consistently across logs. My only request is that we make sure we can > identify a connection uniqule even across Bro runs. Then one can just > scan a whole log archive for a specific connection without needing to > worry about when Bro started etc. What do you think about using UUID/GUID? I don't know about the overhead to create those values and they're probably quite a bit larger than we need (128-bits displayed as hex), but it would be interesting to be able to have unique values per run and per instance. It'd end up being globally unique log identifiers. :) The length would be pretty annoying though. What sort of uniqueness are we aiming for here? I don't think that was ever very clearly laid out in the previous thread. With GUID we could do uniqueness for eternity (or close to it), but if we do something like hash the bytes for the $start_time timestamp and the 4-tuple that may be unique enough for most cases. I don't know what the relative overheads would be for generating that hash or the GUID would be either which could be a concern. .Seth From gregor at icir.org Tue Mar 8 10:37:47 2011 From: gregor at icir.org (Gregor Maier) Date: Tue, 08 Mar 2011 10:37:47 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: References: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> <4D7652F1.9060000@icir.org> <20110308163713.GH88108@icir.org> Message-ID: <4D76777B.7020709@icir.org> On 3/8/11 10:08 , Seth Hall wrote: > > On Mar 8, 2011, at 11:37 AM, Robin Sommer wrote: > >> I like switching from notice tags to a generic conn id used >> consistently across logs. My only request is that we make sure we can >> identify a connection uniqule even across Bro runs. Then one can just >> scan a whole log archive for a specific connection without needing to >> worry about when Bro started etc. > > > What do you think about using UUID/GUID? I don't know about the overhead to create those values and they're probably quite a bit larger than we need (128-bits displayed as hex), but it would be interesting to be able to have unique values per run and per instance. It'd end up being globally unique log identifiers. :) The length would be pretty annoying though. > > What sort of uniqueness are we aiming for here? I don't think that was ever very clearly laid out in the previous thread. With GUID we could do uniqueness for eternity (or close to it), but if we do something like hash the bytes for the $start_time timestamp and the 4-tuple that may be unique enough for most cases. I don't know what the relative overheads would be for generating that hash or the GUID would be either which could be a concern. I don't think we have to go that far. However, I think that using 128bits might be helpful. We could then have a 64-bit counter and generate a 64bit Bro run-ID. We can then concatenate the two 64bit values. This way there's pretty much no cost to create a new conn-id Another small advantage is that this way, one could just strip the run-ID, if one is only searching through the logs of single run. (or there could be a flag to force the run-ID to be 0 for testing) To get the run-ID we could use information like hostname, PID, time-of-day, Bro's host-id-name (for cluster deployments), etc. and hash them together using md5 or sha1 or something. (Or use GUID/UUID to generate the runid and then only use the 64bits with most entropy). just my 2ct -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From bro at tracker.icir.org Tue Mar 8 10:49:59 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 08 Mar 2011 18:49:59 -0000 Subject: [Bro-Dev] #394: all.bro loads non-existent policy and segfaults In-Reply-To: <045.8886ff9e67d9c2f2571dc694c41ec1bd@tracker.icir.org> References: <045.8886ff9e67d9c2f2571dc694c41ec1bd@tracker.icir.org> Message-ID: <060.e2a0f96249433e5633f5249cebe0db21@tracker.icir.org> #394: all.bro loads non-existent policy and segfaults ----------------------+------------------------ Reporter: gregor | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): Not loading `capture-events.bro` gets rid of the seg fault in this case, but looking into it more, there seems to be a problem in the `termination_signal()` function -- `terminate_bro()` will `delete event_serializer`, but `BroFile::CloseCachedFiles()` looks like it still depends on it. Flipping the order of `terminate_bro()` and `BroFile::CloseCachedFiles()` gets rid of the seg fault for me, but I'm not sure if that's right. For one it looks like `BroFile::CloseCachedFiles()` uses events, although the code/comment indicates they get dispatched immediately. Otherwise, I'm just not familiar enough to say... -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Mar 9 10:05:46 2011 From: robin at icir.org (Robin Sommer) Date: Wed, 9 Mar 2011 10:05:46 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <4D76777B.7020709@icir.org> References: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> <4D7652F1.9060000@icir.org> <20110308163713.GH88108@icir.org> <4D76777B.7020709@icir.org> Message-ID: <20110309180546.GL30260@icir.org> On Tue, Mar 08, 2011 at 10:37 -0800, you wrote: > I don't think we have to go that far. However, I think that using > 128bits might be helpful. We could then have a 64-bit counter and > generate a 64bit Bro run-ID. I'm not convinced we need the separate run-id. Note that while it would allow to get all connections from the same run, it doesn't get all the *logs* from the same run (because some logs may not have connection-level semantics). That doesn't seem worth storing an additional 64-bit value with every connection in almost every log to me. Also, 128-bit is really long and ugly. So I propose we go with a single 64-bit value that combines the run-id and the conn-id into a likely unique value, something like in this pseudo-code: struct { uint64 run_id; uint64 conn_count } id; id.run_id = md5(hostname, timeofday, pid); id.conn_count = ++global_conn_counter; uint64 unique_val = crc64(id); Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.icir.org Wed Mar 9 10:09:20 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Wed, 09 Mar 2011 18:09:20 -0000 Subject: [Bro-Dev] #409: topic/jsiwek/cmake-compiler-check Message-ID: <045.462dcb56dd015f236d76a905767cbc82@tracker.icir.org> #409: topic/jsiwek/cmake-compiler-check ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Low | Milestone: Bro1.6 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch exists in the 'bro' git repository and all of its submodules (recursively, except for 'btest') It changes `./configure` output to show nicer error messages when a C or C++ compiler isn't found. Also, I realized I forgot to update some of the INSTALL files when I updated repos to be CMake 2.6 compatible (versus requiring 2.8) -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Wed Mar 9 10:44:06 2011 From: seth at icir.org (Seth Hall) Date: Wed, 9 Mar 2011 13:44:06 -0500 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110309180546.GL30260@icir.org> References: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> <4D7652F1.9060000@icir.org> <20110308163713.GH88108@icir.org> <4D76777B.7020709@icir.org> <20110309180546.GL30260@icir.org> Message-ID: On Mar 9, 2011, at 1:05 PM, Robin Sommer wrote: > struct { uint64 run_id; uint64 conn_count } id; > id.run_id = md5(hostname, timeofday, pid); > id.conn_count = ++global_conn_counter; > > uint64 unique_val = crc64(id); I think I would prefer to leave out the md5. Do you think that we'd ever see conflicts by just adding those values together? Also, would CRC-64 provide enough reliability against collisions considering that some installations may run for a very long time? I don't know the characteristics of CRC algorithms, but I know they weren't designed for this use and I'd be a little worried about collisions. Maybe this is ok though? .Seth From gregor at icir.org Wed Mar 9 14:40:14 2011 From: gregor at icir.org (Gregor Maier) Date: Wed, 09 Mar 2011 14:40:14 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: References: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> <4D7652F1.9060000@icir.org> <20110308163713.GH88108@icir.org> <4D76777B.7020709@icir.org> <20110309180546.GL30260@icir.org> Message-ID: <4D7801CE.50902@icir.org> On 3/9/11 10:44 , Seth Hall wrote: > > On Mar 9, 2011, at 1:05 PM, Robin Sommer wrote: > >> struct { uint64 run_id; uint64 conn_count } id; >> id.run_id = md5(hostname, timeofday, pid); >> id.conn_count = ++global_conn_counter; >> >> uint64 unique_val = crc64(id); > > > I think I would prefer to leave out the md5. Do you think that we'd ever see conflicts by just adding those values together? I would hash them. What's the reason you don't like md5? It would run exactly once at startup. cu Gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From vern at icir.org Wed Mar 9 19:50:42 2011 From: vern at icir.org (Vern Paxson) Date: Wed, 09 Mar 2011 19:50:42 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <4D7801CE.50902@icir.org> (Wed, 09 Mar 2011 14:40:14 PST). Message-ID: <20110310035042.7341C36A520@taffy.ICSI.Berkeley.EDU> > >> uint64 unique_val = crc64(id); (could just take the bottom 64 bits of the MD5 here) From slagell at ncsa.illinois.edu Wed Mar 9 19:54:33 2011 From: slagell at ncsa.illinois.edu (Adam Slagell) Date: Wed, 9 Mar 2011 21:54:33 -0600 (CST) Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110310035042.7341C36A520@taffy.ICSI.Berkeley.EDU> References: <20110310035042.7341C36A520@taffy.ICSI.Berkeley.EDU> Message-ID: <22741C58-41C4-458E-8595-2576350E8080@ncsa.illinois.edu> How often would these hashed be computed? You may need to worry about performance. Sent from my mobile On Mar 9, 2011, at 9:50 PM, Vern Paxson wrote: >>>> uint64 unique_val = crc64(id); > > (could just take the bottom 64 bits of the MD5 here) > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > From robin at icir.org Wed Mar 9 19:54:44 2011 From: robin at icir.org (Robin Sommer) Date: Wed, 9 Mar 2011 19:54:44 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110310035042.7341C36A520@taffy.ICSI.Berkeley.EDU> References: <4D7801CE.50902@icir.org> <20110310035042.7341C36A520@taffy.ICSI.Berkeley.EDU> Message-ID: <20110310035444.GB48856@icir.org> On Wed, Mar 09, 2011 at 19:50 -0800, you wrote: > (could just take the bottom 64 bits of the MD5 here) Sure, but I was trying to avoid the MD5 altogether. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Wed Mar 9 20:15:14 2011 From: seth at icir.org (Seth Hall) Date: Wed, 9 Mar 2011 23:15:14 -0500 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <4D7801CE.50902@icir.org> References: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> <4D7652F1.9060000@icir.org> <20110308163713.GH88108@icir.org> <4D76777B.7020709@icir.org> <20110309180546.GL30260@icir.org> <4D7801CE.50902@icir.org> Message-ID: <6FE4A3A8-768E-4078-BB03-625014495919@icir.org> On Mar 9, 2011, at 5:40 PM, Gregor Maier wrote: > I would hash them. What's the reason you don't like md5? It would run > exactly once at startup. Whoops, you're right. I was thinking it would be done for each connection for some reason. .Seth From robin at icir.org Wed Mar 9 21:50:04 2011 From: robin at icir.org (Robin Sommer) Date: Wed, 9 Mar 2011 21:50:04 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: References: <20110308083315.6200B36A41F@taffy.ICSI.Berkeley.EDU> <4D7652F1.9060000@icir.org> <20110308163713.GH88108@icir.org> <4D76777B.7020709@icir.org> <20110309180546.GL30260@icir.org> Message-ID: <20110310055004.GI48856@icir.org> On Wed, Mar 09, 2011 at 13:44 -0500, you wrote: > Also, would CRC-64 provide enough reliability against collisions > considering that some installations may run for a very long time? I was wondering about that too and the answer is: I don't know ... I was looking for a hash that is cheap to calculate, don't really want to do an MD5 per connection. Other suggestions? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From vern at icir.org Wed Mar 9 21:57:19 2011 From: vern at icir.org (Vern Paxson) Date: Wed, 09 Mar 2011 21:57:19 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110310055004.GI48856@icir.org> (Wed, 09 Mar 2011 21:50:04 PST). Message-ID: <20110310055719.94A9436A030@taffy.ICSI.Berkeley.EDU> > I was wondering about that too and the answer is: I don't know ... I > was looking for a hash that is cheap to calculate, don't really want > to do an MD5 per connection. Other suggestions? I think a universal hash should work here. There's already one in Hash.cc, thanks to Ruoming (I think) and avoiding algorithmic complexity attacks. Vern From robin at icir.org Thu Mar 10 09:34:41 2011 From: robin at icir.org (Robin Sommer) Date: Thu, 10 Mar 2011 09:34:41 -0800 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110310055719.94A9436A030@taffy.ICSI.Berkeley.EDU> References: <20110310055004.GI48856@icir.org> <20110310055719.94A9436A030@taffy.ICSI.Berkeley.EDU> Message-ID: <20110310173441.GE74428@icir.org> On Wed, Mar 09, 2011 at 21:57 -0800, you wrote: > I think a universal hash should work here. There's already one in Hash.cc, Good idea! Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.icir.org Thu Mar 10 12:48:04 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Thu, 10 Mar 2011 20:48:04 -0000 Subject: [Bro-Dev] #410: Extension to init time pattern construction Message-ID: <043.1f9fe5635fad6e39b30fda249ed52c61@tracker.icir.org> #410: Extension to init time pattern construction -----------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Keywords: | -----------------------------+-------------------- I'd like to be able to do this... {{{ const pattern_a = /A/; const pattern_b = /B/; const pattern_ab = pattern_a | pattern_b; }}} This doesn't currently work. -- Ticket URL: Bro Tracker Bro Issue Tracker From appleman at ncsa.illinois.edu Fri Mar 11 14:02:17 2011 From: appleman at ncsa.illinois.edu (Don Appleman) Date: Fri, 11 Mar 2011 16:02:17 -0600 (CST) Subject: [Bro-Dev] capitalization standardization In-Reply-To: <61A16165-5B99-4348-9604-ED432251E988@icir.org> Message-ID: <1510703691.3898.1299880937032.JavaMail.root@zimbra-1.ncsa.uiuc.edu> Seth, What do you recommend as the standard for function names? I'm seeing them named like variables (lower_case_with_underscores). Personally, I'd prefer that they differ in some way from variables. How about camelCase with the first character lower case (to distinguish them from types)? And I don't know how often we need to name new events. Right now, their naming appears to also follow that of variables, lower_case_with_underscores. I assume this won't change, since there are lots and lots of events already defined, but I figured I should ask, for completeness. Thanks, Don ----- Original Message ----- From: "Seth Hall" To: "Bro Dev" Sent: Monday, March 7, 2011 3:02:32 PM Subject: [Bro-Dev] capitalization standardization Now that I'm *finally* digging into these scripts, I've noticed that there isn't a lot of standardization across scripts with regard to capitalization. I want to propose the following style for scripts (keep in mind this won't be enforced by the language, just a common practice). Module names: Camel cased, no underscores (all uppercase for abbreviations) Examples: FTP, SSH, Notice, Remote, Signatures Types: Camel cased, no underscores Examples: Log, Connection, EntropyTestResult, GeoLocation, Packet Enum values: Upper-case with underscores Examples: KNOWN_HOSTS, SSH, NFS3_REG Variables: Lower-case with underscores Examples: example_variable, ftp_sessions This should follow the established conventions of most scripts (unfortunately not all). Did I leave anything out? The enum values doesn't quite work because Notice enum values have been defined as Camel-cased with underscores mostly. I'm sort of inclined to leave this as it is because it pinpoints when a notice type is being defined. I suppose that would be a subcategory of enum values that is just done specially. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ _______________________________________________ bro-dev mailing list bro-dev at bro-ids.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev -- Don Appleman National Center for Supercomputing Applications 2006B NCSA, 1205 W. Clark St. Urbana, IL 61801 217/333-6340 appleman at ncsa.illinois.edu From vern at icir.org Fri Mar 11 14:11:00 2011 From: vern at icir.org (Vern Paxson) Date: Fri, 11 Mar 2011 14:11:00 -0800 Subject: [Bro-Dev] capitalization standardization In-Reply-To: <1510703691.3898.1299880937032.JavaMail.root@zimbra-1.ncsa.uiuc.edu> (Fri, 11 Mar 2011 16:02:17 CST). Message-ID: <20110311221100.AB69A36A406@taffy.ICSI.Berkeley.EDU> > Personally, I'd prefer that they differ in some way from variables. How > about camelCase with the first character lower case (to distinguish them > from types)? I appreciate the point you're making, but I resist this change. It would require extensive alterations (pretty much all event names, for starters) and result in a style that to my eye would be jarring, given how used I am to the current style. Vern From appleman at ncsa.illinois.edu Fri Mar 11 14:45:40 2011 From: appleman at ncsa.illinois.edu (Don Appleman) Date: Fri, 11 Mar 2011 16:45:40 -0600 (CST) Subject: [Bro-Dev] capitalization standardization In-Reply-To: <20110311221100.AB69A36A406@taffy.ICSI.Berkeley.EDU> Message-ID: <1825162663.4096.1299883540982.JavaMail.root@zimbra-1.ncsa.uiuc.edu> Sorry, that paragraph is meant to refer to script functions, not to events. I concede that it would not be a good idea to change the naming convention for events. Thanks, Don ----- Original Message ----- From: "Vern Paxson" To: "Don Appleman" Cc: "Seth Hall" , "Bro Dev" Sent: Friday, March 11, 2011 4:11:00 PM Subject: Re: [Bro-Dev] capitalization standardization > Personally, I'd prefer that they differ in some way from variables. How > about camelCase with the first character lower case (to distinguish them > from types)? I appreciate the point you're making, but I resist this change. It would require extensive alterations (pretty much all event names, for starters) and result in a style that to my eye would be jarring, given how used I am to the current style. Vern -- Don Appleman National Center for Supercomputing Applications 2006B NCSA, 1205 W. Clark St. Urbana, IL 61801 217/333-6340 appleman at ncsa.illinois.edu From vern at icir.org Fri Mar 11 14:49:55 2011 From: vern at icir.org (Vern Paxson) Date: Fri, 11 Mar 2011 14:49:55 -0800 Subject: [Bro-Dev] capitalization standardization In-Reply-To: <1825162663.4096.1299883540982.JavaMail.root@zimbra-1.ncsa.uiuc.edu> (Fri, 11 Mar 2011 16:45:40 CST). Message-ID: <20110311224955.3370836A406@taffy.ICSI.Berkeley.EDU> > Sorry, that paragraph is meant to refer to script functions Understood. > not to events The problem here is that script functions and event handlers are nearly isomorphic in the language, with the only difference being that functions optionally return values. So a change to just script functions would then look peculiar in that most "functions" (really handlers) don't reflect the change. Some might argue that that's a feature, not a bug. If the change were indeed confined to script functions, it would limit its impact / jarring-ness, and maybe I could be talked into it if others are enthused too. Vern From gregor at icir.org Fri Mar 11 16:19:20 2011 From: gregor at icir.org (Gregor Maier) Date: Fri, 11 Mar 2011 16:19:20 -0800 Subject: [Bro-Dev] capitalization standardization In-Reply-To: <1510703691.3898.1299880937032.JavaMail.root@zimbra-1.ncsa.uiuc.edu> References: <1510703691.3898.1299880937032.JavaMail.root@zimbra-1.ncsa.uiuc.edu> Message-ID: <4D7ABC08.2020501@icir.org> On 3/11/11 14:02 , Don Appleman wrote: > Seth, > > What do you recommend as the standard for function names? I'm seeing them named like variables (lower_case_with_underscores). Personally, I'd prefer that they differ in some way from variables. How about camelCase with the first character lower case (to distinguish them from types)? I'd prefer to *not* change the function names. cu Gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From seth at icir.org Fri Mar 11 18:32:53 2011 From: seth at icir.org (Seth Hall) Date: Fri, 11 Mar 2011 21:32:53 -0500 Subject: [Bro-Dev] capitalization standardization In-Reply-To: <4D7ABC08.2020501@icir.org> References: <1510703691.3898.1299880937032.JavaMail.root@zimbra-1.ncsa.uiuc.edu> <4D7ABC08.2020501@icir.org> Message-ID: <35295A17-57ED-4466-903E-E7C281661E56@icir.org> On Mar 11, 2011, at 7:19 PM, Gregor Maier wrote: >> Personally, I'd prefer that they differ in some way from variables. How about camelCase with the first character lower case (to distinguish them from types)? > > I'd prefer to *not* change the function names. I agree with Gregor and Vern. Changing the style for event and function names in the shipped scripts would be a big change (mentally and code-wise). Taking Vern's point one step further about functions and events being isomorphic; functions and events are still just variables so keeping them with the same style as variables actually makes a lot of sense. That's actually why I didn't specify functions and events originally. I should have point that out though. Try printing a function sometime if you want to see what I mean. :) .Seth From bro at tracker.icir.org Mon Mar 14 09:47:27 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 14 Mar 2011 16:47:27 -0000 Subject: [Bro-Dev] #411: Non-binpac analyzer generates incorrect weird Message-ID: <043.92d5260f603652099df92e1b24c3dd57@tracker.icir.org> #411: Non-binpac analyzer generates incorrect weird ---------------------+-------------------- Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Keywords: | ---------------------+-------------------- With the attached tracefile, the non-binpac analyzer raise a weird named unmatched_HTTP_reply when it shouldn't. The tracefile has a request that uses "Expect: 100-continue" which should allow the web server to respond to tell the client if it's ok to send a request body to the avoid the situation of a client sending a lot of data only to be rejected because the request was bad or unallowed for some reason. The second http reply is expected because it comes after the request body. Here are the specs for the command for reference: http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.2.3 I think the right response to this is to remove the weird from the core. If we still want to handle this situation, we can handle it from the appropriate bro script. This should help trim down the number of invalid weird's a little bit since this is probably a fairly common occurrence. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 14 16:34:16 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 14 Mar 2011 23:34:16 -0000 Subject: [Bro-Dev] #382: Design an internal API for logging backends In-Reply-To: <044.1bec2fcec10edc2608651458c92f895a@tracker.icir.org> References: <044.1bec2fcec10edc2608651458c92f895a@tracker.icir.org> Message-ID: <059.08841bbf44a44aafe5f557451d693c76@tracker.icir.org> #382: Design an internal API for logging backends ----------------------+--------------------- Reporter: robin | Owner: robin Type: Problem | Status: testing Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Resolution: | Keywords: logging ----------------------+--------------------- Changes (by robin): * status: accepted => testing -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 14 17:41:40 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 15 Mar 2011 00:41:40 -0000 Subject: [Bro-Dev] #409: topic/jsiwek/cmake-compiler-check In-Reply-To: <045.462dcb56dd015f236d76a905767cbc82@tracker.icir.org> References: <045.462dcb56dd015f236d76a905767cbc82@tracker.icir.org> Message-ID: <060.3550832b786fe2afa575c6f2fc5ed692@tracker.icir.org> #409: topic/jsiwek/cmake-compiler-check -----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: closed Priority: Low | Milestone: Bro1.6 Component: Bro | Version: git/master Resolution: Merged/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Merged/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 14 18:32:42 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 15 Mar 2011 01:32:42 -0000 Subject: [Bro-Dev] #325: Remove ACTIVE_MAPPING code In-Reply-To: <044.35fa7a347d936d84e134297408ded21b@tracker.icir.org> References: <044.35fa7a347d936d84e134297408ded21b@tracker.icir.org> Message-ID: <059.66f180a4516878af4280f7512d4c66d2@tracker.icir.org> #325: Remove ACTIVE_MAPPING code ---------------------+------------------------ Reporter: robin | Owner: robin Type: Task | Status: testing Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: git/master Resolution: | Keywords: cleanup ---------------------+------------------------ Changes (by robin): * status: accepted => testing Comment: This is in topic/robin/cleanup-active-mapping. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 14 18:33:28 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 15 Mar 2011 01:33:28 -0000 Subject: [Bro-Dev] #412: Port the istate tests to btes Message-ID: <044.325f74c6d09693a51bde1e009db18a42@tracker.icir.org> #412: Port the istate tests to btes --------------------+-------------------- Reporter: robin | Owner: robin Type: Task | Status: new Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Keywords: | --------------------+-------------------- They are in testings/istate. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 14 18:57:01 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 15 Mar 2011 01:57:01 -0000 Subject: [Bro-Dev] #324: Remove EXPIRE_DFA_STATES code In-Reply-To: <044.1d76ad3b3cc62054fae99370dccd0543@tracker.icir.org> References: <044.1d76ad3b3cc62054fae99370dccd0543@tracker.icir.org> Message-ID: <059.10c867d774c4639a026fe692d7db80cc@tracker.icir.org> #324: Remove EXPIRE_DFA_STATES code ---------------------+------------------------ Reporter: robin | Owner: robin Type: Task | Status: accepted Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: git/master Resolution: | Keywords: cleanup ---------------------+------------------------ Comment (by robin): Removed in topic/robin/cleanup-dfa-cache. This passes the test-suite, but more some regexp-heavy testing would probably be good ... -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 14 18:57:10 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 15 Mar 2011 01:57:10 -0000 Subject: [Bro-Dev] #324: Remove EXPIRE_DFA_STATES code In-Reply-To: <044.1d76ad3b3cc62054fae99370dccd0543@tracker.icir.org> References: <044.1d76ad3b3cc62054fae99370dccd0543@tracker.icir.org> Message-ID: <059.07c4755a32099cb059ab300065697131@tracker.icir.org> #324: Remove EXPIRE_DFA_STATES code ---------------------+------------------------ Reporter: robin | Owner: robin Type: Task | Status: testing Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: git/master Resolution: | Keywords: cleanup ---------------------+------------------------ Changes (by robin): * status: accepted => testing -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Tue Mar 15 21:22:45 2011 From: robin at icir.org (Robin Sommer) Date: Tue, 15 Mar 2011 21:22:45 -0700 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110310055719.94A9436A030@taffy.ICSI.Berkeley.EDU> References: <20110310055004.GI48856@icir.org> <20110310055719.94A9436A030@taffy.ICSI.Berkeley.EDU> Message-ID: <20110316042245.GA19883@icir.org> On Wed, Mar 09, 2011 at 21:57 -0800, you wrote: > I think a universal hash should work here. There's already one in Hash.cc, > thanks to Ruoming (I think) and avoiding algorithmic complexity attacks. I've added code implementing the unique connection identifiers in a topic branch. However, two questions: - There are actually a number of hash algorithms in Hash.{h,cc}, with only one of them being used (via #ifdefs). Vern, do you remember the story behind having multiple of them, and can we just remove the code for those not enabled? - I'm wondering whether for the unique connection ids it would make sense to make them stable in the case that we're working offline from a trace. It kind of bothers me that when reading the same trace multiple times, I get a different output each time. Should we just fix the instance ID when reading from a trace? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Wed Mar 16 04:28:18 2011 From: seth at icir.org (Seth Hall) Date: Wed, 16 Mar 2011 07:28:18 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/conn-ids: Extending conn_id with a globally unique identifiers. (881071c) In-Reply-To: <201103160343.p2G3hV9e002149@bro-ids.icir.org> References: <201103160343.p2G3hV9e002149@bro-ids.icir.org> Message-ID: On Mar 15, 2011, at 11:43 PM, Robin Sommer wrote: > orig_p: port; > resp_h: addr; > resp_p: port; > + uid: string; Would it make more sense to store this value in the connection record directly? It doesn't really add any information to the conn_id over the existing fields and I suppose it would make comparison operators take a bit longer. However, it would make the logging framework integration really easy that way you have it. .Seth From robin at icir.org Wed Mar 16 08:25:06 2011 From: robin at icir.org (Robin Sommer) Date: Wed, 16 Mar 2011 08:25:06 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/conn-ids: Extending conn_id with a globally unique identifiers. (881071c) In-Reply-To: References: <201103160343.p2G3hV9e002149@bro-ids.icir.org> Message-ID: <20110316152506.GA40299@icir.org> Yeah, I was wondering where to put it, and I'm still undecided. On Wed, Mar 16, 2011 at 07:28 -0400, you wrote: > I suppose it would make comparison operators take a bit longer. Haven't thought about that but could we now change them to use the new uid? > However, it would make the logging framework integration really easy > that way you have it. That's exactly why for now I went with the conn_id. That way, whoever logs the id, will automatically log the unique string as well, which is neat. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Wed Mar 16 09:02:30 2011 From: seth at icir.org (Seth Hall) Date: Wed, 16 Mar 2011 12:02:30 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/conn-ids: Extending conn_id with a globally unique identifiers. (881071c) In-Reply-To: <20110316152506.GA40299@icir.org> References: <201103160343.p2G3hV9e002149@bro-ids.icir.org> <20110316152506.GA40299@icir.org> Message-ID: <1F3B062F-A8F8-4DC8-A388-4E204C816998@icir.org> On Mar 16, 2011, at 11:25 AM, Robin Sommer wrote: >> I suppose it would make comparison operators take a bit longer. > > Haven't thought about that but could we now change them to use the new > uid? I guess direct comparisons aren't really done that frequently, but table and set lookups are done all the time. Would it affect table and set lookups to have the extra string to compare? It shouldn't affect lookup time much (or any) should it? > That's exactly why for now I went with the conn_id. That way, whoever > logs the id, will automatically log the unique string as well, which > is neat. It also gives us the option for changing the default filter to remove either the 4-tuple or the unique string so that sites could choose globally what shows up in their logs if they are just running the default filters. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Wed Mar 16 09:32:13 2011 From: robin at icir.org (Robin Sommer) Date: Wed, 16 Mar 2011 09:32:13 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/conn-ids: Extending conn_id with a globally unique identifiers. (881071c) In-Reply-To: <1F3B062F-A8F8-4DC8-A388-4E204C816998@icir.org> References: <201103160343.p2G3hV9e002149@bro-ids.icir.org> <20110316152506.GA40299@icir.org> <1F3B062F-A8F8-4DC8-A388-4E204C816998@icir.org> Message-ID: <20110316163213.GF40299@icir.org> On Wed, Mar 16, 2011 at 12:02 -0400, you wrote: > I guess direct comparisons aren't really done that frequently, but > table and set lookups are done all the time. Would it affect table > and set lookups to have the extra string to compare? One could now actually use the uid as the table index ... However, that wouldn't be as intuitive as using the whole conn_id and I don't think I want to advocate that. However, here's disruptive alternative: we could move {orig,resp}{_h,_p} into the connection record and then use the unique identifier as the "id" directly ... (Wouldn't do the automatic logging of both though). > It shouldn't affect lookup time much (or any) should it? It could a little bit but nothing signficant I'd guess (but always hard to say). > It also gives us the option for changing the default filter to remove > either the 4-tuple or the unique string so that sites could choose > globally what shows up in their logs if they are just running the > default filters. Assuming fields are named consistenly, that would also work if the uid were explicitly inlcuded into the log record. So, I'm fine going either way, conn_id or connection. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.icir.org Wed Mar 16 09:43:37 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Wed, 16 Mar 2011 16:43:37 -0000 Subject: [Bro-Dev] #413: Headers not added to CMake-generated Xcode project Message-ID: <045.54fd1755a615e8fc1a684c9c99a794ab@tracker.icir.org> #413: Headers not added to CMake-generated Xcode project --------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: new Priority: Low | Milestone: Bro1.6 Component: Bro | Version: git/master Keywords: | --------------------+------------------------ This probably isn't specific to Xcode, but that's where I notice the problem. In order for files to be automatically added to a generated IDE project, CMake has to have specified them in a call to `add_executable()`, `add_library()`, etc. In some cases, headers and other files that one would want to edit within the IDE, but that the compiler doesn't explicitly need to know about, were omitted from these calls, making the user have to manually add them to the project. Fixing this may also be a good time to decide whether to change the way source files are specified to calls to `add_executable()`, `add_library()`, etc. Either we can keep the hardcoded file list in the CMakeLists.txt or we can change it to use file globbing. The pros/cons of either method are discussed here: http://stackoverflow.com/questions/1027247/best-way-to-specify- sourcefiles-in-cmake I can see how one might get easily confused when trying to add a new source file if the globbing method is used. -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Wed Mar 16 09:45:26 2011 From: seth at icir.org (Seth Hall) Date: Wed, 16 Mar 2011 12:45:26 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/conn-ids: Extending conn_id with a globally unique identifiers. (881071c) In-Reply-To: <20110316163213.GF40299@icir.org> References: <201103160343.p2G3hV9e002149@bro-ids.icir.org> <20110316152506.GA40299@icir.org> <1F3B062F-A8F8-4DC8-A388-4E204C816998@icir.org> <20110316163213.GF40299@icir.org> Message-ID: <92263734-F4CA-4B0B-BA86-B5119CD4CB3E@icir.org> On Mar 16, 2011, at 12:32 PM, Robin Sommer wrote: > One could now actually use the uid as the table index ... However, > that wouldn't be as intuitive as using the whole conn_id and I don't > think I want to advocate that. Ok. > However, here's disruptive alternative: we could move > {orig,resp}{_h,_p} into the connection record and then use the unique > identifier as the "id" directly ... (Wouldn't do the automatic logging > of both though). Going the other direction, how about we put the uid in the connection record? We'd then have c$uid and c$id. That causes the least amount of mental and code disruption while accomplishing essentially the same thing. > Assuming fields are named consistenly, that would also work if the uid > were explicitly inlcuded into the log record. I need to think more about how this could be used as seamlessly as possible with the logging framework, I don't have a good idea yet. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.icir.org Wed Mar 16 09:51:37 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Wed, 16 Mar 2011 16:51:37 -0000 Subject: [Bro-Dev] #414: Xcode gives warnings about preprocessor redefinitions Message-ID: <045.54694f79c4c975422186a462eb5ed664@tracker.icir.org> #414: Xcode gives warnings about preprocessor redefinitions --------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: new Priority: Low | Milestone: Bro1.6 Component: Bro | Version: git/master Keywords: | --------------------+------------------------ Might not be limited to Xcode, but that's where I notice them. Many of them are in the CMake-generated `config.h` where the easiest way to convey the result of some of the configure tests was done like: {{{ #define int32_t int32_t }}} Which should be harmless, but could probably be done more cleanly in the CMake scripts. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Wed Mar 16 09:51:38 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Wed, 16 Mar 2011 16:51:38 -0000 Subject: [Bro-Dev] #415: Xcode gives warnings about preprocessor redefinitions Message-ID: <045.a96b79c30c93cd25613bf217f7c0042c@tracker.icir.org> #415: Xcode gives warnings about preprocessor redefinitions --------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: new Priority: Low | Milestone: Bro1.6 Component: Bro | Version: git/master Keywords: | --------------------+------------------------ Might not be limited to Xcode, but that's where I notice them. Many of them are in the CMake-generated `config.h` where the easiest way to convey the result of some of the configure tests was done like: {{{ #define int32_t int32_t }}} Which should be harmless, but could probably be done more cleanly in the CMake scripts. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Wed Mar 16 09:54:33 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Wed, 16 Mar 2011 16:54:33 -0000 Subject: [Bro-Dev] #415: Xcode gives warnings about preprocessor redefinitions In-Reply-To: <045.a96b79c30c93cd25613bf217f7c0042c@tracker.icir.org> References: <045.a96b79c30c93cd25613bf217f7c0042c@tracker.icir.org> Message-ID: <060.daca7dac4764b05db13e8fba04629514@tracker.icir.org> #415: Xcode gives warnings about preprocessor redefinitions ------------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: closed Priority: Low | Milestone: Bro1.6 Component: Bro | Version: git/master Resolution: Duplicate | Keywords: ------------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Duplicate Comment: Not sure how this duplicate got created. See #414 instead -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Wed Mar 16 10:13:23 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Wed, 16 Mar 2011 17:13:23 -0000 Subject: [Bro-Dev] #396: cmake does not create broccoli-config with execute perms In-Reply-To: <044.1a71c7aef9a42b869853de5f3c9b5321@tracker.icir.org> References: <044.1a71c7aef9a42b869853de5f3c9b5321@tracker.icir.org> Message-ID: <059.ce10cce88e5fbccd46d6c2808e778dbc@tracker.icir.org> #396: cmake does not create broccoli-config with execute perms -----------------------+----------------------- Reporter: leres | Owner: jsiwek Type: Problem | Status: closed Priority: Normal | Milestone: Component: Broccoli | Version: git/topic Resolution: Solved | Keywords: -----------------------+----------------------- Changes (by jsiwek): * status: assigned => closed * resolution: => Solved Comment: This was resolved by: http://git.bro- ids.org/broccoli.git/commit/452f6f93486f8937d8ef5cf2edb46bf8926f5ed6 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Wed Mar 16 13:28:56 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Wed, 16 Mar 2011 20:28:56 -0000 Subject: [Bro-Dev] #408: Unique connection ID for bro In-Reply-To: <045.d1dcc001adbef43f9bd34f06ae252340@tracker.icir.org> References: <045.d1dcc001adbef43f9bd34f06ae252340@tracker.icir.org> Message-ID: <060.d998829e1a6c40d6043f16c8e832b55c@tracker.icir.org> #408: Unique connection ID for bro ------------------------------+------------------------ Reporter: gregor | Owner: robin Type: Feature Request | Status: accepted Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => accepted * milestone: => Bro1.6 Comment: This is being worked on in topic/robin/conn-ids. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Wed Mar 16 17:54:46 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Thu, 17 Mar 2011 00:54:46 -0000 Subject: [Bro-Dev] #297: Remove trace rewriter In-Reply-To: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> References: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> Message-ID: <058.25cdaafd2e17aff6add90a7cd73da6ca@tracker.icir.org> #297: Remove trace rewriter ---------------------+--------------------- Reporter: seth | Owner: robin Type: Task | Status: testing Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Resolution: | Keywords: cleanup ---------------------+--------------------- Changes (by robin): * status: accepted => testing Comment: This is in topic/robin/cleanup-rewriter. We can make a patch out of this branch later. However, I'm not sure whether we can really follow the original idea of making a patch that applies to 1.6: with all the scripts being changed, that will probably be quite impractical. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Wed Mar 16 19:22:48 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Thu, 17 Mar 2011 02:22:48 -0000 Subject: [Bro-Dev] #297: Remove trace rewriter In-Reply-To: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> References: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> Message-ID: <058.ff66fcb4b857938212770a1d98e54d57@tracker.icir.org> #297: Remove trace rewriter ---------------------+--------------------- Reporter: seth | Owner: robin Type: Task | Status: testing Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Resolution: | Keywords: cleanup ---------------------+--------------------- Comment (by vern): Hrumph, as that was the original "deal". I don't quite understand the comment about "probably be quite impractical". Why is that? -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Thu Mar 17 06:35:40 2011 From: seth at icir.org (Seth Hall) Date: Thu, 17 Mar 2011 09:35:40 -0400 Subject: [Bro-Dev] enum namespacing Message-ID: <35A08662-D2CF-45D5-A3E8-BE09D1D49091@icir.org> Does it makes sense that an enum would be in the name space it's declared within even if it's extending an enum type from another namespace? My example: module Software; export { type Type: enum { UNKNOWN }; } module HTTP; redef enum Software::Type += { WEB_SERVER }; event bro_init() { # I would expect this to be true, but it seems that WEB_SERVER was placed into the HTTP namespace and this doesn't work. print Software::WEB_SERVER; } .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From jsiwek at ncsa.illinois.edu Thu Mar 17 08:49:58 2011 From: jsiwek at ncsa.illinois.edu (Jonathan Siwek) Date: Thu, 17 Mar 2011 10:49:58 -0500 (CDT) Subject: [Bro-Dev] whitespace convention (tab vs. space) In-Reply-To: <16987863.101.1300375835255.JavaMail.jsiwek@tangent.ncsa.illinois.edu> Message-ID: <29856286.113.1300376994341.JavaMail.jsiwek@tangent.ncsa.illinois.edu> I'm having trouble deciding when to use tabs versus spaces for leading whitespace... * Existing C/C++ source files seem to mostly use tabs, but it's mixed in some places. * Python (broctl) source files seem to be mostly spaces * reST source files for documentation seem to be spaces * Bro policy script source files look like they mostly use tabs Not sure if I missed any cases, but what's standard convention, if there is one? I kind of dislike tabs in any case, but even more than that I dislike mixing tabs/spaces or having to switch between the two frequently. I'm not sure if anyone has strong opinions, but really all I'm looking for is some general rules that I should follow. - Jon From jsiwek at ncsa.illinois.edu Thu Mar 17 08:51:43 2011 From: jsiwek at ncsa.illinois.edu (Jonathan Siwek) Date: Thu, 17 Mar 2011 10:51:43 -0500 (CDT) Subject: [Bro-Dev] whitespace convention (tab vs. space) In-Reply-To: <29856286.113.1300376994341.JavaMail.jsiwek@tangent.ncsa.illinois.edu> Message-ID: <25372530.115.1300377099936.JavaMail.jsiwek@tangent.ncsa.illinois.edu> > Not sure if I missed any cases Oh, * CMake scripts were written with spaces From robin at icir.org Thu Mar 17 09:16:37 2011 From: robin at icir.org (Robin Sommer) Date: Thu, 17 Mar 2011 09:16:37 -0700 Subject: [Bro-Dev] whitespace convention (tab vs. space) In-Reply-To: <29856286.113.1300376994341.JavaMail.jsiwek@tangent.ncsa.illinois.edu> References: <16987863.101.1300375835255.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <29856286.113.1300376994341.JavaMail.jsiwek@tangent.ncsa.illinois.edu> Message-ID: <20110317161637.GE82025@icir.org> On Thu, Mar 17, 2011 at 10:49 -0500, you wrote: > I'm having trouble deciding when to use tabs versus spaces for leading whitespace... Not surprising. :) The underlying rules are: > * Existing C/C++ source files seem to mostly use tabs, but it's mixed in some places. > * Bro policy script source files look like they mostly use tabs Indentation-style due to Vern -> Tabs. However, sometimes spaces are slipping in via contributions from others (including me). That can (and should) be normalized to tabs when noticed. > * Python (broctl) source files seem to be mostly spaces > * reST source files for documentation seem to be spaces Indentation-style due to Robin -> Spaces. > Not sure if I missed any cases, but what's standard convention, if there is one? I guess the implicit rule so far was that subprojects have their own style conventions. > I kind of dislike tabs in any case, but even more than that I dislike > mixing tabs/spaces or having to switch between the two frequently. I don't like tabs either but as you say, it's even worse to mix. Btw, there's an item on our todo-list-once-we-find-a-student: finding a tool that we can pipe all the source code through to canonify formatting. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.icir.org Thu Mar 17 09:34:31 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Thu, 17 Mar 2011 16:34:31 -0000 Subject: [Bro-Dev] #297: Remove trace rewriter In-Reply-To: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> References: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> Message-ID: <058.b01f43717060f24f62cea49cb19b851f@tracker.icir.org> #297: Remove trace rewriter ---------------------+--------------------- Reporter: seth | Owner: robin Type: Task | Status: testing Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Resolution: | Keywords: cleanup ---------------------+--------------------- Comment (by robin): > I don't quite understand the comment about "probably be quite > impractical". Why is that? The problem is that with the scripts changing a lot, a simple patch doesn't really work: if it just adds the rewriting-specific parts to the new code, that is unlikely to work in any reasonable way due to the restructuring of logic and data structures. A patch that gets us back to the current functionality would essentially need to revert all the scripts changes and go back to their old versions. What I could see is: (1) we create a patch that re-adds all the old versions of the scripts in a separate directory, like "old-for-rewriting". (2) our patch does actually not add rewriting to 1.6, but *removes* rewriting from 1.5. Rewriting is already not working correctly in 1.5, and I think the orginal idea was mainly to have reference of the code as it used to be. Having the remove-from-1.5 patch can play that role too. Anybody who actually wants to use it with 1.6 needs to invest quite a bit work anyway. -- Ticket URL: Bro Tracker Bro Issue Tracker From leres at ee.lbl.gov Thu Mar 17 10:10:49 2011 From: leres at ee.lbl.gov (Craig Leres) Date: Thu, 17 Mar 2011 10:10:49 -0700 Subject: [Bro-Dev] whitespace convention (tab vs. space) In-Reply-To: <20110317161637.GE82025@icir.org> References: <16987863.101.1300375835255.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <29856286.113.1300376994341.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <20110317161637.GE82025@icir.org> Message-ID: <4D824099.3060404@ee.lbl.gov> > Btw, there's an item on our todo-list-once-we-find-a-student: finding > a tool that we can pipe all the source code through to canonify > formatting. There's always expand/unexpand. And I would guess that git has a checkin hook so you it should be easy to force this for leading whitespace without being too annoying. Craig From bernhard+bro at net.t-labs.tu-berlin.de Thu Mar 17 10:11:38 2011 From: bernhard+bro at net.t-labs.tu-berlin.de (Bernhard Ager) Date: Thu, 17 Mar 2011 18:11:38 +0100 Subject: [Bro-Dev] #297: Remove trace rewriter In-Reply-To: <058.b01f43717060f24f62cea49cb19b851f@tracker.icir.org> References: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> <058.b01f43717060f24f62cea49cb19b851f@tracker.icir.org> Message-ID: <20110317171138.GN30176@in.tum.de> On Thu, Mar 17, 2011 at 09:16:37AM -0700, Robin Sommer wrote: > > * Python (broctl) source files seem to be mostly spaces reST > > * source files for documentation seem to be spaces > > Indentation-style due to Robin -> Spaces. Spaces only, btw, are recommended for Python in PEP-8, "Style Guide for Python Code", . It's a terribly good idea to adhere to this recommendation because it's the prevalent use and because of Python's definition of blocks with whitespace. Diverting usually results in increased pain when using Python code from multiple origins. For further information check PEP-666 :-) Bernhard -- Technische Universit?t Berlin An-Institut Deutsche Telekom Laboratories FG INET, Research Group Anja Feldmann Sekr. TEL 4 Ernst-Reuter-Platz 7 D-10587 Berlin From robin at icir.org Thu Mar 17 10:17:33 2011 From: robin at icir.org (Robin Sommer) Date: Thu, 17 Mar 2011 10:17:33 -0700 Subject: [Bro-Dev] whitespace convention (tab vs. space) In-Reply-To: <4D824099.3060404@ee.lbl.gov> References: <16987863.101.1300375835255.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <29856286.113.1300376994341.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <20110317161637.GE82025@icir.org> <4D824099.3060404@ee.lbl.gov> Message-ID: <20110317171733.GB86111@icir.org> On Thu, Mar 17, 2011 at 10:10 -0700, you wrote: > There's always expand/unexpand. Without having used them, I think going from spaces to tabs automatically is tricky, isn't it? Or said differently: just by doing the conversion, one doesn't necessarily get the layout "right". > And I would guess that git has a checkin hook so you it should be easy > to force this for leading whitespace without being too annoying. That's right, and if we have a consensus that we want to enforce this, I can activate it. Note however that when I wrote "canonyfing formatting" I was thinking more broadly in terms of all the little things everybody does differently with source code (parentheses placement, level of indentation, etc.). A good candidate for doing that seems to be uncrustify[1] but that needs somebody spending some time with its plenty of options. Robin [1] http://uncrustify.sourceforge.net/ -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Thu Mar 17 14:57:46 2011 From: robin at icir.org (Robin Sommer) Date: Thu, 17 Mar 2011 14:57:46 -0700 Subject: [Bro-Dev] active_conns Message-ID: <20110317215746.GG86111@icir.org> Seth, just looking through the new conn.bro, and I have a philosophical question about this piece: # This is where users can get access to the active Log record for a # connection so they can extend and enhance the logged data. global active_conns: table[conn_id] of Log; What kind of data do you see this extended with? I'm asking because one thing that always struck me as suboptimal is how currently many scripts are maintaining their own session table. E.g., the HTTP analyzer has http_sessions[conn_id] where it's stores its stuff. With the new record extension mechanisms we could instead do the other extreme: no script gets its own table anymore, the additional things just get added to a central record, like this Conn::Log. I'm not sure whether I really want to advocate that change but I was wondering what your (or anybodys) thoughts are. (Note that if it were really Conn::*Log* that gets extended, this would interfere with logging obvioysly. But we could separate the two notions, and just have a central Connection record which everybody extends.). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From gregor at icir.org Thu Mar 17 15:25:25 2011 From: gregor at icir.org (Gregor Maier) Date: Thu, 17 Mar 2011 15:25:25 -0700 Subject: [Bro-Dev] whitespace convention (tab vs. space) In-Reply-To: <20110317171733.GB86111@icir.org> References: <16987863.101.1300375835255.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <29856286.113.1300376994341.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <20110317161637.GE82025@icir.org> <4D824099.3060404@ee.lbl.gov> <20110317171733.GB86111@icir.org> Message-ID: <4D828A55.7080305@icir.org> On 3/17/11 10:17 , Robin Sommer wrote: > > On Thu, Mar 17, 2011 at 10:10 -0700, you wrote: > >> There's always expand/unexpand. > > Without having used them, I think going from spaces to tabs > automatically is tricky, isn't it? Or said differently: just by doing > the conversion, one doesn't necessarily get the layout "right". Yeah. That's usually the problem. Since space<-->tab conversion depends on the tab-width that should be used. So I would not recommend do convert automatically it. But it might be nice to just have a checker, that can notify one if the indentation style in a file is "wrong". Then one can manually check and decide whether expand/unexpand is warranted. (Note that for code that uses tabs, mixing spaces and tabs makes sense in one particular case: continuations: this_is_a_function (it, has, very_very_very_many, ... arguments, more_args, foo, bar); just my 2ct Gregor BTW: I prefer tabs (but python more or less forces one to use spaces....) -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From leres at ee.lbl.gov Thu Mar 17 15:29:24 2011 From: leres at ee.lbl.gov (Craig Leres) Date: Thu, 17 Mar 2011 15:29:24 -0700 Subject: [Bro-Dev] whitespace convention (tab vs. space) In-Reply-To: <4D828A55.7080305@icir.org> References: <16987863.101.1300375835255.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <29856286.113.1300376994341.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <20110317161637.GE82025@icir.org> <4D824099.3060404@ee.lbl.gov> <20110317171733.GB86111@icir.org> <4D828A55.7080305@icir.org> Message-ID: <4D828B44.4030404@ee.lbl.gov> > Yeah. That's usually the problem. Since space<-->tab conversion depends > on the tab-width that should be used. So I would not recommend do > convert automatically it. My suggestion is to balk when you attempt to and commit a change with the leading whitespace that violates the policy. And really: if anybody is using a tabwidth of other than 8, his/her code already is messed up for the rest of us. Craig From gregor at icir.org Thu Mar 17 16:41:49 2011 From: gregor at icir.org (Gregor Maier) Date: Thu, 17 Mar 2011 16:41:49 -0700 Subject: [Bro-Dev] active_conns In-Reply-To: <20110317215746.GG86111@icir.org> References: <20110317215746.GG86111@icir.org> Message-ID: <4D829C3D.6030705@icir.org> > I'm asking because one thing that always struck me as suboptimal is > how currently many scripts are maintaining their own session table. > E.g., the HTTP analyzer has http_sessions[conn_id] where it's stores > its stuff. I agree. > With the new record extension mechanisms we could instead do the other > extreme: no script gets its own table anymore, the additional things > just get added to a central record, like this Conn::Log. I'm not sure > whether I really want to advocate that change but I was wondering what > your (or anybodys) thoughts are. I like the idea, however, I guess one question would be what the memory overhead would be. Assuming that you have many analyzers or scripts that want to add stuff to *some* connections. If the connection record is extended every connection needs to have the additional field. Probably as an optional, so it's only a NULL pointer, but still. cu gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From seth at icir.org Thu Mar 17 17:25:40 2011 From: seth at icir.org (Seth Hall) Date: Thu, 17 Mar 2011 20:25:40 -0400 Subject: [Bro-Dev] whitespace convention (tab vs. space) In-Reply-To: <4D828B44.4030404@ee.lbl.gov> References: <16987863.101.1300375835255.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <29856286.113.1300376994341.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <20110317161637.GE82025@icir.org> <4D824099.3060404@ee.lbl.gov> <20110317171733.GB86111@icir.org> <4D828A55.7080305@icir.org> <4D828B44.4030404@ee.lbl.gov> Message-ID: <797038F9-7D91-4219-ACB6-635706A5B9E7@icir.org> On Mar 17, 2011, at 6:29 PM, Craig Leres wrote: > And really: if anybody is using a tabwidth of other than 8, his/her code > already is messed up for the rest of us. I use the style that Gregor showed so changing the width doesn't affect anything.... > (Note that for code that uses tabs, mixing spaces and tabs makes sense > in one particular case: continuations: > this_is_a_function (it, has, very_very_very_many, > ... arguments, more_args, foo, bar); .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Thu Mar 17 18:34:03 2011 From: seth at icir.org (Seth Hall) Date: Thu, 17 Mar 2011 21:34:03 -0400 Subject: [Bro-Dev] active_conns In-Reply-To: <20110317215746.GG86111@icir.org> References: <20110317215746.GG86111@icir.org> Message-ID: <262D507B-82B3-4D3D-B008-49D1CA7B8827@icir.org> On Mar 17, 2011, at 5:57 PM, Robin Sommer wrote: > # This is where users can get access to the active Log record for a > # connection so they can extend and enhance the logged data. > global active_conns: table[conn_id] of Log; > > What kind of data do you see this extended with? Initially the stuff that would have gone into the $addl field before. I don't like that field much and a lot of different scripts seem to want to add various and fairly unstructured things to it. I'm not *totally* sure how the extension would play out with the conn.bro script, but it also didn't feel right for that script to not be extensible like all of the other scripts are and will be. I've really been trying to push myself toward consistency across all of the scripts so that without even reading the documentation or source, someone would be able to guess how each of the shipped scripts works and what variables will be named if they know the general convention. I think that the existence of the active_conns variable is documented in one of the manuals too, but that may have been from the active.bro script that was removed with release 1.1. I don't exactly like the extension model applied to the conn.bro script either, but it has the benefit of being regular (i.e. like the other new scripts). > I'm asking because one thing that always struck me as suboptimal is > how currently many scripts are maintaining their own session table. > E.g., the HTTP analyzer has http_sessions[conn_id] where it's stores > its stuff. Hm, I guess I hadn't even thought about that at all > With the new record extension mechanisms we could instead do the other > extreme: no script gets its own table anymore, the additional things > just get added to a central record, like this Conn::Log. Hm, I hadn't even considered it from that approach. I'll think about it some more. > (Note that if it were really Conn::*Log* that gets extended, this > would interfere with logging obvioysly. But we could separate the two > notions, and just have a central Connection record which everybody > extends.). This is actually the model I've already moved to with most of the other scripts already. I create an ::Info type that is kept in a conn_id indexed table, then inside the Info variable, there is an instance of a ::Log type. Data is stored sort of haphazardly (which I don't particularly like) in either ::Info or ::Log depending on if it's an internal state tracking detail or if it's something that would conceivably ever need to be logged. What if we make a Conn::Info (maybe different name? I'm awful at naming) type that contains a Conn::Log type record that we could extend that with all of the ::Info typed variables for each individual script? I think I may have to implement it to see how it looks and how it would be used. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From jsiwek at ncsa.illinois.edu Fri Mar 18 07:46:51 2011 From: jsiwek at ncsa.illinois.edu (Jonathan Siwek) Date: Fri, 18 Mar 2011 09:46:51 -0500 (CDT) Subject: [Bro-Dev] whitespace convention (tab vs. space) In-Reply-To: <4D828A55.7080305@icir.org> Message-ID: <5002259.130.1300459611581.JavaMail.jsiwek@tangent.ncsa.illinois.edu> > (Note that for code that uses tabs, mixing spaces and tabs makes sense > in one particular case: continuations: > this_is_a_function (it, has, very_very_very_many, > ... arguments, more_args, foo, bar); Yeah, I agree about that. The case I think we were talking about would be mixing leading whitespace styles. e.g. if there's a function formatted like you had with tabs, then it would be bad to mix in some more code that looks like ... this_is_another_function (it, has, very_very_very_many, ......................... arguments, more_args, foo, bar); - Jon From robin at icir.org Fri Mar 18 08:30:05 2011 From: robin at icir.org (Robin Sommer) Date: Fri, 18 Mar 2011 08:30:05 -0700 Subject: [Bro-Dev] active_conns In-Reply-To: <4D829C3D.6030705@icir.org> References: <20110317215746.GG86111@icir.org> <4D829C3D.6030705@icir.org> Message-ID: <20110318153005.GE6542@icir.org> On Thu, Mar 17, 2011 at 16:41 -0700, you wrote: > I like the idea, however, I guess one question would be what the memory > overhead would be. Wouldn't be very concerned about that. It's not much per connection (compared to how much we already store ...), and just by not having some of the other high volume tables (like http_sessions) we'd probably compensate for quite a bit alredy. That said, I'm still not argueing for this. I also like Seth's current model. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Fri Mar 18 08:40:33 2011 From: robin at icir.org (Robin Sommer) Date: Fri, 18 Mar 2011 08:40:33 -0700 Subject: [Bro-Dev] whitespace convention (tab vs. space) In-Reply-To: <4D828A55.7080305@icir.org> References: <16987863.101.1300375835255.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <29856286.113.1300376994341.JavaMail.jsiwek@tangent.ncsa.illinois.edu> <20110317161637.GE82025@icir.org> <4D824099.3060404@ee.lbl.gov> <20110317171733.GB86111@icir.org> <4D828A55.7080305@icir.org> Message-ID: <20110318154033.GG6542@icir.org> On Thu, Mar 17, 2011 at 15:25 -0700, you wrote: > this_is_a_function (it, has, very_very_very_many, > ... arguments, more_args, foo, bar); Just for the record, Bro code is not using this: it uses TABs exclusively also for the second line. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Fri Mar 18 18:48:51 2011 From: seth at icir.org (Seth Hall) Date: Fri, 18 Mar 2011 21:48:51 -0400 Subject: [Bro-Dev] active_conns In-Reply-To: <20110318153005.GE6542@icir.org> References: <20110317215746.GG86111@icir.org> <4D829C3D.6030705@icir.org> <20110318153005.GE6542@icir.org> Message-ID: <5134EB6C-96AC-48E5-84A7-89117AD33C94@icir.org> On Mar 18, 2011, at 11:30 AM, Robin Sommer wrote: > On Thu, Mar 17, 2011 at 16:41 -0700, you wrote: > >> I like the idea, however, I guess one question would be what the memory >> overhead would be. > > Wouldn't be very concerned about that. It's not much per connection > (compared to how much we already store ...) I'm sort of tempted in the base scripts to maintain fairly minimal logging and then have other scripts that add lots of extra state if people want it, but I also really want to avoid making things complicated again. I like being able to do http logging by just loading "http". Perhaps we could have these state extension scripts loaded by default, but if you load some other script (minimal-logging.bro?), it would not load the extended state scripts and give you very minimal state tracking for if you are doing something that doesn't need the full load of information and you have some memory constraint? Of course, that's increasing the complexity of the base scripts again. If someone really has the need to conserve memory that badly, they could always trim code out of the shipped scripts which should be quite a bit easier with the new scripts (not that I'd *want* people to edit the base scripts, it's just that they could). Most of those state tables don't occupy *that* much memory at any one point in time anyway since they're cleaned up after the connection. > , and just by not having > some of the other high volume tables (like http_sessions) we'd > probably compensate for quite a bit alredy. I was just about to write here that I would like to keep it as it is for now, but then I started writing some example code to see how it would look if it was stored in the connection record and I sort of like it. State stored in connection record: print c$http_session$log$method; State stored in separate global table: print HTTP::active_conns[c$id]$log$method; What I *don't* like about this model is that it breaks down when you have data that is stored about something outside of a connection. My current example for this is TLS session IDs which are tracked in a state table so that the same previously established TLS session ID can be used in a different TCP connection. You would then have to resort to storing the sessions IDs in a global table which makes the data storage less regular. It may be worth it to get the syntax benefits though. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Fri Mar 18 19:22:44 2011 From: seth at icir.org (Seth Hall) Date: Fri, 18 Mar 2011 22:22:44 -0400 Subject: [Bro-Dev] template language Message-ID: <764A07DC-923C-4C65-A56C-CB15C844B4D5@icir.org> I know some of you will laugh about this feature request, but I'm going to ask for it anyway. :) I would like a BiF (or something) that would implement a minimal template language. If it used the state tracking records to pull field values from, that would be even better! I'll give an example... type Log: record { a: count; b: string; c: addr; }; global abc: Log = [$a=1, $b="test", $c=1.2.3.4]; print template_fmt("{{a}}-{{b}}-{{c}}", abc); This would print... 1-test-1.2.3.4 I already want to use something like this to implement file extraction one-liners so that people can arbitrarily name extracted files from the command line like this: bro -r traffic.trace -e "HTTP_Extract::file_types=/application.*/ HTTP_Extract::file_name_template=\"{{id}}-{{url}}-{{filename}}.exe\"" http-extract I think it would be useful in other ways too. Especially if we end up moving to these state records that have a lot of data in them. Any thoughts? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.icir.org Sat Mar 19 20:21:08 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Sun, 20 Mar 2011 03:21:08 -0000 Subject: [Bro-Dev] #297: Remove trace rewriter In-Reply-To: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> References: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> Message-ID: <058.b8c4aab418018f3a6a1845f1c026184b@tracker.icir.org> #297: Remove trace rewriter ---------------------+--------------------- Reporter: seth | Owner: robin Type: Task | Status: testing Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Resolution: | Keywords: cleanup ---------------------+--------------------- Comment (by vern): Okay, I see your point. I still find it a pity to lose the bragging rights to this functionality, since it's pretty darn cool .... -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Sun Mar 20 21:33:27 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 21 Mar 2011 04:33:27 -0000 Subject: [Bro-Dev] #297: Remove trace rewriter In-Reply-To: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> References: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> Message-ID: <058.25d1717c22c967ed010dcdece22a2b8c@tracker.icir.org> #297: Remove trace rewriter ---------------------+--------------------- Reporter: seth | Owner: robin Type: Task | Status: testing Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Resolution: | Keywords: cleanup ---------------------+--------------------- Comment (by robin): On Sun, Mar 20, 2011 at 03:21 -0000, you wrote: > Okay, I see your point. I still find it a pity to lose the bragging > rights to this functionality, since it's pretty darn cool .... Long-term that functionality might actually fit very nicely into Binpac++ ... -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Sun Mar 20 21:37:17 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 21 Mar 2011 04:37:17 -0000 Subject: [Bro-Dev] #297: Remove trace rewriter In-Reply-To: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> References: <043.f4e0eb0a9924070cf852ace700beb6bf@tracker.icir.org> Message-ID: <058.a76212c45476bb1c7581fe26088ec7d3@tracker.icir.org> #297: Remove trace rewriter ---------------------+--------------------- Reporter: seth | Owner: robin Type: Task | Status: testing Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Resolution: | Keywords: cleanup ---------------------+--------------------- Comment (by vern): Re BinPAC++: That's indeed exactly my hope, too. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 21 08:05:30 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 21 Mar 2011 15:05:30 -0000 Subject: [Bro-Dev] #416: topic/jsiwek/config-file-clobber-fixes Message-ID: <045.5ff90cc821a3d42b5c51497b6878605b@tracker.icir.org> #416: topic/jsiwek/config-file-clobber-fixes ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Low | Milestone: Bro1.6 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch exists in the broccoli, broctl, and bro repositories. The changes/fixes are listed in ticket:405#comment:4 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 21 08:07:30 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 21 Mar 2011 15:07:30 -0000 Subject: [Bro-Dev] #405: Broccoli's `make install` may clobber existing user-modified broccoli.conf In-Reply-To: <045.4bf2ca2c1472064fe96ebac9cd928d79@tracker.icir.org> References: <045.4bf2ca2c1472064fe96ebac9cd928d79@tracker.icir.org> Message-ID: <060.3b92a609ab2c70d700844db7cdd2f71c@tracker.icir.org> #405: Broccoli's `make install` may clobber existing user-modified broccoli.conf -----------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Problem | Status: new Priority: Normal | Milestone: Bro1.6 Component: Broccoli | Version: git/master Resolution: | Keywords: -----------------------+------------------------ Comment (by jsiwek): Craig confirmed the `topic/jsiwek/config-file-clobber-fixes` branch looks like it does as he expects. Merge request is #416. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Mon Mar 21 10:09:54 2011 From: robin at icir.org (Robin Sommer) Date: Mon, 21 Mar 2011 10:09:54 -0700 Subject: [Bro-Dev] active_conns In-Reply-To: <262D507B-82B3-4D3D-B008-49D1CA7B8827@icir.org> References: <20110317215746.GG86111@icir.org> <262D507B-82B3-4D3D-B008-49D1CA7B8827@icir.org> Message-ID: <20110321170954.GF620@icir.org> On Thu, Mar 17, 2011 at 21:34 -0400, you wrote: > instance of a ::Log type. Data is stored sort of haphazardly (which I > don't particularly like) in either ::Info or ::Log depending on if Yeah, that's bothering me a bit too, but I'm not sure how to solve it either. Well, here's an idea: with the new record-to-sub-record coercion, I believe we could store *all* information in the Info record, and still define a separate Log record for determining what gets logged (by default): type Info = record { count a; count b; count c; } type Log = record { count a; count b; } We use Log to create the logging stream, and Info to fill in the data. When time for logging comes around, we simply pass the Info instance into the Log:write() function and it will log whatever is defined in Log, via coercion. (Actually I don't believe this would work right now because of internals of the log bif, but it should be doable) Advantage: Only one type of record to fill in. Disadvantage: Two record types to define and maintain, both of which need to match for the logged fields (i.e., redundancy). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Mon Mar 21 10:11:26 2011 From: robin at icir.org (Robin Sommer) Date: Mon, 21 Mar 2011 10:11:26 -0700 Subject: [Bro-Dev] active_conns In-Reply-To: <5134EB6C-96AC-48E5-84A7-89117AD33C94@icir.org> References: <20110317215746.GG86111@icir.org> <4D829C3D.6030705@icir.org> <20110318153005.GE6542@icir.org> <5134EB6C-96AC-48E5-84A7-89117AD33C94@icir.org> Message-ID: <20110321171126.GG620@icir.org> On Fri, Mar 18, 2011 at 21:48 -0400, you wrote: > print c$http_session$log$method; This is kind of neat ... We could even extend c?$http_session to check whether the record type has that field at all, and then use that as a replacement for all the "is this script loaded?" hacks currently in use ... Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Mon Mar 21 10:21:13 2011 From: seth at icir.org (Seth Hall) Date: Mon, 21 Mar 2011 13:21:13 -0400 Subject: [Bro-Dev] active_conns In-Reply-To: <20110321171126.GG620@icir.org> References: <20110317215746.GG86111@icir.org> <4D829C3D.6030705@icir.org> <20110318153005.GE6542@icir.org> <5134EB6C-96AC-48E5-84A7-89117AD33C94@icir.org> <20110321171126.GG620@icir.org> Message-ID: On Mar 21, 2011, at 1:11 PM, Robin Sommer wrote: > On Fri, Mar 18, 2011 at 21:48 -0400, you wrote: > >> print c$http_session$log$method; > > This is kind of neat ... We could even extend c?$http_session to check > whether the record type has that field at all, and then use that as a > replacement for all the "is this script loaded?" hacks currently in > use ... Yeah, that's a good point. The approach I've been taking with the "is this script loaded?" hacks is to solve the problem a different way. It seems that many of those hacks are due to one of two things: 1. There is some sort of general library functionality that should probably always be loaded as a sort of base library. 2. The functionality was hacked into an existing script the fastest way possible. I think the script extension model should make it possible for us to extract a lot of the circular dependencies into separate scripts, but like you said, in the cases where it does make sense to use a "is this script loaded?" hack, checking based on the existence of the protocol specific field certainly makes things cleaner and more regular across all of the scripts. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From jsiwek at ncsa.illinois.edu Mon Mar 21 10:41:41 2011 From: jsiwek at ncsa.illinois.edu (Jonathan Siwek) Date: Mon, 21 Mar 2011 12:41:41 -0500 (CDT) Subject: [Bro-Dev] active_conns In-Reply-To: <20110321171126.GG620@icir.org> Message-ID: <16459762.228.1300729296661.JavaMail.jsiwek@tangent.ncsa.illinois.edu> > > print c$http_session$log$method; > > This is kind of neat ... We could even extend c?$http_session to check > whether the record type has that field at all, and then use that as a > replacement for all the "is this script loaded?" hacks currently in use ... Why can't the "is this script loaded?" functionality be implemented as a BIF that queries some global state that the lexical scanner built up at scan/parse-time ? - Jon From bro at tracker.icir.org Mon Mar 21 13:50:50 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 21 Mar 2011 20:50:50 -0000 Subject: [Bro-Dev] #417: Bro Crash: internal_error () at SSLInterpreter.cc:30 Message-ID: <046.25880635a2f99f9d4bc87ec6cbed2a63@tracker.icir.org> #417: Bro Crash: internal_error () at SSLInterpreter.cc:30 ---------------------+--------------------- Reporter: aashish | Type: Problem Status: new | Priority: Normal Milestone: | Component: Bro Version: 1.5.2 | Keywords: ---------------------+--------------------- Clusters 1 and 2 both crashed simultaneously with the following crash report: bro.30276.20201.core Core was generated by `bro'. Program terminated with signal 6, Aborted. #0 0x0000000081776fcc in kill () from /lib/libc.so.7 #0 0x0000000081776fcc in kill () from /lib/libc.so.7 #1 0x0000000081775dcb in abort () from /lib/libc.so.7 #2 0x000000000040b319 in internal_error () at SSLInterpreter.cc:30 #3 0x00000000005126ce in RemoteSerializer::InternalCommError (this=) at RemoteSerializer.cc:2726 #4 0x000000000051a581 in RemoteSerializer::Poll (this=0x7bb868, may_block=false) at RemoteSerializer.cc:1478 #5 0x000000000051ac53 in RemoteSerializer::NextTimestamp (this=0x7bb868, local_network_time=0x7fffffffe178) at RemoteSerializer.cc:1294 #6 0x00000000004d9055 in IOSourceRegistry::FindSoonest (this=0x783e60, ts=0x7fffffffe1d8) at IOSource.cc:61 #7 0x00000000004f4d1d in net_run () at Net.cc:509 #8 0x000000000040956c in main (argc=) at main.cc:999 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 21 13:59:06 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 21 Mar 2011 20:59:06 -0000 Subject: [Bro-Dev] #417: Bro Crash: internal_error () at SSLInterpreter.cc:30 In-Reply-To: <046.25880635a2f99f9d4bc87ec6cbed2a63@tracker.icir.org> References: <046.25880635a2f99f9d4bc87ec6cbed2a63@tracker.icir.org> Message-ID: <061.7201d7630da5874a470891b783e0bf1f@tracker.icir.org> #417: Bro Crash: internal_error () at SSLInterpreter.cc:30 ----------------------+------------------- Reporter: aashish | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: 1.5.2 Resolution: | Keywords: ----------------------+------------------- Comment (by seth): Did you have anything in your stdout.log or stderr.log logs? This looks like a communications system overload to me. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Mon Mar 21 15:32:29 2011 From: robin at icir.org (Robin Sommer) Date: Mon, 21 Mar 2011 15:32:29 -0700 Subject: [Bro-Dev] active_conns In-Reply-To: <16459762.228.1300729296661.JavaMail.jsiwek@tangent.ncsa.illinois.edu> References: <20110321171126.GG620@icir.org> <16459762.228.1300729296661.JavaMail.jsiwek@tangent.ncsa.illinois.edu> Message-ID: <20110321223229.GB34894@icir.org> On Mon, Mar 21, 2011 at 12:41 -0500, you wrote: > Why can't the "is this script loaded?" functionality be implemented as > a BIF that queries some global state that the lexical scanner built up > at scan/parse-time ? I could, but I'd still count that as a hack because to be more precise, it's not really about the *script* being loaded but more about the functionality being available. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From asharma at lbl.gov Mon Mar 21 15:41:57 2011 From: asharma at lbl.gov (Aashish Sharma) Date: Mon, 21 Mar 2011 17:41:57 -0500 Subject: [Bro-Dev] #417: Bro Crash: internal_error () at SSLInterpreter.cc:30 In-Reply-To: <061.7201d7630da5874a470891b783e0bf1f@tracker.icir.org> References: <046.25880635a2f99f9d4bc87ec6cbed2a63@tracker.icir.org> <061.7201d7630da5874a470891b783e0bf1f@tracker.icir.org> Message-ID: <20110321224156.GJ3802@yaksha.lbl.gov> Seth: Nope. there isn't much in the stderr.log or stdout.log in either cluster. I do see following but I don't belive that has to do with the crashes. /home/users/bro/bro-152/share/broctl/scripts/create-link-for-log: line 60: .link./home/users/bro/logs/tm-contents/conn.46-4c59-cdb1.213.239.193.15.56667-128.3.28.180.80.resp.dat: No such file or directory As per Robin's email (off thread), I am planning to remove &synchronized from cluster IRC analyzer and see how system behaves. Aashish On Mon, Mar 21, 2011 at 08:59:06PM -0000, Bro Tracker wrote: > #417: Bro Crash: internal_error () at SSLInterpreter.cc:30 > ----------------------+------------------- > Reporter: aashish | Owner: > Type: Problem | Status: new > Priority: Normal | Milestone: > Component: Bro | Version: 1.5.2 > Resolution: | Keywords: > ----------------------+------------------- > > Comment (by seth): > > Did you have anything in your stdout.log or stderr.log logs? This looks > like a communications system overload to me. > > -- > Ticket URL: > Bro Tracker > Bro Issue Tracker -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Information Technology Division Lawrence Berkeley National Laboratory http://www.lbl.gov/cyber/pgp-aashish.txt Office: (510)-495-2680 Cell: (510)-457-1525 From bro at tracker.icir.org Mon Mar 21 15:42:49 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 21 Mar 2011 22:42:49 -0000 Subject: [Bro-Dev] #417: Bro Crash: internal_error () at SSLInterpreter.cc:30 In-Reply-To: <046.25880635a2f99f9d4bc87ec6cbed2a63@tracker.icir.org> References: <046.25880635a2f99f9d4bc87ec6cbed2a63@tracker.icir.org> Message-ID: <061.91a60ffaa4fc4d5a3be2755bbf1eb6b4@tracker.icir.org> #417: Bro Crash: internal_error () at SSLInterpreter.cc:30 ----------------------+------------------- Reporter: aashish | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: 1.5.2 Resolution: | Keywords: ----------------------+------------------- Comment (by aashish): Seth: Nope. there isn't much in the stderr.log or stdout.log in either cluster. I do see following but I don't belive that has to do with the crashes. /home/users/bro/bro-152/share/broctl/scripts/create-link-for-log: line 60: .link./home/users/bro/logs/tm- contents/conn.46-4c59-cdb1.213.239.193.15.56667-128.3.28.180.80.resp.dat: No such file or directory As per Robin's email (off thread), I am planning to remove &synchronized from cluster IRC analyzer and see how system behaves. Aashish On Mon, Mar 21, 2011 at 08:59:06PM -0000, Bro Tracker wrote: > #417: Bro Crash: internal_error () at SSLInterpreter.cc:30 > ----------------------+------------------- > Reporter: aashish | Owner: > Type: Problem | Status: new > Priority: Normal | Milestone: > Component: Bro | Version: 1.5.2 > Resolution: | Keywords: > ----------------------+------------------- > > Comment (by seth): > > Did you have anything in your stdout.log or stderr.log logs? This looks > like a communications system overload to me. > > -- > Ticket URL: > Bro Tracker > Bro Issue Tracker -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 21 16:35:52 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 21 Mar 2011 23:35:52 -0000 Subject: [Bro-Dev] #416: topic/jsiwek/config-file-clobber-fixes In-Reply-To: <045.5ff90cc821a3d42b5c51497b6878605b@tracker.icir.org> References: <045.5ff90cc821a3d42b5c51497b6878605b@tracker.icir.org> Message-ID: <060.2a610d8f14b6992bf5f86899e7eb5b5b@tracker.icir.org> #416: topic/jsiwek/config-file-clobber-fixes -----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: closed Priority: Low | Milestone: Bro1.6 Component: Bro | Version: git/master Resolution: Merged/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Merged/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 21 17:27:52 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 22 Mar 2011 00:27:52 -0000 Subject: [Bro-Dev] #374: Add type testing operator In-Reply-To: <043.8b359cb3ae787df001044e74fe896074@tracker.icir.org> References: <043.8b359cb3ae787df001044e74fe896074@tracker.icir.org> Message-ID: <058.acff30b4c07c49c7978743818716a958@tracker.icir.org> #374: Add type testing operator ------------------------------+--------------------- Reporter: seth | Owner: robin Type: Feature Request | Status: testing Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Resolution: | Keywords: logging ------------------------------+--------------------- Comment (by robin): The code has now been merged into `topic/robin/logging-internals`, and the original branch deleted. This was necessary because of overlapping changes in the two branches leading to serious conflicts. -- Ticket URL: Bro Tracker Bro Issue Tracker From vern at icir.org Mon Mar 21 20:12:39 2011 From: vern at icir.org (Vern Paxson) Date: Mon, 21 Mar 2011 20:12:39 -0700 Subject: [Bro-Dev] enum namespacing In-Reply-To: <35A08662-D2CF-45D5-A3E8-BE09D1D49091@icir.org> (Thu, 17 Mar 2011 09:35:40 EDT). Message-ID: <20110322031239.9A6C536A501@taffy.ICSI.Berkeley.EDU> > Does it makes sense that an enum would be in the name space it's declared > within even if it's extending an enum type from another namespace? Huh. I have a nagging sense that there's no correct answer to this one ... However, the behavior you show matches what seems to me more likely to accord with Principle Of Least Surprise. Vern From vern at icir.org Mon Mar 21 20:12:46 2011 From: vern at icir.org (Vern Paxson) Date: Mon, 21 Mar 2011 20:12:46 -0700 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110316042245.GA19883@icir.org> (Tue, 15 Mar 2011 21:22:45 PDT). Message-ID: <20110322031246.6B5CD36A50D@taffy.ICSI.Berkeley.EDU> > - There are actually a number of hash algorithms in Hash.{h,cc}, with > only one of them being used (via #ifdefs). Vern, do you remember the > story behind having multiple of them I'm pretty sure that Ruoming (or maybe even Yin Zhang, come to think of it) had several so he could do performance testing across them. > and can we just remove the code for those not enabled? Yeah, though hopefully we'll remember that we have them buried in the repository if at some point we find ourselves wanting to play with hash functions again. > - I'm wondering whether for the unique connection ids it would make > sense to make them stable in the case that we're working offline from > a trace. I would definitely like that! Vern From seth at icir.org Tue Mar 22 06:24:49 2011 From: seth at icir.org (Seth Hall) Date: Tue, 22 Mar 2011 09:24:49 -0400 Subject: [Bro-Dev] &log attribute Message-ID: There is a clarity issue with the current style of the new scripts I've been writing with where the state is stored before it's logged. My typical model has been to define two record types; Module::Info and Module::Log. An example looks like this.... type Log: record { uri: string; host: string; }; type Info: record { log: Log; pending_requests: SomeOtherDataType; }; The state is stored rather haphazardly in either the Log record or the Info record depending on if it's ever going to be logged. The Log value is what ends up being sent onward to the logging framework, so by default all of it's fields are logged. The $pending_requests field can't be logging in any reasonable form and never really has any need to be logged so it's stored in the parent ::Info record. Robin an I discussed creating a new attribute to support this notion of data for use by the logging framework. If we make a &log attribute, that could transmit the intent of the field to the logging framework. Here's an example of how this would look... type Info: record { uri: string &log; host: string &log; pending_requests: SomeOtherDataType; }; Although that attribute is annoying use case specific, I think it really makes for a much cleaner interface for creating these records that are meant for carrying data forward to the point of logging considering that in many cases there is other data that needs carried forward but is never logged (like pending_requests). I'll give some examples of the two styles in use now.. Existing style.. active_http_session$log$uri = unescaped_URI; New style... active_http_session$uri = unescaped_URI; Those are sort of silly examples but I wanted to show them to point out how often the $log$ fragment is reused, but it really isn't anything but syntactical noise that shouldn't really be there. I think that the &log attribute would make the interfaces to these scripts quite a bit more palatable and enjoyable to use. Thoughts? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Tue Mar 22 06:38:28 2011 From: seth at icir.org (Seth Hall) Date: Tue, 22 Mar 2011 09:38:28 -0400 Subject: [Bro-Dev] extending connection record type Message-ID: This is sort of an extension on the previous email, but sort of a different topic at the same time. In the active_conns thread, Robin mentioned the idea of extending the connection record to include these state tracking records directly within it. At first I was going to say that I didn't like the idea because it made state tracking less "regular" in that the cases where state not related to a connection is being tracked, the fall back would be to use a global table with some index. As I thought about it more and wrote some code for how it would look to use it, I realized that I liked the idea more and more. For starters, non-connection oriented state tracking is a bit of an exception in many cases and secondly, there is quite a bit less indexing into tables that needs to be done, and lastly, the syntax is much cleaner and nicer to look at and use. Here are some examples: Currently... HTTP::active_conns[c$id]$log$uri; By extending the connection record... c$http$uri; Using Robin's code to extend records is easy and fairly clean looking too... redef record connection += { http: HTTP::Info; }; This has the additional benefit of providing a very easy way for script writers to find out if some certain functionality is available with the field existence testing operator (?$) by checking to see if the http structure is available (for example) by doing c?$http. It's much clearer in it's intent than some of the existing hacks like using @ifdef to find if some variable exists. If we combine this proposal with the &log attribute proposal, I think we can make some very clean interfaces for interacting with data and making it really easy to understand and expand on the scripts that we ship. Thoughts? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From jsiwek at ncsa.illinois.edu Tue Mar 22 07:27:05 2011 From: jsiwek at ncsa.illinois.edu (Jonathan Siwek) Date: Tue, 22 Mar 2011 09:27:05 -0500 (CDT) Subject: [Bro-Dev] active_conns In-Reply-To: <20110321223229.GB34894@icir.org> Message-ID: <1244765394.613.1300804025267.JavaMail.root@zimbra-1.ncsa.uiuc.edu> > > Why can't the "is this script loaded?" functionality be implemented > > as a BIF that queries some global state that the lexical scanner built > > up at scan/parse-time ? > > I could, but I'd still count that as a hack because to be more > precise, it's not really about the *script* being loaded but more > about the functionality being available. Ah ok. Yeah, when you rephrase it like that, this solution becomes less intuitive than I thought. - Jon From jsiwek at ncsa.illinois.edu Tue Mar 22 07:57:20 2011 From: jsiwek at ncsa.illinois.edu (Jonathan Siwek) Date: Tue, 22 Mar 2011 09:57:20 -0500 (CDT) Subject: [Bro-Dev] &log attribute In-Reply-To: Message-ID: <1587872840.904.1300805840995.JavaMail.root@zimbra-1.ncsa.uiuc.edu> > I think that the &log attribute would make the interfaces to these scripts > quite a bit more palatable and enjoyable to use. > > Thoughts? I haven't worked w/ the new upcoming log framework for my opinion to carry much weight, but it does look like a &log attribute might make more sense and simplify things for a script writer. > Although that attribute is annoying use case specific I don't really get why it has to be use-case specific and not a general way to convey to the logging framework that "only these record fields" are meant to be logged. Is it just a problem because it requires more verbosity in the case that all fields are meant to be logged (i.e. script writer might forget to add the &log attribute) ? Or is there an example you can give to explain what you mean? - Jon From seth at icir.org Tue Mar 22 08:14:53 2011 From: seth at icir.org (Seth Hall) Date: Tue, 22 Mar 2011 11:14:53 -0400 Subject: [Bro-Dev] &log attribute In-Reply-To: <1587872840.904.1300805840995.JavaMail.root@zimbra-1.ncsa.uiuc.edu> References: <1587872840.904.1300805840995.JavaMail.root@zimbra-1.ncsa.uiuc.edu> Message-ID: On Mar 22, 2011, at 10:57 AM, Jonathan Siwek wrote: >> Is it just a problem because it requires more verbosity in the case that all fields are meant to be logged (i.e. script writer might forget to add the &log attribute) ? Nah, I don't think that needing the &log attribute explicitly needed for each field is a problem. I think it's better than many alternatives. > Or is there an example you can give to explain what you mean? It's a non-reusable attribute. It only applies in this one scenario, but I suppose the same can also be said for most of the file based attributes like &raw_output. I guess I don't have a really good example and the attribute really does make for clearer intentions when writing scripts compared to the current model of the two separate records. It would be nice for documentation generation too because there isn't any question about what fields/records are intended for the logging framework. In the existing model, there would need to be some sort of indicator for a record to indicate that the record is intended for the logging framework because you don't want to guess based on the type name. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Tue Mar 22 08:54:50 2011 From: seth at icir.org (Seth Hall) Date: Tue, 22 Mar 2011 11:54:50 -0400 Subject: [Bro-Dev] enum namespacing In-Reply-To: <20110322031239.9A6C536A501@taffy.ICSI.Berkeley.EDU> References: <20110322031239.9A6C536A501@taffy.ICSI.Berkeley.EDU> Message-ID: On Mar 21, 2011, at 11:12 PM, Vern Paxson wrote: >> Does it makes sense that an enum would be in the name space it's declared >> within even if it's extending an enum type from another namespace? > > Huh. I have a nagging sense that there's no correct answer to this one > ... However, the behavior you show matches what seems to me more likely > to accord with Principle Of Least Surprise. Fair enough. I'll have to think about how this some more then. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From jsiwek at ncsa.illinois.edu Tue Mar 22 11:15:50 2011 From: jsiwek at ncsa.illinois.edu (Jonathan Siwek) Date: Tue, 22 Mar 2011 13:15:50 -0500 (CDT) Subject: [Bro-Dev] &log attribute In-Reply-To: Message-ID: <12952287.261.1300817750010.JavaMail.jsiwek@tangent.ncsa.illinois.edu> > It's a non-reusable attribute. It only applies in this one scenario, I think I understand now, tell me if this rephrasing is wrong -- in practice you'd only ever use &log in record types that get passed to the logging framework and it doesn't make sense to tag any other identifiers with it (which, if done, shouldn't actually do anything). > It would be nice for documentation generation too because there isn't > any question about what fields/records are intended for the logging > framework. In the existing model, there would need to be some sort of > indicator for a record to indicate that the record is intended for the > logging framework because you don't want to guess based on the type > name. Yeah. It does seem to simplify (at least part of) this task as well. - Jon From seth at icir.org Tue Mar 22 13:49:34 2011 From: seth at icir.org (Seth Hall) Date: Tue, 22 Mar 2011 16:49:34 -0400 Subject: [Bro-Dev] &log attribute In-Reply-To: <12952287.261.1300817750010.JavaMail.jsiwek@tangent.ncsa.illinois.edu> References: <12952287.261.1300817750010.JavaMail.jsiwek@tangent.ncsa.illinois.edu> Message-ID: <1B05B02B-BC45-4AA8-A858-86D791D3E42B@icir.org> On Mar 22, 2011, at 2:15 PM, Jonathan Siwek wrote: >> It's a non-reusable attribute. It only applies in this one scenario, > > I think I understand now, tell me if this rephrasing is wrong -- in practice you'd only ever use &log in record types that get passed to the logging framework and it doesn't make sense to tag any other identifiers with it (which, if done, shouldn't actually do anything). Yes, exactly. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Tue Mar 22 12:01:20 2011 From: robin at icir.org (Robin Sommer) Date: Tue, 22 Mar 2011 12:01:20 -0700 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110322031246.6B5CD36A50D@taffy.ICSI.Berkeley.EDU> References: <20110316042245.GA19883@icir.org> <20110322031246.6B5CD36A50D@taffy.ICSI.Berkeley.EDU> Message-ID: <20110322190120.GA20996@icir.org> On Mon, Mar 21, 2011 at 20:12 -0700, you wrote: > - I'm wondering whether for the unique connection ids it would make > > sense to make them stable in the case that we're working offline from > > a trace. > > I would definitely like that! What I've now done is making them stable if a hash seed is provided. That seems in line with how things are currently: when running from a trace results are non-deterministic by default, but seeding gets rid of that. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From vern at icir.org Tue Mar 22 17:08:55 2011 From: vern at icir.org (Vern Paxson) Date: Tue, 22 Mar 2011 17:08:55 -0700 Subject: [Bro-Dev] &log attribute In-Reply-To: (Tue, 22 Mar 2011 09:24:49 EDT). Message-ID: <20110323000855.7749F36A50C@taffy.ICSI.Berkeley.EDU> > type Info: record { > uri: string &log; > host: string &log; This in general sounds better to me than the two-separate-records approach. However, I wonder if this could be done in a non-use-case-specific manner. You could picture something like: type attribute &log; # declares "&log" as a new attribute, # in this case without an associated value and then (as I guess you already have in mind) the logging framework can first test for whether $uri is present, and, if so, whether it has &log set (via the existing ?$$ operator). Something like that ... Vern From seth at icir.org Tue Mar 22 17:34:57 2011 From: seth at icir.org (Seth Hall) Date: Tue, 22 Mar 2011 20:34:57 -0400 Subject: [Bro-Dev] &log attribute In-Reply-To: <20110323000855.7749F36A50C@taffy.ICSI.Berkeley.EDU> References: <20110323000855.7749F36A50C@taffy.ICSI.Berkeley.EDU> Message-ID: On Mar 22, 2011, at 8:08 PM, Vern Paxson wrote: > type attribute &log; # declares "&log" as a new attribute, > # in this case without an associated value Ah! Nice. I like that too. Now you're going to get me thinking about other ways this can be used. I generally like this for one reason already, it brings more of the "core" functionality (creating new attributes) to the scripting layer. I'm probably always going to be in favor of anything that does that. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Tue Mar 22 19:26:24 2011 From: seth at icir.org (Seth Hall) Date: Tue, 22 Mar 2011 22:26:24 -0400 Subject: [Bro-Dev] syntax consistency question Message-ID: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> set[string] and vector of string Why do these have such differing syntaxes? I understand that internally vectors are really much more similar to tables than sets, but in usage they're more equivalent to an ordered set than a table (in my mind at least). If we add FIFO-type operations to vectors so that they're easier to work with, they would feel even more like ordered sets. For example, using a totally made up code snippet... global stuff: vector[string] = vector(); enqueue stuff["foo"]; enqueue stuff["bar"]; print |stuff|; ==> 2 print dequeue stuff; ==> foo print dequeue stuff; ==> bar print |stuff|; ==> 0 Although it's essentially used as a FIFO here, it's still feels sort of set-y. BTW, this code or something like it working would be awesome. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Tue Mar 22 19:42:47 2011 From: seth at icir.org (Seth Hall) Date: Tue, 22 Mar 2011 22:42:47 -0400 Subject: [Bro-Dev] &log attribute In-Reply-To: <20110323000855.7749F36A50C@taffy.ICSI.Berkeley.EDU> References: <20110323000855.7749F36A50C@taffy.ICSI.Berkeley.EDU> Message-ID: <7044DC4A-BEF8-479F-A9BC-6BA563E254B0@icir.org> On Mar 22, 2011, at 8:08 PM, Vern Paxson wrote: > type attribute &log; # declares "&log" as a new attribute, > # in this case without an associated value > > and then (as I guess you already have in mind) the logging framework > can first test for whether $uri is present, and, if so, whether it > has &log set (via the existing ?$$ operator). One more thought on this. If we did the script level implementation, it would only immediately create another script dependency with the core. Almost all of the new logging framework code that Robin has written is core code in C++ so we'd never actually be using the ?$ operator for check for the attribute in the script layer. It seems like we'd *have* to think of another use case for it to justify implementing the script level attribute creation unless it's really easy since since implementing yet another attribute is so easy. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.icir.org Wed Mar 23 07:34:03 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Wed, 23 Mar 2011 14:34:03 -0000 Subject: [Bro-Dev] #418: Add analysis support for SCTP Message-ID: <043.6ce9179c4fcc6bc76c85ab4b160fb400@tracker.icir.org> #418: Add analysis support for SCTP -----------------------------+----------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Component: Bro | Version: Keywords: | -----------------------------+----------------- Like the summary says. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Mar 23 08:21:06 2011 From: robin at icir.org (Robin Sommer) Date: Wed, 23 Mar 2011 08:21:06 -0700 Subject: [Bro-Dev] &log attribute In-Reply-To: <7044DC4A-BEF8-479F-A9BC-6BA563E254B0@icir.org> References: <20110323000855.7749F36A50C@taffy.ICSI.Berkeley.EDU> <7044DC4A-BEF8-479F-A9BC-6BA563E254B0@icir.org> Message-ID: <20110323152106.GB39112@icir.org> On Tue, Mar 22, 2011 at 22:42 -0400, you wrote: > attribute in the script layer. It seems like we'd *have* to think of > another use case for it to justify implementing the script level > attribute creation I was about to say something similar: why don't we go with the &log attribute for now and generalize to user-definable attributes if/when we find more use cases? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From baxterw3232 at gmail.com Wed Mar 23 08:52:17 2011 From: baxterw3232 at gmail.com (Will) Date: Wed, 23 Mar 2011 11:52:17 -0400 Subject: [Bro-Dev] bro-ids.org Message-ID: Is the website (bro-ids.org) is offline for maintanence or anything? I have tried accessing it from two different networks, so I don't *think* its just me. Thanks, Will -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20110323/2fd64c0a/attachment.html From vern at icir.org Wed Mar 23 09:14:07 2011 From: vern at icir.org (Vern Paxson) Date: Wed, 23 Mar 2011 09:14:07 -0700 Subject: [Bro-Dev] &log attribute In-Reply-To: <20110323152106.GB39112@icir.org> (Wed, 23 Mar 2011 08:21:06 PDT). Message-ID: <20110323161407.E1D4836A4FA@taffy.ICSI.Berkeley.EDU> > I was about to say something similar: why don't we go with the &log > attribute for now and generalize to user-definable attributes if/when > we find more use cases? I had (for some reason) thought that the processing of this attribute was mostly at script-level. If it's indeed instead in the core [*], then I agree it makes sense to not generalize at this point. Vern [*] Yes, you got me to use the term "core". "Event engine" doesn't sound right for something like this which isn't at all about generating events. (Which just makes me think it should in fact be implemented at the script level. ;-) From leres at ee.lbl.gov Wed Mar 23 10:16:48 2011 From: leres at ee.lbl.gov (Craig Leres) Date: Wed, 23 Mar 2011 10:16:48 -0700 Subject: [Bro-Dev] bro-ids.org In-Reply-To: References: Message-ID: <4D8A2B00.8080701@ee.lbl.gov> On 03/23/11 08:52, Will wrote: > Is the website (bro-ids.org ) is offline for > maintanence or anything? The server crashed early this morning but is back up now. Craig From jsiwek at ncsa.illinois.edu Thu Mar 24 09:02:26 2011 From: jsiwek at ncsa.illinois.edu (Jonathan Siwek) Date: Thu, 24 Mar 2011 11:02:26 -0500 (CDT) Subject: [Bro-Dev] bro script "public interface" In-Reply-To: <1547412.281.1300981305716.JavaMail.jsiwek@tangent.ncsa.illinois.edu> Message-ID: <4936037.291.1300982544336.JavaMail.jsiwek@tangent.ncsa.illinois.edu> WRT my work on auto-generation of script documentation... I'm taking a bro script's "public interface" to mean: any identifier declared in the GLOBAL namespace/module (regardless of whether it's in an export section) OR any identifier declared in an export section for a namespace/module other than GLOBAL from that, it follows that "private interface" is: any identifier declared outside an export section AND not in the GLOBAL namespace/module Aside from commenting on the correctness of those definitions: 1) Is this in line with the way standard scripts are being "modernized" ? I think a big part of it involves updating them to use the namespacing capabilities, but I think there's still desire to declare some things in the GLOBAL namespace? That's why I'm still considering that situation as part of the script's "public interface". 2) Are there any initial thoughts on whether the auto-generated documentation should even include the "private interface" stuff? Since "private" things are accessible only from within the given script, it's implied that the person who is interested in that stuff is one who is actually working on modifying the script itself and they could easily rely on the old-fashioned script comments for documentation. On the other hand, the "private" stuff could be utilizing the "public" stuff of other scripts and the auto-generated docs might provide more convenient cross-references for that. - Jon From seth at icir.org Thu Mar 24 09:11:30 2011 From: seth at icir.org (Seth Hall) Date: Thu, 24 Mar 2011 12:11:30 -0400 Subject: [Bro-Dev] bro script "public interface" In-Reply-To: <4936037.291.1300982544336.JavaMail.jsiwek@tangent.ncsa.illinois.edu> References: <4936037.291.1300982544336.JavaMail.jsiwek@tangent.ncsa.illinois.edu> Message-ID: On Mar 24, 2011, at 12:02 PM, Jonathan Siwek wrote: > 1) Is this in line with the way standard scripts are being "modernized" ? I think a big part of it involves updating them to use the namespacing capabilities, but I think there's still desire to declare some things in the GLOBAL namespace? That's why I'm still considering that situation as part of the script's "public interface". There will at least be a number of things in bro.init declared in the global namespace that we really need to document, but I think you generally have it right. I'm trying to avoid anything in the global namespace as I work on the script rewrite though. The one exception is the notice.bro script because there is a well known function in the global namespace named NOTICE that is sort of the entry point for generating notices. > 2) Are there any initial thoughts on whether the auto-generated documentation should even include the "private interface" stuff? No, I wouldn't bother with it for this release. If it turns out that it's something that people would like to have then we can add it, but I don't expect there will be a huge demand for it. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From jsiwek at ncsa.illinois.edu Thu Mar 24 09:34:06 2011 From: jsiwek at ncsa.illinois.edu (Jonathan Siwek) Date: Thu, 24 Mar 2011 11:34:06 -0500 (CDT) Subject: [Bro-Dev] bro script "public interface" In-Reply-To: Message-ID: <25005191.297.1300984443763.JavaMail.jsiwek@tangent.ncsa.illinois.edu> > There will at least be a number of things in bro.init declared in the > global namespace that we really need to document Yeah, that's exactly the stuff I found missing from my first run of auto-generated docs that I had in mind when asking the question. > > 2) Are there any initial thoughts on whether the auto-generated > > documentation should even include the "private interface" stuff? > > No, I wouldn't bother with it for this release. If it turns out that > it's something that people would like to have then we can add it, but > I don't expect there will be a huge demand for it. Good to know in case it turns out to be a mess to get that to work, but I'm thinking it might be simple. It could probably even be an extra "verbose documentation" command line switch to bro that will create both the public and the private interface docs and then we can decide later if it's actually useful to provide. - Jon From gregor at icir.org Thu Mar 24 18:59:39 2011 From: gregor at icir.org (Gregor Maier) Date: Thu, 24 Mar 2011 18:59:39 -0700 Subject: [Bro-Dev] enum namespacing In-Reply-To: <35A08662-D2CF-45D5-A3E8-BE09D1D49091@icir.org> References: <35A08662-D2CF-45D5-A3E8-BE09D1D49091@icir.org> Message-ID: <4D8BF70B.6070502@icir.org> Note that currently the NOTICE framework is the part the mostly makes use of extended Enums and there it makes sense for the extended elements to live in the enclosing name-space (in your example HTTP). I guess the current rule is: if an identifier does not have a Foo:: module specifier, it's placed in the current module. So, module HTTP; redef enum Software::Type += { Software::UNKNOWN }; # UNKNOWN in Software namespace redef enum Software::Type += { UNKNOWN }; # unknown in HTTP namespace just my 2ct Gregor On 3/17/11 6:35 , Seth Hall wrote: > Does it makes sense that an enum would be in the name space it's declared within even if it's extending an enum type from another namespace? > > My example: > > module Software; > > export { > type Type: enum { UNKNOWN }; > } > > module HTTP; > > redef enum Software::Type += { WEB_SERVER }; > > event bro_init() > { > # I would expect this to be true, but it seems that WEB_SERVER was placed into the HTTP namespace and this doesn't work. > print Software::WEB_SERVER; > } > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From gregor at icir.org Thu Mar 24 19:03:57 2011 From: gregor at icir.org (Gregor Maier) Date: Thu, 24 Mar 2011 19:03:57 -0700 Subject: [Bro-Dev] $tag in notice_info In-Reply-To: <20110322190120.GA20996@icir.org> References: <20110316042245.GA19883@icir.org> <20110322031246.6B5CD36A50D@taffy.ICSI.Berkeley.EDU> <20110322190120.GA20996@icir.org> Message-ID: <4D8BF80D.7060805@icir.org> You could also use the first packet's timestamp and/or a hash over it's content + hostname or such to generate the 64bit run-ID. This way we would always get consistent behavior even if no seed is sets..... but ymmv gregor On 3/22/11 12:01 , Robin Sommer wrote: > > On Mon, Mar 21, 2011 at 20:12 -0700, you wrote: > >> - I'm wondering whether for the unique connection ids it would make >>> sense to make them stable in the case that we're working offline from >>> a trace. >> >> I would definitely like that! > > What I've now done is making them stable if a hash seed is provided. > That seems in line with how things are currently: when running from a > trace results are non-deterministic by default, but seeding gets rid > of that. > > Robin > -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From gregor at icir.org Thu Mar 24 19:10:32 2011 From: gregor at icir.org (Gregor Maier) Date: Thu, 24 Mar 2011 19:10:32 -0700 Subject: [Bro-Dev] syntax consistency question In-Reply-To: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> References: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> Message-ID: <4D8BF998.7090201@icir.org> Yeah, I also discovered that vectors are kinda limited in their functionality. I think it would be great to have two different data types in Bro. Possibly both supporting the same. * array or vector (pretty much like the current vector type): + O(1) random element access + O(1) (armotized) adding to / removing from *end* + O(n) adding to / removing from *beginning* + grows (shrinks?) dynamically. * queue, deque (for FIFOs, stacks, etc) + O(n) random element access + O(1) adding/removing from *either end* cu Gregor On 3/22/11 19:26 , Seth Hall wrote: > set[string] > and > vector of string > > Why do these have such differing syntaxes? I understand that internally vectors are really much more similar to tables than sets, but in usage they're more equivalent to an ordered set than a table (in my mind at least). If we add FIFO-type operations to vectors so that they're easier to work with, they would feel even more like ordered sets. For example, using a totally made up code snippet... > > global stuff: vector[string] = vector(); > enqueue stuff["foo"]; > enqueue stuff["bar"]; > print |stuff|; > ==> 2 > print dequeue stuff; > ==> foo > print dequeue stuff; > ==> bar > print |stuff|; > ==> 0 > > Although it's essentially used as a FIFO here, it's still feels sort of set-y. BTW, this code or something like it working would be awesome. :) > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From seth at icir.org Thu Mar 24 19:12:03 2011 From: seth at icir.org (Seth Hall) Date: Thu, 24 Mar 2011 22:12:03 -0400 Subject: [Bro-Dev] enum namespacing In-Reply-To: <4D8BF70B.6070502@icir.org> References: <35A08662-D2CF-45D5-A3E8-BE09D1D49091@icir.org> <4D8BF70B.6070502@icir.org> Message-ID: <4006E800-5D17-4745-B32C-C4BE790E56BB@icir.org> On Mar 24, 2011, at 9:59 PM, Gregor Maier wrote: > I guess the current rule is: if an identifier does not have a Foo:: > module specifier, it's placed in the current module. So, > > module HTTP; > redef enum Software::Type += { Software::UNKNOWN }; # UNKNOWN in > Software namespace > redef enum Software::Type += { UNKNOWN }; # unknown in HTTP namespace This is what would seem to make the most sense to me, but doesn't seem to be true currently. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From gregor at icir.org Thu Mar 24 21:06:34 2011 From: gregor at icir.org (Gregor Maier) Date: Thu, 24 Mar 2011 21:06:34 -0700 Subject: [Bro-Dev] enum namespacing In-Reply-To: <4006E800-5D17-4745-B32C-C4BE790E56BB@icir.org> References: <35A08662-D2CF-45D5-A3E8-BE09D1D49091@icir.org> <4D8BF70B.6070502@icir.org> <4006E800-5D17-4745-B32C-C4BE790E56BB@icir.org> Message-ID: <4D8C14CA.8030701@icir.org> >> module HTTP; >> redef enum Software::Type += { Software::UNKNOWN }; # UNKNOWN in >> Software namespace >> redef enum Software::Type += { UNKNOWN }; # unknown in HTTP namespace > > > This is what would seem to make the most sense to me, but doesn't seem to be true currently. Huh. I thought that's what it was doing. At least that's what I thought your mail said. I must have misunderstood it. Can you briefly explain the current behavior? thx gregor -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From robin at icir.org Sun Mar 27 10:07:02 2011 From: robin at icir.org (Robin Sommer) Date: Sun, 27 Mar 2011 10:07:02 -0700 Subject: [Bro-Dev] syntax consistency question In-Reply-To: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> References: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> Message-ID: <20110327170702.GI89534@icir.org> On Tue, Mar 22, 2011 at 22:26 -0400, you wrote: > print dequeue stuff; > print dequeue stuff; Don't really like this syntax but also don't see another more intuitive operator. Could we just add more built-ins for vector operations like these? (e.g., vector_dequeue(v)). As usual, type-safety would be a problem though. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Sun Mar 27 10:08:07 2011 From: robin at icir.org (Robin Sommer) Date: Sun, 27 Mar 2011 10:08:07 -0700 Subject: [Bro-Dev] syntax consistency question In-Reply-To: <4D8BF998.7090201@icir.org> References: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> <4D8BF998.7090201@icir.org> Message-ID: <20110327170807.GJ89534@icir.org> On Thu, Mar 24, 2011 at 19:10 -0700, you wrote: > * queue, deque (for FIFOs, stacks, etc) Or more generally, lists. I agree, but adding that is a major task, I don't see that near-term. Perhaps for the 1.7 timeline? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Sun Mar 27 10:55:34 2011 From: robin at icir.org (Robin Sommer) Date: Sun, 27 Mar 2011 10:55:34 -0700 Subject: [Bro-Dev] &log attribute In-Reply-To: <20110323161407.E1D4836A4FA@taffy.ICSI.Berkeley.EDU> References: <20110323152106.GB39112@icir.org> <20110323161407.E1D4836A4FA@taffy.ICSI.Berkeley.EDU> Message-ID: <20110327175534.GA2973@icir.org> One more question regarding this: Seth and I had discussed earlier that if a record does not have any "&log" fields, the logging framework could by default log all fields. We dismissed that as "too much magic", but thinking about it again, I'd like to reconsider. The reason is that if I pass a record in for logging, chances are that I want something logged. If there's no explicit &log attribute, it seems natural to assume that one wants to log everything. That also looks nicer and is less cumbersome than having to add &log to every single field to get the same effect. And I don't really see how one would pass a record to logging with the intention of *not* logging anything. Makes sense? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From gregor at icir.org Sun Mar 27 16:04:04 2011 From: gregor at icir.org (Gregor Maier) Date: Sun, 27 Mar 2011 16:04:04 -0700 Subject: [Bro-Dev] syntax consistency question In-Reply-To: <20110327170807.GJ89534@icir.org> References: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> <4D8BF998.7090201@icir.org> <20110327170807.GJ89534@icir.org> Message-ID: <4D8FC264.4030602@icir.org> On 3/27/11 10:08 , Robin Sommer wrote: > > On Thu, Mar 24, 2011 at 19:10 -0700, you wrote: > >> * queue, deque (for FIFOs, stacks, etc) > > Or more generally, lists. I agree, but adding that is a major task, I > don't see that near-term. Perhaps for the 1.7 timeline? sounds good to me -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From seth at icir.org Sun Mar 27 19:15:58 2011 From: seth at icir.org (Seth Hall) Date: Sun, 27 Mar 2011 22:15:58 -0400 Subject: [Bro-Dev] syntax consistency question In-Reply-To: <4D8FC264.4030602@icir.org> References: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> <4D8BF998.7090201@icir.org> <20110327170807.GJ89534@icir.org> <4D8FC264.4030602@icir.org> Message-ID: On Mar 27, 2011, at 7:04 PM, Gregor Maier wrote: >> Or more generally, lists. I agree, but adding that is a major task, I >> don't see that near-term. Perhaps for the 1.7 timeline? > > sounds good to me Me too. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Sun Mar 27 19:42:27 2011 From: seth at icir.org (Seth Hall) Date: Sun, 27 Mar 2011 22:42:27 -0400 Subject: [Bro-Dev] &log attribute In-Reply-To: <20110327175534.GA2973@icir.org> References: <20110323152106.GB39112@icir.org> <20110323161407.E1D4836A4FA@taffy.ICSI.Berkeley.EDU> <20110327175534.GA2973@icir.org> Message-ID: On Mar 27, 2011, at 1:55 PM, Robin Sommer wrote: > The reason is that if I pass a record in for logging, chances are that > I want something logged. If there's no explicit &log attribute, it > seems natural to assume that one wants to log everything. That also > looks nicer and is less cumbersome than having to add &log to every > single field to get the same effect. And I don't really see how one > would pass a record to logging with the intention of *not* logging > anything. > > Makes sense? Haha! That does make sense. I still sort of think I'd rather have the attribute be required though. Oh! I just thought of an example. Someone could write that it collects state, but none of it is ever logged because it's only intended to extended and built upon in further scripts. If we had the "absence of &log means log everything" convention, it would want to log our state tracking variables. It also becomes really confusing since we can extend record types in other locations and things that would formerly log are suddenly not being logged because the type was extended and a one of the extended values had the &log attribute. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From slagell at ncsa.illinois.edu Sun Mar 27 19:56:03 2011 From: slagell at ncsa.illinois.edu (Adam J. Slagell) Date: Sun, 27 Mar 2011 21:56:03 -0500 Subject: [Bro-Dev] &log attribute In-Reply-To: References: <20110323152106.GB39112@icir.org> <20110323161407.E1D4836A4FA@taffy.ICSI.Berkeley.EDU> <20110327175534.GA2973@icir.org> Message-ID: <1FD7E2A3-B3D3-4329-B0B4-FCB7C82A4A3F@ncsa.illinois.edu> On Mar 27, 2011, at 9:42 PM, Seth Hall wrote: > > On Mar 27, 2011, at 1:55 PM, Robin Sommer wrote: > >> The reason is that if I pass a record in for logging, chances are that >> I want something logged. If there's no explicit &log attribute, it >> seems natural to assume that one wants to log everything. That also >> looks nicer and is less cumbersome than having to add &log to every >> single field to get the same effect. And I don't really see how one >> would pass a record to logging with the intention of *not* logging >> anything. >> >> Makes sense? > > Haha! That does make sense. I still sort of think I'd rather have the attribute be required though. > > Oh! I just thought of an example. Someone could write that it collects state, but none of it is ever logged because it's only intended to extended and built upon in further scripts. If we had the "absence of &log means log everything" convention, it would want to log our state tracking variables. It also becomes really confusing since we can extend record types in other locations and things that would formerly log are suddenly not being logged because the type was extended and a one of the extended values had the &log attribute. I'd rather have a separate syntax to log all fields than to just assume that not using the attribute anywhere has this opposite behavior. From seth at icir.org Sun Mar 27 20:06:02 2011 From: seth at icir.org (Seth Hall) Date: Sun, 27 Mar 2011 23:06:02 -0400 Subject: [Bro-Dev] &log attribute In-Reply-To: <1FD7E2A3-B3D3-4329-B0B4-FCB7C82A4A3F@ncsa.illinois.edu> References: <20110323152106.GB39112@icir.org> <20110323161407.E1D4836A4FA@taffy.ICSI.Berkeley.EDU> <20110327175534.GA2973@icir.org> <1FD7E2A3-B3D3-4329-B0B4-FCB7C82A4A3F@ncsa.illinois.edu> Message-ID: On Mar 27, 2011, at 10:56 PM, Adam J. Slagell wrote: > I'd rather have a separate syntax to log all fields than to just assume that not using the attribute anywhere has this opposite behavior. Just for the record, I don't think this would be appropriate either due to the recent record extension mechanism. It would essentially prohibit someone from including non-logged state information in a script extending one of the logging records with the separate syntax indicating "log all". .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Mon Mar 28 08:17:56 2011 From: robin at icir.org (Robin Sommer) Date: Mon, 28 Mar 2011 08:17:56 -0700 Subject: [Bro-Dev] &log attribute In-Reply-To: References: <20110323152106.GB39112@icir.org> <20110323161407.E1D4836A4FA@taffy.ICSI.Berkeley.EDU> <20110327175534.GA2973@icir.org> <1FD7E2A3-B3D3-4329-B0B4-FCB7C82A4A3F@ncsa.illinois.edu> Message-ID: <20110328151756.GD23493@icir.org> Ok, convinced, I see the record-extension argumnet. The main point I'm worried about is that we'll end up with lots of &log attributes in these combined log/state records. But ok, let's see how it turns out. I'll add the &log attribute as discussed. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Mon Mar 28 08:27:23 2011 From: seth at icir.org (Seth Hall) Date: Mon, 28 Mar 2011 11:27:23 -0400 Subject: [Bro-Dev] New SSL analyzer Message-ID: As I was working on the ssl.bro script last week, I realized that the core SSL analyzers (hand written and binpac) generated really strange events that seemed very different than the events that most of the other core analyzers generate. It turns out that those analyzers were storing some state about existing SSL sessions and generating events based on that information. I think it was storing too much internally and not even making all of it available at the scripting layer. I'll go through bullet points of changes and where I think a bit more work needs to be done. - SSL & X509 related events are completely rewritten. event ssl_client_hello(c: connection, version: count, possible_ts: time, session_id: string, ciphers: count_set) event ssl_server_hello(c: connection, version: count, possible_ts: time, session_id: string, cipher: count, comp_method: count) event ssl_extension(c: connection, code: count, val: string) event ssl_established(c: connection) event ssl_alert(c: connection, level: count, desc: count) event x509_certificate(c: connection, cert: X509, is_server: bool, chain_idx: count, chain_len: count) event x509_extension(c: connection, data: string) event x509_error(c: connection, err: int, err_string: string) event x509_cert_validated(c: connection) event x509_cert_body(c: connection, cert: string) - In ssl_client_hello, I'm not currently extracting the compression methods supported for some reason, but I'll try to remember to add that soon. - The possible_ts fields in both "hello" events are the first 32bits of the random data section which some SSL libraries populate with a current time stamp. - I changed the session_id to a string instead of a table[count] of count like it was before (since it can be up to 32 bytes long). The table was really difficult to work with but it's much easier now that it's a string. - There is a small issue where the ssl_extension event is be generated before the ssl_client_hello event due to how the parser is parsing the hello record which is a little backward since the extensions are at the end of the hello record. Should I just include the SSL extensions in the hello events as a table? The follow up question is if it makes sense to just include the raw data for the extension even though there is a data structure there? Here's an example of printing the server_name extension (i have to rip off the first 5 bytes which is a piece of the data structure for the extension)... event ssl_extension(c: connection, code: count, val: string) { if ( code == 0 ) print fmt("ssl extension %d %s", code, sub(val, /^...../, "")); } - The X509 data type has had some extra fields added to it about the certificate (like the serial number, notValidBefore, and notValidAfter fields). - The chain_idx and chain_len values indicate which certificate is being looked at if it's part of a certificate full signing chain. The server's certificate will be chain_idx == 1 and the root signing key will be chain_idx==chain_len. Intermediate certs will be somewhere between. :) Is there a better way to represent this in the script land? I doesn't look very clean and isn't terribly obvious but I wasn't coming up with anything better. - Certificate extraction now happens in scripting land with the x509_cert_body event (as a DER certificate). It can work on a cluster deployment this way (by printing remotely) and it's just generally more flexible. My initial testing doesn't actually show much difference in pushing the cert to script land or not pushing it there. - Certificate and certificate chain validation can be controlled at the scripting land a bit more easily because I added a table with root CAs. Sites can even add local CAs to Bro's root CA list if they want to trust locally signed keys. It's just a table that yields DER encoded certificates. I have a script that can automatically generate the Bro script with all of Mozilla's root CAs too so we can ship the same trust model as Mozilla. The problem is that I'm building this X509_STACK (OpenSSL terminology) with each instantiation of the SSL analyzer and it should probably only be done when certs are added or removed from the scripting land table and stored globally. Any guidance I could get on where/how to do this would help. There is probably more, but this is all I can think of at the moment. Any and all feedback would be great! I'll get the code committed and pushed today. Thanks! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.icir.org Mon Mar 28 08:36:39 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 28 Mar 2011 15:36:39 -0000 Subject: [Bro-Dev] #419: Add a "real" list type to the scripting language. Message-ID: <044.fe402075ead7854f881029eeb278dad9@tracker.icir.org> #419: Add a "real" list type to the scripting language. ----------------------+-------------------- Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro1.7 Component: Bro | Version: Keywords: language | ----------------------+-------------------- See subject. Include convenience support for using it as a stack and fifo. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 28 09:22:59 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 28 Mar 2011 16:22:59 -0000 Subject: [Bro-Dev] #420: Always associate &default with record fields Message-ID: <044.fd88dd449ce707c62224c042cbd0b906@tracker.icir.org> #420: Always associate &default with record fields --------------------+-------------------- Reporter: robin | Owner: robin Type: Task | Status: new Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Keywords: | --------------------+-------------------- In record definitions, sets/table fields associate a &default attribute with the type, not the record field. That's inconsistent and should be changed. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 28 09:24:12 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Mon, 28 Mar 2011 16:24:12 -0000 Subject: [Bro-Dev] #421: Initialize set/table field in records. Message-ID: <044.43058f6fba84734b712b1fcb46d48a68@tracker.icir.org> #421: Initialize set/table field in records. --------------------+-------------------- Reporter: robin | Owner: robin Type: Task | Status: new Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Keywords: | --------------------+-------------------- We should initialize set/tables in instantiated records to empty values if they don't have an &optional attribute. -- Ticket URL: Bro Tracker Bro Issue Tracker From vern at icir.org Mon Mar 28 09:45:10 2011 From: vern at icir.org (Vern Paxson) Date: Mon, 28 Mar 2011 09:45:10 -0700 Subject: [Bro-Dev] &log attribute In-Reply-To: <20110327175534.GA2973@icir.org> (Sun, 27 Mar 2011 10:55:34 PDT). Message-ID: <20110328164510.8EB4836A416@taffy.ICSI.Berkeley.EDU> > The reason is that if I pass a record in for logging, chances are that > I want something logged. If there's no explicit &log attribute What about explicitly associating an &log with the entire record, rather than with each of it fields? (I'm not quite picturing the usage you have in mind here, so this may or may not be appropriate.) Vern From slagell at ncsa.illinois.edu Mon Mar 28 10:21:18 2011 From: slagell at ncsa.illinois.edu (Adam J. Slagell) Date: Mon, 28 Mar 2011 12:21:18 -0500 Subject: [Bro-Dev] &log attribute In-Reply-To: <20110328164510.8EB4836A416@taffy.ICSI.Berkeley.EDU> References: <20110328164510.8EB4836A416@taffy.ICSI.Berkeley.EDU> Message-ID: <76BCA0BE-53EE-4A14-95B6-ABDCE196DBC1@ncsa.illinois.edu> On Mar 28, 2011, at 11:45 AM, Vern Paxson wrote: >> The reason is that if I pass a record in for logging, chances are that >> I want something logged. If there's no explicit &log attribute > > What about explicitly associating an &log with the entire record, rather > than with each of it fields? (I'm not quite picturing the usage you have > in mind here, so this may or may not be appropriate.) That was my suggestion that Seth didn't like. ------ Adam J. Slagell, CISSP Sr. Security Engineer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info 217.244.8965 From seth at icir.org Mon Mar 28 13:42:20 2011 From: seth at icir.org (Seth Hall) Date: Mon, 28 Mar 2011 16:42:20 -0400 Subject: [Bro-Dev] &log attribute In-Reply-To: <76BCA0BE-53EE-4A14-95B6-ABDCE196DBC1@ncsa.illinois.edu> References: <20110328164510.8EB4836A416@taffy.ICSI.Berkeley.EDU> <76BCA0BE-53EE-4A14-95B6-ABDCE196DBC1@ncsa.illinois.edu> Message-ID: <605B73E0-427B-4C45-9892-C2211C6C0575@icir.org> On Mar 28, 2011, at 1:21 PM, Adam J. Slagell wrote: > > On Mar 28, 2011, at 11:45 AM, Vern Paxson wrote: > >>> The reason is that if I pass a record in for logging, chances are that >>> I want something logged. If there's no explicit &log attribute >> >> What about explicitly associating an &log with the entire record, rather >> than with each of it fields? (I'm not quite picturing the usage you have >> in mind here, so this may or may not be appropriate.) > > That was my suggestion that Seth didn't like. I can give a little detail about why I didn't like it. :P type Info: record { ts: time &log; id: conn_id &log; }; Given the above type, I wouldn't want someone giving &log to the entire record because if later, someone else comes along with a script that extends that record to include some extra fields like this... redef record Info += { did_something: bool &default=F &log; ready_to_log: bool &default=F; }; ...the whole record would be logged and &ready_to_log might be a variable that they included for tracking if the whole record is prepared to be logged. This is a technique I use in a few scripts to let users decide which point they'd like to log at based on costs/benefits on early or late logging. The author of the extension script wouldn't want the $ready_to_log attribute logged since it's really just an internal tracking variable. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From vern at icir.org Mon Mar 28 13:49:58 2011 From: vern at icir.org (Vern Paxson) Date: Mon, 28 Mar 2011 13:49:58 -0700 Subject: [Bro-Dev] &log attribute In-Reply-To: <605B73E0-427B-4C45-9892-C2211C6C0575@icir.org> (Mon, 28 Mar 2011 16:42:20 EDT). Message-ID: <20110328204958.DE01236A4F1@taffy.ICSI.Berkeley.EDU> > redef record Info += { > did_something: bool &default=F &log; > ready_to_log: bool &default=F; > }; What about requiring &log to be added to extensions (if one wants the entire extension logged by default), otherwise they go back to the normal behavior of only associating &log with fields for which it's explicitly mentioned? This is a bit different from how attributes currently propagate, but we don't (I think) have other maybe-it's-the-whole-aggregate-maybe-it's-a-field attributes, so the above wouldn't be inconsistent with existing practice. Vern From seth at icir.org Mon Mar 28 14:00:12 2011 From: seth at icir.org (Seth Hall) Date: Mon, 28 Mar 2011 17:00:12 -0400 Subject: [Bro-Dev] &log attribute In-Reply-To: <20110328204958.DE01236A4F1@taffy.ICSI.Berkeley.EDU> References: <20110328204958.DE01236A4F1@taffy.ICSI.Berkeley.EDU> Message-ID: <2DA79694-717B-4594-A5C3-21AAB8C24ECF@icir.org> On Mar 28, 2011, at 4:49 PM, Vern Paxson wrote: > What about requiring &log to be added to extensions (if one wants the > entire extension logged by default), otherwise they go back to the normal > behavior of only associating &log with fields for which it's explicitly > mentioned? Ah, I see what you're saying. I suppose that would work just fine. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.icir.org Mon Mar 28 17:26:49 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 29 Mar 2011 00:26:49 -0000 Subject: [Bro-Dev] #305: Replace communication system with 0mq In-Reply-To: <043.40694402c58225da14515b3c0a74bb6a@tracker.icir.org> References: <043.40694402c58225da14515b3c0a74bb6a@tracker.icir.org> Message-ID: <058.7b645148fb9beac710444a6d02aaa038@tracker.icir.org> #305: Replace communication system with 0mq ---------------------+-------------------- Reporter: seth | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro1.7 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Changes (by seth): * milestone: Bro1.6 => Bro1.7 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 28 17:30:50 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 29 Mar 2011 00:30:50 -0000 Subject: [Bro-Dev] #300: Document default policy scripts In-Reply-To: <043.9b3b25d0ae8ee2368b2b397389a52829@tracker.icir.org> References: <043.9b3b25d0ae8ee2368b2b397389a52829@tracker.icir.org> Message-ID: <058.b8a9256b8606184cc5ec7604eccc82fb@tracker.icir.org> #300: Document default policy scripts ---------------------+-------------------- Reporter: seth | Owner: Type: Task | Status: closed Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Resolution: Solved | Keywords: ---------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Solved Comment: I'm going to close this ticket since these projects are already being worked on. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Mon Mar 28 17:48:20 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 29 Mar 2011 00:48:20 -0000 Subject: [Bro-Dev] #422: Array-style index accessor for strings Message-ID: <043.2a1b39b410bf7a8fe9f1fba567a76afb@tracker.icir.org> #422: Array-style index accessor for strings -----------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro1.7 Component: Bro | Version: Keywords: language | -----------------------------+-------------------- String octets should be addressable by index. Like this: {{{ local abc = "abc"; print abc[1]; ==> "b" }}} Having reverse indexes would be nice too. Like this: {{{ local alpha = "abcdefghijklmnopqrstuvwxyz"; print alpha[-2]; ==> "y" }}} It would be nice to do substring extraction with it as well. Like this: {{{ local alpha = "abcdefghijklmnopqrstuvwxyz"; print alpha[4,9]; ==> "efghij" print alpha[-5,-3]; ==> "vwx" }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Mon Mar 28 17:49:24 2011 From: robin at icir.org (Robin Sommer) Date: Mon, 28 Mar 2011 17:49:24 -0700 Subject: [Bro-Dev] vectors (Re: syntax consistency question) In-Reply-To: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> References: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> Message-ID: <20110329004924.GB43817@icir.org> Another vector wish list item: can we please make them zero-based? I propose to do that immediately while we're working on scripts anyway. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Mon Mar 28 18:00:22 2011 From: seth at icir.org (Seth Hall) Date: Mon, 28 Mar 2011 21:00:22 -0400 Subject: [Bro-Dev] vectors (Re: syntax consistency question) In-Reply-To: <20110329004924.GB43817@icir.org> References: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> <20110329004924.GB43817@icir.org> Message-ID: <013A5B49-EADE-4263-A489-C706162689EC@icir.org> On Mar 28, 2011, at 8:49 PM, Robin Sommer wrote: > Another vector wish list item: can we please make them zero-based? I > propose to do that immediately while we're working on scripts anyway. Seems reasonable to me. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.icir.org Mon Mar 28 18:39:03 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 29 Mar 2011 01:39:03 -0000 Subject: [Bro-Dev] #423: Additional dynamic init time pattern construction Message-ID: <043.644ac9f515c363c23c854c588d9b5ddb@tracker.icir.org> #423: Additional dynamic init time pattern construction ----------------------+-------------------- Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro1.7 Component: Bro | Version: Keywords: language | ----------------------+-------------------- The pattern building BiF are now working, but I think that pattern construction operators need to be supported with variables too. {{{ const a = /abc/; const b = /def/; const c = a | b; }}} This doesn't seem to work currently. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Tue Mar 29 12:03:39 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 29 Mar 2011 19:03:39 -0000 Subject: [Bro-Dev] #424: Loading sub-pathed bro script at command line Message-ID: <043.05cdb9fb2220c4f6e6eb13b477e902b4@tracker.icir.org> #424: Loading sub-pathed bro script at command line ---------------------+-------------------- Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Keywords: | ---------------------+-------------------- This works in Bro scripts: {{{ @load http/base-extended }}} You can't do this at the command line though: {{{ ./src/bro http/base-extended }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From jmellander at lbl.gov Tue Mar 29 13:49:48 2011 From: jmellander at lbl.gov (Jim Mellander) Date: Tue, 29 Mar 2011 13:49:48 -0700 Subject: [Bro-Dev] Bug in drop.bro and patch Message-ID: Hi folks: In drop.bro, if use_catch_release is F (indicating that you don't want to use catch & release), bro will still attempt to unblock hosts after a 1 day timeout by executing the clear_host function (see the drop_info table), and if there is a restore-connectivity script in the path, it will get executed, so you actually get a pseudo catch & release. The fix is to add a one liner to the clear_host function, which returns immediately if catch & release is not enabled. See patch below: ==================================== *** drop.bro Tue Mar 29 13:39:44 2011 --- drop.bro.new Tue Mar 29 13:37:16 2011 *************** *** 283,288 **** --- 283,289 ---- function clear_host(t: table[addr] of drop_rec, a: addr): interval { + if ( ! use_catch_release ) return 0 secs; if ( is_dropped(a) ) # Restore address. do_restore(a, T); From bro at tracker.icir.org Tue Mar 29 13:57:36 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Tue, 29 Mar 2011 20:57:36 -0000 Subject: [Bro-Dev] #425: Vector initializer is broken Message-ID: <043.b61a318108876868f025119b95815c8b@tracker.icir.org> #425: Vector initializer is broken ---------------------+-------------------- Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Keywords: | ---------------------+-------------------- vector() can't be used for initializing an empty vector.. {{{ global a: vector of string = vector(); }}} {{{ ./test2.bro, line 1: error: no type can be inferred for empty list }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Tue Mar 29 21:56:01 2011 From: robin at icir.org (Robin Sommer) Date: Tue, 29 Mar 2011 21:56:01 -0700 Subject: [Bro-Dev] Initial btest setup merged into master Message-ID: <20110330045601.GB3475@icir.org> Fyi, per the commit mails, I've merged the initial btest setup from Don's branch into master so that we can start collecting tests centrally. Don, when you merge this back, please note that I changed some paths; see the commit messages. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From appleman at ncsa.illinois.edu Wed Mar 30 08:51:33 2011 From: appleman at ncsa.illinois.edu (Don Appleman) Date: Wed, 30 Mar 2011 10:51:33 -0500 (CDT) Subject: [Bro-Dev] Initial btest setup merged into master In-Reply-To: <20110330045601.GB3475@icir.org> Message-ID: <453328114.1608.1301500293502.JavaMail.root@zimbra-1.ncsa.uiuc.edu> Got it, Robin, thanks. ----- Original Message ----- From: "Robin Sommer" To: bro-dev at bro-ids.org Sent: Tuesday, March 29, 2011 11:56:01 PM Subject: [Bro-Dev] Initial btest setup merged into master Fyi, per the commit mails, I've merged the initial btest setup from Don's branch into master so that we can start collecting tests centrally. Don, when you merge this back, please note that I changed some paths; see the commit messages. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org _______________________________________________ bro-dev mailing list bro-dev at bro-ids.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev -- Don Appleman National Center for Supercomputing Applications 2006B NCSA, 1205 W. Clark St. Urbana, IL 61801 217/333-6340 appleman at ncsa.illinois.edu From bro at tracker.icir.org Wed Mar 30 09:44:40 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Wed, 30 Mar 2011 16:44:40 -0000 Subject: [Bro-Dev] #426: Bug in regular expressions Message-ID: <043.1195061ba5040fe0ca11aff9502ae504@tracker.icir.org> #426: Bug in regular expressions ---------------------+----------------- Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: Keywords: | ---------------------+----------------- This code: {{{ const test_regex = /([f]{1,4}){7}/ &redef; redef test_regex += /a/; }}} Results in: {{{ (gdb) r Starting program: /Users/seth/bro.git9/build/src/bro test3.bro Reading symbols for shared libraries .+++++++ done ./test3.bro, line 2: internal error: bad reference count [2] Program received signal SIGABRT, Aborted. 0x00007fff87fa7616 in __kill () (gdb) bt #0 0x00007fff87fa7616 in __kill () #1 0x00007fff88047cca in abort () #2 0x0000000100003c5a in internal_error (fmt=Could not find the frame base for "internal_error(char const*, ...)". ) at /Users/seth/bro.git9/src/util.cc:747 #3 0x000000010019e65e in bad_ref (type=Could not find the frame base for "bad_ref(int)". ) at /Users/seth/bro.git9/src/Obj.cc:263 #4 0x0000000100045838 in Unref (o=0x100cca150) at Obj.h:219 #5 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc9f70) at /Users/seth/bro.git9/src/NFA.cc:46 #6 0x0000000100045853 in Unref (o=0x100cc9f70) at Obj.h:220 #7 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc9e80) at /Users/seth/bro.git9/src/NFA.cc:46 #8 0x0000000100045853 in Unref (o=0x100cc9e80) at Obj.h:220 #9 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc9d90) at /Users/seth/bro.git9/src/NFA.cc:46 #10 0x0000000100045853 in Unref (o=0x100cc9d90) at Obj.h:220 #11 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc9ca0) at /Users/seth/bro.git9/src/NFA.cc:46 #12 0x0000000100045853 in Unref (o=0x100cc9ca0) at Obj.h:220 #13 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc9bb0) at /Users/seth/bro.git9/src/NFA.cc:46 #14 0x0000000100045853 in Unref (o=0x100cc9bb0) at Obj.h:220 #15 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc9ac0) at /Users/seth/bro.git9/src/NFA.cc:46 #16 0x0000000100045853 in Unref (o=0x100cc9ac0) at Obj.h:220 #17 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc99d0) at /Users/seth/bro.git9/src/NFA.cc:46 #18 0x0000000100045853 in Unref (o=0x100cc99d0) at Obj.h:220 #19 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc98e0) at /Users/seth/bro.git9/src/NFA.cc:46 #20 0x0000000100045853 in Unref (o=0x100cc98e0) at Obj.h:220 #21 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc97f0) at /Users/seth/bro.git9/src/NFA.cc:46 #22 0x0000000100045853 in Unref (o=0x100cc97f0) at Obj.h:220 #23 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc9700) at /Users/seth/bro.git9/src/NFA.cc:46 #24 0x0000000100045853 in Unref (o=0x100cc9700) at Obj.h:220 #25 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc9610) at /Users/seth/bro.git9/src/NFA.cc:46 #26 0x0000000100045853 in Unref (o=0x100cc9610) at Obj.h:220 #27 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cca670) at /Users/seth/bro.git9/src/NFA.cc:46 #28 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cca670) at NFA.h:73 #29 0x0000000100045853 in Unref (o=0x100cca670) at Obj.h:220 #30 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc94a0) at /Users/seth/bro.git9/src/NFA.cc:46 #31 0x0000000100045853 in Unref (o=0x100cc94a0) at Obj.h:220 #32 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc93b0) at /Users/seth/bro.git9/src/NFA.cc:46 #33 0x0000000100045853 in Unref (o=0x100cc93b0) at Obj.h:220 #34 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc92c0) at /Users/seth/bro.git9/src/NFA.cc:46 #35 0x0000000100045853 in Unref (o=0x100cc92c0) at Obj.h:220 #36 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc91d0) at /Users/seth/bro.git9/src/NFA.cc:46 #37 0x0000000100045853 in Unref (o=0x100cc91d0) at Obj.h:220 #38 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc90e0) at /Users/seth/bro.git9/src/NFA.cc:46 #39 0x0000000100045853 in Unref (o=0x100cc90e0) at Obj.h:220 #40 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8ff0) at /Users/seth/bro.git9/src/NFA.cc:46 #41 0x0000000100045853 in Unref (o=0x100cc8ff0) at Obj.h:220 #42 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8f00) at /Users/seth/bro.git9/src/NFA.cc:46 #43 0x0000000100045853 in Unref (o=0x100cc8f00) at Obj.h:220 #44 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8e10) at /Users/seth/bro.git9/src/NFA.cc:46 #45 0x0000000100045853 in Unref (o=0x100cc8e10) at Obj.h:220 #46 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8d20) at /Users/seth/bro.git9/src/NFA.cc:46 #47 0x0000000100045853 in Unref (o=0x100cc8d20) at Obj.h:220 #48 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8c30) at /Users/seth/bro.git9/src/NFA.cc:46 #49 0x0000000100045853 in Unref (o=0x100cc8c30) at Obj.h:220 #50 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8b40) at /Users/seth/bro.git9/src/NFA.cc:46 #51 0x0000000100045853 in Unref (o=0x100cc8b40) at Obj.h:220 #52 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8a50) at /Users/seth/bro.git9/src/NFA.cc:46 #53 0x0000000100045853 in Unref (o=0x100cc8a50) at Obj.h:220 #54 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8960) at /Users/seth/bro.git9/src/NFA.cc:46 #55 0x0000000100045853 in Unref (o=0x100cc8960) at Obj.h:220 #56 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cca5c0) at /Users/seth/bro.git9/src/NFA.cc:46 #57 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cca5c0) at NFA.h:73 #58 0x0000000100045853 in Unref (o=0x100cca5c0) at Obj.h:220 #59 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc87f0) at /Users/seth/bro.git9/src/NFA.cc:46 #60 0x0000000100045853 in Unref (o=0x100cc87f0) at Obj.h:220 #61 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8700) at /Users/seth/bro.git9/src/NFA.cc:46 #62 0x0000000100045853 in Unref (o=0x100cc8700) at Obj.h:220 #63 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8610) at /Users/seth/bro.git9/src/NFA.cc:46 #64 0x0000000100045853 in Unref (o=0x100cc8610) at Obj.h:220 #65 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8520) at /Users/seth/bro.git9/src/NFA.cc:46 #66 0x0000000100045853 in Unref (o=0x100cc8520) at Obj.h:220 #67 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8430) at /Users/seth/bro.git9/src/NFA.cc:46 #68 0x0000000100045853 in Unref (o=0x100cc8430) at Obj.h:220 #69 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8340) at /Users/seth/bro.git9/src/NFA.cc:46 #70 0x0000000100045853 in Unref (o=0x100cc8340) at Obj.h:220 #71 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8250) at /Users/seth/bro.git9/src/NFA.cc:46 #72 0x0000000100045853 in Unref (o=0x100cc8250) at Obj.h:220 #73 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8160) at /Users/seth/bro.git9/src/NFA.cc:46 #74 0x0000000100045853 in Unref (o=0x100cc8160) at Obj.h:220 #75 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc8070) at /Users/seth/bro.git9/src/NFA.cc:46 #76 0x0000000100045853 in Unref (o=0x100cc8070) at Obj.h:220 #77 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc7f80) at /Users/seth/bro.git9/src/NFA.cc:46 #78 0x0000000100045853 in Unref (o=0x100cc7f80) at Obj.h:220 #79 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc7e90) at /Users/seth/bro.git9/src/NFA.cc:46 #80 0x0000000100045853 in Unref (o=0x100cc7e90) at Obj.h:220 #81 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc7da0) at /Users/seth/bro.git9/src/NFA.cc:46 #82 0x0000000100045853 in Unref (o=0x100cc7da0) at Obj.h:220 #83 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc7cb0) at /Users/seth/bro.git9/src/NFA.cc:46 #84 0x0000000100045853 in Unref (o=0x100cc7cb0) at Obj.h:220 #85 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cca510) at /Users/seth/bro.git9/src/NFA.cc:46 #86 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cca510) at NFA.h:73 #87 0x0000000100045853 in Unref (o=0x100cca510) at Obj.h:220 #88 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc7b40) at /Users/seth/bro.git9/src/NFA.cc:46 #89 0x0000000100045853 in Unref (o=0x100cc7b40) at Obj.h:220 #90 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc7a50) at /Users/seth/bro.git9/src/NFA.cc:46 #91 0x0000000100045853 in Unref (o=0x100cc7a50) at Obj.h:220 #92 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc7960) at /Users/seth/bro.git9/src/NFA.cc:46 #93 0x0000000100045853 in Unref (o=0x100cc7960) at Obj.h:220 #94 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc7870) at /Users/seth/bro.git9/src/NFA.cc:46 #95 0x0000000100045853 in Unref (o=0x100cc7870) at Obj.h:220 #96 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc7780) at /Users/seth/bro.git9/src/NFA.cc:46 #97 0x0000000100045853 in Unref (o=0x100cc7780) at Obj.h:220 #98 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc7690) at /Users/seth/bro.git9/src/NFA.cc:46 #99 0x0000000100045853 in Unref (o=0x100cc7690) at Obj.h:220 #100 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc75a0) at /Users/seth/bro.git9/src/NFA.cc:46 #101 0x0000000100045853 in Unref (o=0x100cc75a0) at Obj.h:220 #102 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc74b0) at /Users/seth/bro.git9/src/NFA.cc:46 #103 0x0000000100045853 in Unref (o=0x100cc74b0) at Obj.h:220 #104 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc73c0) at /Users/seth/bro.git9/src/NFA.cc:46 #105 0x0000000100045853 in Unref (o=0x100cc73c0) at Obj.h:220 #106 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc72d0) at /Users/seth/bro.git9/src/NFA.cc:46 #107 0x0000000100045853 in Unref (o=0x100cc72d0) at Obj.h:220 #108 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc71e0) at /Users/seth/bro.git9/src/NFA.cc:46 #109 0x0000000100045853 in Unref (o=0x100cc71e0) at Obj.h:220 #110 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc70f0) at /Users/seth/bro.git9/src/NFA.cc:46 #111 0x0000000100045853 in Unref (o=0x100cc70f0) at Obj.h:220 #112 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc7000) at /Users/seth/bro.git9/src/NFA.cc:46 #113 0x0000000100045853 in Unref (o=0x100cc7000) at Obj.h:220 #114 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cca460) at /Users/seth/bro.git9/src/NFA.cc:46 #115 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cca460) at NFA.h:73 #116 0x0000000100045853 in Unref (o=0x100cca460) at Obj.h:220 #117 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6e90) at /Users/seth/bro.git9/src/NFA.cc:46 #118 0x0000000100045853 in Unref (o=0x100cc6e90) at Obj.h:220 #119 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6da0) at /Users/seth/bro.git9/src/NFA.cc:46 #120 0x0000000100045853 in Unref (o=0x100cc6da0) at Obj.h:220 #121 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6cb0) at /Users/seth/bro.git9/src/NFA.cc:46 #122 0x0000000100045853 in Unref (o=0x100cc6cb0) at Obj.h:220 #123 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6bc0) at /Users/seth/bro.git9/src/NFA.cc:46 #124 0x0000000100045853 in Unref (o=0x100cc6bc0) at Obj.h:220 #125 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6ad0) at /Users/seth/bro.git9/src/NFA.cc:46 #126 0x0000000100045853 in Unref (o=0x100cc6ad0) at Obj.h:220 #127 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc69e0) at /Users/seth/bro.git9/src/NFA.cc:46 #128 0x0000000100045853 in Unref (o=0x100cc69e0) at Obj.h:220 #129 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc68f0) at /Users/seth/bro.git9/src/NFA.cc:46 #130 0x0000000100045853 in Unref (o=0x100cc68f0) at Obj.h:220 #131 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6800) at /Users/seth/bro.git9/src/NFA.cc:46 #132 0x0000000100045853 in Unref (o=0x100cc6800) at Obj.h:220 #133 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6710) at /Users/seth/bro.git9/src/NFA.cc:46 #134 0x0000000100045853 in Unref (o=0x100cc6710) at Obj.h:220 #135 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6620) at /Users/seth/bro.git9/src/NFA.cc:46 #136 0x0000000100045853 in Unref (o=0x100cc6620) at Obj.h:220 #137 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6530) at /Users/seth/bro.git9/src/NFA.cc:46 #138 0x0000000100045853 in Unref (o=0x100cc6530) at Obj.h:220 #139 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6440) at /Users/seth/bro.git9/src/NFA.cc:46 #140 0x0000000100045853 in Unref (o=0x100cc6440) at Obj.h:220 #141 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6350) at /Users/seth/bro.git9/src/NFA.cc:46 #142 0x0000000100045853 in Unref (o=0x100cc6350) at Obj.h:220 #143 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cca3b0) at /Users/seth/bro.git9/src/NFA.cc:46 #144 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cca3b0) at NFA.h:73 #145 0x0000000100045853 in Unref (o=0x100cca3b0) at Obj.h:220 #146 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc61e0) at /Users/seth/bro.git9/src/NFA.cc:46 #147 0x0000000100045853 in Unref (o=0x100cc61e0) at Obj.h:220 #148 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc60f0) at /Users/seth/bro.git9/src/NFA.cc:46 #149 0x0000000100045853 in Unref (o=0x100cc60f0) at Obj.h:220 #150 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc6000) at /Users/seth/bro.git9/src/NFA.cc:46 #151 0x0000000100045853 in Unref (o=0x100cc6000) at Obj.h:220 #152 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc5f10) at /Users/seth/bro.git9/src/NFA.cc:46 #153 0x0000000100045853 in Unref (o=0x100cc5f10) at Obj.h:220 #154 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc5e20) at /Users/seth/bro.git9/src/NFA.cc:46 #155 0x0000000100045853 in Unref (o=0x100cc5e20) at Obj.h:220 #156 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc5d30) at /Users/seth/bro.git9/src/NFA.cc:46 #157 0x0000000100045853 in Unref (o=0x100cc5d30) at Obj.h:220 #158 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc5c40) at /Users/seth/bro.git9/src/NFA.cc:46 #159 0x0000000100045853 in Unref (o=0x100cc5c40) at Obj.h:220 #160 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc5b50) at /Users/seth/bro.git9/src/NFA.cc:46 #161 0x0000000100045853 in Unref (o=0x100cc5b50) at Obj.h:220 #162 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc5a60) at /Users/seth/bro.git9/src/NFA.cc:46 #163 0x0000000100045853 in Unref (o=0x100cc5a60) at Obj.h:220 #164 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc5970) at /Users/seth/bro.git9/src/NFA.cc:46 #165 0x0000000100045853 in Unref (o=0x100cc5970) at Obj.h:220 #166 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc5880) at /Users/seth/bro.git9/src/NFA.cc:46 #167 0x0000000100045853 in Unref (o=0x100cc5880) at Obj.h:220 #168 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc5790) at /Users/seth/bro.git9/src/NFA.cc:46 #169 0x0000000100045853 in Unref (o=0x100cc5790) at Obj.h:220 #170 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc56e0) at /Users/seth/bro.git9/src/NFA.cc:46 #171 0x0000000100045853 in Unref (o=0x100cc56e0) at Obj.h:220 #172 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cca2c0) at /Users/seth/bro.git9/src/NFA.cc:46 #173 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cca2c0) at NFA.h:73 #174 0x0000000100045853 in Unref (o=0x100cca2c0) at Obj.h:220 #175 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cc5540) at /Users/seth/bro.git9/src/NFA.cc:46 #176 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cc5540) at NFA.h:73 #177 0x0000000100045853 in Unref (o=0x100cc5540) at Obj.h:220 #178 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cc5490) at /Users/seth/bro.git9/src/NFA.cc:46 #179 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cc5490) at NFA.h:73 #180 0x0000000100045853 in Unref (o=0x100cc5490) at Obj.h:220 #181 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cc55f0) at /Users/seth/bro.git9/src/NFA.cc:46 #182 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cc55f0) at NFA.h:73 #183 0x0000000100045853 in Unref (o=0x100cc55f0) at Obj.h:220 #184 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cc52b0) at /Users/seth/bro.git9/src/NFA.cc:46 #185 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cc52b0) at NFA.h:73 #186 0x0000000100045853 in Unref (o=0x100cc52b0) at Obj.h:220 #187 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cc51c0) at /Users/seth/bro.git9/src/NFA.cc:46 #188 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cc51c0) at NFA.h:73 #189 0x0000000100045853 in Unref (o=0x100cc51c0) at Obj.h:220 #190 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cc53a0) at /Users/seth/bro.git9/src/NFA.cc:46 #191 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cc53a0) at NFA.h:73 #192 0x0000000100045853 in Unref (o=0x100cc53a0) at Obj.h:220 #193 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cc4ef0) at /Users/seth/bro.git9/src/NFA.cc:46 #194 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cc4ef0) at NFA.h:73 #195 0x0000000100045853 in Unref (o=0x100cc4ef0) at Obj.h:220 #196 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cc4e00) at /Users/seth/bro.git9/src/NFA.cc:46 #197 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cc4e00) at NFA.h:73 #198 0x0000000100045853 in Unref (o=0x100cc4e00) at Obj.h:220 #199 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cc4fe0) at /Users/seth/bro.git9/src/NFA.cc:46 #200 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cc4fe0) at NFA.h:73 #201 0x0000000100045853 in Unref (o=0x100cc4fe0) at Obj.h:220 #202 0x0000000100190672 in NFA_State::~NFA_State (this=0x100cc49b0) at /Users/seth/bro.git9/src/NFA.cc:46 #203 0x0000000100045853 in Unref (o=0x100cc49b0) at Obj.h:220 #204 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cca240) at /Users/seth/bro.git9/src/NFA.cc:46 #205 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cca240) at NFA.h:73 #206 0x0000000100045853 in Unref (o=0x100cca240) at Obj.h:220 #207 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cc47b0) at /Users/seth/bro.git9/src/NFA.cc:46 #208 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cc47b0) at NFA.h:73 #209 0x0000000100045853 in Unref (o=0x100cc47b0) at Obj.h:220 #210 0x0000000100190836 in NFA_State::~NFA_State (this=0x100cc46c0) at /Users/seth/bro.git9/src/NFA.cc:46 #211 0x000000010007dddd in EpsilonState::~EpsilonState (this=0x100cc46c0) at NFA.h:73 #212 0x0000000100045853 in Unref (o=0x100cc46c0) at Obj.h:220 #213 0x00000001001900ff in NFA_Machine::~NFA_Machine (this=0x100cc4640) at /Users/seth/bro.git9/src/NFA.cc:162 #214 0x0000000100045853 in Unref (o=0x100cc4640) at Obj.h:220 #215 0x00000001000cc611 in DFA_Machine::~DFA_Machine (this=0x100cc4ae0) at /Users/seth/bro.git9/src/DFA.cc:534 #216 0x0000000100045853 in Unref (o=0x100cc4ae0) at Obj.h:220 #217 0x00000001001b6150 in Specific_RE_Matcher::~Specific_RE_Matcher (this=0x100cbd120) at /Users/seth/bro.git9/src/RE.cc:44 #218 0x00000001001b62ef in RE_Matcher::~RE_Matcher (this=0x100cbccf0) at /Users/seth/bro.git9/src/RE.cc:446 #219 0x0000000100236256 in PatternVal::SetMatcher (this=0x100ccae60, re=0x100ccdd70) at /Users/seth/bro.git9/src/Val.cc:1423 #220 0x000000010023a802 in PatternVal::AddTo (this=0x100ccdc60, v=0x100ccae60) at /Users/seth/bro.git9/src/Val.cc:1416 #221 0x0000000100169c1f in ID::SetVal (this=0x100cbcd20, v=0x100ccdc60, c=INIT_EXTRA) at /Users/seth/bro.git9/src/ID.cc:157 #222 0x0000000100246e0f in make_var (id=0x100cbcd20, t=0x100b4b560, c=INIT_EXTRA, init=0x100ccdcf0, attr=0x0, dt=VAR_REDEF, do_init=1) at /Users/seth/bro.git9/src/Var.cc:160 #223 0x00000001002472fe in add_global (id=0x100cbcd20, t=0x0, c=INIT_EXTRA, init=0x100ccdcf0, attr=0x0, dt=VAR_REDEF) at /Users/seth/bro.git9/src/Var.cc:189 #224 0x00000001000825d0 in yyparse () at parse.y:859 #225 0x000000010009288f in main (argc=2, argv=0x7fff5fbfef20) at /Users/seth/bro.git9/src/main.cc:756 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Wed Mar 30 14:36:04 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Wed, 30 Mar 2011 21:36:04 -0000 Subject: [Bro-Dev] #337: BroCtl's top has trouble with large values In-Reply-To: <044.194bdf0be7a34bfafe07f70c1ae7871f@tracker.icir.org> References: <044.194bdf0be7a34bfafe07f70c1ae7871f@tracker.icir.org> Message-ID: <059.8083113fbc21d2bf8d1c523b5b1e1564@tracker.icir.org> #337: BroCtl's top has trouble with large values -------------------------+------------------------ Reporter: robin | Owner: robin Type: Problem | Status: reopened Priority: Normal | Milestone: Bro1.6 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Changes (by leres): * status: closed => reopened * resolution: Merged => Comment: It turns out "rss" has the same problem: {{{ File "/home/users/bro/bro-152/lib/broctl/BroControl/control.py", line 589, in getTopOutput d["rss"] = int(p[2]) ValueError: invalid literal for int(): 2.15063e+09 }}} I'll attach a simple patch for this. -- Ticket URL: Bro Tracker Bro Issue Tracker From leres at ee.lbl.gov Wed Mar 30 14:44:46 2011 From: leres at ee.lbl.gov (Craig Leres) Date: Wed, 30 Mar 2011 14:44:46 -0700 Subject: [Bro-Dev] http://tracker.icir.org/bro/newticket Message-ID: <4D93A44E.5000105@ee.lbl.gov> It would be good to add 1.5.3 to the version pulldown for net tickets. Craig From bro at tracker.icir.org Wed Mar 30 14:59:33 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Wed, 30 Mar 2011 21:59:33 -0000 Subject: [Bro-Dev] #427: Assertion failed: (!v), function Eval, file Trigger.cc Message-ID: <044.bdaec04b30d46256de33127427d55fbf@tracker.icir.org> #427: Assertion failed: (!v), function Eval, file Trigger.cc -------------------+--------------------- Reporter: leres | Type: Problem Status: new | Priority: Normal Milestone: | Component: Bro Version: 1.5.2 | Keywords: -------------------+--------------------- We're running a standalone bro/broctl config (1.5.3) to watch a 10G link that has IPv4 and IPv6 traffic on it and have seen several crashes over the last week. I recompiled with -g so there are symbols for the stack trace. I think it's probably a coincidence that the "lookup_addr() only supports IPv4 addresses" error appears right before the assert() failure so I opened a new ticket. {{{ Variable "arg_cond" is not available. Variable "flow" is not available. Variable "no_remote" is not available. Variable "this" is not available. Variable "argc" is not available. ==== stderr.log listening on nxge0 /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.36622-131.243.2.88.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.88.23-131.243.64.209.36622: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.36872-131.243.2.237.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.237.23-131.243.64.209.36872: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.33337-131.243.2.88.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.88.23-131.243.64.209.33337: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.237.23-131.243.64.209.33525: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.33525-131.243.2.237.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.56557-131.243.2.88.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.88.23-131.243.64.209.56557: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.56926-131.243.2.237.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.237.23-131.243.64.209.56926: No such file or directory 1301383820.573102 run-time error: cannot find connection /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.52266-131.243.2.88.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.88.23-131.243.64.209.52266: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.52426-131.243.2.237.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.237.23-131.243.64.209.52426: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.88.23-131.243.64.209.49884: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.49884-131.243.2.88.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.49986-131.243.2.237.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.237.23-131.243.64.209.49986: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.88.23-131.243.64.209.46119: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.46119-131.243.2.88.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/keys.trb.131.243.64.209.46265-131.243.2.237.23: No such file or directory /usr/local/share/broctl/scripts/create-link-for-log: line 60: .link.xscript.log/telnet.trb.131.243.2.237.23-131.243.64.209.46265: No such file or directory 1301449356.137171 error: bad dotted address: 2001:da8:200:900e:0:5efe:166.111.66.23|1032 1301449356.137171 /usr/local/share/bro/broctl/mail-alarms.bro, line 79 (lookup_addr(MailAlarms::host)): run-time error, lookup_addr() only supports IPv4 addresses Assertion failed: (!v), function Eval, file Trigger.cc, line 194. /usr/local/share/broctl/scripts/run-bro: line 73: 79724 Abort trap: 6 (core dumped) nohup $tmpbro $@ ==== stdout.log X509: Using the default trusted cert path. ==== .status RUNNING [net_run] ==== No prof.log. bro.core Core was generated by `bro'. Program terminated with signal 6, Aborted. #0 0x000000080177dfcc in kill () from /lib/libc.so.7 #0 0x000000080177dfcc in kill () from /lib/libc.so.7 #1 0x000000080177cdcb in abort () from /lib/libc.so.7 #2 0x00000008017661a5 in __assert () from /lib/libc.so.7 #3 0x00000000005619ab in Trigger::Eval (this=0x373fd88) at Trigger.cc:194 #4 0x0000000000561f12 in Trigger (this=0x373fd88, arg_cond=) at Trigger.cc:133 #5 0x000000000054ae11 in WhenStmt::Exec (this=0xb79960, f=0x4f0fbc0, flow=) at Stmt.cc:1826 #6 0x00000000005482d1 in StmtList::Exec (this=0xb71e68, f=0x4f0fbc0, flow=@0x7fffffffd87c) at Stmt.cc:1432 #7 0x00000000004b1470 in BroFunc::Call (this=0xb79a10, args=0x4b1dbf8, parent=0x343a548) at Func.cc:308 #8 0x000000000049ea03 in CallExpr::Eval (this=0xb82de0, f=0x343a548) at Expr.cc:4629 #9 0x000000000054e538 in ExprStmt::Exec (this=0xb82e70, f=0x343a548, flow=@0x7fffffffda7c) at Stmt.cc:397 #10 0x00000000005482d1 in StmtList::Exec (this=0xb7a730, f=0x343a548, flow=@0x7fffffffda7c) at Stmt.cc:1432 #11 0x00000000004b1470 in BroFunc::Call (this=0xb82f00, args=0x567af50, parent=0x4cbe248) at Func.cc:308 #12 0x000000000049ea03 in CallExpr::Eval (this=0xb85410, f=0x4cbe248) at Expr.cc:4629 #13 0x000000000054e538 in ExprStmt::Exec (this=0xb854a0, f=0x4cbe248, flow=@0x7fffffffdc7c) at Stmt.cc:397 #14 0x00000000005482d1 in StmtList::Exec (this=0xb84970, f=0x4cbe248, flow=@0x7fffffffdc7c) at Stmt.cc:1432 #15 0x00000000004b1470 in BroFunc::Call (this=0x9bfc38, args=0x45a6950, parent=0x0) at Func.cc:308 #16 0x0000000000468caa in EventHandler::Call (this=0x9bfd80, vl=0x45a6950, no_remote=) at EventHandler.cc:67 #17 0x0000000000468521 in EventMgr::Dispatch (this=) at Event.h:43 #18 0x0000000000468638 in EventMgr::Drain (this=0x788f40) at Event.cc:119 #19 0x00000000004f74b9 in net_packet_dispatch (t=1301449356.137171, hdr=0x1922210, pkt=0x8018cd032 "", hdr_size=14, src_ps=0x19221d0, pkt_elem=0x0) at Net.cc:436 #20 0x0000000000505c38 in PktSrc::Process (this=0x19221d0) at PktSrc.cc:199 #21 0x00000000004f7725 in net_run () at Net.cc:528 #22 0x00000000004095bc in main (argc=) at main.cc:999 -- [Automatically generated.] }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From gregor at icir.org Wed Mar 30 15:57:31 2011 From: gregor at icir.org (Gregor Maier) Date: Wed, 30 Mar 2011 15:57:31 -0700 Subject: [Bro-Dev] vectors (Re: syntax consistency question) In-Reply-To: <013A5B49-EADE-4263-A489-C706162689EC@icir.org> References: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> <20110329004924.GB43817@icir.org> <013A5B49-EADE-4263-A489-C706162689EC@icir.org> Message-ID: <4D93B55B.6000704@icir.org> IIRC, there's a #define that decides whether it's 0 or 1. (And yes, 0 based seems great). IIRC, 0 might be used as a special value in C++ code but I'm not sure.... cu gregor On 3/28/11 18:00 , Seth Hall wrote: > > On Mar 28, 2011, at 8:49 PM, Robin Sommer wrote: > >> Another vector wish list item: can we please make them zero-based? I >> propose to do that immediately while we're working on scripts anyway. > > > Seems reasonable to me. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -- Gregor Maier Int. Computer Science Institute (ICSI) 1947 Center St., Ste. 600 Berkeley, CA 94704, USA http://www.icir.org/gregor/ From bro at tracker.icir.org Wed Mar 30 19:32:40 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Thu, 31 Mar 2011 02:32:40 -0000 Subject: [Bro-Dev] #428: Cleanup aux Message-ID: <044.a14838bdcd19a5e2ec52599102884d78@tracker.icir.org> #428: Cleanup aux --------------------+-------------------- Reporter: robin | Owner: robin Type: Task | Status: new Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Keywords: | --------------------+-------------------- Remove hf/cf and the SSL cert scripts. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Mar 30 20:45:59 2011 From: robin at icir.org (Robin Sommer) Date: Wed, 30 Mar 2011 20:45:59 -0700 Subject: [Bro-Dev] vectors (Re: syntax consistency question) In-Reply-To: <4D93B55B.6000704@icir.org> References: <13786C9F-0E1C-49A2-B7FE-C2365ABCED34@icir.org> <20110329004924.GB43817@icir.org> <013A5B49-EADE-4263-A489-C706162689EC@icir.org> <4D93B55B.6000704@icir.org> Message-ID: <20110331034559.GC58238@icir.org> On Wed, Mar 30, 2011 at 15:57 -0700, you wrote: > IIRC, there's a #define that decides whether it's 0 or 1. Right, technically the change is trivial, I was mainly looking for a consensus. Seem we have one. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.icir.org Wed Mar 30 20:46:42 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Thu, 31 Mar 2011 03:46:42 -0000 Subject: [Bro-Dev] #429: Change vector indices to be zero-based Message-ID: <044.3f36644cc028ca5ce232deb62f75f6cb@tracker.icir.org> #429: Change vector indices to be zero-based --------------------+-------------------- Reporter: robin | Owner: robin Type: Task | Status: new Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: Keywords: | --------------------+-------------------- See subject. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.icir.org Wed Mar 30 20:59:20 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Thu, 31 Mar 2011 03:59:20 -0000 Subject: [Bro-Dev] #427: Assertion failed: (!v), function Eval, file Trigger.cc In-Reply-To: <044.bdaec04b30d46256de33127427d55fbf@tracker.icir.org> References: <044.bdaec04b30d46256de33127427d55fbf@tracker.icir.org> Message-ID: <059.0ae737d6460f30038e9f993e16594970@tracker.icir.org> #427: Assertion failed: (!v), function Eval, file Trigger.cc ----------------------+-------------------- Reporter: leres | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro1.6 Component: Bro | Version: 1.5.2 Resolution: | Keywords: ipv6 ----------------------+-------------------- Changes (by robin): * keywords: => ipv6 * milestone: => Bro1.6 Comment: On Wed, Mar 30, 2011 at 21:59 -0000, you wrote: > I think it's probably a coincidence that the "lookup_addr() only supports > IPv4 addresses" error appears right before the assert() failure so I > opened a new ticket. Without looking more closely yet, my guess is it's still related to IPv6 in some form, perhaps the trigger setup doesn't handle the error cases correctly in some cases. Tagging this with "ipv6" and marking for 1.6 to at least check whether it's something obvious we can fix quickly. -- Ticket URL: Bro Tracker Bro Issue Tracker From vern at icir.org Wed Mar 30 21:00:28 2011 From: vern at icir.org (Vern Paxson) Date: Wed, 30 Mar 2011 21:00:28 -0700 Subject: [Bro-Dev] vectors (Re: syntax consistency question) In-Reply-To: <20110331034559.GC58238@icir.org> (Wed, 30 Mar 2011 20:45:59 PDT). Message-ID: <20110331040028.5169236A365@taffy.ICSI.Berkeley.EDU> > Right, technically the change is trivial, I was mainly looking for a > consensus. Seem we have one. Well, depends on how you define consensus. I prefer indexing to start with 1. I think use of 0-basing is a holdover from C's too-low-level model equating arrays with pointers. But given I'm the lone holdout, and vectors aren't something we've used much, I'm not going to fight hard. However, I can picture an attribute &base=1 or whatever to re-base particular vectors. Vern From robin at icir.org Wed Mar 30 21:05:03 2011 From: robin at icir.org (Robin Sommer) Date: Wed, 30 Mar 2011 21:05:03 -0700 Subject: [Bro-Dev] http://tracker.icir.org/bro/newticket In-Reply-To: <4D93A44E.5000105@ee.lbl.gov> References: <4D93A44E.5000105@ee.lbl.gov> Message-ID: <20110331040503.GG58238@icir.org> On Wed, Mar 30, 2011 at 14:44 -0700, you wrote: > It would be good to add 1.5.3 to the version pulldown for net tickets. That's already there, do you mean 1.5.4? I've now added that and also adapted the 1.5.x in the home page accordingly. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Wed Mar 30 21:09:41 2011 From: robin at icir.org (Robin Sommer) Date: Wed, 30 Mar 2011 21:09:41 -0700 Subject: [Bro-Dev] vectors (Re: syntax consistency question) In-Reply-To: <20110331040028.5169236A365@taffy.ICSI.Berkeley.EDU> References: <20110331034559.GC58238@icir.org> <20110331040028.5169236A365@taffy.ICSI.Berkeley.EDU> Message-ID: <20110331040941.GH58238@icir.org> On Wed, Mar 30, 2011 at 21:00 -0700, you wrote: > Well, depends on how you define consensus. Defined as "hearing no objections so far". :) > I prefer indexing to start with 1. I think use of 0-basing is a > holdover from C's too-low-level model equating arrays with pointers. Maybe, but it's so common also with many other languages that I find it not worth the confusion that a different base causes. > much, I'm not going to fight hard. However, I can picture an attribute > &base=1 or whatever to re-base particular vectors. Oh, please not. That would be even worse because now one has to check everytime whether a vector has that attribute. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From leres at ee.lbl.gov Wed Mar 30 21:28:25 2011 From: leres at ee.lbl.gov (Craig Leres) Date: Wed, 30 Mar 2011 21:28:25 -0700 Subject: [Bro-Dev] http://tracker.icir.org/bro/newticket In-Reply-To: <20110331040503.GG58238@icir.org> References: <4D93A44E.5000105@ee.lbl.gov> <20110331040503.GG58238@icir.org> Message-ID: <4D9402E9.4020404@ee.lbl.gov> On 3/30/2011 9:05 PM, Robin Sommer wrote: > > On Wed, Mar 30, 2011 at 14:44 -0700, you wrote: > >> It would be good to add 1.5.3 to the version pulldown for net tickets. > > That's already there, do you mean 1.5.4? I've now added that and also > adapted the 1.5.x in the home page accordingly. Actually I might have been confused because they're out of order; I saw 1.5.1 followed by 1.5.2 at the bottom... Craig -------------- next part -------------- A non-text attachment was scrubbed... Name: tracker.png Type: image/png Size: 10056 bytes Desc: not available Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20110330/e9a29e7c/attachment.bin From robin at icir.org Wed Mar 30 21:33:22 2011 From: robin at icir.org (Robin Sommer) Date: Wed, 30 Mar 2011 21:33:22 -0700 Subject: [Bro-Dev] http://tracker.icir.org/bro/newticket In-Reply-To: <4D940282.1080302@ee.lbl.gov> References: <4D93A44E.5000105@ee.lbl.gov> <20110331040503.GG58238@icir.org> <4D940282.1080302@ee.lbl.gov> Message-ID: <20110331043322.GN58238@icir.org> On Wed, Mar 30, 2011 at 21:26 -0700, you wrote: > Actually I might have been confused because they're out of order; I saw > 1.5.1 followed by 1.5.2 at the bottom... Yeah, that's kind of weird. Not sure what determines the order but don't think we can change it. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Wed Mar 30 21:53:05 2011 From: robin at icir.org (Robin Sommer) Date: Wed, 30 Mar 2011 21:53:05 -0700 Subject: [Bro-Dev] Bug in drop.bro and patch In-Reply-To: References: Message-ID: <20110331045305.GR58238@icir.org> Can you please file a ticket for this and set the milestone to 1.6? Thanks, Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.icir.org Thu Mar 31 12:12:46 2011 From: bro at tracker.icir.org (Bro Tracker) Date: Thu, 31 Mar 2011 19:12:46 -0000 Subject: [Bro-Dev] #430: Build error in LogMgr.cc Message-ID: <047.b7495de40c3e3ce4a9d440149e22b540@tracker.icir.org> #430: Build error in LogMgr.cc ----------------------+----------------------- Reporter: appleman | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/topic Keywords: | ----------------------+----------------------- I cloned the repository as master and built bro. I then did ... git checkout topic/robin/logging-internals make ... and it died with the following errors: [ 16%] Building CXX object src/CMakeFiles/bro.dir/LogMgr.cc.o /home/appleman/bro/trunk/scripts/bro/src/LogMgr.cc: In member function ?bool LogMgr::AddFilter(EnumVal*, RecordVal*)?: /home/appleman/bro/trunk/scripts/bro/src/LogMgr.cc:691: error: ?transform? is not a member of ?std? make[3]: *** [src/CMakeFiles/bro.dir/LogMgr.cc.o] Error 1 make[3]: Leaving directory `/home/appleman/bro/trunk/scripts/bro/build' make[2]: *** [src/CMakeFiles/bro.dir/all] Error 2 make[2]: Leaving directory `/home/appleman/bro/trunk/scripts/bro/build' make[1]: *** [all] Error 2 make[1]: Leaving directory `/home/appleman/bro/trunk/scripts/bro/build' make: *** [all] Error 2 My gcc version is 4.4.3 Problem was resolved for me by adding #include to LogMgr.cc -- Ticket URL: Bro Tracker Bro Issue Tracker