From jira at bro-tracker.atlassian.net Wed Oct 1 16:51:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 1 Oct 2014 18:51:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1266) PktSrc rewrite introduces packet drops In-Reply-To: References: Message-ID: Robin Sommer created BIT-1266: --------------------------------- Summary: PktSrc rewrite introduces packet drops Key: BIT-1266 URL: https://bro-tracker.atlassian.net/browse/BIT-1266 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Reporter: Robin Sommer Assignee: Robin Sommer Priority: High Current master drops significantly more packets than 2.3. Looks like it's the packet source reorg: going back to 3caecadf0a3028e1dad7f446a92ffb7560c9a734 lets drop rates go back to normal. Reported by John Donaldson. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Wed Oct 1 18:56:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 1 Oct 2014 20:56:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1266) PktSrc rewrite introduces packet drops In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18219#comment-18219 ] Jon Siwek commented on BIT-1266: -------------------------------- I haven't reproduced the problem exactly, but I was monitoring the behavior of -Bmain-loop for various commits and noticed a significant difference that might be due to a (unintended) difference in iosource::Manager::FindSoonest's fd_set construction -- quickly hacking out my FD_Set wrapper returned things to "normal". Still in process of figuring that out, but thought I'd mention it since that change wasn't specific to the PktSrc reorg. > PktSrc rewrite introduces packet drops > -------------------------------------- > > Key: BIT-1266 > URL: https://bro-tracker.atlassian.net/browse/BIT-1266 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Assignee: Robin Sommer > Priority: High > > Current master drops significantly more packets than 2.3. Looks like it's the packet source reorg: going back to 3caecadf0a3028e1dad7f446a92ffb7560c9a734 lets drop rates go back to normal. > Reported by John Donaldson. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Wed Oct 1 19:06:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 1 Oct 2014 21:06:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1266) PktSrc rewrite introduces packet drops In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18220#comment-18220 ] Jon Siwek commented on BIT-1266: -------------------------------- Oh, and another difference I think is that PktSrc isn't doing a SetIdle(false) when a packet is available? Didn't appear related to the behavior I mentioned in the last comment, but maybe it does factor in to the packet drop problem. > PktSrc rewrite introduces packet drops > -------------------------------------- > > Key: BIT-1266 > URL: https://bro-tracker.atlassian.net/browse/BIT-1266 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Assignee: Robin Sommer > Priority: High > > Current master drops significantly more packets than 2.3. Looks like it's the packet source reorg: going back to 3caecadf0a3028e1dad7f446a92ffb7560c9a734 lets drop rates go back to normal. > Reported by John Donaldson. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Wed Oct 1 20:06:07 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 1 Oct 2014 22:06:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1267) packet source prefix separator in node.cfg breaks broctl In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1267: ---------------------------------- Summary: packet source prefix separator in node.cfg breaks broctl Key: BIT-1267 URL: https://bro-tracker.atlassian.net/browse/BIT-1267 Project: Bro Issue Tracker Issue Type: Problem Components: BroControl Reporter: Daniel Thayer Attempting to use the new packet source prefix separator (see BIT-1249) in node.cfg prevents broctl from being able to start due to the use of Python's ConfigParser (it interprets % as a special character). For example: interface=test%eth0 -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Wed Oct 1 20:31:07 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 1 Oct 2014 22:31:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1267) packet source prefix separator in node.cfg breaks broctl In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18221#comment-18221 ] Daniel Thayer commented on BIT-1267: ------------------------------------ Using a different class (instead of SafeConfigParser) from the ConfigParser module seems to fix the problem. > packet source prefix separator in node.cfg breaks broctl > -------------------------------------------------------- > > Key: BIT-1267 > URL: https://bro-tracker.atlassian.net/browse/BIT-1267 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > > Attempting to use the new packet source prefix separator (see BIT-1249) > in node.cfg prevents broctl from being able to start due to the use of > Python's ConfigParser (it interprets % as a special character). > For example: > interface=test%eth0 -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From robin at icir.org Thu Oct 2 08:29:46 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 2 Oct 2014 08:29:46 -0700 Subject: [Bro-Dev] [JIRA] (BIT-1266) PktSrc rewrite introduces packet drops In-Reply-To: References: Message-ID: <20141002152946.GB76556@icir.org> > Oh, and another difference I think is that PktSrc isn't doing a > SetIdle(false) when a packet is available? Good catch, indeed. The old code did that, and from a quick look it seems that got lost now. I'll lock more closely in a bit. From jira at bro-tracker.atlassian.net Thu Oct 2 08:30:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 2 Oct 2014 10:30:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1266) PktSrc rewrite introduces packet drops In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18222#comment-18222 ] Robin Sommer commented on BIT-1266: ----------------------------------- Good catch, indeed. The old code did that, and from a quick look it seems that got lost now. I'll lock more closely in a bit. > PktSrc rewrite introduces packet drops > -------------------------------------- > > Key: BIT-1266 > URL: https://bro-tracker.atlassian.net/browse/BIT-1266 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Assignee: Robin Sommer > Priority: High > > Current master drops significantly more packets than 2.3. Looks like it's the packet source reorg: going back to 3caecadf0a3028e1dad7f446a92ffb7560c9a734 lets drop rates go back to normal. > Reported by John Donaldson. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From robin at icir.org Thu Oct 2 08:45:05 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 2 Oct 2014 08:45:05 -0700 Subject: [Bro-Dev] Pktsrc prefix separator (Re: [JIRA] (BIT-1267) packet source prefix separator in node.cfg breaks broctl) In-Reply-To: References: Message-ID: <20141002154505.GD76556@icir.org> (Taking this to bro-dev for discussion.) On Wed, Oct 01, 2014 at 22:31 -0500, Daniel wrote for https://bro-tracker.atlassian.net/browse/BIT-1267: > Using a different class (instead of SafeConfigParser) from the > ConfigParser module seems to fix the problem. Seems I hit another prefix separator that already had a meaning. :) SafeConfigParser implements some interpolation features using % as the control character. I don't think we're using that anywhere, so it might be fine; on the other hand the docs say: Derived class of ConfigParser that implements a more-sane variant of the magical interpolation feature. This implementation is more predictable as well. New applications should prefer this version if they don?t need to be compatible with older versions of Python." So we can either ignore that, or change the prefix separator once more. If we did the latter, any idea what would be a good one? As a reminder, the problem is specifying the packet source plugin to use with an interface, e.g., currently you'd say "-i netmap%eth1" to monitor eth1 via the netmap plugin. It used to be "-i netmap:eth1" but that conflicts with some BPF devices that use the colon already. So, what would be better than ':' or '%', both visually and in terms of not conflicting with existing interface naming conventions? Robin From dnthayer at illinois.edu Thu Oct 2 08:56:14 2014 From: dnthayer at illinois.edu (Daniel Thayer) Date: Thu, 2 Oct 2014 10:56:14 -0500 Subject: [Bro-Dev] Pktsrc prefix separator (Re: [JIRA] (BIT-1267) packet source prefix separator in node.cfg breaks broctl) In-Reply-To: <20141002154505.GD76556@icir.org> References: <20141002154505.GD76556@icir.org> Message-ID: <542D759E.4040002@illinois.edu> On 10/02/2014 10:45 AM, Robin Sommer wrote: > (Taking this to bro-dev for discussion.) > > On Wed, Oct 01, 2014 at 22:31 -0500, Daniel wrote for > https://bro-tracker.atlassian.net/browse/BIT-1267: > >> Using a different class (instead of SafeConfigParser) from the >> ConfigParser module seems to fix the problem. > > Seems I hit another prefix separator that already had a meaning. :) > > SafeConfigParser implements some interpolation features using % as the > control character. I don't think we're using that anywhere, so it > might be fine; on the other hand the docs say: > > Derived class of ConfigParser that implements a more-sane variant > of the magical interpolation feature. This implementation is more > predictable as well. New applications should prefer this version > if they don?t need to be compatible with older versions of > Python." > > So we can either ignore that, or change the prefix separator once > more. If we did the latter, any idea what would be a good one? As a > reminder, the problem is specifying the packet source plugin to use > with an interface, e.g., currently you'd say "-i netmap%eth1" to > monitor eth1 via the netmap plugin. It used to be "-i netmap:eth1" but > that conflicts with some BPF devices that use the colon already. So, > what would be better than ':' or '%', both visually and in terms of > not conflicting with existing interface naming conventions? > > Robin Just out of curiosity, I'm wondering why it was implemented as a prefix to the interface name, as opposed to just specifying the packet source with a different cmd-line option? (for example: bro --pktsrc netmap -i eth0) Is it expected that someone will want to do something like this: bro -i netmap:eth0 -i pcap:eth1 From robin at icir.org Thu Oct 2 09:10:15 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 2 Oct 2014 09:10:15 -0700 Subject: [Bro-Dev] Pktsrc prefix separator (Re: [JIRA] (BIT-1267) packet source prefix separator in node.cfg breaks broctl) In-Reply-To: <542D759E.4040002@illinois.edu> References: <20141002154505.GD76556@icir.org> <542D759E.4040002@illinois.edu> Message-ID: <20141002161015.GI76556@icir.org> On Thu, Oct 02, 2014 at 10:56 -0500, you wrote: > Is it expected that someone will want to do something > like this: bro -i netmap:eth0 -i pcap:eth1 I wouldn't call it "expected", but I don't want to rule it out either. I see the plugin through which an interface gets routed as a property of the interface itself. A global setting saying "all interfaces are to go through netmap" feels to coarse to me. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From dnthayer at illinois.edu Thu Oct 2 09:14:02 2014 From: dnthayer at illinois.edu (Daniel Thayer) Date: Thu, 2 Oct 2014 11:14:02 -0500 Subject: [Bro-Dev] Pktsrc prefix separator (Re: [JIRA] (BIT-1267) packet source prefix separator in node.cfg breaks broctl) In-Reply-To: <20141002161015.GI76556@icir.org> References: <20141002154505.GD76556@icir.org> <542D759E.4040002@illinois.edu> <20141002161015.GI76556@icir.org> Message-ID: <542D79CA.9070208@illinois.edu> On 10/02/2014 11:10 AM, Robin Sommer wrote: > > > On Thu, Oct 02, 2014 at 10:56 -0500, you wrote: > >> Is it expected that someone will want to do something >> like this: bro -i netmap:eth0 -i pcap:eth1 > > I wouldn't call it "expected", but I don't want to rule it out either. > I see the plugin through which an interface gets routed as a property > of the interface itself. A global setting saying "all interfaces are > to go through netmap" feels to coarse to me. > > Robin > How about a double colon: netmap::eth0 From jira at bro-tracker.atlassian.net Thu Oct 2 11:18:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 2 Oct 2014 13:18:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1266) PktSrc rewrite introduces packet drops In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18223#comment-18223 ] Jon Siwek commented on BIT-1266: -------------------------------- He confirmed off-list to me that the patch in topic/jsiwek/pktsrc-idle was enough to put drop rates back down, but more review couldn't hurt (now I understand how easy it is to break things when making modifications to the main loop :)) > PktSrc rewrite introduces packet drops > -------------------------------------- > > Key: BIT-1266 > URL: https://bro-tracker.atlassian.net/browse/BIT-1266 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Assignee: Robin Sommer > Priority: High > > Current master drops significantly more packets than 2.3. Looks like it's the packet source reorg: going back to 3caecadf0a3028e1dad7f446a92ffb7560c9a734 lets drop rates go back to normal. > Reported by John Donaldson. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From robin at icir.org Thu Oct 2 11:48:51 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 2 Oct 2014 11:48:51 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/pktsrc-idle: Fix packet sources being treated as idle when a packet is available. (31b7e98) In-Reply-To: <201410021825.s92IPUtH002954@bro-ids.icir.org> References: <201410021825.s92IPUtH002954@bro-ids.icir.org> Message-ID: <20141002184851.GP76556@icir.org> On Thu, Oct 02, 2014 at 11:25 -0700, Jonathan Siwek wrote: > + SetIdle(false); The 2.1 code was also doing the opposite: setting to true if we have a packet. Not immediately sure if that's necessary. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Thu Oct 2 12:34:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 2 Oct 2014 14:34:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1266) PktSrc rewrite introduces packet drops In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18224#comment-18224 ] Robin Sommer commented on BIT-1266: ----------------------------------- I'll go ahead and merge this in. > PktSrc rewrite introduces packet drops > -------------------------------------- > > Key: BIT-1266 > URL: https://bro-tracker.atlassian.net/browse/BIT-1266 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Assignee: Robin Sommer > Priority: High > > Current master drops significantly more packets than 2.3. Looks like it's the packet source reorg: going back to 3caecadf0a3028e1dad7f446a92ffb7560c9a734 lets drop rates go back to normal. > Reported by John Donaldson. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From robin at icir.org Thu Oct 2 12:34:24 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 2 Oct 2014 12:34:24 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/pktsrc-idle: Fix packet sources being treated as idle when a packet is available. (31b7e98) In-Reply-To: <20141002184851.GP76556@icir.org> References: <201410021825.s92IPUtH002954@bro-ids.icir.org> <20141002184851.GP76556@icir.org> Message-ID: <20141002193424.GA64364@icir.org> Never mind, I see that's there already. Robin On Thu, Oct 02, 2014 at 11:48 -0700, I wrote: > > > On Thu, Oct 02, 2014 at 11:25 -0700, Jonathan Siwek wrote: > > > + SetIdle(false); > > The 2.1 code was also doing the opposite: setting to true if we have a > packet. Not immediately sure if that's necessary. > > Robin > -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jsiwek at illinois.edu Thu Oct 2 12:41:34 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 2 Oct 2014 19:41:34 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/pktsrc-idle: Fix packet sources being treated as idle when a packet is available. (31b7e98) In-Reply-To: <20141002193424.GA64364@icir.org> References: <201410021825.s92IPUtH002954@bro-ids.icir.org> <20141002184851.GP76556@icir.org> <20141002193424.GA64364@icir.org> Message-ID: <80E090B6-FE99-4666-94DF-CCCD793665CB@illinois.edu> Yeah, if it reports itself as idle while a packet was just retrieved, then whether or not it?s actually a candidate to be Process()?d can depend on the result of a subsequent select() ? seems problematic :) - Jon On Oct 2, 2014, at 2:34 PM, Robin Sommer wrote: > Never mind, I see that's there already. > > Robin > > On Thu, Oct 02, 2014 at 11:48 -0700, I wrote: > >> >> >> On Thu, Oct 02, 2014 at 11:25 -0700, Jonathan Siwek wrote: >> >>> + SetIdle(false); >> >> The 2.1 code was also doing the opposite: setting to true if we have a >> packet. Not immediately sure if that's necessary. >> >> Robin >> > > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > _______________________________________________ > bro-commits mailing list > bro-commits at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits > From robin at icir.org Thu Oct 2 13:05:25 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 2 Oct 2014 13:05:25 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/pktsrc-idle: Fix packet sources being treated as idle when a packet is available. (31b7e98) In-Reply-To: <80E090B6-FE99-4666-94DF-CCCD793665CB@illinois.edu> References: <201410021825.s92IPUtH002954@bro-ids.icir.org> <20141002184851.GP76556@icir.org> <20141002193424.GA64364@icir.org> <80E090B6-FE99-4666-94DF-CCCD793665CB@illinois.edu> Message-ID: <20141002200525.GT76556@icir.org> (I had mixed up the cases in my mail :) On Thu, Oct 02, 2014 at 19:41 +0000, Jonathan Siwek wrote: > Yeah, if it reports itself as idle while a packet was just retrieved, then whether or not it?s actually a candidate to be Process()?d can depend on the result of a subsequent select() ? seems problematic :) > > - Jon > > On Oct 2, 2014, at 2:34 PM, Robin Sommer wrote: > > > Never mind, I see that's there already. > > > > Robin > > > > On Thu, Oct 02, 2014 at 11:48 -0700, I wrote: > > > >> > >> > >> On Thu, Oct 02, 2014 at 11:25 -0700, Jonathan Siwek wrote: > >> > >>> + SetIdle(false); > >> > >> The 2.1 code was also doing the opposite: setting to true if we have a > >> packet. Not immediately sure if that's necessary. > >> > >> Robin > >> > > > > > > -- > > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > > _______________________________________________ > > bro-commits mailing list > > bro-commits at bro.org > > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits > > > > -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Thu Oct 2 13:57:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 2 Oct 2014 15:57:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1266) PktSrc rewrite introduces packet drops In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1266: ------------------------------ Resolution: Fixed Status: Closed (was: Open) > PktSrc rewrite introduces packet drops > -------------------------------------- > > Key: BIT-1266 > URL: https://bro-tracker.atlassian.net/browse/BIT-1266 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Assignee: Robin Sommer > Priority: High > > Current master drops significantly more packets than 2.3. Looks like it's the packet source reorg: going back to 3caecadf0a3028e1dad7f446a92ffb7560c9a734 lets drop rates go back to normal. > Reported by John Donaldson. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From robin at icir.org Thu Oct 2 15:23:22 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 2 Oct 2014 15:23:22 -0700 Subject: [Bro-Dev] Pktsrc prefix separator (Re: [JIRA] (BIT-1267) packet source prefix separator in node.cfg breaks broctl) In-Reply-To: <542D79CA.9070208@illinois.edu> References: <20141002154505.GD76556@icir.org> <542D759E.4040002@illinois.edu> <20141002161015.GI76556@icir.org> <542D79CA.9070208@illinois.edu> Message-ID: <20141002222322.GF75302@icir.org> On Thu, Oct 02, 2014 at 11:14 -0500, you wrote: > How about a double colon: netmap::eth0 I like that actually. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From seth at icir.org Thu Oct 2 18:28:38 2014 From: seth at icir.org (Seth Hall) Date: Thu, 2 Oct 2014 21:28:38 -0400 Subject: [Bro-Dev] Pktsrc prefix separator (Re: [JIRA] (BIT-1267) packet source prefix separator in node.cfg breaks broctl) In-Reply-To: <20141002222322.GF75302@icir.org> References: <20141002154505.GD76556@icir.org> <542D759E.4040002@illinois.edu> <20141002161015.GI76556@icir.org> <542D79CA.9070208@illinois.edu> <20141002222322.GF75302@icir.org> Message-ID: <25038780-FBBC-4C02-A7D5-A1785FF38B49@icir.org> On Oct 2, 2014, at 6:23 PM, Robin Sommer wrote: > On Thu, Oct 02, 2014 at 11:14 -0500, you wrote: > >> How about a double colon: netmap::eth0 > > I like that actually. Me too! Looks like a namespace. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From jira at bro-tracker.atlassian.net Thu Oct 2 20:28:07 2014 From: jira at bro-tracker.atlassian.net (gclark (JIRA)) Date: Thu, 2 Oct 2014 22:28:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1268) Crash when tracing output In-Reply-To: References: Message-ID: gclark created BIT-1268: --------------------------- Summary: Crash when tracing output Key: BIT-1268 URL: https://bro-tracker.atlassian.net/browse/BIT-1268 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.3 Environment: $ uname -a Linux linux-xq75.site 3.16.3-1.gd2bbe7f-desktop #1 SMP PREEMPT Thu Sep 18 06:32:16 UTC 2014 (d2bbe7f) x86_64 x86_64 x86_64 GNU/Linux $ clang++ -v clang version 3.3 (branches/release_33 183898) Target: x86_64-suse-linux Thread model: posix Reporter: gclark Running the following against a fresh build of 2.3.1 (also reproduced against a copy of the current master): /usr/bro-2.3.1/bin/bro -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out generates a segmentation fault. Based on the gdb output below, looks like this happens in a call to the dhcp_ack script function. Also, args->entry[3] is NULL, which seems suspicious to me. GDB says: (gdb) run -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out Starting program: /usr/bro-2.3.1/bin/bro -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out Execution tracing ON. Program received signal SIGSEGV, Segmentation fault. Func::DescribeDebug (this=, d=0x7fffffffd198, args=0x2dc5090) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:207 207 (*args)[i]->Describe(d); (gdb) bt #0 Func::DescribeDebug (this=, d=0x7fffffffd198, args=0x2dc5090) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:207 #1 0x00000000005812c2 in BroFunc::Call (this=0xdc4fd0, args=0x2dc5090, parent=0x0) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:309 #2 0x000000000055e095 in EventHandler::Call (this=, vl=0x2dc5090, no_remote=) at /home/clarkg1/Code/bro-2.3.1/src/EventHandler.cc:77 #3 0x000000000053449b in Event::Dispatch (this=0x343f5d0, no_remote=false) at /home/clarkg1/Code/bro-2.3.1/src/Event.h:48 #4 0x000000000055d73d in EventMgr::Dispatch (this=0xb0e260 ) at /home/clarkg1/Code/bro-2.3.1/src/Event.cc:105 #5 0x000000000055d878 in EventMgr::Drain (this=0xb0e260 ) at /home/clarkg1/Code/bro-2.3.1/src/Event.cc:120 #6 0x00000000005b6d6b in net_packet_dispatch (t=1374432420.191205, hdr=0x200b0c0, pkt=0x3408570 "\220\261\034\231I)", hdr_size=14, src_ps=0x200b080) at /home/clarkg1/Code/bro-2.3.1/src/Net.cc:347 #7 0x00000000005c3049 in PktSrc::Process (this=0x200b080) at /home/clarkg1/Code/bro-2.3.1/src/PktSrc.cc:326 #8 0x00000000005b6f09 in net_run () at /home/clarkg1/Code/bro-2.3.1/src/Net.cc:389 #9 0x0000000000533343 in main (argc=0, argv=0x7fffffffdcc8) at /home/clarkg1/Code/bro-2.3.1/src/main.cc:1165 (gdb) frame 1 #1 0x00000000005812c2 in BroFunc::Call (this=0xdc4fd0, args=0x2dc5090, parent=0x0) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:309 309 DescribeDebug(&d, args); (gdb) print *this $1 = { = { = { = {_vptr$SerialObj = 0x7f04e0 , static NEVER = 0, static ALWAYS = 1, static factories = 0xb1b1c0, static names = 0xb1b200, static time_counter = 2758}, in_ser_cache = false, location = 0xdc58f0, ref_cnt = 3, static suppress_errors = 0}, bodies = std::vector of length 1, capacity 1 = {{stmts = 0x1855130, priority = 0}}, scope = 0x184f870, kind = Func::BRO_FUNC, type = 0xdc5770, name = "dhcp_ack", unique_id = 127, static unique_ids = { >> = {_M_impl = {> = {<__gnu_cxx::new_allocator> = {}, }, _M_start = 0x19cde80, _M_finish = 0x19d0190, _M_end_of_storage = 0x19d1e80}}, }}, static register_type = {}, tid = {id = 15093, static counter = 224901}, frame_size = 8} (gdb) print (*args) $6 = { = {entry = 0x343f520, chunk_size = 10, max_entries = 10, num_entries = 7}, } (gdb) print args->entry[0] $7 = (ent) 0x343d310 (gdb) print args->entry[1] $8 = (ent) 0x343f3a0 (gdb) print args->entry[2] $9 = (ent) 0x343f480 (gdb) print args->entry[3] $10 = (ent) 0x0 (gdb) print args->entry[4] $11 = (ent) 0x343cb50 (gdb) print args->entry[5] $12 = (ent) 0x343f4d0 (gdb) print args->entry[6] $13 = (ent) 0x343f430 -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Thu Oct 2 20:31:07 2014 From: jira at bro-tracker.atlassian.net (gclark (JIRA)) Date: Thu, 2 Oct 2014 22:31:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1268) Crash when tracing script execution (invalid argument to script method?) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gclark updated BIT-1268: ------------------------ Summary: Crash when tracing script execution (invalid argument to script method?) (was: Crash when tracing output) > Crash when tracing script execution (invalid argument to script method?) > ------------------------------------------------------------------------ > > Key: BIT-1268 > URL: https://bro-tracker.atlassian.net/browse/BIT-1268 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: $ uname -a > Linux linux-xq75.site 3.16.3-1.gd2bbe7f-desktop #1 SMP PREEMPT Thu Sep 18 06:32:16 UTC 2014 (d2bbe7f) x86_64 x86_64 x86_64 GNU/Linux > $ clang++ -v > clang version 3.3 (branches/release_33 183898) > Target: x86_64-suse-linux > Thread model: posix > Reporter: gclark > > Running the following against a fresh build of 2.3.1 (also reproduced against a copy of the current master): > /usr/bro-2.3.1/bin/bro -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out > generates a segmentation fault. > Based on the gdb output below, looks like this happens in a call to the dhcp_ack script function. Also, args->entry[3] is NULL, which seems suspicious to me. > GDB says: > (gdb) run -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out > Starting program: /usr/bro-2.3.1/bin/bro -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out > Execution tracing ON. > Program received signal SIGSEGV, Segmentation fault. > Func::DescribeDebug (this=, d=0x7fffffffd198, args=0x2dc5090) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:207 > 207 (*args)[i]->Describe(d); > (gdb) bt > #0 Func::DescribeDebug (this=, d=0x7fffffffd198, args=0x2dc5090) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:207 > #1 0x00000000005812c2 in BroFunc::Call (this=0xdc4fd0, args=0x2dc5090, parent=0x0) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:309 > #2 0x000000000055e095 in EventHandler::Call (this=, vl=0x2dc5090, no_remote=) at /home/clarkg1/Code/bro-2.3.1/src/EventHandler.cc:77 > #3 0x000000000053449b in Event::Dispatch (this=0x343f5d0, no_remote=false) at /home/clarkg1/Code/bro-2.3.1/src/Event.h:48 > #4 0x000000000055d73d in EventMgr::Dispatch (this=0xb0e260 ) at /home/clarkg1/Code/bro-2.3.1/src/Event.cc:105 > #5 0x000000000055d878 in EventMgr::Drain (this=0xb0e260 ) at /home/clarkg1/Code/bro-2.3.1/src/Event.cc:120 > #6 0x00000000005b6d6b in net_packet_dispatch (t=1374432420.191205, hdr=0x200b0c0, pkt=0x3408570 "\220\261\034\231I)", hdr_size=14, src_ps=0x200b080) at /home/clarkg1/Code/bro-2.3.1/src/Net.cc:347 > #7 0x00000000005c3049 in PktSrc::Process (this=0x200b080) at /home/clarkg1/Code/bro-2.3.1/src/PktSrc.cc:326 > #8 0x00000000005b6f09 in net_run () at /home/clarkg1/Code/bro-2.3.1/src/Net.cc:389 > #9 0x0000000000533343 in main (argc=0, argv=0x7fffffffdcc8) at /home/clarkg1/Code/bro-2.3.1/src/main.cc:1165 > (gdb) frame 1 > #1 0x00000000005812c2 in BroFunc::Call (this=0xdc4fd0, args=0x2dc5090, parent=0x0) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:309 > 309 DescribeDebug(&d, args); > (gdb) print *this > $1 = { = { = { = {_vptr$SerialObj = 0x7f04e0 , static NEVER = 0, static ALWAYS = 1, static factories = 0xb1b1c0, static names = 0xb1b200, static time_counter = 2758}, in_ser_cache = false, location = 0xdc58f0, > ref_cnt = 3, static suppress_errors = 0}, bodies = std::vector of length 1, capacity 1 = {{stmts = 0x1855130, priority = 0}}, scope = 0x184f870, kind = Func::BRO_FUNC, type = 0xdc5770, name = "dhcp_ack", unique_id = 127, > static unique_ids = { >> = {_M_impl = {> = {<__gnu_cxx::new_allocator> = {}, }, _M_start = 0x19cde80, _M_finish = 0x19d0190, > _M_end_of_storage = 0x19d1e80}}, }}, static register_type = {}, tid = {id = 15093, static counter = 224901}, frame_size = 8} > (gdb) print (*args) > $6 = { = {entry = 0x343f520, chunk_size = 10, max_entries = 10, num_entries = 7}, } > (gdb) print args->entry[0] > $7 = (ent) 0x343d310 > (gdb) print args->entry[1] > $8 = (ent) 0x343f3a0 > (gdb) print args->entry[2] > $9 = (ent) 0x343f480 > (gdb) print args->entry[3] > $10 = (ent) 0x0 > (gdb) print args->entry[4] > $11 = (ent) 0x343cb50 > (gdb) print args->entry[5] > $12 = (ent) 0x343f4d0 > (gdb) print args->entry[6] > $13 = (ent) 0x343f430 -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 3 07:42:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 3 Oct 2014 09:42:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1268) Crash when tracing script execution (invalid argument to script method?) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1268: --------------------------- Fix Version/s: 2.4 > Crash when tracing script execution (invalid argument to script method?) > ------------------------------------------------------------------------ > > Key: BIT-1268 > URL: https://bro-tracker.atlassian.net/browse/BIT-1268 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: $ uname -a > Linux linux-xq75.site 3.16.3-1.gd2bbe7f-desktop #1 SMP PREEMPT Thu Sep 18 06:32:16 UTC 2014 (d2bbe7f) x86_64 x86_64 x86_64 GNU/Linux > $ clang++ -v > clang version 3.3 (branches/release_33 183898) > Target: x86_64-suse-linux > Thread model: posix > Reporter: gclark > Fix For: 2.4 > > > Running the following against a fresh build of 2.3.1 (also reproduced against a copy of the current master): > /usr/bro-2.3.1/bin/bro -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out > generates a segmentation fault. > Based on the gdb output below, looks like this happens in a call to the dhcp_ack script function. Also, args->entry[3] is NULL, which seems suspicious to me. > GDB says: > (gdb) run -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out > Starting program: /usr/bro-2.3.1/bin/bro -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out > Execution tracing ON. > Program received signal SIGSEGV, Segmentation fault. > Func::DescribeDebug (this=, d=0x7fffffffd198, args=0x2dc5090) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:207 > 207 (*args)[i]->Describe(d); > (gdb) bt > #0 Func::DescribeDebug (this=, d=0x7fffffffd198, args=0x2dc5090) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:207 > #1 0x00000000005812c2 in BroFunc::Call (this=0xdc4fd0, args=0x2dc5090, parent=0x0) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:309 > #2 0x000000000055e095 in EventHandler::Call (this=, vl=0x2dc5090, no_remote=) at /home/clarkg1/Code/bro-2.3.1/src/EventHandler.cc:77 > #3 0x000000000053449b in Event::Dispatch (this=0x343f5d0, no_remote=false) at /home/clarkg1/Code/bro-2.3.1/src/Event.h:48 > #4 0x000000000055d73d in EventMgr::Dispatch (this=0xb0e260 ) at /home/clarkg1/Code/bro-2.3.1/src/Event.cc:105 > #5 0x000000000055d878 in EventMgr::Drain (this=0xb0e260 ) at /home/clarkg1/Code/bro-2.3.1/src/Event.cc:120 > #6 0x00000000005b6d6b in net_packet_dispatch (t=1374432420.191205, hdr=0x200b0c0, pkt=0x3408570 "\220\261\034\231I)", hdr_size=14, src_ps=0x200b080) at /home/clarkg1/Code/bro-2.3.1/src/Net.cc:347 > #7 0x00000000005c3049 in PktSrc::Process (this=0x200b080) at /home/clarkg1/Code/bro-2.3.1/src/PktSrc.cc:326 > #8 0x00000000005b6f09 in net_run () at /home/clarkg1/Code/bro-2.3.1/src/Net.cc:389 > #9 0x0000000000533343 in main (argc=0, argv=0x7fffffffdcc8) at /home/clarkg1/Code/bro-2.3.1/src/main.cc:1165 > (gdb) frame 1 > #1 0x00000000005812c2 in BroFunc::Call (this=0xdc4fd0, args=0x2dc5090, parent=0x0) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:309 > 309 DescribeDebug(&d, args); > (gdb) print *this > $1 = { = { = { = {_vptr$SerialObj = 0x7f04e0 , static NEVER = 0, static ALWAYS = 1, static factories = 0xb1b1c0, static names = 0xb1b200, static time_counter = 2758}, in_ser_cache = false, location = 0xdc58f0, > ref_cnt = 3, static suppress_errors = 0}, bodies = std::vector of length 1, capacity 1 = {{stmts = 0x1855130, priority = 0}}, scope = 0x184f870, kind = Func::BRO_FUNC, type = 0xdc5770, name = "dhcp_ack", unique_id = 127, > static unique_ids = { >> = {_M_impl = {> = {<__gnu_cxx::new_allocator> = {}, }, _M_start = 0x19cde80, _M_finish = 0x19d0190, > _M_end_of_storage = 0x19d1e80}}, }}, static register_type = {}, tid = {id = 15093, static counter = 224901}, frame_size = 8} > (gdb) print (*args) > $6 = { = {entry = 0x343f520, chunk_size = 10, max_entries = 10, num_entries = 7}, } > (gdb) print args->entry[0] > $7 = (ent) 0x343d310 > (gdb) print args->entry[1] > $8 = (ent) 0x343f3a0 > (gdb) print args->entry[2] > $9 = (ent) 0x343f480 > (gdb) print args->entry[3] > $10 = (ent) 0x0 > (gdb) print args->entry[4] > $11 = (ent) 0x343cb50 > (gdb) print args->entry[5] > $12 = (ent) 0x343f4d0 > (gdb) print args->entry[6] > $13 = (ent) 0x343f430 -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 3 07:45:07 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Fri, 3 Oct 2014 09:45:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1242) Add a page which lists and describes all Bro logs In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1242: ------------------------------- Summary: Add a page which lists and describes all Bro logs (was: Logs: add page listing logs to /script-reference/index.html) > Add a page which lists and describes all Bro logs > ------------------------------------------------- > > Key: BIT-1242 > URL: https://bro-tracker.atlassian.net/browse/BIT-1242 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Documentation, Website > Reporter: Jeannette Dopheide > Assignee: Jeannette Dopheide > Fix For: 2.4 > > > Add a page to documentation listing the Bro log files. > There's already a page listing common log files: > http://www.bro.org/sphinx/logs/index.html#common-log-files > This page could be moved to http://www.bro.org/sphinx/script-reference/index.html and be expanded to list all known log files. > If this goes well, Robin suggested adding a btest that heuristically checks if something changes with regards to which logs we have. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 3 07:45:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 3 Oct 2014 09:45:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1268) Crash when tracing script execution (invalid argument to script method?) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1268: --------------------------- Resolution: Fixed Status: Closed (was: Open) > Crash when tracing script execution (invalid argument to script method?) > ------------------------------------------------------------------------ > > Key: BIT-1268 > URL: https://bro-tracker.atlassian.net/browse/BIT-1268 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: $ uname -a > Linux linux-xq75.site 3.16.3-1.gd2bbe7f-desktop #1 SMP PREEMPT Thu Sep 18 06:32:16 UTC 2014 (d2bbe7f) x86_64 x86_64 x86_64 GNU/Linux > $ clang++ -v > clang version 3.3 (branches/release_33 183898) > Target: x86_64-suse-linux > Thread model: posix > Reporter: gclark > Fix For: 2.4 > > > Running the following against a fresh build of 2.3.1 (also reproduced against a copy of the current master): > /usr/bro-2.3.1/bin/bro -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out > generates a segmentation fault. > Based on the gdb output below, looks like this happens in a call to the dhcp_ack script function. Also, args->entry[3] is NULL, which seems suspicious to me. > GDB says: > (gdb) run -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out > Starting program: /usr/bro-2.3.1/bin/bro -r ~/Code/bro/testing/btest/Traces/dhcp/dhcp_inform.trace -t /tmp/trace.out > Execution tracing ON. > Program received signal SIGSEGV, Segmentation fault. > Func::DescribeDebug (this=, d=0x7fffffffd198, args=0x2dc5090) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:207 > 207 (*args)[i]->Describe(d); > (gdb) bt > #0 Func::DescribeDebug (this=, d=0x7fffffffd198, args=0x2dc5090) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:207 > #1 0x00000000005812c2 in BroFunc::Call (this=0xdc4fd0, args=0x2dc5090, parent=0x0) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:309 > #2 0x000000000055e095 in EventHandler::Call (this=, vl=0x2dc5090, no_remote=) at /home/clarkg1/Code/bro-2.3.1/src/EventHandler.cc:77 > #3 0x000000000053449b in Event::Dispatch (this=0x343f5d0, no_remote=false) at /home/clarkg1/Code/bro-2.3.1/src/Event.h:48 > #4 0x000000000055d73d in EventMgr::Dispatch (this=0xb0e260 ) at /home/clarkg1/Code/bro-2.3.1/src/Event.cc:105 > #5 0x000000000055d878 in EventMgr::Drain (this=0xb0e260 ) at /home/clarkg1/Code/bro-2.3.1/src/Event.cc:120 > #6 0x00000000005b6d6b in net_packet_dispatch (t=1374432420.191205, hdr=0x200b0c0, pkt=0x3408570 "\220\261\034\231I)", hdr_size=14, src_ps=0x200b080) at /home/clarkg1/Code/bro-2.3.1/src/Net.cc:347 > #7 0x00000000005c3049 in PktSrc::Process (this=0x200b080) at /home/clarkg1/Code/bro-2.3.1/src/PktSrc.cc:326 > #8 0x00000000005b6f09 in net_run () at /home/clarkg1/Code/bro-2.3.1/src/Net.cc:389 > #9 0x0000000000533343 in main (argc=0, argv=0x7fffffffdcc8) at /home/clarkg1/Code/bro-2.3.1/src/main.cc:1165 > (gdb) frame 1 > #1 0x00000000005812c2 in BroFunc::Call (this=0xdc4fd0, args=0x2dc5090, parent=0x0) at /home/clarkg1/Code/bro-2.3.1/src/Func.cc:309 > 309 DescribeDebug(&d, args); > (gdb) print *this > $1 = { = { = { = {_vptr$SerialObj = 0x7f04e0 , static NEVER = 0, static ALWAYS = 1, static factories = 0xb1b1c0, static names = 0xb1b200, static time_counter = 2758}, in_ser_cache = false, location = 0xdc58f0, > ref_cnt = 3, static suppress_errors = 0}, bodies = std::vector of length 1, capacity 1 = {{stmts = 0x1855130, priority = 0}}, scope = 0x184f870, kind = Func::BRO_FUNC, type = 0xdc5770, name = "dhcp_ack", unique_id = 127, > static unique_ids = { >> = {_M_impl = {> = {<__gnu_cxx::new_allocator> = {}, }, _M_start = 0x19cde80, _M_finish = 0x19d0190, > _M_end_of_storage = 0x19d1e80}}, }}, static register_type = {}, tid = {id = 15093, static counter = 224901}, frame_size = 8} > (gdb) print (*args) > $6 = { = {entry = 0x343f520, chunk_size = 10, max_entries = 10, num_entries = 7}, } > (gdb) print args->entry[0] > $7 = (ent) 0x343d310 > (gdb) print args->entry[1] > $8 = (ent) 0x343f3a0 > (gdb) print args->entry[2] > $9 = (ent) 0x343f480 > (gdb) print args->entry[3] > $10 = (ent) 0x0 > (gdb) print args->entry[4] > $11 = (ent) 0x343cb50 > (gdb) print args->entry[5] > $12 = (ent) 0x343f4d0 > (gdb) print args->entry[6] > $13 = (ent) 0x343f430 -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 3 08:27:07 2014 From: jira at bro-tracker.atlassian.net (Jimmy Jones (JIRA)) Date: Fri, 3 Oct 2014 10:27:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1264) HTTP response not detected on nonstandard port In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18225#comment-18225 ] Jimmy Jones commented on BIT-1264: ---------------------------------- Is it possible for bro to infer the packets belong to a responder, because the connection started with a SYN+ACK rather than just a SYN? Or is that a major change for an edge case, although not unheard of on SPAN ports? > HTTP response not detected on nonstandard port > ---------------------------------------------- > > Key: BIT-1264 > URL: https://bro-tracker.atlassian.net/browse/BIT-1264 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: CentOS 6 > Reporter: Jimmy Jones > Attachments: relaxed.bro, relaxed-http.sig, sample-small2-rsp.pcap, sample-small-rsp.pcap > > > Using the attached bro script I've tweaked the HTTP signature to match on http responses without the corresponding HTTP request TCP session. I know in a proper setup you should never get single sided traffic, but certainly when using bro as a tool you have to deal with it sometimes. > Bro handles this fine when the HTTP is on port 80, but not when on port 4321 (see attached PCAPs). I'm curious as to why? -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 3 08:43:07 2014 From: jira at bro-tracker.atlassian.net (Jimmy Jones (JIRA)) Date: Fri, 3 Oct 2014 10:43:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1265) Single sided HTTP POST split In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18226#comment-18226 ] Jimmy Jones commented on BIT-1265: ---------------------------------- Thanks! Might it be better to mark the connection as successful if data is sent? Again, for the single sided case, which I'm not sure how many people are worried about/notice? Otherwise have to set this to a large number, to cover longest possible TCP sessions, but presumably has a big impact on memory usage, as "lone" SYN's will keep state? > Single sided HTTP POST split > ---------------------------- > > Key: BIT-1265 > URL: https://bro-tracker.atlassian.net/browse/BIT-1265 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: CentOS 6 > Reporter: Jimmy Jones > Attachments: sample-upload2-all.pcap, sample-upload2-req.pcap > > > Attached two pcap samples, one is a single sided version of the other, an HTTP POST. > When I process the single sided version (sample-upload2-req) conn.log shows two sessions (the HTTP POST tcp connection that has been split) and http.log shows a partial upload. However processing the original sample (sample-upload2-all) everything is as expected - one connection in conn.log and a complete http.log > Are there any parameters I can tweak to make this work? -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 3 09:03:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 3 Oct 2014 11:03:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1267) packet source prefix separator in node.cfg breaks broctl In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1267: --------------------------------- Assignee: Robin Sommer I'm going to change the prefix separator once more to {{::}}, so that BroControl then won't need a change. > packet source prefix separator in node.cfg breaks broctl > -------------------------------------------------------- > > Key: BIT-1267 > URL: https://bro-tracker.atlassian.net/browse/BIT-1267 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Assignee: Robin Sommer > > Attempting to use the new packet source prefix separator (see BIT-1249) > in node.cfg prevents broctl from being able to start due to the use of > Python's ConfigParser (it interprets % as a special character). > For example: > interface=test%eth0 -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 3 09:41:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 3 Oct 2014 11:41:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1264) HTTP response not detected on nonstandard port In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18228#comment-18228 ] Jon Siwek commented on BIT-1264: -------------------------------- {quote} Is it possible for bro to infer the packets belong to a responder, because the connection started with a SYN+ACK rather than just a SYN? Or is that a major change for an edge case, although not unheard of on SPAN ports? {quote} It is possible to do that: you can take a look at BIT-1236 which mentions a branch that implements that change, but it isn't 100% accurate (check out the github pull request comments also linked in that ticket). Haven't yet revisited to see if something more can be done and not sure right now how deep the changes would be to improve it. > HTTP response not detected on nonstandard port > ---------------------------------------------- > > Key: BIT-1264 > URL: https://bro-tracker.atlassian.net/browse/BIT-1264 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: CentOS 6 > Reporter: Jimmy Jones > Attachments: relaxed.bro, relaxed-http.sig, sample-small2-rsp.pcap, sample-small-rsp.pcap > > > Using the attached bro script I've tweaked the HTTP signature to match on http responses without the corresponding HTTP request TCP session. I know in a proper setup you should never get single sided traffic, but certainly when using bro as a tool you have to deal with it sometimes. > Bro handles this fine when the HTTP is on port 80, but not when on port 4321 (see attached PCAPs). I'm curious as to why? -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 3 09:54:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 3 Oct 2014 11:54:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1265) Single sided HTTP POST split In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18229#comment-18229 ] Jon Siwek commented on BIT-1265: -------------------------------- {quote} Might it be better to mark the connection as successful if data is sent? {quote} Yeah, I think that's a nice idea -- seems kind of arbitrary for Bro to close the session if it knows one side is still actively sending data. {quote} Otherwise have to set this to a large number, to cover longest possible TCP sessions, but presumably has a big impact on memory usage, as "lone" SYN's will keep state? {quote} Yes, I think that would be a concern, but there's also several other timeout mechanisms (which are also tuneable) that I'm not immediately sure would come to the rescue even if the one in question was set high. > Single sided HTTP POST split > ---------------------------- > > Key: BIT-1265 > URL: https://bro-tracker.atlassian.net/browse/BIT-1265 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: CentOS 6 > Reporter: Jimmy Jones > Attachments: sample-upload2-all.pcap, sample-upload2-req.pcap > > > Attached two pcap samples, one is a single sided version of the other, an HTTP POST. > When I process the single sided version (sample-upload2-req) conn.log shows two sessions (the HTTP POST tcp connection that has been split) and http.log shows a partial upload. However processing the original sample (sample-upload2-all) everything is as expected - one connection in conn.log and a complete http.log > Are there any parameters I can tweak to make this work? -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 3 12:10:07 2014 From: jira at bro-tracker.atlassian.net (Jeannette Dopheide (JIRA)) Date: Fri, 3 Oct 2014 14:10:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1242) Add a page which lists and describes all Bro logs In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18230#comment-18230 ] Jeannette Dopheide commented on BIT-1242: ----------------------------------------- On branch : topic/jdopheid/BIT-1242 Changing ticket status to Request Merge > Add a page which lists and describes all Bro logs > ------------------------------------------------- > > Key: BIT-1242 > URL: https://bro-tracker.atlassian.net/browse/BIT-1242 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Documentation, Website > Reporter: Jeannette Dopheide > Assignee: Jeannette Dopheide > Fix For: 2.4 > > > Add a page to documentation listing the Bro log files. > There's already a page listing common log files: > http://www.bro.org/sphinx/logs/index.html#common-log-files > This page could be moved to http://www.bro.org/sphinx/script-reference/index.html and be expanded to list all known log files. > If this goes well, Robin suggested adding a btest that heuristically checks if something changes with regards to which logs we have. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 3 12:11:07 2014 From: jira at bro-tracker.atlassian.net (Jeannette Dopheide (JIRA)) Date: Fri, 3 Oct 2014 14:11:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1242) Add a page which lists and describes all Bro logs In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeannette Dopheide updated BIT-1242: ------------------------------------ Status: Merge Request (was: In Progress) > Add a page which lists and describes all Bro logs > ------------------------------------------------- > > Key: BIT-1242 > URL: https://bro-tracker.atlassian.net/browse/BIT-1242 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Documentation, Website > Reporter: Jeannette Dopheide > Assignee: Jeannette Dopheide > Fix For: 2.4 > > > Add a page to documentation listing the Bro log files. > There's already a page listing common log files: > http://www.bro.org/sphinx/logs/index.html#common-log-files > This page could be moved to http://www.bro.org/sphinx/script-reference/index.html and be expanded to list all known log files. > If this goes well, Robin suggested adding a btest that heuristically checks if something changes with regards to which logs we have. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From noreply at bro.org Sat Oct 4 00:00:41 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 4 Oct 2014 00:00:41 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410040700.s9470fVc009763@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ --------------------- ------------------ ------------------ ---------- ------------- ---------- ------------------------------------------------- BIT-1242 [1] Documentation,Website Jeannette Dopheide Jeannette Dopheide 2014-10-03 2.4 Normal Add a page which lists and describes all Bro logs [1] BIT-1242 https://bro-tracker.atlassian.net/browse/BIT-1242 From noreply at bro.org Sun Oct 5 00:00:16 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 5 Oct 2014 00:00:16 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410050700.s9570Ghm029226@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ --------------------- ------------------ ------------------ ---------- ------------- ---------- ------------------------------------------------- BIT-1242 [1] Documentation,Website Jeannette Dopheide Jeannette Dopheide 2014-10-03 2.4 Normal Add a page which lists and describes all Bro logs [1] BIT-1242 https://bro-tracker.atlassian.net/browse/BIT-1242 From ak at opendns.com Sun Oct 5 02:30:10 2014 From: ak at opendns.com (Anthony Kasza) Date: Sun, 5 Oct 2014 02:30:10 -0700 Subject: [Bro-Dev] Geo Location Plugin Message-ID: Hello All, I recently have been looking at how Bro's plugins work. I was able to recreate the rot13 functionality from the online documentation and wanted to do something a little more challenging in hopes of learning more about plugin development. On a suggestion from Seth, I've been attempting to move the geoIP stuff, which is currently baked into the core, into a self contained plugin. I've gotten Bro and my plugin to compile together (mostly by simply moving existing code into their proper files within aux/bro-aux/plugin-support/Geo). However, I cannot seem to find the plugin listed in 'bro -NN' and I cannot use the data types or functions my bif defines. I'm hoping someone has a suggestion or a tip on troubleshooting the interaction between plugins and the core. Many thanks, -AK From robin at icir.org Sun Oct 5 08:16:33 2014 From: robin at icir.org (Robin Sommer) Date: Sun, 5 Oct 2014 08:16:33 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: Message-ID: <20141005151633.GF31437@icir.org> On Sun, Oct 05, 2014 at 02:30 -0700, you wrote: > I'm hoping someone has a suggestion or a tip > on troubleshooting the interaction between plugins and the core. If you haven't yet, try compiling Bro with --enable-debug (and then recompiling your plugin as well). Then run with "-B plugins" and see if debug.log says anything helpful. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From anthony.kasza at gmail.com Sun Oct 5 23:00:37 2014 From: anthony.kasza at gmail.com (anthony kasza) Date: Sun, 5 Oct 2014 23:00:37 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: <20141005151633.GF31437@icir.org> References: <20141005151633.GF31437@icir.org> Message-ID: I can at least now see why my plugin isn't working as excepted. Thanks Robin. -AK On Oct 5, 2014 8:23 AM, "Robin Sommer" wrote: > > > On Sun, Oct 05, 2014 at 02:30 -0700, you wrote: > > > I'm hoping someone has a suggestion or a tip > > on troubleshooting the interaction between plugins and the core. > > If you haven't yet, try compiling Bro with --enable-debug (and then > recompiling your plugin as well). Then run with "-B plugins" and see > if debug.log says anything helpful. > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141005/2c25b472/attachment.html From noreply at bro.org Mon Oct 6 00:00:16 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 6 Oct 2014 00:00:16 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410060700.s9670GQ4008500@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ --------------------- ------------------ ------------------ ---------- ------------- ---------- ------------------------------------------------- BIT-1242 [1] Documentation,Website Jeannette Dopheide Jeannette Dopheide 2014-10-03 2.4 Normal Add a page which lists and describes all Bro logs [1] BIT-1242 https://bro-tracker.atlassian.net/browse/BIT-1242 From robin at icir.org Mon Oct 6 08:10:04 2014 From: robin at icir.org (Robin Sommer) Date: Mon, 6 Oct 2014 08:10:04 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <20141005151633.GF31437@icir.org> Message-ID: <20141006151004.GM31437@icir.org> On Sun, Oct 05, 2014 at 23:00 -0700, you wrote: > I can at least now see why my plugin isn't working as excepted. Great, let me know if it's anything that the plugin documentation could make more clear. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Mon Oct 6 11:24:07 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Mon, 6 Oct 2014 13:24:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1269) Add more Bro script language reference documentation In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1269: ---------------------------------- Summary: Add more Bro script language reference documentation Key: BIT-1269 URL: https://bro-tracker.atlassian.net/browse/BIT-1269 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Daniel Thayer Fix For: 2.4 The Bro script language needs more documentation. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Mon Oct 6 13:10:07 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Mon, 6 Oct 2014 15:10:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1269) Add more Bro script language reference documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18231#comment-18231 ] Daniel Thayer commented on BIT-1269: ------------------------------------ Branch topic/dnthayer/langref adds new sections to the reference documentation that document operators, declarations and statements, and directives. > Add more Bro script language reference documentation > ---------------------------------------------------- > > Key: BIT-1269 > URL: https://bro-tracker.atlassian.net/browse/BIT-1269 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Daniel Thayer > Fix For: 2.4 > > > The Bro script language needs more documentation. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Mon Oct 6 13:11:07 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Mon, 6 Oct 2014 15:11:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1269) Add more Bro script language reference documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1269: ------------------------------- Status: Merge Request (was: Open) > Add more Bro script language reference documentation > ---------------------------------------------------- > > Key: BIT-1269 > URL: https://bro-tracker.atlassian.net/browse/BIT-1269 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Daniel Thayer > Fix For: 2.4 > > > The Bro script language needs more documentation. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From anthony.kasza at gmail.com Mon Oct 6 21:46:41 2014 From: anthony.kasza at gmail.com (anthony kasza) Date: Mon, 6 Oct 2014 21:46:41 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: <20141006151004.GM31437@icir.org> References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> Message-ID: Thanks Robin. Everything on the "Writing Bro Plugins" page is clear although the debugging section seems thin to me. Then again, I've have never compiled Bro with debugging enabled before. I'm running into issues with my plugin's shared object. Bro is complaining about an undefined symbol (likely due to a bug in my source). Are there plans to expand the Types section of that page? I think I'm not declaring a scriptland record type correctly. -AK On Oct 6, 2014 8:10 AM, "Robin Sommer" wrote: > On Sun, Oct 05, 2014 at 23:00 -0700, you wrote: > > > I can at least now see why my plugin isn't working as excepted. > > Great, let me know if it's anything that the plugin documentation > could make more clear. > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141006/3547f13f/attachment.html From noreply at bro.org Tue Oct 7 00:00:20 2014 From: noreply at bro.org (Merge Tracker) Date: Tue, 7 Oct 2014 00:00:20 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410070700.s9770KEB020959@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ --------------------- ------------------ ------------------ ---------- ------------- ---------- ---------------------------------------------------- BIT-1269 [1] Bro Daniel Thayer - 2014-10-06 2.4 Normal Add more Bro script language reference documentation BIT-1242 [2] Documentation,Website Jeannette Dopheide Jeannette Dopheide 2014-10-03 2.4 Normal Add a page which lists and describes all Bro logs [1] BIT-1269 https://bro-tracker.atlassian.net/browse/BIT-1269 [2] BIT-1242 https://bro-tracker.atlassian.net/browse/BIT-1242 From gc355804 at ohio.edu Tue Oct 7 06:36:34 2014 From: gc355804 at ohio.edu (Clark, Gilbert) Date: Tue, 7 Oct 2014 13:36:34 +0000 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org>, Message-ID: <6332d438669d40f197d91521f55ee59d@CO1PR01MB238.prod.exchangelabs.com> Hi: A mismatched function definition in .h / implementation in .cc usually causes this issue for me. Maybe try running the mangled symbol that's reported as missing through c++filt to see what it really is (if you haven't already) and use nm or the like to make sure the plugin library file contains that symbol? -Gilbert ________________________________________ From: bro-dev-bounces at bro.org [bro-dev-bounces at bro.org] on behalf of anthony kasza [anthony.kasza at gmail.com] Sent: Tuesday, October 07, 2014 12:46 AM To: Robin Sommer Cc: Subject: Re: [Bro-Dev] Geo Location Plugin Thanks Robin. Everything on the "Writing Bro Plugins" page is clear although the debugging section seems thin to me. Then again, I've have never compiled Bro with debugging enabled before. I'm running into issues with my plugin's shared object. Bro is complaining about an undefined symbol (likely due to a bug in my source). Are there plans to expand the Types section of that page? I think I'm not declaring a scriptland record type correctly. -AK On Oct 6, 2014 8:10 AM, "Robin Sommer" > wrote: On Sun, Oct 05, 2014 at 23:00 -0700, you wrote: > I can at least now see why my plugin isn't working as excepted. Great, let me know if it's anything that the plugin documentation could make more clear. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From seth at icir.org Tue Oct 7 08:05:11 2014 From: seth at icir.org (Seth Hall) Date: Tue, 7 Oct 2014 11:05:11 -0400 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: <6332d438669d40f197d91521f55ee59d@CO1PR01MB238.prod.exchangelabs.com> References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org>, <6332d438669d40f197d91521f55ee59d@CO1PR01MB238.prod.exchangelabs.com> Message-ID: <829665C8-BA5B-49E3-AA06-2D4D18BA5428@icir.org> On Oct 7, 2014, at 9:36 AM, Clark, Gilbert wrote: > A mismatched function definition in .h / implementation in .cc usually causes this issue for me. Maybe try running the mangled symbol that's reported as missing through c++filt to see what it really is (if you haven't already) and use nm or the like to make sure the plugin library file contains that symbol? I ran into a similar issue recently with the approxidate plugin I've been working on. Jon reminded me that I need to do this... #ifdef __cplusplus extern "C" { #endif unsigned long approxidate_relative(const char *date, const struct timeval *tv); #ifdef __cplusplus } #endif .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From robin at icir.org Tue Oct 7 08:25:55 2014 From: robin at icir.org (Robin Sommer) Date: Tue, 7 Oct 2014 08:25:55 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> Message-ID: <20141007152555.GC63065@icir.org> On Mon, Oct 06, 2014 at 21:46 -0700, you wrote: > Thanks Robin. Everything on the "Writing Bro Plugins" page is clear > although the debugging section seems thin to me. Then again, I've have > never compiled Bro with debugging enabled before. Ok, I'll try to extend that a bit more, it's written mainly from the perspective of adding debugging output to your plugin; not in terms of leveraging the existing output to see why something's not working. > Are there plans to expand the Types section of that page? I would like to, but it's quite a bit of work to fill in the TODOs, so honestly I don't see that coming very soon. The problem is that most of the missing TODOs are actually not really a matter per se of writing a plugin or not, but require text on how to write the corresponding code for Bro in general (in other words, even before moving to plugins, much of that would have been quite similar). For now, the best way to figure this out is to look at existing code, i.e., for the builtin-in stuff other *.bif files. > I think I'm not declaring a scriptland record type correctly. Feel free to post error messages here. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Tue Oct 7 13:59:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 7 Oct 2014 15:59:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1269) Add more Bro script language reference documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1269: --------------------------------- Assignee: Robin Sommer > Add more Bro script language reference documentation > ---------------------------------------------------- > > Key: BIT-1269 > URL: https://bro-tracker.atlassian.net/browse/BIT-1269 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Daniel Thayer > Assignee: Robin Sommer > Fix For: 2.4 > > > The Bro script language needs more documentation. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Tue Oct 7 13:59:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 7 Oct 2014 15:59:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1242) Add a page which lists and describes all Bro logs In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1242: --------------------------------- Assignee: Robin Sommer (was: Jeannette Dopheide) > Add a page which lists and describes all Bro logs > ------------------------------------------------- > > Key: BIT-1242 > URL: https://bro-tracker.atlassian.net/browse/BIT-1242 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Documentation, Website > Reporter: Jeannette Dopheide > Assignee: Robin Sommer > Fix For: 2.4 > > > Add a page to documentation listing the Bro log files. > There's already a page listing common log files: > http://www.bro.org/sphinx/logs/index.html#common-log-files > This page could be moved to http://www.bro.org/sphinx/script-reference/index.html and be expanded to list all known log files. > If this goes well, Robin suggested adding a btest that heuristically checks if something changes with regards to which logs we have. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From robin at icir.org Tue Oct 7 15:43:01 2014 From: robin at icir.org (Robin Sommer) Date: Tue, 7 Oct 2014 15:43:01 -0700 Subject: [Bro-Dev] Plugins providing threads? Message-ID: <20141007224301.GC37792@icir.org> I'm wondering if we should add another type of plugin component: threads. This would be for functionality that's to run in parallel with Bro's main thread, and communicate with it via message passing. We have the structure for that in place already, logging and reading are using it as already. But this would formalize the notion a bit more directly that a plugin can provide a new thread, with its own logic; and also extend the interface that it has available from inside that thread (e.g., being able to raise events; have bif functions that, when called, get passed through to the thread). One use case is Christian's OpenFlow plugin: if we went the route of integrating an external library for speaking OpenFlow directly, that communication needs to be handled somewhere. Traditionally, that would be an IOSource hooked into the main loop. The plugin model could support that, too, but being able to fully decouple it inside its own thread seems appealing. Jon, how are you planing to integrate Broker into Bro? Would this help there as well if you could just follow a similar structure with Broker running inside its own thread? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From asharma at lbl.gov Tue Oct 7 16:11:09 2014 From: asharma at lbl.gov (Aashish Sharma) Date: Tue, 7 Oct 2014 16:11:09 -0700 Subject: [Bro-Dev] Plugins providing threads? In-Reply-To: <20141007224301.GC37792@icir.org> References: <20141007224301.GC37792@icir.org> Message-ID: <20141007231106.GP5935@yaksha.lbl.gov> Very timely question, I've be mulling over this and I would like to vote for adding thread component. This may allow us to do a lot more processing of data in the script land. Now my use case may not be likely an ideal one. I am *experimenting* with a policy to flag very long sustained persistant connections between two hosts (irrespective of ports). For this, I am storing connection information [src, dst] in a table and based on various conditions/heuristics I am expiring these at different dynamic times (so read_expire isn't helpful). For 'spike' elimination from scanners, I iterate through the table every min. (unoptimal but stay with me). At present when table stores about 4 Million elements, iterating through it freezes entire bro (no growth of conn log etc). A seperate thread could allow me to delegate these kinds of tasks outside Bro's main thread/event queue. Aashish On Tue, Oct 07, 2014 at 03:43:01PM -0700, Robin Sommer wrote: > I'm wondering if we should add another type of plugin component: > threads. This would be for functionality that's to run in parallel > with Bro's main thread, and communicate with it via message passing. > > We have the structure for that in place already, logging and reading > are using it as already. But this would formalize the notion a bit > more directly that a plugin can provide a new thread, with its own > logic; and also extend the interface that it has available from inside > that thread (e.g., being able to raise events; have bif functions > that, when called, get passed through to the thread). > > One use case is Christian's OpenFlow plugin: if we went the route of > integrating an external library for speaking OpenFlow directly, that > communication needs to be handled somewhere. Traditionally, that would > be an IOSource hooked into the main loop. The plugin model could > support that, too, but being able to fully decouple it inside its own > thread seems appealing. > > Jon, how are you planing to integrate Broker into Bro? Would this help > there as well if you could just follow a similar structure with Broker > running inside its own thread? > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Lawrence Berkeley National Laboratory http://go.lbl.gov/pgp-aashish Office: (510)-495-2680 Cell: (510)-612-7971 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141007/32bf996e/attachment.bin From robin at icir.org Tue Oct 7 17:15:47 2014 From: robin at icir.org (Robin Sommer) Date: Tue, 7 Oct 2014 17:15:47 -0700 Subject: [Bro-Dev] Plugins providing threads? In-Reply-To: <20141007231106.GP5935@yaksha.lbl.gov> References: <20141007224301.GC37792@icir.org> <20141007231106.GP5935@yaksha.lbl.gov> Message-ID: <20141008001547.GJ37792@icir.org> On Tue, Oct 07, 2014 at 16:11 -0700, you wrote: > This may allow us to do a lot more processing of data in the script land. Well, that's a different thing. I was talking about plugins in core land, written in C++. Threading in script-land is nothing I see happening soon unfortunately ... Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Tue Oct 7 17:44:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 7 Oct 2014 19:44:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1267) packet source prefix separator in node.cfg breaks broctl In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1267: ------------------------------ Resolution: Fixed Status: Closed (was: Open) > packet source prefix separator in node.cfg breaks broctl > -------------------------------------------------------- > > Key: BIT-1267 > URL: https://bro-tracker.atlassian.net/browse/BIT-1267 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Assignee: Robin Sommer > > Attempting to use the new packet source prefix separator (see BIT-1249) > in node.cfg prevents broctl from being able to start due to the use of > Python's ConfigParser (it interprets % as a special character). > For example: > interface=test%eth0 -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Tue Oct 7 17:44:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 7 Oct 2014 19:44:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1242) Add a page which lists and describes all Bro logs In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1242: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Add a page which lists and describes all Bro logs > ------------------------------------------------- > > Key: BIT-1242 > URL: https://bro-tracker.atlassian.net/browse/BIT-1242 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Documentation, Website > Reporter: Jeannette Dopheide > Assignee: Robin Sommer > Fix For: 2.4 > > > Add a page to documentation listing the Bro log files. > There's already a page listing common log files: > http://www.bro.org/sphinx/logs/index.html#common-log-files > This page could be moved to http://www.bro.org/sphinx/script-reference/index.html and be expanded to list all known log files. > If this goes well, Robin suggested adding a btest that heuristically checks if something changes with regards to which logs we have. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Tue Oct 7 17:44:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 7 Oct 2014 19:44:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1269) Add more Bro script language reference documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1269: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Add more Bro script language reference documentation > ---------------------------------------------------- > > Key: BIT-1269 > URL: https://bro-tracker.atlassian.net/browse/BIT-1269 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Daniel Thayer > Assignee: Robin Sommer > Fix For: 2.4 > > > The Bro script language needs more documentation. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Tue Oct 7 19:04:07 2014 From: jira at bro-tracker.atlassian.net (gclark (JIRA)) Date: Tue, 7 Oct 2014 21:04:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: References: Message-ID: gclark created BIT-1270: --------------------------- Summary: topic/gilbert/plugin-api-tweak Key: BIT-1270 URL: https://bro-tracker.atlassian.net/browse/BIT-1270 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Reporter: gclark -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Tue Oct 7 19:10:07 2014 From: jira at bro-tracker.atlassian.net (gclark (JIRA)) Date: Tue, 7 Oct 2014 21:10:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gclark updated BIT-1270: ------------------------ Description: This branch makes a few changes to the API: * Wraps values in a simple class (ValWrapper) that include an explicit processed / not processed flag (to avoid confusion with delayed / opaque invocations). * Adds a Frame argument to HookCallFunction * Adds support for Frame argument types to HookArgument * Adds support for ValWrapper argument types to HookArgument * Tweaks the plugin.hooks tests a bit to include new output (from additional argument) * Tweaks the plugin.api-version-mismatch to remove explicit home directory path via simple regex > topic/gilbert/plugin-api-tweak > ------------------------------ > > Key: BIT-1270 > URL: https://bro-tracker.atlassian.net/browse/BIT-1270 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: gclark > > This branch makes a few changes to the API: > * Wraps values in a simple class (ValWrapper) that include an explicit processed / not processed flag (to avoid confusion with delayed / opaque invocations). > * Adds a Frame argument to HookCallFunction > * Adds support for Frame argument types to HookArgument > * Adds support for ValWrapper argument types to HookArgument > * Tweaks the plugin.hooks tests a bit to include new output (from additional argument) > * Tweaks the plugin.api-version-mismatch to remove explicit home directory path via simple regex -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Tue Oct 7 19:14:07 2014 From: jira at bro-tracker.atlassian.net (gclark (JIRA)) Date: Tue, 7 Oct 2014 21:14:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gclark updated BIT-1270: ------------------------ Status: Merge Request (was: Open) > topic/gilbert/plugin-api-tweak > ------------------------------ > > Key: BIT-1270 > URL: https://bro-tracker.atlassian.net/browse/BIT-1270 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: gclark > > This branch makes a few changes to the API: > * Wraps values in a simple class (ValWrapper) that include an explicit processed / not processed flag (to avoid confusion with delayed / opaque invocations). > * Adds a Frame argument to HookCallFunction > * Adds support for Frame argument types to HookArgument > * Adds support for ValWrapper argument types to HookArgument > * Tweaks the plugin.hooks tests a bit to include new output (from additional argument) > * Tweaks the plugin.api-version-mismatch to remove explicit home directory path via simple regex -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From seth at icir.org Tue Oct 7 19:39:45 2014 From: seth at icir.org (Seth Hall) Date: Tue, 7 Oct 2014 22:39:45 -0400 Subject: [Bro-Dev] Plugins providing threads? In-Reply-To: <20141007224301.GC37792@icir.org> References: <20141007224301.GC37792@icir.org> Message-ID: <2DF964B5-9B6B-45AA-9449-B932C47868DE@icir.org> On Oct 7, 2014, at 6:43 PM, Robin Sommer wrote: > I'm wondering if we should add another type of plugin component: > threads. This would be for functionality that's to run in parallel > with Bro's main thread, and communicate with it via message passing. I think it makes sense. Another use for it could be as a mechanism for creating async BiFs for integrating with libraries that are blocking. ;) Sounds cool. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From noreply at bro.org Wed Oct 8 00:00:19 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 8 Oct 2014 00:00:19 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410080700.s9870J1H023527@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ---------------------------------- BIT-1270 [1] Bro gclark - 2014-10-07 - Normal topic/gilbert/plugin-api-tweak [2] [1] BIT-1270 https://bro-tracker.atlassian.net/browse/BIT-1270 [2] plugin-api-tweak https://github.com/bro/bro/tree/topic/gilbert/plugin-api-tweak From jsiwek at illinois.edu Wed Oct 8 08:29:30 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Wed, 8 Oct 2014 15:29:30 +0000 Subject: [Bro-Dev] Plugins providing threads? In-Reply-To: <20141007224301.GC37792@icir.org> References: <20141007224301.GC37792@icir.org> Message-ID: <58B850C9-EF1F-4E79-80A5-42C2BEE3401B@illinois.edu> On Oct 7, 2014, at 5:43 PM, Robin Sommer wrote: > Jon, how are you planing to integrate Broker into Bro? Would this help > there as well if you could just follow a similar structure with Broker > running inside its own thread? Mostly planning to integrate as IOSource(s) in the main thread as I expect many of the messages either originate or terminate there due to script-land interactivity. Broker?s internal processing is to be scheduled to threads automatically by actor-framework. Wrapping all Broker-related messaging as a Bro-thread that does its own message passing seems like an unneeded indirection as Broker already facilitates its own message passing between endpoints (as it?s built on top of the more-generalized actor-framework message passing). - Jon From gc355804 at ohio.edu Wed Oct 8 10:47:38 2014 From: gc355804 at ohio.edu (Clark, Gilbert) Date: Wed, 8 Oct 2014 17:47:38 +0000 Subject: [Bro-Dev] Plugins providing threads? In-Reply-To: <58B850C9-EF1F-4E79-80A5-42C2BEE3401B@illinois.edu> References: <20141007224301.GC37792@icir.org>, <58B850C9-EF1F-4E79-80A5-42C2BEE3401B@illinois.edu> Message-ID: <1412790457386.11662@ohio.edu> Would it make sense to replace the existing inter-thread communication code with the broker / porting the existing writers and readers to use the actor framework? This way, there would only be a single, shared message-passing mechanism inside of bro, instead of having one for core <-> threads, and another for core <-> external -Gilbert ________________________________________ From: bro-dev-bounces at bro.org on behalf of Siwek, Jon Sent: Wednesday, October 08, 2014 11:29 AM To: Robin Sommer Cc: bro-dev at bro.org Subject: Re: [Bro-Dev] Plugins providing threads? On Oct 7, 2014, at 5:43 PM, Robin Sommer wrote: > Jon, how are you planing to integrate Broker into Bro? Would this help > there as well if you could just follow a similar structure with Broker > running inside its own thread? Mostly planning to integrate as IOSource(s) in the main thread as I expect many of the messages either originate or terminate there due to script-land interactivity. Broker?s internal processing is to be scheduled to threads automatically by actor-framework. Wrapping all Broker-related messaging as a Bro-thread that does its own message passing seems like an unneeded indirection as Broker already facilitates its own message passing between endpoints (as it?s built on top of the more-generalized actor-framework message passing). - Jon _______________________________________________ bro-dev mailing list bro-dev at bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From jsiwek at illinois.edu Wed Oct 8 12:08:26 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Wed, 8 Oct 2014 19:08:26 +0000 Subject: [Bro-Dev] Plugins providing threads? In-Reply-To: <1412790457386.11662@ohio.edu> References: <20141007224301.GC37792@icir.org>, <58B850C9-EF1F-4E79-80A5-42C2BEE3401B@illinois.edu> <1412790457386.11662@ohio.edu> Message-ID: On Oct 8, 2014, at 12:47 PM, Clark, Gilbert wrote: > Would it make sense to replace the existing inter-thread communication code with the broker / porting the existing writers and readers to use the actor framework? This way, there would only be a single, shared message-passing mechanism inside of bro, instead of having one for core <-> threads, and another for core <-> external I think Broker is going to end up best addressing "core <-> external? by providing (hopefully) simple interfaces to a few common/specific messaging patterns and may not make sense to use it generally for ?core <-> threads?. But yes, using actor-framework directly to replace the current threading approach is an option to consider. - Jon From jira at bro-tracker.atlassian.net Wed Oct 8 15:15:07 2014 From: jira at bro-tracker.atlassian.net (Todd C (JIRA)) Date: Wed, 8 Oct 2014 17:15:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1271) Installing from source fails on Xubuntu 14.04 In-Reply-To: References: Message-ID: Todd C created BIT-1271: --------------------------- Summary: Installing from source fails on Xubuntu 14.04 Key: BIT-1271 URL: https://bro-tracker.atlassian.net/browse/BIT-1271 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Environment: Xubuntu 14.04 3.13.0-36-generic x86_64 Reporter: Todd C Attachments: bashlog.txt, CMakeError.log, CMakeOutput.log Attempting to install a fresh instance of bro from source, cloned the master repository with git according to instructions on https://www.bro.org/sphinx/install/install.html Made sure to install prereqs using the command listed, attempted to run the configure script and am receiving a few errors in regards to CMake. Attached the bash output, CMake output, and error logs. The error causing the break is as follows: CMake Error at cmake/PCAPTests.cmake:58 (message): Can't determine if pcap_compile_nopcap takes an error parameter -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Wed Oct 8 18:39:08 2014 From: jira at bro-tracker.atlassian.net (Todd C (JIRA)) Date: Wed, 8 Oct 2014 20:39:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1271) Installing from source fails on Xubuntu 14.04 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd C updated BIT-1271: ------------------------ Resolution: No longer applies Status: Closed (was: Open) Ended up going back and removing all dependencies and re-installing them fresh. Also reverted back to stable 2.3.1 from Sep. 5 > Installing from source fails on Xubuntu 14.04 > --------------------------------------------- > > Key: BIT-1271 > URL: https://bro-tracker.atlassian.net/browse/BIT-1271 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Xubuntu 14.04 3.13.0-36-generic x86_64 > Reporter: Todd C > Labels: installation > Attachments: bashlog.txt, CMakeError.log, CMakeOutput.log > > > Attempting to install a fresh instance of bro from source, cloned the master repository with git according to instructions on https://www.bro.org/sphinx/install/install.html > Made sure to install prereqs using the command listed, attempted to run the configure script and am receiving a few errors in regards to CMake. Attached the bash output, CMake output, and error logs. > The error causing the break is as follows: > CMake Error at cmake/PCAPTests.cmake:58 (message): > Can't determine if pcap_compile_nopcap takes an error parameter -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Wed Oct 8 18:39:08 2014 From: jira at bro-tracker.atlassian.net (Todd C (JIRA)) Date: Wed, 8 Oct 2014 20:39:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1271) Installing from source fails on Xubuntu 14.04 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18232#comment-18232 ] Todd C commented on BIT-1271: ----------------------------- Ended up going back and removing all dependencies and re-installing them fresh. Also reverted back to stable 2.3.1 from Sep. 5 > Installing from source fails on Xubuntu 14.04 > --------------------------------------------- > > Key: BIT-1271 > URL: https://bro-tracker.atlassian.net/browse/BIT-1271 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Xubuntu 14.04 3.13.0-36-generic x86_64 > Reporter: Todd C > Labels: installation > Attachments: bashlog.txt, CMakeError.log, CMakeOutput.log > > > Attempting to install a fresh instance of bro from source, cloned the master repository with git according to instructions on https://www.bro.org/sphinx/install/install.html > Made sure to install prereqs using the command listed, attempted to run the configure script and am receiving a few errors in regards to CMake. Attached the bash output, CMake output, and error logs. > The error causing the break is as follows: > CMake Error at cmake/PCAPTests.cmake:58 (message): > Can't determine if pcap_compile_nopcap takes an error parameter -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From noreply at bro.org Thu Oct 9 00:00:28 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 9 Oct 2014 00:00:28 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410090700.s9970S66027926@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ---------------------------------- BIT-1270 [1] Bro gclark - 2014-10-07 - Normal topic/gilbert/plugin-api-tweak [2] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ---------------------------------------------- 072dad6 [3] bro Daniel Thayer 2014-10-08 Add error checks and messages to a test script [1] BIT-1270 https://bro-tracker.atlassian.net/browse/BIT-1270 [2] plugin-api-tweak https://github.com/bro/bro/tree/topic/gilbert/plugin-api-tweak [3] 072dad6 https://github.com/bro/bro/commit/072dad6508c1ff984cbe4fe8ba4e23c3a1fdb640 From noreply at bro.org Fri Oct 10 00:00:18 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 10 Oct 2014 00:00:18 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410100700.s9A70I58008306@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ---------------------------------- BIT-1270 [1] Bro gclark - 2014-10-07 - Normal topic/gilbert/plugin-api-tweak [2] [1] BIT-1270 https://bro-tracker.atlassian.net/browse/BIT-1270 [2] plugin-api-tweak https://github.com/bro/bro/tree/topic/gilbert/plugin-api-tweak From jira at bro-tracker.atlassian.net Fri Oct 10 17:37:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 10 Oct 2014 19:37:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18300#comment-18300 ] Robin Sommer commented on BIT-1270: ----------------------------------- A few comments: - instead if using a separate wrapper class, what would think about just using std::tuple? I think that would simplify things quite a bit. - reporter->InternalError() takes a format string, so you can skip the snprintf. - please try to follow Bro's coding conventions for formatting, the diff is a bit messy > topic/gilbert/plugin-api-tweak > ------------------------------ > > Key: BIT-1270 > URL: https://bro-tracker.atlassian.net/browse/BIT-1270 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: gclark > > This branch makes a few changes to the API: > * Wraps values in a simple class (ValWrapper) that include an explicit processed / not processed flag (to avoid confusion with delayed / opaque invocations). > * Adds a Frame argument to HookCallFunction > * Adds support for Frame argument types to HookArgument > * Adds support for ValWrapper argument types to HookArgument > * Tweaks the plugin.hooks tests a bit to include new output (from additional argument) > * Tweaks the plugin.api-version-mismatch to remove explicit home directory path via simple regex -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 10 17:37:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 10 Oct 2014 19:37:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1270: --------------------------------- Assignee: gclark > topic/gilbert/plugin-api-tweak > ------------------------------ > > Key: BIT-1270 > URL: https://bro-tracker.atlassian.net/browse/BIT-1270 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: gclark > Assignee: gclark > > This branch makes a few changes to the API: > * Wraps values in a simple class (ValWrapper) that include an explicit processed / not processed flag (to avoid confusion with delayed / opaque invocations). > * Adds a Frame argument to HookCallFunction > * Adds support for Frame argument types to HookArgument > * Adds support for ValWrapper argument types to HookArgument > * Tweaks the plugin.hooks tests a bit to include new output (from additional argument) > * Tweaks the plugin.api-version-mismatch to remove explicit home directory path via simple regex -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 10 17:37:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 10 Oct 2014 19:37:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1270: ------------------------------ Status: Open (was: Merge Request) > topic/gilbert/plugin-api-tweak > ------------------------------ > > Key: BIT-1270 > URL: https://bro-tracker.atlassian.net/browse/BIT-1270 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: gclark > Assignee: gclark > > This branch makes a few changes to the API: > * Wraps values in a simple class (ValWrapper) that include an explicit processed / not processed flag (to avoid confusion with delayed / opaque invocations). > * Adds a Frame argument to HookCallFunction > * Adds support for Frame argument types to HookArgument > * Adds support for ValWrapper argument types to HookArgument > * Tweaks the plugin.hooks tests a bit to include new output (from additional argument) > * Tweaks the plugin.api-version-mismatch to remove explicit home directory path via simple regex -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Fri Oct 10 19:57:07 2014 From: jira at bro-tracker.atlassian.net (gclark (JIRA)) Date: Fri, 10 Oct 2014 21:57:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18301#comment-18301 ] gclark commented on BIT-1270: ----------------------------- * Only bad thing about a tuple is that the members would no longer be named, which can make accesses more difficult to read in the code. Assuming I'm understanding this suggestion properly, what about modifying this to pass / return the existing ValWrapper by value instead? This way, the members would retain readable names, but it would still eliminate some complexity. * Re: internalerror - good to know, thanks * Re: coding conventions - sorry about that: will get things cleaned up. > topic/gilbert/plugin-api-tweak > ------------------------------ > > Key: BIT-1270 > URL: https://bro-tracker.atlassian.net/browse/BIT-1270 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: gclark > Assignee: gclark > > This branch makes a few changes to the API: > * Wraps values in a simple class (ValWrapper) that include an explicit processed / not processed flag (to avoid confusion with delayed / opaque invocations). > * Adds a Frame argument to HookCallFunction > * Adds support for Frame argument types to HookArgument > * Adds support for ValWrapper argument types to HookArgument > * Tweaks the plugin.hooks tests a bit to include new output (from additional argument) > * Tweaks the plugin.api-version-mismatch to remove explicit home directory path via simple regex -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Sat Oct 11 05:36:07 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Sat, 11 Oct 2014 07:36:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18302#comment-18302 ] Brian O'Berry commented on BIT-1238: ------------------------------------ Played around with regex101.com, which shows the following string matches the regex. Ignore line wrapping, it does not contain a newline. {code} This sequence is exactly 100 printable characters, followed by 3 groups of 8-character digits/spaces 23 5 78 23456781 3 5 78 {code} I guess we see a lot of text files with strings like that in our environment. I'll try to research tar file structure to understand where the regex came from. In the meantime, we'll try excluding the {{file-tar}} signature by adding it to the {{Signatures::ignored_ids}} pattern. > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Labels: file, mime, signature > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Sat Oct 11 05:55:07 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Sat, 11 Oct 2014 07:55:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18303#comment-18303 ] Brian O'Berry commented on BIT-1238: ------------------------------------ I guess more likely, any regular text file containing lines padded with at least 24 spaces after the 100th character will match, including a line of at least 124 spaces. Or, lines padded with spaces followed by digits in the case of left-justified descriptions followed by right-justified integers. Point being this pattern seems likely to match many text files, period, not just those in our environment. > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Labels: file, mime, signature > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Sat Oct 11 08:09:08 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Sat, 11 Oct 2014 10:09:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18304#comment-18304 ] Brian O'Berry commented on BIT-1238: ------------------------------------ After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7} ?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include: * don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code} * don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code} * change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}} * change the possible digits for mode, uid, gid to octal -- change {{\[:digits:\]}} to {{\[0-7\]}} * move the space and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end * change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group * make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Labels: file, mime, signature > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Sat Oct 11 08:28:07 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Sat, 11 Oct 2014 10:28:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18304#comment-18304 ] Brian O'Berry edited comment on BIT-1238 at 10/11/14 10:27 AM: --------------------------------------------------------------- After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7} ?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include: * don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code} * don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code} * change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}} * change the possible digits for mode, uid, gid to octal -- change {{\[:digits:\]}} to {{0-7}} * move the space and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end * change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group * make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly was (Author: boberry): After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7} ?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include: * don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code} * don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code} * change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}} * change the possible digits for mode, uid, gid to octal -- change {{\[:digits:\]}} to {{\[0-7\]}} * move the space and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end * change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group * make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Labels: file, mime, signature > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Sat Oct 11 08:28:08 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Sat, 11 Oct 2014 10:28:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18304#comment-18304 ] Brian O'Berry edited comment on BIT-1238 at 10/11/14 10:28 AM: --------------------------------------------------------------- After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7} ?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include: * don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code} * don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code} * change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}} * change the possible digits for mode, uid, gid to octal -- change {{\[:digit:\]}} to {{0-7}} * move the space and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end * change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group * make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly was (Author: boberry): After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7} ?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include: * don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code} * don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code} * change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}} * change the possible digits for mode, uid, gid to octal -- change {{\[:digits:\]}} to {{0-7}} * move the space and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end * change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group * make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Labels: file, mime, signature > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Sat Oct 11 08:35:07 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Sat, 11 Oct 2014 10:35:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18304#comment-18304 ] Brian O'Berry edited comment on BIT-1238 at 10/11/14 10:34 AM: --------------------------------------------------------------- After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7}\x20?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include: * don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code} * don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code} * change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}} * change the possible digits for mode, uid, gid to octal -- change {{\[:digit:\]}} to {{0-7}} * move the space ({{\x20}} and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end * change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group * make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly UPDATE: corrected the 4th bullet, and changed the pattern {{' '}} back to {{\x20}} to avoid possible errors if someone's editor converts the blank to a tab was (Author: boberry): After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7} ?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include: * don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code} * don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code} * change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}} * change the possible digits for mode, uid, gid to octal -- change {{\[:digit:\]}} to {{0-7}} * move the space and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end * change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group * make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Labels: file, mime, signature > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Sat Oct 11 08:41:08 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Sat, 11 Oct 2014 10:41:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18304#comment-18304 ] Brian O'Berry edited comment on BIT-1238 at 10/11/14 10:40 AM: --------------------------------------------------------------- After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7}[[:space:]]?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include: * don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code} * don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code} * change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}} * change the possible digits for mode, uid, gid to octal -- change {{\[:digit:\]}} to {{0-7}} * move the space ({{\x20}} and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end * change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group * make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly UPDATE: corrected the 4th bullet, and changed the space in the pattern to {{\[\[:space:]]}} to avoid possible errors if someone's editor converts the blank to a tab was (Author: boberry): After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7}\x20?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include: * don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code} * don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code} * change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}} * change the possible digits for mode, uid, gid to octal -- change {{\[:digit:\]}} to {{0-7}} * move the space ({{\x20}} and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end * change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group * make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly UPDATE: corrected the 4th bullet, and changed the pattern {{' '}} back to {{\x20}} to avoid possible errors if someone's editor converts the blank to a tab > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Labels: file, mime, signature > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Sat Oct 11 08:42:07 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Sat, 11 Oct 2014 10:42:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18304#comment-18304 ] Brian O'Berry edited comment on BIT-1238 at 10/11/14 10:41 AM: --------------------------------------------------------------- After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7}[[:space:]]?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include: * don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code} * don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code} * change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}} * change the possible digits for mode, uid, gid to octal -- change {{\[:digit:\]}} to {{0-7}} * move the space and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end * change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group * make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly UPDATE: corrected the 4th bullet, and changed the space in the pattern to {{\[\[:space:]]}} to avoid possible errors if someone's editor converts the blank to a tab was (Author: boberry): After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7}[[:space:]]?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include: * don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code} * don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code} * change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}} * change the possible digits for mode, uid, gid to octal -- change {{\[:digit:\]}} to {{0-7}} * move the space ({{\x20}} and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end * change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group * make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly UPDATE: corrected the 4th bullet, and changed the space in the pattern to {{\[\[:space:]]}} to avoid possible errors if someone's editor converts the blank to a tab > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Labels: file, mime, signature > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Sat Oct 11 10:00:07 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Sat, 11 Oct 2014 12:00:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1235) HTTP multipart POST request alters file contents In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18305#comment-18305 ] Brian O'Berry commented on BIT-1235: ------------------------------------ We've run this through our QA process and it's been running on our network for almost a week without issue. From our standpoint, it's ready for merge. Can we help with anything else? Thank you! > HTTP multipart POST request alters file contents > ------------------------------------------------ > > Key: BIT-1235 > URL: https://bro-tracker.atlassian.net/browse/BIT-1235 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: CentOS 6.5, file extract analyzer > Reporter: Brian O'Berry > Fix For: 2.4 > > Attachments: bro-2.3-HTTP.patch, gdb.log, upload-api-http.pcap > > > HTTP POST multipart processing converts bare CR or LF chars to CRLF pairs, corrupting most files when extracted with Files::ANALYZER_EXTRACT. This is clear in the attached gdb.log, which has a backtrace that shows a buffer with the start of a PDF file entering MIME/HTTP entity processing at frame 25, and emerging with LF chars converted to CRLF at frame 6. > Also attached are the pcap file associated with the backtrace, and an initial patch that we've barely begun to test. A point of concern with the patch is that it changes a weird.log entry from "line_terminated_with_single_CR" to "http_no_crlf_in_header_list". It does enable Files::ANALYZER_EXTRACT to correctly extract the PDF file from the attached pcap. > Please let me know if we can provide anything else to help with this. -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Sun Oct 12 10:15:08 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 12 Oct 2014 12:15:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall reassigned BIT-1238: ------------------------------ Assignee: Seth Hall > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-05-009#64003) From jira at bro-tracker.atlassian.net Sun Oct 12 19:07:07 2014 From: jira at bro-tracker.atlassian.net (steve smoot (JIRA)) Date: Sun, 12 Oct 2014 21:07:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1272) doc fixes In-Reply-To: References: Message-ID: steve smoot created BIT-1272: -------------------------------- Summary: doc fixes Key: BIT-1272 URL: https://bro-tracker.atlassian.net/browse/BIT-1272 Project: Bro Issue Tracker Issue Type: Patch Components: Documentation Reporter: steve smoot Priority: Low https://www.bro.org/sphinx/scripting/index.html#understanding-bro-scripts fig. data_struct_vector_declaration.bro line 15 should say v2 in text not just argument (and change line 4 of output) fig. data_type_pattern_01.bro should really be "jumps" not "jumped" (-; (and also in output) -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Mon Oct 13 07:52:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 13 Oct 2014 09:52:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1235) HTTP multipart POST request alters file contents In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1235: --------------------------- Assignee: Jon Siwek > HTTP multipart POST request alters file contents > ------------------------------------------------ > > Key: BIT-1235 > URL: https://bro-tracker.atlassian.net/browse/BIT-1235 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: CentOS 6.5, file extract analyzer > Reporter: Brian O'Berry > Assignee: Jon Siwek > Fix For: 2.4 > > Attachments: bro-2.3-HTTP.patch, gdb.log, upload-api-http.pcap > > > HTTP POST multipart processing converts bare CR or LF chars to CRLF pairs, corrupting most files when extracted with Files::ANALYZER_EXTRACT. This is clear in the attached gdb.log, which has a backtrace that shows a buffer with the start of a PDF file entering MIME/HTTP entity processing at frame 25, and emerging with LF chars converted to CRLF at frame 6. > Also attached are the pcap file associated with the backtrace, and an initial patch that we've barely begun to test. A point of concern with the patch is that it changes a weird.log entry from "line_terminated_with_single_CR" to "http_no_crlf_in_header_list". It does enable Files::ANALYZER_EXTRACT to correctly extract the PDF file from the attached pcap. > Please let me know if we can provide anything else to help with this. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Mon Oct 13 07:54:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 13 Oct 2014 09:54:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1235) HTTP multipart POST request alters file contents In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18400#comment-18400 ] Jon Siwek commented on BIT-1235: -------------------------------- I'll make sure it still merges cleanly with the master branch, then ask Robin to review/merge. Thanks for helping test/confirm. > HTTP multipart POST request alters file contents > ------------------------------------------------ > > Key: BIT-1235 > URL: https://bro-tracker.atlassian.net/browse/BIT-1235 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: CentOS 6.5, file extract analyzer > Reporter: Brian O'Berry > Assignee: Jon Siwek > Fix For: 2.4 > > Attachments: bro-2.3-HTTP.patch, gdb.log, upload-api-http.pcap > > > HTTP POST multipart processing converts bare CR or LF chars to CRLF pairs, corrupting most files when extracted with Files::ANALYZER_EXTRACT. This is clear in the attached gdb.log, which has a backtrace that shows a buffer with the start of a PDF file entering MIME/HTTP entity processing at frame 25, and emerging with LF chars converted to CRLF at frame 6. > Also attached are the pcap file associated with the backtrace, and an initial patch that we've barely begun to test. A point of concern with the patch is that it changes a weird.log entry from "line_terminated_with_single_CR" to "http_no_crlf_in_header_list". It does enable Files::ANALYZER_EXTRACT to correctly extract the PDF file from the attached pcap. > Please let me know if we can provide anything else to help with this. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Mon Oct 13 11:20:08 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Mon, 13 Oct 2014 13:20:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1235) HTTP multipart POST request alters file contents In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18305#comment-18305 ] Brian O'Berry edited comment on BIT-1235 at 10/13/14 1:19 PM: -------------------------------------------------------------- We've run this through our QA process and we've been running it for almost a week without issue. From our standpoint, it's ready for merge. Can we help with anything else? Thank you! was (Author: boberry): We've run this through our QA process and it's been running on our network for almost a week without issue. From our standpoint, it's ready for merge. Can we help with anything else? Thank you! > HTTP multipart POST request alters file contents > ------------------------------------------------ > > Key: BIT-1235 > URL: https://bro-tracker.atlassian.net/browse/BIT-1235 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: CentOS 6.5, file extract analyzer > Reporter: Brian O'Berry > Assignee: Jon Siwek > Fix For: 2.4 > > Attachments: bro-2.3-HTTP.patch, gdb.log, upload-api-http.pcap > > > HTTP POST multipart processing converts bare CR or LF chars to CRLF pairs, corrupting most files when extracted with Files::ANALYZER_EXTRACT. This is clear in the attached gdb.log, which has a backtrace that shows a buffer with the start of a PDF file entering MIME/HTTP entity processing at frame 25, and emerging with LF chars converted to CRLF at frame 6. > Also attached are the pcap file associated with the backtrace, and an initial patch that we've barely begun to test. A point of concern with the patch is that it changes a weird.log entry from "line_terminated_with_single_CR" to "http_no_crlf_in_header_list". It does enable Files::ANALYZER_EXTRACT to correctly extract the PDF file from the attached pcap. > Please let me know if we can provide anything else to help with this. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 14 11:17:08 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Tue, 14 Oct 2014 13:17:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18401#comment-18401 ] Seth Hall commented on BIT-1238: -------------------------------- Do you have an example text file that incorrectly matches the current pattern? I'd like to have something for our tests. > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 14 12:34:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 14 Oct 2014 14:34:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1272) doc fixes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek reassigned BIT-1272: ------------------------------ Assignee: Jon Siwek > doc fixes > --------- > > Key: BIT-1272 > URL: https://bro-tracker.atlassian.net/browse/BIT-1272 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Documentation > Reporter: steve smoot > Assignee: Jon Siwek > Priority: Low > > https://www.bro.org/sphinx/scripting/index.html#understanding-bro-scripts > fig. data_struct_vector_declaration.bro > line 15 should say v2 in text not just argument > (and change line 4 of output) > fig. data_type_pattern_01.bro > should really be "jumps" not "jumped" (-; > (and also in output) -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 14 12:42:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 14 Oct 2014 14:42:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1272) doc fixes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1272: --------------------------- Fix Version/s: 2.4 > doc fixes > --------- > > Key: BIT-1272 > URL: https://bro-tracker.atlassian.net/browse/BIT-1272 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Documentation > Reporter: steve smoot > Assignee: Jon Siwek > Priority: Low > Fix For: 2.4 > > > https://www.bro.org/sphinx/scripting/index.html#understanding-bro-scripts > fig. data_struct_vector_declaration.bro > line 15 should say v2 in text not just argument > (and change line 4 of output) > fig. data_type_pattern_01.bro > should really be "jumps" not "jumped" (-; > (and also in output) -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 14 12:44:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 14 Oct 2014 14:44:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1272) doc fixes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1272: --------------------------- Resolution: Fixed Status: Closed (was: Open) > doc fixes > --------- > > Key: BIT-1272 > URL: https://bro-tracker.atlassian.net/browse/BIT-1272 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Documentation > Reporter: steve smoot > Assignee: Jon Siwek > Priority: Low > Fix For: 2.4 > > > https://www.bro.org/sphinx/scripting/index.html#understanding-bro-scripts > fig. data_struct_vector_declaration.bro > line 15 should say v2 in text not just argument > (and change line 4 of output) > fig. data_type_pattern_01.bro > should really be "jumps" not "jumped" (-; > (and also in output) -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 14 12:45:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 14 Oct 2014 14:45:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1272) doc fixes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18402#comment-18402 ] Jon Siwek commented on BIT-1272: -------------------------------- Fixed, thanks. > doc fixes > --------- > > Key: BIT-1272 > URL: https://bro-tracker.atlassian.net/browse/BIT-1272 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Documentation > Reporter: steve smoot > Assignee: Jon Siwek > Priority: Low > Fix For: 2.4 > > > https://www.bro.org/sphinx/scripting/index.html#understanding-bro-scripts > fig. data_struct_vector_declaration.bro > line 15 should say v2 in text not just argument > (and change line 4 of output) > fig. data_type_pattern_01.bro > should really be "jumps" not "jumped" (-; > (and also in output) -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 14 13:21:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 14 Oct 2014 15:21:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1235) HTTP multipart POST request alters file contents In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1235: --------------------------- Status: Merge Request (was: Open) > HTTP multipart POST request alters file contents > ------------------------------------------------ > > Key: BIT-1235 > URL: https://bro-tracker.atlassian.net/browse/BIT-1235 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: CentOS 6.5, file extract analyzer > Reporter: Brian O'Berry > Assignee: Jon Siwek > Fix For: 2.4 > > Attachments: bro-2.3-HTTP.patch, gdb.log, upload-api-http.pcap > > > HTTP POST multipart processing converts bare CR or LF chars to CRLF pairs, corrupting most files when extracted with Files::ANALYZER_EXTRACT. This is clear in the attached gdb.log, which has a backtrace that shows a buffer with the start of a PDF file entering MIME/HTTP entity processing at frame 25, and emerging with LF chars converted to CRLF at frame 6. > Also attached are the pcap file associated with the backtrace, and an initial patch that we've barely begun to test. A point of concern with the patch is that it changes a weird.log entry from "line_terminated_with_single_CR" to "http_no_crlf_in_header_list". It does enable Files::ANALYZER_EXTRACT to correctly extract the PDF file from the attached pcap. > Please let me know if we can provide anything else to help with this. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From anthony.kasza at gmail.com Tue Oct 14 17:48:56 2014 From: anthony.kasza at gmail.com (anthony kasza) Date: Tue, 14 Oct 2014 17:48:56 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: <20141007152555.GC63065@icir.org> References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> Message-ID: Thanks all. After moving declarations around, nm shows all my expected symbols to be defined. I'm now receiving "internal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1: internal type geo_location missing". It seems this error is being caused by the check on the return value of lookup_ID() done within the internal_type() function in Var.cc. From what I can tell, I don't have geo_location = internal_type("geo_location")->AsRecordType(); in the right location. This line is from the init_net_var() function from NetVar.cc, which gets called by main.cc. I thought it might be a clash in name/module spaces and tried using the init-plugin script with some unique values, but I still receive the same error. I'm not sure if it matters but I when I run "bro -NNb" I can see my inactive dynamic plugin, so I know Bro is aware of it. I currently have everything sitting in a single .bif file. Would it be useful to post that? -AK On Tue, Oct 7, 2014 at 8:25 AM, Robin Sommer wrote: > > > On Mon, Oct 06, 2014 at 21:46 -0700, you wrote: > >> Thanks Robin. Everything on the "Writing Bro Plugins" page is clear >> although the debugging section seems thin to me. Then again, I've have >> never compiled Bro with debugging enabled before. > > Ok, I'll try to extend that a bit more, it's written mainly from the > perspective of adding debugging output to your plugin; not in terms of > leveraging the existing output to see why something's not working. > >> Are there plans to expand the Types section of that page? > > I would like to, but it's quite a bit of work to fill in the TODOs, so > honestly I don't see that coming very soon. The problem is that most > of the missing TODOs are actually not really a matter per se of > writing a plugin or not, but require text on how to write the > corresponding code for Bro in general (in other words, even before > moving to plugins, much of that would have been quite similar). For > now, the best way to figure this out is to look at existing code, > i.e., for the builtin-in stuff other *.bif files. > >> I think I'm not declaring a scriptland record type correctly. > > Feel free to post error messages here. > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Tue Oct 14 18:44:07 2014 From: jira at bro-tracker.atlassian.net (Struck (JIRA)) Date: Tue, 14 Oct 2014 20:44:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1273) Broscript crashes Bro with segfault when defining a global enum In-Reply-To: References: Message-ID: Struck created BIT-1273: --------------------------- Summary: Broscript crashes Bro with segfault when defining a global enum Key: BIT-1273 URL: https://bro-tracker.atlassian.net/browse/BIT-1273 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Reporter: Struck Attachments: test.bro broscript crashes bro with segfault, when trying to define a *global* : enum { } instead of giving just a simple error on parsing. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From noreply at bro.org Wed Oct 15 00:00:17 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 15 Oct 2014 00:00:17 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410150700.s9F70HVF031470@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- ------------------------------------------------ BIT-1235 [1] Bro Brian O'Berry Jon Siwek 2014-10-14 2.4 Normal HTTP multipart POST request alters file contents [1] BIT-1235 https://bro-tracker.atlassian.net/browse/BIT-1235 From jira at bro-tracker.atlassian.net Wed Oct 15 06:43:08 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Wed, 15 Oct 2014 08:43:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian O'Berry updated BIT-1238: ------------------------------- Attachment: test.tar.gz Test results that show text file (test.txt) classified as a tar file > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > Attachments: test.tar.gz > > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 15 06:55:07 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Wed, 15 Oct 2014 08:55:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18404#comment-18404 ] Brian O'Berry commented on BIT-1238: ------------------------------------ Attached test.tar.gz, with test results of an HTTP GET for which the extracted text file is classified as a tar file. * test.pcap contains the HTTP GET session (you'll need to disable checksums to use it) * test.txt is the original file * extract_files/ contains the extracted file * etc/ holds config files used for the test, for which bro was started as a daemon via 'broctl start' * logs/ includes files.*.log.gz showing the incorrect mime type * spool/ includes the local.bro policy file used for the test > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > Attachments: test.tar.gz > > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jsiwek at illinois.edu Wed Oct 15 07:46:37 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Wed, 15 Oct 2014 14:46:37 +0000 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> Message-ID: On Oct 14, 2014, at 7:48 PM, anthony kasza wrote: > From what I can tell, I don't have > > geo_location = internal_type("geo_location")->AsRecordType(); > > in the right location. This line is from the init_net_var() function > from NetVar.cc, which gets called by main.cc. Maybe that can just be completely removed if Bro proper no longer relies on that type since all the related functionality is now provided by the plugin? > everything sitting in a single .bif file. Would it be useful to post > that? How about putting the entire plugin source directory in a github repo? That should make it easy for others to start poking at the same code as you. - Jon From jira at bro-tracker.atlassian.net Wed Oct 15 08:23:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 15 Oct 2014 10:23:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1273) Broscript crashes Bro with segfault when defining a global enum In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1273: --------------------------- Fix Version/s: 2.4 > Broscript crashes Bro with segfault when defining a global enum > --------------------------------------------------------------- > > Key: BIT-1273 > URL: https://bro-tracker.atlassian.net/browse/BIT-1273 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Struck > Labels: language > Fix For: 2.4 > > Attachments: test.bro > > > broscript crashes bro with segfault, when trying to define a > *global* : enum { > } > instead of giving just a simple error on parsing. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 15 08:24:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 15 Oct 2014 10:24:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1273) Broscript crashes Bro with segfault when defining a global enum In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1273: --------------------------- Resolution: Fixed Status: Closed (was: Open) > Broscript crashes Bro with segfault when defining a global enum > --------------------------------------------------------------- > > Key: BIT-1273 > URL: https://bro-tracker.atlassian.net/browse/BIT-1273 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Struck > Labels: language > Fix For: 2.4 > > Attachments: test.bro > > > broscript crashes bro with segfault, when trying to define a > *global* : enum { > } > instead of giving just a simple error on parsing. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 15 08:25:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 15 Oct 2014 10:25:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1273) Broscript crashes Bro with segfault when defining a global enum In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18405#comment-18405 ] Jon Siwek commented on BIT-1273: -------------------------------- Fixed in master. Thanks. > Broscript crashes Bro with segfault when defining a global enum > --------------------------------------------------------------- > > Key: BIT-1273 > URL: https://bro-tracker.atlassian.net/browse/BIT-1273 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Struck > Labels: language > Fix For: 2.4 > > Attachments: test.bro > > > broscript crashes bro with segfault, when trying to define a > *global* : enum { > } > instead of giving just a simple error on parsing. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From seth at icir.org Wed Oct 15 08:28:35 2014 From: seth at icir.org (Seth Hall) Date: Wed, 15 Oct 2014 11:28:35 -0400 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> Message-ID: <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> On Oct 15, 2014, at 10:46 AM, Siwek, Jon wrote: > Maybe that can just be completely removed if Bro proper no longer relies on that type since all the related functionality is now provided by the plugin? Agreed. Any code related to GeoIP should move into the plugin, including types. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From anthony.kasza at gmail.com Wed Oct 15 10:22:34 2014 From: anthony.kasza at gmail.com (anthony kasza) Date: Wed, 15 Oct 2014 10:22:34 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> Message-ID: I moved that line from NetVar to my plugin source. I also moved some things from bifs to the plugin source (honestly, just copied and pasted). Posting just the plugin directory won't show which lines in the core I moved to the plugin. Should I make a branch off master? What are the general conventions for doing this? -AK On Oct 15, 2014 8:28 AM, "Seth Hall" wrote: > > On Oct 15, 2014, at 10:46 AM, Siwek, Jon wrote: > > > Maybe that can just be completely removed if Bro proper no longer relies > on that type since all the related functionality is now provided by the > plugin? > > Agreed. Any code related to GeoIP should move into the plugin, including > types. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141015/4f8fb736/attachment.html From seth at icir.org Wed Oct 15 10:29:25 2014 From: seth at icir.org (Seth Hall) Date: Wed, 15 Oct 2014 13:29:25 -0400 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> Message-ID: On Oct 15, 2014, at 1:22 PM, anthony kasza wrote: > I moved that line from NetVar to my plugin source. I also moved some things from bifs to the plugin source (honestly, just copied and pasted). Posting just the plugin directory won't show which lines in the core I moved to the plugin. Should I make a branch off master? What are the general conventions for doing this? Yes, you'd make a branch of master and make your changes there and build your plugin against that branch. When your plugin is merged, it will also require a merge of your branch against Bro as well. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From seth at icir.org Wed Oct 15 10:29:58 2014 From: seth at icir.org (Seth Hall) Date: Wed, 15 Oct 2014 13:29:58 -0400 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> Message-ID: <6A96733F-1897-445A-B6D7-6199BB1D02E8@icir.org> On Oct 15, 2014, at 1:22 PM, anthony kasza wrote: > What are the general conventions for doing this? Oh, also: https://www.bro.org/development/howtos/process.html .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From jira at bro-tracker.atlassian.net Wed Oct 15 23:01:07 2014 From: jira at bro-tracker.atlassian.net (AK (JIRA)) Date: Thu, 16 Oct 2014 01:01:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1274) Moving GeoIP Code to Plugin In-Reply-To: References: Message-ID: AK created BIT-1274: ----------------------- Summary: Moving GeoIP Code to Plugin Key: BIT-1274 URL: https://bro-tracker.atlassian.net/browse/BIT-1274 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Reporter: AK I've started moving the GeoIP code to a plugin. The branch of Bro I'm working from is here: https://github.com/anthonykasza/bro/tree/topic/akasza/geoplugin. The source for the plugin is here: https://github.com/anthonykasza/Bro_GeoIP. Any pointers would be appreciated. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From noreply at bro.org Thu Oct 16 00:00:27 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 16 Oct 2014 00:00:27 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410160700.s9G70RxI002880@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- ------------------------------------------------ BIT-1235 [1] Bro Brian O'Berry Jon Siwek 2014-10-14 2.4 Normal HTTP multipart POST request alters file contents [1] BIT-1235 https://bro-tracker.atlassian.net/browse/BIT-1235 From jira at bro-tracker.atlassian.net Thu Oct 16 05:40:07 2014 From: jira at bro-tracker.atlassian.net (jdonnelly (JIRA)) Date: Thu, 16 Oct 2014 07:40:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1275) README (or install) need better build instructions In-Reply-To: References: Message-ID: jdonnelly created BIT-1275: ------------------------------ Summary: README (or install) need better build instructions Key: BIT-1275 URL: https://bro-tracker.atlassian.net/browse/BIT-1275 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.3 Environment: Ubuntu 14.04 LTS Reporter: jdonnelly The build instructions (specifically the required packages) from https://www.bro.org/sphinx-git/install/install.html should be added to the README file so the product can be built from source without have to consult (or search for on the web) how to cure the ./configure errors. Thank you, -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From robin at icir.org Thu Oct 16 06:02:23 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 16 Oct 2014 06:02:23 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> Message-ID: <20141016130223.GE48006@icir.org> On Wed, Oct 15, 2014 at 13:29 -0400, you wrote: > Yes, you'd make a branch of master and make your changes there and > build your plugin against that branch. When your plugin is merged, it > will also require a merge of your branch against Bro as well. One question here: are we prepated to ask people to install a plugin if they want geoip functionality? I'm asking because while generally I think it's the right way going forward to move external dependencies into plugins, this one we have offered for a while so, in some way, not having it out of the box is a regression (unless we decide to build (some) plugins by default, which currently isn't happening). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Thu Oct 16 06:11:08 2014 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Thu, 16 Oct 2014 08:11:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18406#comment-18406 ] Brian O'Berry commented on BIT-1238: ------------------------------------ Apologies, no idea why I thought bro patterns were pcre-compliant. (Restricted to Users role) > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > Attachments: test.tar.gz > > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From slagell at illinois.edu Thu Oct 16 06:21:02 2014 From: slagell at illinois.edu (Slagell, Adam J) Date: Thu, 16 Oct 2014 13:21:02 +0000 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: <20141016130223.GE48006@icir.org> References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> <20141016130223.GE48006@icir.org> Message-ID: <647511B6-2CFB-4376-9D57-3B9A8FDE88F4@illinois.edu> I think that is reasonable for some things. On Oct 16, 2014, at 8:02 AM, Robin Sommer robin at icir.org wrote: > (unless we decide to > build (some) plugins by default, which currently isn't happening). ------ Adam J. Slagell Chief Information Security Officer Assistant Director, Cybersecurity Directorate National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From jira at bro-tracker.atlassian.net Thu Oct 16 06:38:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 16 Oct 2014 08:38:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1235) HTTP multipart POST request alters file contents In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1235: --------------------------------- Assignee: Robin Sommer (was: Jon Siwek) > HTTP multipart POST request alters file contents > ------------------------------------------------ > > Key: BIT-1235 > URL: https://bro-tracker.atlassian.net/browse/BIT-1235 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: CentOS 6.5, file extract analyzer > Reporter: Brian O'Berry > Assignee: Robin Sommer > Fix For: 2.4 > > Attachments: bro-2.3-HTTP.patch, gdb.log, upload-api-http.pcap > > > HTTP POST multipart processing converts bare CR or LF chars to CRLF pairs, corrupting most files when extracted with Files::ANALYZER_EXTRACT. This is clear in the attached gdb.log, which has a backtrace that shows a buffer with the start of a PDF file entering MIME/HTTP entity processing at frame 25, and emerging with LF chars converted to CRLF at frame 6. > Also attached are the pcap file associated with the backtrace, and an initial patch that we've barely begun to test. A point of concern with the patch is that it changes a weird.log entry from "line_terminated_with_single_CR" to "http_no_crlf_in_header_list". It does enable Files::ANALYZER_EXTRACT to correctly extract the PDF file from the attached pcap. > Please let me know if we can provide anything else to help with this. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 16 06:59:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 16 Oct 2014 08:59:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1235) HTTP multipart POST request alters file contents In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18407#comment-18407 ] Robin Sommer commented on BIT-1235: ----------------------------------- Yeah, the fix looks reasonable. It feels a bit hackish, as it further complicates what's going on based on current state; however I think that's really primarily a problem of the existing HTTP code, don't see a better way to fix it either. Merging. > HTTP multipart POST request alters file contents > ------------------------------------------------ > > Key: BIT-1235 > URL: https://bro-tracker.atlassian.net/browse/BIT-1235 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: CentOS 6.5, file extract analyzer > Reporter: Brian O'Berry > Assignee: Robin Sommer > Fix For: 2.4 > > Attachments: bro-2.3-HTTP.patch, gdb.log, upload-api-http.pcap > > > HTTP POST multipart processing converts bare CR or LF chars to CRLF pairs, corrupting most files when extracted with Files::ANALYZER_EXTRACT. This is clear in the attached gdb.log, which has a backtrace that shows a buffer with the start of a PDF file entering MIME/HTTP entity processing at frame 25, and emerging with LF chars converted to CRLF at frame 6. > Also attached are the pcap file associated with the backtrace, and an initial patch that we've barely begun to test. A point of concern with the patch is that it changes a weird.log entry from "line_terminated_with_single_CR" to "http_no_crlf_in_header_list". It does enable Files::ANALYZER_EXTRACT to correctly extract the PDF file from the attached pcap. > Please let me know if we can provide anything else to help with this. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 16 07:01:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 16 Oct 2014 09:01:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1235) HTTP multipart POST request alters file contents In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1235: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > HTTP multipart POST request alters file contents > ------------------------------------------------ > > Key: BIT-1235 > URL: https://bro-tracker.atlassian.net/browse/BIT-1235 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: CentOS 6.5, file extract analyzer > Reporter: Brian O'Berry > Assignee: Robin Sommer > Fix For: 2.4 > > Attachments: bro-2.3-HTTP.patch, gdb.log, upload-api-http.pcap > > > HTTP POST multipart processing converts bare CR or LF chars to CRLF pairs, corrupting most files when extracted with Files::ANALYZER_EXTRACT. This is clear in the attached gdb.log, which has a backtrace that shows a buffer with the start of a PDF file entering MIME/HTTP entity processing at frame 25, and emerging with LF chars converted to CRLF at frame 6. > Also attached are the pcap file associated with the backtrace, and an initial patch that we've barely begun to test. A point of concern with the patch is that it changes a weird.log entry from "line_terminated_with_single_CR" to "http_no_crlf_in_header_list". It does enable Files::ANALYZER_EXTRACT to correctly extract the PDF file from the attached pcap. > Please let me know if we can provide anything else to help with this. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From anthony.kasza at gmail.com Thu Oct 16 08:40:44 2014 From: anthony.kasza at gmail.com (anthony kasza) Date: Thu, 16 Oct 2014 08:40:44 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: <647511B6-2CFB-4376-9D57-3B9A8FDE88F4@illinois.edu> References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> <20141016130223.GE48006@icir.org> <647511B6-2CFB-4376-9D57-3B9A8FDE88F4@illinois.edu> Message-ID: I think having some plugins distributed and built with Bro is reasonable, too. As long as functionality and interfaces don't change current users shouldn't notice a difference. How eventually moving the plugin to a package manager will affect users is something else to consider. -AK On Oct 16, 2014 6:21 AM, "Slagell, Adam J" wrote: > I think that is reasonable for some things. > > On Oct 16, 2014, at 8:02 AM, Robin Sommer robin at icir.org wrote: > > > (unless we decide to > > build (some) plugins by default, which currently isn't happening). > > ------ > > Adam J. Slagell > Chief Information Security Officer > Assistant Director, Cybersecurity Directorate > National Center for Supercomputing Applications > University of Illinois at Urbana-Champaign > www.slagell.info > > "Under the Illinois Freedom of Information Act (FOIA), any written > communication to or from University employees regarding University business > is a public record and may be subject to public disclosure." > > > > > > > > > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141016/8b308e6f/attachment.html From gc355804 at ohio.edu Thu Oct 16 09:21:24 2014 From: gc355804 at ohio.edu (Clark, Gilbert) Date: Thu, 16 Oct 2014 16:21:24 +0000 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> <20141016130223.GE48006@icir.org> <647511B6-2CFB-4376-9D57-3B9A8FDE88F4@illinois.edu>, Message-ID: <180c719f0e97494a80987fdbdb700ef5@CO1PR01MB238.prod.exchangelabs.com> +1 that core plugins be built / distributed with bro. Re: package / plugin management, coming from a Java EE background, one of the things I don't like about some of the OSGi software (read: highly modular, plugin-driven) I've worked with is how confusing it can get to track versions of each specific bundle / plugin to know how / when to update them. Binding a specific set of core plugin versions to application releases mitigates some of that complexity in my experience. In other words, I'd be a bit hesitant to agree that bro should individually package *core* plugins and offer them independently through a repo, just because it will probably require a great deal of care to update those (since there could be e.g. side effects / dependencies that aren't immediately obvious). An external repository / package manager seems like a good fit for non-core plugins, though. -Gilbert On Oct 16, 2014 11:41 AM, anthony kasza wrote: I think having some plugins distributed and built with Bro is reasonable, too. As long as functionality and interfaces don't change current users shouldn't notice a difference. How eventually moving the plugin to a package manager will affect users is something else to consider. -AK On Oct 16, 2014 6:21 AM, "Slagell, Adam J" > wrote: I think that is reasonable for some things. On Oct 16, 2014, at 8:02 AM, Robin Sommer robin at icir.org wrote: > (unless we decide to > build (some) plugins by default, which currently isn't happening). ------ Adam J. Slagell Chief Information Security Officer Assistant Director, Cybersecurity Directorate National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." _______________________________________________ bro-dev mailing list bro-dev at bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141016/c6a37f7f/attachment.html From anthony.kasza at gmail.com Thu Oct 16 11:54:05 2014 From: anthony.kasza at gmail.com (anthony kasza) Date: Thu, 16 Oct 2014 11:54:05 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: <180c719f0e97494a80987fdbdb700ef5@CO1PR01MB238.prod.exchangelabs.com> References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> <20141016130223.GE48006@icir.org> <647511B6-2CFB-4376-9D57-3B9A8FDE88F4@illinois.edu> <180c719f0e97494a80987fdbdb700ef5@CO1PR01MB238.prod.exchangelabs.com> Message-ID: That makes sense. The next logical question would be, what makes a plugin special enough to be included the core? I think GeoIP, being legacy like Robin mentioned, qualifies. -AK On Oct 16, 2014 9:21 AM, "Clark, Gilbert" wrote: > +1 that core plugins be built / distributed with bro. > > Re: package / plugin management, coming from a Java EE background, one of > the things I don't like about some of the OSGi software (read: highly > modular, plugin-driven) I've worked with is how confusing it can get to > track versions of each specific bundle / plugin to know how / when to > update them. Binding a specific set of core plugin versions to application > releases mitigates some of that complexity in my experience. In other > words, I'd be a bit hesitant to agree that bro should individually package > *core* plugins and offer them independently through a repo, just because it > will probably require a great deal of care to update those (since there > could be e.g. side effects / dependencies that aren't immediately obvious). > > An external repository / package manager seems like a good fit for > non-core plugins, though. > > -Gilbert > On Oct 16, 2014 11:41 AM, anthony kasza wrote: > > I think having some plugins distributed and built with Bro is reasonable, > too. As long as functionality and interfaces don't change current users > shouldn't notice a difference. > How eventually moving the plugin to a package manager will affect users is > something else to consider. > > -AK > On Oct 16, 2014 6:21 AM, "Slagell, Adam J" wrote: > >> I think that is reasonable for some things. >> >> On Oct 16, 2014, at 8:02 AM, Robin Sommer robin at icir.org wrote: >> >> > (unless we decide to >> > build (some) plugins by default, which currently isn't happening). >> >> ------ >> >> Adam J. Slagell >> Chief Information Security Officer >> Assistant Director, Cybersecurity Directorate >> National Center for Supercomputing Applications >> University of Illinois at Urbana-Champaign >> www.slagell.info >> >> "Under the Illinois Freedom of Information Act (FOIA), any written >> communication to or from University employees regarding University business >> is a public record and may be subject to public disclosure." >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> bro-dev mailing list >> bro-dev at bro.org >> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141016/39bf2ebe/attachment.html From seth at icir.org Thu Oct 16 13:29:17 2014 From: seth at icir.org (Seth Hall) Date: Thu, 16 Oct 2014 16:29:17 -0400 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <20141005151633.GF31437@icir.org> <20141006151004.GM31437@icir.org> <20141007152555.GC63065@icir.org> <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> <20141016130223.GE48006@icir.org> <647511B6-2CFB-4376-9D57-3B9A8FDE88F4@illinois.edu> <180c719f0e97494a80987fdbdb700ef5@CO1PR01MB238.prod.exchangelabs.com> Message-ID: <26C9A1EC-3678-453C-BBFB-3CA0AB1C3884@icir.org> On Oct 16, 2014, at 2:54 PM, anthony kasza wrote: > That makes sense. The next logical question would be, what makes a plugin special enough to be included the core? I think GeoIP, being legacy like Robin mentioned, qualifies. Yep. This is going to just remain as-always a careful balance between providing extra functionality and keeping the number of dependencies low for Bro. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From jira at bro-tracker.atlassian.net Thu Oct 16 14:18:07 2014 From: jira at bro-tracker.atlassian.net (jdonnelly (JIRA)) Date: Thu, 16 Oct 2014 16:18:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1276) DEB package(s) won't install on Ubuntu 14.04 - LTS In-Reply-To: References: Message-ID: jdonnelly created BIT-1276: ------------------------------ Summary: DEB package(s) won't install on Ubuntu 14.04 - LTS Key: BIT-1276 URL: https://bro-tracker.atlassian.net/browse/BIT-1276 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.3 Reporter: jdonnelly Starting with Ubtuntu 14.01 x64 installed and using Bro-2.3.1-Linux-x86_64.deb dpkg --install Bro-2.3.1-Linux-x86_64.deb (Reading database ... 122099 files and directories currently installed.) Preparing to unpack Bro-2.3.1-Linux-x86_64.deb ... Unpacking bro (2.3.1) over (2.3.1) ... dpkg: dependency problems prevent configuration of bro: bro depends on libc6 (<< 2.12); however: Version of libc6:amd64 on system is 2.19-0ubuntu6.3. bro depends on libpython2.6 (>= 2.6); however: Package libpython2.6 is not installed. dpkg: error processing package bro (--install): dependency problems - leaving unconfigured Errors were encountered while processing Method 2: root at dyn-x64-01:~# gdebi Bro-*.deb Reading package lists... Done Building dependency tree Reading state information... Done Building data structures... Done Building data structures... Done This package is uninstallable Dependency is not satisfiable: libc6 (< 2.12) root at dyn-x64-01:~# Is "git clone" from source the only reasonable way to install Bro ? Thank you for Bro. Jd -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From robin at icir.org Fri Oct 17 06:16:23 2014 From: robin at icir.org (Robin Sommer) Date: Fri, 17 Oct 2014 06:16:23 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> <20141016130223.GE48006@icir.org> <647511B6-2CFB-4376-9D57-3B9A8FDE88F4@illinois.edu> <180c719f0e97494a80987fdbdb700ef5@CO1PR01MB238.prod.exchangelabs.com> Message-ID: <20141017131623.GF62872@icir.org> On Thu, Oct 16, 2014 at 11:54 -0700, you wrote: > That makes sense. The next logical question would be, what makes a plugin > special enough to be included the core? I think GeoIP, being legacy like > Robin mentioned, qualifies. Yeah, as Seth writes, it'll be a fine line. The other thing is infrastructure: right now we have one bro-plugins repository, which is pulled into aux/ but nothing in there is built automatically. I see three options: (1) Leave as it is; i.e., not build any plugins by default. (2) Select some plugins in bro-plugins that build automatically with Bro. (3) Split bro-plugins into two repositories. One set would build with Bro, the other not. We'd need to decide if both would be pulled into the Bro source tree, or not. Sounds we don't want (1), but I'm undecided between (2) and (3). Opinions? For both (2) and (3) we'll need to integrate the selected plugins into Bro's build process, including passing through configure options. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Fri Oct 17 06:33:09 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 17 Oct 2014 08:33:09 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18408#comment-18408 ] Robin Sommer commented on BIT-1270: ----------------------------------- Yes, passing by value is the main part. Though still, having the separate ValWrapper seems just a tad too complex to me. Though actually a std::pair is probably better than a a tuple, as it allows access via "first" and "second", and using std::pair is a pretty common pattern for flagging success/failure in addition to passing an X back. I'm not super-strong on this, but I think I'd still find that more intuitive than a separate wrapper class. > topic/gilbert/plugin-api-tweak > ------------------------------ > > Key: BIT-1270 > URL: https://bro-tracker.atlassian.net/browse/BIT-1270 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: gclark > Assignee: gclark > > This branch makes a few changes to the API: > * Wraps values in a simple class (ValWrapper) that include an explicit processed / not processed flag (to avoid confusion with delayed / opaque invocations). > * Adds a Frame argument to HookCallFunction > * Adds support for Frame argument types to HookArgument > * Adds support for ValWrapper argument types to HookArgument > * Tweaks the plugin.hooks tests a bit to include new output (from additional argument) > * Tweaks the plugin.api-version-mismatch to remove explicit home directory path via simple regex -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 17 07:31:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 17 Oct 2014 09:31:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1276) DEB package(s) won't install on Ubuntu 14.04 - LTS In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18409#comment-18409 ] Jon Siwek commented on BIT-1276: -------------------------------- Currently, the DEB package from bro.org is just targeted for Debian 6 (and RPM for EL6), so yes, installing from from source (either via `git clone` for latest development version or a source-release tarball from bro.org) would be the way to go right now. > DEB package(s) won't install on Ubuntu 14.04 - LTS > --------------------------------------------------- > > Key: BIT-1276 > URL: https://bro-tracker.atlassian.net/browse/BIT-1276 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: jdonnelly > > Starting with Ubtuntu 14.01 x64 installed > and using Bro-2.3.1-Linux-x86_64.deb > dpkg --install Bro-2.3.1-Linux-x86_64.deb > (Reading database ... 122099 files and directories currently installed.) > Preparing to unpack Bro-2.3.1-Linux-x86_64.deb ... > Unpacking bro (2.3.1) over (2.3.1) ... > dpkg: dependency problems prevent configuration of bro: > bro depends on libc6 (<< 2.12); however: > Version of libc6:amd64 on system is 2.19-0ubuntu6.3. > bro depends on libpython2.6 (>= 2.6); however: > Package libpython2.6 is not installed. > dpkg: error processing package bro (--install): > dependency problems - leaving unconfigured > Errors were encountered while processing > Method 2: > root at dyn-x64-01:~# gdebi Bro-*.deb > Reading package lists... Done > Building dependency tree > Reading state information... Done > Building data structures... Done > Building data structures... Done > This package is uninstallable > Dependency is not satisfiable: libc6 (< 2.12) > root at dyn-x64-01:~# > Is "git clone" from source the only reasonable way to install Bro ? > Thank you for Bro. > Jd -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jsiwek at illinois.edu Fri Oct 17 08:04:41 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Fri, 17 Oct 2014 15:04:41 +0000 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: <20141017131623.GF62872@icir.org> References: <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> <20141016130223.GE48006@icir.org> <647511B6-2CFB-4376-9D57-3B9A8FDE88F4@illinois.edu> <180c719f0e97494a80987fdbdb700ef5@CO1PR01MB238.prod.exchangelabs.com> <20141017131623.GF62872@icir.org> Message-ID: <87C43A31-B7E1-463C-94F6-FA8C6224D823@illinois.edu> On Oct 17, 2014, at 8:16 AM, Robin Sommer wrote: > (2) Select some plugins in bro-plugins that build automatically > with Bro. > > (3) Split bro-plugins into two repositories. One set would build > with Bro, the other not. We'd need to decide if both would be > pulled into the Bro source tree, or not. > > Sounds we don't want (1), but I'm undecided between (2) and (3). > Opinions? Maybe a minor downside to (3) would be it?s less smooth of a transition if you ever wanted to promote/demote a plugin ? involves moving code across repo as well as changing configure/CMake glue whereas (2) I?m guessing just involves the later with plugin code staying unchanged/unmoved. Maybe try approach (2) first as it seems simpler, then find out along the way what improvements are needed. - Jon From robin at icir.org Fri Oct 17 08:14:15 2014 From: robin at icir.org (Robin Sommer) Date: Fri, 17 Oct 2014 08:14:15 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: <87C43A31-B7E1-463C-94F6-FA8C6224D823@illinois.edu> References: <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> <20141016130223.GE48006@icir.org> <647511B6-2CFB-4376-9D57-3B9A8FDE88F4@illinois.edu> <180c719f0e97494a80987fdbdb700ef5@CO1PR01MB238.prod.exchangelabs.com> <20141017131623.GF62872@icir.org> <87C43A31-B7E1-463C-94F6-FA8C6224D823@illinois.edu> Message-ID: <20141017151415.GR62872@icir.org> On Fri, Oct 17, 2014 at 15:04 +0000, you wrote: > Maybe try approach (2) first as it seems simpler, then find out along the way what improvements are needed. Yeah, I can see that. I wondering if we could find a mechanism to automatically hook selected plugins into the Bro build without changing anythign there. Maybe they'd ship with a configure hook that the main configure searches all plugins for; or something to that effect. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Fri Oct 17 08:53:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 17 Oct 2014 10:53:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18410#comment-18410 ] Jon Siwek commented on BIT-1270: -------------------------------- Whether to pass by value and whether to use ValWrapper vs. std::pair seem like separate issues/suggestions? I'd also make the suggestion to pass by value. Whether that value is ValWrapper or std::pair I don't have a good feel for without reviewing the code more. Maybe std::pair if it's used infrequently else ValWrapper just to be able to chose the names of the fields (reading/writing code that uses "x.first" and "x.second" can get confusing for me personally). (And std::tuple is C++11, right? Did we decide that's generally ok to use in Bro now?) > topic/gilbert/plugin-api-tweak > ------------------------------ > > Key: BIT-1270 > URL: https://bro-tracker.atlassian.net/browse/BIT-1270 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: gclark > Assignee: gclark > > This branch makes a few changes to the API: > * Wraps values in a simple class (ValWrapper) that include an explicit processed / not processed flag (to avoid confusion with delayed / opaque invocations). > * Adds a Frame argument to HookCallFunction > * Adds support for Frame argument types to HookArgument > * Adds support for ValWrapper argument types to HookArgument > * Tweaks the plugin.hooks tests a bit to include new output (from additional argument) > * Tweaks the plugin.api-version-mismatch to remove explicit home directory path via simple regex -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From seth at icir.org Fri Oct 17 08:59:06 2014 From: seth at icir.org (Seth Hall) Date: Fri, 17 Oct 2014 11:59:06 -0400 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: <20141017151415.GR62872@icir.org> References: <5C71E139-0781-4D86-8A61-74BE992DB3E8@icir.org> <20141016130223.GE48006@icir.org> <647511B6-2CFB-4376-9D57-3B9A8FDE88F4@illinois.edu> <180c719f0e97494a80987fdbdb700ef5@CO1PR01MB238.prod.exchangelabs.com> <20141017131623.GF62872@icir.org> <87C43A31-B7E1-463C-94F6-FA8C6224D823@illinois.edu> <20141017151415.GR62872@icir.org> Message-ID: On Oct 17, 2014, at 11:14 AM, Robin Sommer wrote: > Maybe they'd ship with a configure hook that > the main configure searches all plugins for; or something to that > effect. What I don't like about this is that eventually these should all be pushed into external plugins so I feel like we should just keep the amount of work minimal for now. We're just fighting against the non-existence of a package manager right now. :) I like plan #2 the best for now too. Although I guess without doing the configure option thing, it would turn things like geoip into a required dependency. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From gc355804 at ohio.edu Fri Oct 17 11:34:35 2014 From: gc355804 at ohio.edu (Gilbert Clark) Date: Fri, 17 Oct 2014 14:34:35 -0400 Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: References: Message-ID: <5441613B.4000208@ohio.edu> Yeah, I think we're in agreement that pass-by-value would be a better method here. To be honest, I'm not sure if expecting a user to return a std::pair from a plugin function hook is more or less obvious than an explicit wrapper. One thing that makes this a little less obvious to me is that the usage of this construct won't be limited to bro folks reviewing this code: plugin authors will need to craft one of these (either ValWrapper or std::pair) and return them from the hook. std::pair might actually be a little easier, since that might make it more obvious what's happening there (since std::pair is vanilla C++) ... but it also might not. How about I give std::pair try and we can move back to an explicit wrapper if it makes the code too difficult to work with? It should be a pretty small change either way. Also, Jon: primary use of this construct (that I can think of) will be in HandlePluginResult and BroFunc::Call in Func.cc (since those deal with the HookFunctionCall interface) if you'd like to take a look. There's also https://github.com/cubic1271/bro-plugin-instrumentation to offer an example of a plugin that uses the updated interface - once I make this change in core, I'll update that plugin to reflect the new usage. On a related note, C++11 would be cool for a lot of reasons (e.g. std::atomic), but that might be another ticket / a longer discussion :) -Gilbert On 10/17/2014 11:53 AM, Jon Siwek (JIRA) wrote: > [ https://bro-tracker.atlassian.net/browse/BIT-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18410#comment-18410 ] > > Jon Siwek commented on BIT-1270: > -------------------------------- > > Whether to pass by value and whether to use ValWrapper vs. std::pair seem like separate issues/suggestions? > > I'd also make the suggestion to pass by value. Whether that value is ValWrapper or std::pair I don't have a good feel for without reviewing the code more. Maybe std::pair if it's used infrequently else ValWrapper just to be able to chose the names of the fields (reading/writing code that uses "x.first" and "x.second" can get confusing for me personally). > > (And std::tuple is C++11, right? Did we decide that's generally ok to use in Bro now?) > >> topic/gilbert/plugin-api-tweak >> ------------------------------ >> >> Key: BIT-1270 >> URL: https://bro-tracker.atlassian.net/browse/BIT-1270 >> Project: Bro Issue Tracker >> Issue Type: Improvement >> Components: Bro >> Reporter: gclark >> Assignee: gclark >> >> This branch makes a few changes to the API: >> * Wraps values in a simple class (ValWrapper) that include an explicit processed / not processed flag (to avoid confusion with delayed / opaque invocations). >> * Adds a Frame argument to HookCallFunction >> * Adds support for Frame argument types to HookArgument >> * Adds support for ValWrapper argument types to HookArgument >> * Tweaks the plugin.hooks tests a bit to include new output (from additional argument) >> * Tweaks the plugin.api-version-mismatch to remove explicit home directory path via simple regex > > > -- > This message was sent by Atlassian JIRA > (v6.4-OD-07-004#64005) > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From jsiwek at illinois.edu Fri Oct 17 12:31:48 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Fri, 17 Oct 2014 19:31:48 +0000 Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: <5441613B.4000208@ohio.edu> References: <5441613B.4000208@ohio.edu> Message-ID: On Oct 17, 2014, at 1:34 PM, Gilbert Clark wrote: > To be honest, I'm not sure if expecting a user to return a > std::pair from a plugin function hook is more or less > obvious than an explicit wrapper. One thing that makes this a little > less obvious to me is that the usage of this construct won't be limited > to bro folks reviewing this code: plugin authors will need to craft one > of these (either ValWrapper or std::pair) and return them from the > hook. std::pair might actually be a little easier, since that might > make it more obvious what's happening there (since std::pair is vanilla > C++) ... but it also might not. I think unless the API is designed to be generic, one reason std::pair may be less obvious is that if it shows up in multiple places, one doesn?t really know at a glance that they express the same meaning/intention. - Jon From jira at bro-tracker.atlassian.net Fri Oct 17 19:09:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Fri, 17 Oct 2014 21:09:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1277) activeHTTP's input read crashes when the returning content-length is 0 In-Reply-To: References: Message-ID: Christian Struck created BIT-1277: ------------------------------------- Summary: activeHTTP's input read crashes when the returning content-length is 0 Key: BIT-1277 URL: https://bro-tracker.atlassian.net/browse/BIT-1277 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Environment: Debian 7 with bro/master and curl 7.26.0 Reporter: Christian Struck curl does not write an bodyfile if the returned content length is 0. therefore the fileread of the bodyfile will fail. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Mon Oct 20 07:13:07 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Mon, 20 Oct 2014 09:13:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1278) topic/seth/dnp3-wrong-sizeof-argument In-Reply-To: References: Message-ID: Seth Hall created BIT-1278: ------------------------------ Summary: topic/seth/dnp3-wrong-sizeof-argument Key: BIT-1278 URL: https://bro-tracker.atlassian.net/browse/BIT-1278 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.4 Reporter: Seth Hall This branch fixes some issues reported by coverity about the bytestring_to_time function not being implemented correctly. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Mon Oct 20 07:13:08 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Mon, 20 Oct 2014 09:13:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1278) topic/seth/dnp3-wrong-sizeof-argument In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1278: --------------------------- Status: Merge Request (was: Open) > topic/seth/dnp3-wrong-sizeof-argument > ------------------------------------- > > Key: BIT-1278 > URL: https://bro-tracker.atlassian.net/browse/BIT-1278 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > > This branch fixes some issues reported by coverity about the bytestring_to_time function not being implemented correctly. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From robin at icir.org Mon Oct 20 08:05:24 2014 From: robin at icir.org (Robin Sommer) Date: Mon, 20 Oct 2014 08:05:24 -0700 Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: References: <5441613B.4000208@ohio.edu> Message-ID: <20141020150524.GI11800@icir.org> So one thing to keep in mind is that not many people will ever write code using this; it's just for function hooks, and I imagine their use will be limited. I'm personaly still leaning towards the pair, but it's not a big deal, the main point was the pass-by-value, and I'm fine either way. If we stick with the wrapper, we should however at least rename to something more semantically meaninful than ValWrapper, like ValOrNull (better ideas appreciated :). Gilbert, please reset to merge request once you're done updating the branch. Robin On Fri, Oct 17, 2014 at 19:31 +0000, you wrote: > > On Oct 17, 2014, at 1:34 PM, Gilbert Clark wrote: > > > To be honest, I'm not sure if expecting a user to return a > > std::pair from a plugin function hook is more or less > > obvious than an explicit wrapper. One thing that makes this a little > > less obvious to me is that the usage of this construct won't be limited > > to bro folks reviewing this code: plugin authors will need to craft one > > of these (either ValWrapper or std::pair) and return them from the > > hook. std::pair might actually be a little easier, since that might > > make it more obvious what's happening there (since std::pair is vanilla > > C++) ... but it also might not. > > I think unless the API is designed to be generic, one reason std::pair may be less obvious is that if it shows up in multiple places, one doesn?t really know at a glance that they express the same meaning/intention. > > - Jon > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From robin at icir.org Mon Oct 20 08:28:41 2014 From: robin at icir.org (Robin Sommer) Date: Mon, 20 Oct 2014 08:28:41 -0700 Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: <20141020150524.GI11800@icir.org> References: <5441613B.4000208@ohio.edu> <20141020150524.GI11800@icir.org> Message-ID: <20141020152841.GA40969@icir.org> On Mon, Oct 20, 2014 at 08:05 -0700, I wrote: > like ValOrNull (better ideas appreciated :). Maybe OptionalVal. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jsiwek at illinois.edu Mon Oct 20 12:53:34 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Mon, 20 Oct 2014 19:53:34 +0000 Subject: [Bro-Dev] [JIRA] (BIT-1270) topic/gilbert/plugin-api-tweak In-Reply-To: <20141020150524.GI11800@icir.org> References: <5441613B.4000208@ohio.edu> <20141020150524.GI11800@icir.org> Message-ID: <97591659-067F-4BEC-915A-5C3AA67E68A8@illinois.edu> > On Oct 20, 2014, at 10:05 AM, Robin Sommer wrote: > > So one thing to keep in mind is that not many people will ever write > code using this; it's just for function hooks, and I imagine their use > will be limited. I'm personaly still leaning towards the pair, but > it's not a big deal, the main point was the pass-by-value, and I'm > fine either way. Me too (I was just in a ?let me help brainstorm pros/cons? mood that day). > If we stick with the wrapper, we should however at > least rename to something more semantically meaninful than ValWrapper, > like ValOrNull (better ideas appreciated :). Not having to choose a name I guess is a ?pro? for std::pair :) - Jon From jira at bro-tracker.atlassian.net Mon Oct 20 16:46:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Mon, 20 Oct 2014 18:46:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1277) activeHTTP's input read crashes when the returning content-length is 0 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Struck updated BIT-1277: ---------------------------------- Status: In Progress (was: Open) > activeHTTP's input read crashes when the returning content-length is 0 > ---------------------------------------------------------------------- > > Key: BIT-1277 > URL: https://bro-tracker.atlassian.net/browse/BIT-1277 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 with bro/master and curl 7.26.0 > Reporter: Christian Struck > Labels: language > > curl does not write an bodyfile if the returned content length is 0. therefore the fileread of the bodyfile will fail. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Mon Oct 20 16:47:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Mon, 20 Oct 2014 18:47:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1277) activeHTTP's input read crashes when the returning content-length is 0 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Struck updated BIT-1277: ---------------------------------- Status: Merge Request (was: In Progress) > activeHTTP's input read crashes when the returning content-length is 0 > ---------------------------------------------------------------------- > > Key: BIT-1277 > URL: https://bro-tracker.atlassian.net/browse/BIT-1277 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 with bro/master and curl 7.26.0 > Reporter: Christian Struck > Labels: language > > curl does not write an bodyfile if the returned content length is 0. therefore the fileread of the bodyfile will fail. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Mon Oct 20 17:19:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Mon, 20 Oct 2014 19:19:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1277) activeHTTP's input read crashes when the returning content-length is 0 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18500#comment-18500 ] Christian Struck commented on BIT-1277: --------------------------------------- fixed in branch topic/struck/BIT-1277 > activeHTTP's input read crashes when the returning content-length is 0 > ---------------------------------------------------------------------- > > Key: BIT-1277 > URL: https://bro-tracker.atlassian.net/browse/BIT-1277 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 with bro/master and curl 7.26.0 > Reporter: Christian Struck > Labels: language > > curl does not write an bodyfile if the returned content length is 0. therefore the fileread of the bodyfile will fail. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From noreply at bro.org Tue Oct 21 00:00:20 2014 From: noreply at bro.org (Merge Tracker) Date: Tue, 21 Oct 2014 00:00:20 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410210700.s9L70K6b032566@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------------- ---------- ---------- ------------- ---------- ---------------------------------------------------------------------- BIT-1278 [1] Bro Seth Hall - 2014-10-20 - Normal topic/seth/dnp3-wrong-sizeof-argument [2] BIT-1277 [3] Bro Christian Struck - 2014-10-20 - Normal activeHTTP's input read crashes when the returning content-length is 0 [1] BIT-1278 https://bro-tracker.atlassian.net/browse/BIT-1278 [2] dnp3-wrong-sizeof-argument https://github.com/bro/bro/tree/topic/seth/dnp3-wrong-sizeof-argument [3] BIT-1277 https://bro-tracker.atlassian.net/browse/BIT-1277 From jira at bro-tracker.atlassian.net Tue Oct 21 08:00:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 21 Oct 2014 10:00:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1263) Implementing three event handlers for supported data structure in Modbus Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1263: --------------------------------- Assignee: Robin Sommer > Implementing three event handlers for supported data structure in Modbus Analyzer > --------------------------------------------------------------------------------- > > Key: BIT-1263 > URL: https://bro-tracker.atlassian.net/browse/BIT-1263 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: hui > Assignee: Robin Sommer > Priority: Low > Labels: analyzer, modbus > > Three support data structures are defined in Modbus analyzer: > FileRecordRequest, > FileRecordResponse, > ReferenceWithData > Three event handlers are declared for them. > The changes are already made and pushed into the branch: > topic/hui/modbus-events2 -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 08:00:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 21 Oct 2014 10:00:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1263) Implementing three event handlers for supported data structure in Modbus Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1263: ------------------------------ Status: Merge Request (was: Open) > Implementing three event handlers for supported data structure in Modbus Analyzer > --------------------------------------------------------------------------------- > > Key: BIT-1263 > URL: https://bro-tracker.atlassian.net/browse/BIT-1263 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: hui > Priority: Low > Labels: analyzer, modbus > > Three support data structures are defined in Modbus analyzer: > FileRecordRequest, > FileRecordResponse, > ReferenceWithData > Three event handlers are declared for them. > The changes are already made and pushed into the branch: > topic/hui/modbus-events2 -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 08:14:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Tue, 21 Oct 2014 10:14:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1277) activeHTTP's input read crashes when the returning content-length is 0 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18501#comment-18501 ] Johanna Amann commented on BIT-1277: ------------------------------------ Could you perhaps add a test-case for that? Especially for the empty-file-part of the exec framework. There already are tests for the exec and the active-http framework - either adding a small new one or extending one of them to cover this case too should do the job :). > activeHTTP's input read crashes when the returning content-length is 0 > ---------------------------------------------------------------------- > > Key: BIT-1277 > URL: https://bro-tracker.atlassian.net/browse/BIT-1277 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 with bro/master and curl 7.26.0 > Reporter: Christian Struck > Labels: language > > curl does not write an bodyfile if the returned content length is 0. therefore the fileread of the bodyfile will fail. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 08:35:08 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Tue, 21 Oct 2014 10:35:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1275) README (or install) need better build instructions In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18502#comment-18502 ] Johanna Amann commented on BIT-1275: ------------------------------------ The readme points to the install file, which in turn is just a pointer to doc/install/install.rst which contains all installation instructions, including the required dependencies. I think that is ok and I like the short readme we have at the moment. We could perhaps think about making INSTALL a symlink to doc/install/install.rst instead of the current file. > README (or install) need better build instructions > --------------------------------------------------- > > Key: BIT-1275 > URL: https://bro-tracker.atlassian.net/browse/BIT-1275 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Ubuntu 14.04 LTS > Reporter: jdonnelly > > The build instructions (specifically the required packages) from > https://www.bro.org/sphinx-git/install/install.html > should be added to the README file so the product can be built from source without have to consult (or search for on the web) how to cure the > ./configure errors. > Thank you, -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 08:37:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Tue, 21 Oct 2014 10:37:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1253) Bro 2.3 - 2.3.1 manager dieing on Bivio hardware In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann reassigned BIT-1253: ---------------------------------- Assignee: Johanna Amann > Bro 2.3 - 2.3.1 manager dieing on Bivio hardware > ------------------------------------------------ > > Key: BIT-1253 > URL: https://bro-tracker.atlassian.net/browse/BIT-1253 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Bro 2.3 and Bro 2.3.1 > bivio hardwareLinux CPU.2.6.31-45 has curl 7.36 gperftools 2.2 flex 2.5.39 bison 3.0.2 libpcap 1.1 swig 2.0.8 > Reporter: Larry Leviton > Assignee: Johanna Amann > Fix For: 2.4 > > > After starting bro up, the bro manager crashes in less than 60 seconds. > Thanks for any help you can give. > Sent stack trace to vendor (at bottom), and here was their response: > Comment(s): Hello Larry, > We have duplicated a crash in our lab setup that seems to be identical to that experienced by you. The code has changed quite a bit from 2.1 to 2.3.1, and we suspect a bug was introduced. > What is going on, seems to be that a writer thread is being terminated, and the destructor for the Ascii writer is called eventually. However, the destructor code does some checks and finds out that proper cleanup has not been done, so it aborts. This does not seem to be due to any library incompatibility, and looks more like maybe a race condition was introduced. > Since you knows the Bro developers, can you please ask them to take a look this and get back to us? We think it requires their expertise at this point. > Thank You, > Hassan. > > Bivio Case Information: > Bivio Case #: 4566243 > Date Created: 9/02/2014 08:02 AM PDT > Stack trace below: > GNU gdb (GDB) Fedora (6.8.50.20090302-40.fc11) Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "ppc-redhat-linux-gnu". > For bug reporting instructions, please see: > ... > backtrace > [New Thread 25501] > [New Thread 25328] > [New Thread 25378] > [New Thread 25379] > [New Thread 25380] > [New Thread 25381] > [New Thread 25382] > [New Thread 25383] > [New Thread 25384] > [New Thread 25385] > [New Thread 25386] > [New Thread 25389] > [New Thread 25442] > warning: Can't read pathname for load map: Input/output error. > Missing separate debuginfo for /usr/local/lib/libz.so.1 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/a2/0a0d1fc0d48c2a303af1417ccc03308b9de04a > Missing separate debuginfo for /usr/local/lib/libtcmalloc.so.4 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/27/eaf56bc64810920d55b9530156c1e8ffbfd43e > Missing separate debuginfo for /usr/local/lib/libcurl.so.4 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/a7/9a2cebb4abc156495ec0806b1c18015c8eba01 > Reading symbols from /usr/lib/libpcap.so.1...done. > Loaded symbols for /usr/lib/libpcap.so.1 Reading symbols from /usr/lib/libssl.so.10...done. > Loaded symbols for /usr/lib/libssl.so.10 Reading symbols from /usr/lib/libcrypto.so.10...done. > Loaded symbols for /usr/lib/libcrypto.so.10 Reading symbols from /usr/lib/libbind.so.4...done. > Loaded symbols for /usr/lib/libbind.so.4 Reading symbols from /usr/local/lib/libz.so.1...done. > Loaded symbols for /usr/local/lib/libz.so.1 Reading symbols from /usr/local/lib/libtcmalloc.so.4...done. > Loaded symbols for /usr/local/lib/libtcmalloc.so.4 Reading symbols from /usr/local/lib/libcurl.so.4...done. > Loaded symbols for /usr/local/lib/libcurl.so.4 Reading symbols from /lib/libpthread.so.0...done. > Loaded symbols for /lib/libpthread.so.0 > Reading symbols from /lib/libdl.so.2...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /usr/lib/libstdc++.so.6...done. > Loaded symbols for /usr/lib/libstdc++.so.6 Reading symbols from /lib/libm.so.6...done. > Loaded symbols for /lib/libm.so.6 > Reading symbols from /lib/libgcc_s.so.1...done. > Loaded symbols for /lib/libgcc_s.so.1 > Reading symbols from /lib/libc.so.6...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libzcp.so...done. > Loaded symbols for /usr/lib/libzcp.so > Reading symbols from /lib/libgssapi_krb5.so.2...done. > Loaded symbols for /lib/libgssapi_krb5.so.2 Reading symbols from /lib/libkrb5.so.3...done. > Loaded symbols for /lib/libkrb5.so.3 > Reading symbols from /lib/libcom_err.so.2...done. > Loaded symbols for /lib/libcom_err.so.2 > Reading symbols from /lib/libk5crypto.so.3...done. > Loaded symbols for /lib/libk5crypto.so.3 Reading symbols from /lib/libresolv.so.2...done. > Loaded symbols for /lib/libresolv.so.2 > Reading symbols from /lib/librt.so.1...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/ld.so.1...done. > Loaded symbols for /lib/ld.so.1 > Reading symbols from /lib/libbvsp.so...done. > Loaded symbols for /lib/libbvsp.so > Reading symbols from /lib/libbcon.so...done. > Loaded symbols for /lib/libbcon.so > Reading symbols from /lib/libkrb5support.so.0...done. > Loaded symbols for /lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. > Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /usr/lib/libxml2.so.2...done. > Loaded symbols for /usr/lib/libxml2.so.2 Reading symbols from /lib/libhmlibs.so...done. > Loaded symbols for /lib/libhmlibs.so > Reading symbols from /lib/libhmolddb.so...done. > Loaded symbols for /lib/libhmolddb.so > Reading symbols from /lib/libcf.so...done. > Loaded symbols for /lib/libcf.so > Reading symbols from /lib/libbvsep.so...done. > Loaded symbols for /lib/libbvsep.so > Reading symbols from /usr/lib/libnrddi.so...done. > Loaded symbols for /usr/lib/libnrddi.so > Reading symbols from /lib/libselinux.so.1...done. > Loaded symbols for /lib/libselinux.so.1 > Core was generated by `/var/tmp/bro/spool/tmp/bro -U .status -p broctl -p broctl-live -p local -p mana'. > Program terminated with signal 6, Aborted. > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. > in ../nptl/sysdeps/unix/sysv/linux/raise.c > Missing separate debuginfos, use: debuginfo-install e2fsprogs-libs-1.41.9-2.fc11.ppc glibc-2.17-4.fc11.ppc keyutils-libs-1.2-5.fc11.ppc krb5-libs-1.9.3-1.fc11.ppc libbind-6.0-1.fc11.ppc libgcc-4.4.1-2.fc11.ppc libselinux-2.0.80-1.fc11.ppc libstdc++-4.4.1-2.fc11.ppc libxml2-2.7.6-1.fc11.ppc openssl-libs-1.0.1e-37.fc11.1.ppc > (gdb) backtrace > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > #1 0x0f6d0de0 in *__GI_abort () at abort.c:90 > #2 0x1024be70 in logging::writer::Ascii::~Ascii (this=0x11a87200, __in_chrg=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/logging/writers/Ascii.cc:186 > #3 0x10236b70 in threading::Manager::Process (this=0x10dae180) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/threading/Manager.cc:171 > #4 0x101a5400 in net_run () at /bivio/scsi/b/levitonl/bro-2.3.1/src/Net.cc:389 > #5 0x100f7554 in main (argc=, argv=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/main.cc:1165 > Current language: auto; currently minimal > (gdb) > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > #1 0x0f6d0de0 in *__GI_abort () at abort.c:90 > #2 0x1024be70 in logging::writer::Ascii::~Ascii (this=0x11a87200, __in_chrg=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/logging/writers/Ascii.cc:186 > #3 0x10236b70 in threading::Manager::Process (this=0x10dae180) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/threading/Manager.cc:171 > #4 0x101a5400 in net_run () at /bivio/scsi/b/levitonl/bro-2.3.1/src/Net.cc:389 > #5 0x100f7554 in main (argc=, argv=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/main.cc:1165 > (gdb) quit -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 08:37:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Tue, 21 Oct 2014 10:37:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1253) Bro 2.3 - 2.3.1 manager dieing on Bivio hardware In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18503#comment-18503 ] Johanna Amann commented on BIT-1253: ------------------------------------ I just wanted to check if you managed to get more information on this? > Bro 2.3 - 2.3.1 manager dieing on Bivio hardware > ------------------------------------------------ > > Key: BIT-1253 > URL: https://bro-tracker.atlassian.net/browse/BIT-1253 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Bro 2.3 and Bro 2.3.1 > bivio hardwareLinux CPU.2.6.31-45 has curl 7.36 gperftools 2.2 flex 2.5.39 bison 3.0.2 libpcap 1.1 swig 2.0.8 > Reporter: Larry Leviton > Assignee: Johanna Amann > Fix For: 2.4 > > > After starting bro up, the bro manager crashes in less than 60 seconds. > Thanks for any help you can give. > Sent stack trace to vendor (at bottom), and here was their response: > Comment(s): Hello Larry, > We have duplicated a crash in our lab setup that seems to be identical to that experienced by you. The code has changed quite a bit from 2.1 to 2.3.1, and we suspect a bug was introduced. > What is going on, seems to be that a writer thread is being terminated, and the destructor for the Ascii writer is called eventually. However, the destructor code does some checks and finds out that proper cleanup has not been done, so it aborts. This does not seem to be due to any library incompatibility, and looks more like maybe a race condition was introduced. > Since you knows the Bro developers, can you please ask them to take a look this and get back to us? We think it requires their expertise at this point. > Thank You, > Hassan. > > Bivio Case Information: > Bivio Case #: 4566243 > Date Created: 9/02/2014 08:02 AM PDT > Stack trace below: > GNU gdb (GDB) Fedora (6.8.50.20090302-40.fc11) Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "ppc-redhat-linux-gnu". > For bug reporting instructions, please see: > ... > backtrace > [New Thread 25501] > [New Thread 25328] > [New Thread 25378] > [New Thread 25379] > [New Thread 25380] > [New Thread 25381] > [New Thread 25382] > [New Thread 25383] > [New Thread 25384] > [New Thread 25385] > [New Thread 25386] > [New Thread 25389] > [New Thread 25442] > warning: Can't read pathname for load map: Input/output error. > Missing separate debuginfo for /usr/local/lib/libz.so.1 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/a2/0a0d1fc0d48c2a303af1417ccc03308b9de04a > Missing separate debuginfo for /usr/local/lib/libtcmalloc.so.4 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/27/eaf56bc64810920d55b9530156c1e8ffbfd43e > Missing separate debuginfo for /usr/local/lib/libcurl.so.4 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/a7/9a2cebb4abc156495ec0806b1c18015c8eba01 > Reading symbols from /usr/lib/libpcap.so.1...done. > Loaded symbols for /usr/lib/libpcap.so.1 Reading symbols from /usr/lib/libssl.so.10...done. > Loaded symbols for /usr/lib/libssl.so.10 Reading symbols from /usr/lib/libcrypto.so.10...done. > Loaded symbols for /usr/lib/libcrypto.so.10 Reading symbols from /usr/lib/libbind.so.4...done. > Loaded symbols for /usr/lib/libbind.so.4 Reading symbols from /usr/local/lib/libz.so.1...done. > Loaded symbols for /usr/local/lib/libz.so.1 Reading symbols from /usr/local/lib/libtcmalloc.so.4...done. > Loaded symbols for /usr/local/lib/libtcmalloc.so.4 Reading symbols from /usr/local/lib/libcurl.so.4...done. > Loaded symbols for /usr/local/lib/libcurl.so.4 Reading symbols from /lib/libpthread.so.0...done. > Loaded symbols for /lib/libpthread.so.0 > Reading symbols from /lib/libdl.so.2...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /usr/lib/libstdc++.so.6...done. > Loaded symbols for /usr/lib/libstdc++.so.6 Reading symbols from /lib/libm.so.6...done. > Loaded symbols for /lib/libm.so.6 > Reading symbols from /lib/libgcc_s.so.1...done. > Loaded symbols for /lib/libgcc_s.so.1 > Reading symbols from /lib/libc.so.6...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libzcp.so...done. > Loaded symbols for /usr/lib/libzcp.so > Reading symbols from /lib/libgssapi_krb5.so.2...done. > Loaded symbols for /lib/libgssapi_krb5.so.2 Reading symbols from /lib/libkrb5.so.3...done. > Loaded symbols for /lib/libkrb5.so.3 > Reading symbols from /lib/libcom_err.so.2...done. > Loaded symbols for /lib/libcom_err.so.2 > Reading symbols from /lib/libk5crypto.so.3...done. > Loaded symbols for /lib/libk5crypto.so.3 Reading symbols from /lib/libresolv.so.2...done. > Loaded symbols for /lib/libresolv.so.2 > Reading symbols from /lib/librt.so.1...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/ld.so.1...done. > Loaded symbols for /lib/ld.so.1 > Reading symbols from /lib/libbvsp.so...done. > Loaded symbols for /lib/libbvsp.so > Reading symbols from /lib/libbcon.so...done. > Loaded symbols for /lib/libbcon.so > Reading symbols from /lib/libkrb5support.so.0...done. > Loaded symbols for /lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. > Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /usr/lib/libxml2.so.2...done. > Loaded symbols for /usr/lib/libxml2.so.2 Reading symbols from /lib/libhmlibs.so...done. > Loaded symbols for /lib/libhmlibs.so > Reading symbols from /lib/libhmolddb.so...done. > Loaded symbols for /lib/libhmolddb.so > Reading symbols from /lib/libcf.so...done. > Loaded symbols for /lib/libcf.so > Reading symbols from /lib/libbvsep.so...done. > Loaded symbols for /lib/libbvsep.so > Reading symbols from /usr/lib/libnrddi.so...done. > Loaded symbols for /usr/lib/libnrddi.so > Reading symbols from /lib/libselinux.so.1...done. > Loaded symbols for /lib/libselinux.so.1 > Core was generated by `/var/tmp/bro/spool/tmp/bro -U .status -p broctl -p broctl-live -p local -p mana'. > Program terminated with signal 6, Aborted. > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. > in ../nptl/sysdeps/unix/sysv/linux/raise.c > Missing separate debuginfos, use: debuginfo-install e2fsprogs-libs-1.41.9-2.fc11.ppc glibc-2.17-4.fc11.ppc keyutils-libs-1.2-5.fc11.ppc krb5-libs-1.9.3-1.fc11.ppc libbind-6.0-1.fc11.ppc libgcc-4.4.1-2.fc11.ppc libselinux-2.0.80-1.fc11.ppc libstdc++-4.4.1-2.fc11.ppc libxml2-2.7.6-1.fc11.ppc openssl-libs-1.0.1e-37.fc11.1.ppc > (gdb) backtrace > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > #1 0x0f6d0de0 in *__GI_abort () at abort.c:90 > #2 0x1024be70 in logging::writer::Ascii::~Ascii (this=0x11a87200, __in_chrg=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/logging/writers/Ascii.cc:186 > #3 0x10236b70 in threading::Manager::Process (this=0x10dae180) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/threading/Manager.cc:171 > #4 0x101a5400 in net_run () at /bivio/scsi/b/levitonl/bro-2.3.1/src/Net.cc:389 > #5 0x100f7554 in main (argc=, argv=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/main.cc:1165 > Current language: auto; currently minimal > (gdb) > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > #1 0x0f6d0de0 in *__GI_abort () at abort.c:90 > #2 0x1024be70 in logging::writer::Ascii::~Ascii (this=0x11a87200, __in_chrg=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/logging/writers/Ascii.cc:186 > #3 0x10236b70 in threading::Manager::Process (this=0x10dae180) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/threading/Manager.cc:171 > #4 0x101a5400 in net_run () at /bivio/scsi/b/levitonl/bro-2.3.1/src/Net.cc:389 > #5 0x100f7554 in main (argc=, argv=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/main.cc:1165 > (gdb) quit -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 11:57:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Tue, 21 Oct 2014 13:57:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1279) Merge topic/johanna/ssl-resumption In-Reply-To: References: Message-ID: Johanna Amann created BIT-1279: ---------------------------------- Summary: Merge topic/johanna/ssl-resumption Key: BIT-1279 URL: https://bro-tracker.atlassian.net/browse/BIT-1279 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: 2.4 Reporter: Johanna Amann Fix For: 2.4 Please merge topic/johanna/ssl-resumption in bro and testing. It changes the current ssl.log in two ways. It removes the session_id string, which is rather useless in its current presence and replaces it with a boolean signifying if a connection was or was not resumed. Connection resumption is determined no matter if stateful or stateless session resumption was used. Furthermore, it adds a field next_protocol which logs the protocol chosen using the application_layer_next_protocol extension (if present). This is e.g. used to negotiate SPDY or HTTP/2. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 11:57:08 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Tue, 21 Oct 2014 13:57:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1279) Merge topic/johanna/ssl-resumption In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1279: ------------------------------- Status: Merge Request (was: Open) > Merge topic/johanna/ssl-resumption > ---------------------------------- > > Key: BIT-1279 > URL: https://bro-tracker.atlassian.net/browse/BIT-1279 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.4 > Reporter: Johanna Amann > Labels: ssl > Fix For: 2.4 > > > Please merge topic/johanna/ssl-resumption in bro and testing. > It changes the current ssl.log in two ways. It removes the session_id string, which is rather useless in its current presence and replaces it with a boolean signifying if a connection was or was not resumed. Connection resumption is determined no matter if stateful or stateless session resumption was used. > Furthermore, it adds a field next_protocol which logs the protocol chosen using the application_layer_next_protocol extension (if present). This is e.g. used to negotiate SPDY or HTTP/2. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 13:21:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 21 Oct 2014 15:21:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1263) Implementing three event handlers for supported data structure in Modbus Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1263: ------------------------------ Status: Open (was: Merge Request) > Implementing three event handlers for supported data structure in Modbus Analyzer > --------------------------------------------------------------------------------- > > Key: BIT-1263 > URL: https://bro-tracker.atlassian.net/browse/BIT-1263 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: hui > Assignee: Robin Sommer > Priority: Low > Labels: analyzer, modbus > > Three support data structures are defined in Modbus analyzer: > FileRecordRequest, > FileRecordResponse, > ReferenceWithData > Three event handlers are declared for them. > The changes are already made and pushed into the branch: > topic/hui/modbus-events2 -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 13:21:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 21 Oct 2014 15:21:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1263) Implementing three event handlers for supported data structure in Modbus Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18504#comment-18504 ] Robin Sommer commented on BIT-1263: ----------------------------------- I've pushed some small tweaks to topic/robin/modbus-events2-merge. Please update the test scripts.base.protocols.modbus.events to log the new events. In case the existing traces don't trigger these events (which is my guess) please add one that does (seems you must have one, right?); and check that the output is correct. > Implementing three event handlers for supported data structure in Modbus Analyzer > --------------------------------------------------------------------------------- > > Key: BIT-1263 > URL: https://bro-tracker.atlassian.net/browse/BIT-1263 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: hui > Assignee: Robin Sommer > Priority: Low > Labels: analyzer, modbus > > Three support data structures are defined in Modbus analyzer: > FileRecordRequest, > FileRecordResponse, > ReferenceWithData > Three event handlers are declared for them. > The changes are already made and pushed into the branch: > topic/hui/modbus-events2 -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 13:22:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 21 Oct 2014 15:22:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1263) Implementing three event handlers for supported data structure in Modbus Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1263: --------------------------------- Assignee: hui (was: Robin Sommer) > Implementing three event handlers for supported data structure in Modbus Analyzer > --------------------------------------------------------------------------------- > > Key: BIT-1263 > URL: https://bro-tracker.atlassian.net/browse/BIT-1263 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: hui > Assignee: hui > Priority: Low > Labels: analyzer, modbus > > Three support data structures are defined in Modbus analyzer: > FileRecordRequest, > FileRecordResponse, > ReferenceWithData > Three event handlers are declared for them. > The changes are already made and pushed into the branch: > topic/hui/modbus-events2 -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 13:24:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 21 Oct 2014 15:24:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1278) topic/seth/dnp3-wrong-sizeof-argument In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1278: --------------------------------- Assignee: Robin Sommer > topic/seth/dnp3-wrong-sizeof-argument > ------------------------------------- > > Key: BIT-1278 > URL: https://bro-tracker.atlassian.net/browse/BIT-1278 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Assignee: Robin Sommer > > This branch fixes some issues reported by coverity about the bytestring_to_time function not being implemented correctly. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 13:51:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 21 Oct 2014 15:51:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1274) Moving GeoIP Code to Plugin In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18505#comment-18505 ] Robin Sommer commented on BIT-1274: ----------------------------------- I've lost track if there's still a particular problem you're having? > Moving GeoIP Code to Plugin > --------------------------- > > Key: BIT-1274 > URL: https://bro-tracker.atlassian.net/browse/BIT-1274 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: AK > > I've started moving the GeoIP code to a plugin. The branch of Bro I'm working from is here: https://github.com/anthonykasza/bro/tree/topic/akasza/geoplugin. > The source for the plugin is here: https://github.com/anthonykasza/Bro_GeoIP. > Any pointers would be appreciated. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From robin at icir.org Tue Oct 21 14:16:27 2014 From: robin at icir.org (Robin Sommer) Date: Tue, 21 Oct 2014 14:16:27 -0700 Subject: [Bro-Dev] Geo Location Plugin In-Reply-To: References: <20141016130223.GE48006@icir.org> <647511B6-2CFB-4376-9D57-3B9A8FDE88F4@illinois.edu> <180c719f0e97494a80987fdbdb700ef5@CO1PR01MB238.prod.exchangelabs.com> <20141017131623.GF62872@icir.org> <87C43A31-B7E1-463C-94F6-FA8C6224D823@illinois.edu> <20141017151415.GR62872@icir.org> Message-ID: <20141021211627.GA66809@icir.org> On Fri, Oct 17, 2014 at 11:59 -0400, you wrote: > > Maybe they'd ship with a configure hook that the main configure > > searches all plugins for; or something to that effect. > > We're just fighting against the non-existence of a package manager > right now. :) True, but I don't see us having one for the next release, and I'd rather have a solid mechanism in place that we remove later, than just some hack for which we don't know how long it will have to last. Seems the consensus is that we want some plugins compile along with Bro by default. I have a few smaller things on my list for the plugin stuff that I still want to do, I'll add this to it and mull over it a bit more then. (Btw, another piece here is getting plugins that compile with Bro to also install with the global "make install"). > I like plan #2 the best for now too. Although I guess without doing > the configure option thing, it would turn things like geoip into a > required dependency. It could still be an optional dependency; the plugin just wouldn't be compiled if it doesn't find geoip. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Tue Oct 21 14:18:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 21 Oct 2014 16:18:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1278) topic/seth/dnp3-wrong-sizeof-argument In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1278: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/seth/dnp3-wrong-sizeof-argument > ------------------------------------- > > Key: BIT-1278 > URL: https://bro-tracker.atlassian.net/browse/BIT-1278 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Assignee: Robin Sommer > > This branch fixes some issues reported by coverity about the bytestring_to_time function not being implemented correctly. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 14:18:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 21 Oct 2014 16:18:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1277) activeHTTP's input read crashes when the returning content-length is 0 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1277: ------------------------------ Status: Open (was: Merge Request) > activeHTTP's input read crashes when the returning content-length is 0 > ---------------------------------------------------------------------- > > Key: BIT-1277 > URL: https://bro-tracker.atlassian.net/browse/BIT-1277 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 with bro/master and curl 7.26.0 > Reporter: Christian Struck > Labels: language > > curl does not write an bodyfile if the returned content length is 0. therefore the fileread of the bodyfile will fail. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 14:18:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 21 Oct 2014 16:18:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1279) Merge topic/johanna/ssl-resumption In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1279: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Merge topic/johanna/ssl-resumption > ---------------------------------- > > Key: BIT-1279 > URL: https://bro-tracker.atlassian.net/browse/BIT-1279 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.4 > Reporter: Johanna Amann > Labels: ssl > Fix For: 2.4 > > > Please merge topic/johanna/ssl-resumption in bro and testing. > It changes the current ssl.log in two ways. It removes the session_id string, which is rather useless in its current presence and replaces it with a boolean signifying if a connection was or was not resumed. Connection resumption is determined no matter if stateful or stateless session resumption was used. > Furthermore, it adds a field next_protocol which logs the protocol chosen using the application_layer_next_protocol extension (if present). This is e.g. used to negotiate SPDY or HTTP/2. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 21 14:18:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 21 Oct 2014 16:18:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1277) activeHTTP's input read crashes when the returning content-length is 0 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1277: --------------------------------- Assignee: Christian Struck > activeHTTP's input read crashes when the returning content-length is 0 > ---------------------------------------------------------------------- > > Key: BIT-1277 > URL: https://bro-tracker.atlassian.net/browse/BIT-1277 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 with bro/master and curl 7.26.0 > Reporter: Christian Struck > Assignee: Christian Struck > Labels: language > > curl does not write an bodyfile if the returned content length is 0. therefore the fileread of the bodyfile will fail. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jsiwek at illinois.edu Wed Oct 22 12:39:19 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Wed, 22 Oct 2014 19:39:19 +0000 Subject: [Bro-Dev] libbroker status/plans Message-ID: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> If anyone has time/interest, I feel like the main components of Broker are established now and deserving feedback/critique. Rather than try to detail how things work here, it?s probably best for people to try figuring things out from the repo (e.g. source code comments and unit test examples) and ask questions about what's unclear. But it would be helpful to start a discussion on some of the planned features and open questions. I?ll try literally pasting my TODO list and hope it?s readable. The items are roughly ordered from most-certainty to least-certainty. Feedback welcome generally, but particularly where questions are posed. Broker TODO =========== - C API - Python bindings - Persistent storage backend - SSL/IPv6 (dependent on actor-framework support) - Need/want overload or flow-control mechanisms? E.g. a simple policy for handling overload is to let a user specify a threshold for how many items are allowed in a queue before new messages are dropped. - In-place data store value modifications Plan to support increment/decrement on integral values. Need any other operations? What to do when applying an operation to invalid data type? Plan to just send error message back to sender and leave further decisions up to them. - Data store support for optional expiry model What are the desired mechanims? Options: (1) Inserter may specify "expire this entry at time X" ? (2) Inserter may specify "expire this entry based on create/read/modification access time" ? Going this route, seems read access times would need to be shared across clones (contradicts goal of lightweight, local read-access)? (3) Other hooks to make expiry conditional? - Data typing model Currently data in Broker is similar to Bro's threading::Value in that full type info (from Bro's perspective) isn't available, just the raw storage required for the types. Broker currently differs in that it doesn't use any tag to distinguish between primitives that share the same storage (e.g. enum/string or double/interval). Interpretation of types is left entirely up to the receiver. Do we need more strict typing than this? Options: (1) Data holds additional type tag to suggest how to interpret (2) Fully implement separate Bro-types. Planning to try integrating w/ Bro as it is and see what specific problems arise. I think (1) may end up being helpful, but maybe not required and I'd like to avoid (2) if possible. - Bro integration Is Broker the default in Bro 2.4 ? That implies requiring C++11. Also I'm requiring CMake 2.8.12+ and may be hard to go below 2.8. Bro is still happy with 2.6.3. From dnthayer at illinois.edu Wed Oct 22 13:45:25 2014 From: dnthayer at illinois.edu (Daniel Thayer) Date: Wed, 22 Oct 2014 15:45:25 -0500 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> Message-ID: <54481765.7080401@illinois.edu> I tried building it on the newest version of debian, and got this error: CMake Error at CMakeLists.txt:2 (cmake_minimum_required): CMake 2.8.12 or higher is required. You are running version 2.8.9 Why does it need version 2.8.12 of cmake? On 10/22/2014 02:39 PM, Siwek, Jon wrote: > If anyone has time/interest, I feel like the main components of Broker are established now and deserving feedback/critique. Rather than try to detail how things work here, it?s probably best for people to try figuring things out from the repo (e.g. source code comments and unit test examples) and ask questions about what's unclear. > > But it would be helpful to start a discussion on some of the planned features and open questions. I?ll try literally pasting my TODO list and hope it?s readable. The items are roughly ordered from most-certainty to least-certainty. Feedback welcome generally, but particularly where questions are posed. > > Broker TODO > =========== > > - C API > > - Python bindings > > - Persistent storage backend > > - SSL/IPv6 (dependent on actor-framework support) > > - Need/want overload or flow-control mechanisms? > > E.g. a simple policy for handling overload is to let a user specify > a threshold for how many items are allowed in a queue before new > messages are dropped. > > - In-place data store value modifications > > Plan to support increment/decrement on integral values. > Need any other operations? > > What to do when applying an operation to invalid data type? > Plan to just send error message back to sender and leave further > decisions up to them. > > - Data store support for optional expiry model > > What are the desired mechanims? Options: > > (1) Inserter may specify "expire this entry at time X" ? > > (2) Inserter may specify "expire this entry based on > create/read/modification access time" ? Going this route, > seems read access times would need to be shared across > clones (contradicts goal of lightweight, local read-access)? > > (3) Other hooks to make expiry conditional? > > - Data typing model > > Currently data in Broker is similar to Bro's threading::Value in > that full type info (from Bro's perspective) isn't available, just > the raw storage required for the types. Broker currently differs in > that it doesn't use any tag to distinguish between primitives that > share the same storage (e.g. enum/string or double/interval). > Interpretation of types is left entirely up to the receiver. > > Do we need more strict typing than this? Options: > > (1) Data holds additional type tag to suggest how to interpret > > (2) Fully implement separate Bro-types. > > Planning to try integrating w/ Bro as it is and see what specific > problems arise. I think (1) may end up being helpful, but maybe not > required and I'd like to avoid (2) if possible. > > - Bro integration > > Is Broker the default in Bro 2.4 ? That implies requiring C++11. > Also I'm requiring CMake 2.8.12+ and may be hard to go below 2.8. > Bro is still happy with 2.6.3. > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > From jsiwek at illinois.edu Wed Oct 22 14:51:14 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Wed, 22 Oct 2014 21:51:14 +0000 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <54481765.7080401@illinois.edu> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <54481765.7080401@illinois.edu> Message-ID: <19BE4571-97D7-424A-98E0-ABA64BE15530@illinois.edu> > On Oct 22, 2014, at 3:45 PM, Thayer, Daniel N wrote: > > Why does it need version 2.8.12 of cmake? 2.8.8+ for object library targets and 2.8.12+ for MACOSX_RPATH (http://www.kitware.com/blog/home/post/510). Don?t see these as strict requirements, but simplifies some things. - Jon From gc355804 at ohio.edu Wed Oct 22 15:07:48 2014 From: gc355804 at ohio.edu (Gilbert Clark) Date: Wed, 22 Oct 2014 18:07:48 -0400 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> Message-ID: <54482AB4.6060707@ohio.edu> Two thoughts inline: On 10/22/2014 3:39 PM, Siwek, Jon wrote: > - C API Do we need a vanilla C API in addition to the C++ API offered? Could be that this was a requirement, so won't argue the point: just making sure someone asked this question. > - Need/want overload or flow-control mechanisms? > > E.g. a simple policy for handling overload is to let a user specify > a threshold for how many items are allowed in a queue before new > messages are dropped. More general question: are there expectations of reliability on broker messages? I'm not really familiar enough with the actor model in general / this library to know. Depending on the protocol being used, it could also be that the transport is going to provide flow-control of its own. Also, one thing that might concern me a little is that, based on a very limited understanding of CAF, messages appear to be garbage collected [1]. How are the messages GC'd? Depending on how the GC works, my concern there would be that a poorly-timed GC cycle could lead to drops. Cheers, Gilbert [1] http://actor-framework.org/pdf/cshw-nassp-13.pdf From 4.1 - "Message Handling and Processing: Messages shall (a) be garbage collected, (b) not be limited to particular types, and (c) provide pattern matching. Requirement (a) is owed to the experience that manual memory management in concurrent systems is error-prone and thus impractical, while the alternative approach of copying a message for each single recipient, leads to suboptimal performance if a message has multiple recipients. Requirement (b) re?ects the common experience that message passing with restricted types is of limited use in practice. However, unrestricted messaging requires ef?cient and expressive facilities such as pattern matching (c), because message handling is a continuously recurring task to implement." From jira at bro-tracker.atlassian.net Wed Oct 22 16:11:08 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Wed, 22 Oct 2014 18:11:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1277) activeHTTP's input read crashes when the returning content-length is 0 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18506#comment-18506 ] Christian Struck commented on BIT-1277: --------------------------------------- Updated tests for both cases. And fixed a minor error. > activeHTTP's input read crashes when the returning content-length is 0 > ---------------------------------------------------------------------- > > Key: BIT-1277 > URL: https://bro-tracker.atlassian.net/browse/BIT-1277 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 with bro/master and curl 7.26.0 > Reporter: Christian Struck > Assignee: Christian Struck > Labels: language > > curl does not write an bodyfile if the returned content length is 0. therefore the fileread of the bodyfile will fail. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 22 16:12:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Wed, 22 Oct 2014 18:12:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1277) activeHTTP's input read crashes when the returning content-length is 0 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Struck updated BIT-1277: ---------------------------------- Status: Merge Request (was: Open) > activeHTTP's input read crashes when the returning content-length is 0 > ---------------------------------------------------------------------- > > Key: BIT-1277 > URL: https://bro-tracker.atlassian.net/browse/BIT-1277 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 with bro/master and curl 7.26.0 > Reporter: Christian Struck > Assignee: Christian Struck > Labels: language > > curl does not write an bodyfile if the returned content length is 0. therefore the fileread of the bodyfile will fail. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From anthony.kasza at gmail.com Wed Oct 22 20:18:36 2014 From: anthony.kasza at gmail.com (anthony kasza) Date: Wed, 22 Oct 2014 20:18:36 -0700 Subject: [Bro-Dev] [JIRA] (BIT-1274) Moving GeoIP Code to Plugin In-Reply-To: References: Message-ID: I am having two issues. The first is how global bro configure options should affect plugins (e.g. USE_GEOIP in CMakeList.txt). This seems like a relatively large architectural decision and I have no solution to this. The second is the following error when I run bro with my dynamic plugin: internal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1: internal type geo_location missing Steps to replicate this error: git clone --recursive https://github.com/anthonykasza/bro.git cd bro git checkout remotes/origin/topic/akasza/geoplugin ./configure --enable-debug && make make install cd aux/bro-aux/plugin-support/ git clone https://github.com/anthonykasza/Bro_GeoIP.git cd Bro_GeoIP/ ./configure --bro-dist=%BRO GIT CLONE DIR% make export BRO_PLUGIN_PATH=%BRO GIT CLONE DIR%/aux/bro-aux/plugin-support/Bro_GeoIP bro -NN -AK On Tue, Oct 21, 2014 at 1:51 PM, Robin Sommer (JIRA) wrote: > > [ https://bro-tracker.atlassian.net/browse/BIT-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18505#comment-18505 ] > > Robin Sommer commented on BIT-1274: > ----------------------------------- > > I've lost track if there's still a particular problem you're having? > >> Moving GeoIP Code to Plugin >> --------------------------- >> >> Key: BIT-1274 >> URL: https://bro-tracker.atlassian.net/browse/BIT-1274 >> Project: Bro Issue Tracker >> Issue Type: Improvement >> Components: Bro >> Reporter: AK >> >> I've started moving the GeoIP code to a plugin. The branch of Bro I'm working from is here: https://github.com/anthonykasza/bro/tree/topic/akasza/geoplugin. >> The source for the plugin is here: https://github.com/anthonykasza/Bro_GeoIP. >> Any pointers would be appreciated. > > > > -- > This message was sent by Atlassian JIRA > (v6.4-OD-07-004#64005) > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From jira at bro-tracker.atlassian.net Wed Oct 22 20:20:07 2014 From: jira at bro-tracker.atlassian.net (AK (JIRA)) Date: Wed, 22 Oct 2014 22:20:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1274) Moving GeoIP Code to Plugin In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18507#comment-18507 ] AK commented on BIT-1274: ------------------------- I am having two issues. The first is how global bro configure options should affect plugins (e.g. USE_GEOIP in CMakeList.txt). This seems like a relatively large architectural decision and I have no solution to this. The second is the following error when I run bro with my dynamic plugin: internal error in /usr/local/bro/share/bro/base/init-bare.bro, line 1: internal type geo_location missing Steps to replicate this error: git clone --recursive https://github.com/anthonykasza/bro.git cd bro git checkout remotes/origin/topic/akasza/geoplugin ./configure --enable-debug && make make install cd aux/bro-aux/plugin-support/ git clone https://github.com/anthonykasza/Bro_GeoIP.git cd Bro_GeoIP/ ./configure --bro-dist=%BRO GIT CLONE DIR% make export BRO_PLUGIN_PATH=%BRO GIT CLONE DIR%/aux/bro-aux/plugin-support/Bro_GeoIP bro -NN -AK On Tue, Oct 21, 2014 at 1:51 PM, Robin Sommer (JIRA) > Moving GeoIP Code to Plugin > --------------------------- > > Key: BIT-1274 > URL: https://bro-tracker.atlassian.net/browse/BIT-1274 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: AK > > I've started moving the GeoIP code to a plugin. The branch of Bro I'm working from is here: https://github.com/anthonykasza/bro/tree/topic/akasza/geoplugin. > The source for the plugin is here: https://github.com/anthonykasza/Bro_GeoIP. > Any pointers would be appreciated. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From noreply at bro.org Thu Oct 23 00:00:17 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 23 Oct 2014 00:00:17 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410230700.s9N70HLE030019@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------------- ---------------- ---------- ------------- ---------- ---------------------------------------------------------------------- BIT-1277 [1] Bro Christian Struck Christian Struck 2014-10-22 - Normal activeHTTP's input read crashes when the returning content-length is 0 [1] BIT-1277 https://bro-tracker.atlassian.net/browse/BIT-1277 From jira at bro-tracker.atlassian.net Thu Oct 23 08:00:07 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 23 Oct 2014 10:00:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1274) Moving GeoIP Code to Plugin In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18508#comment-18508 ] Seth Hall commented on BIT-1274: -------------------------------- It looks like you've placed the geo_location type into the GeoIP namespace. You need to make sure when you reference that type from your C code that your include the full namespace which is GeoIP::geo_location. Maybe you could change it to GeoIP::location? Like this... RecordType* geo_location = internal_type("GeoIP::location")->AsRecordType(); I think you also want to put that type definition into an export section. > Moving GeoIP Code to Plugin > --------------------------- > > Key: BIT-1274 > URL: https://bro-tracker.atlassian.net/browse/BIT-1274 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: AK > > I've started moving the GeoIP code to a plugin. The branch of Bro I'm working from is here: https://github.com/anthonykasza/bro/tree/topic/akasza/geoplugin. > The source for the plugin is here: https://github.com/anthonykasza/Bro_GeoIP. > Any pointers would be appreciated. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jsiwek at illinois.edu Thu Oct 23 08:41:49 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 23 Oct 2014 15:41:49 +0000 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <54482AB4.6060707@ohio.edu> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <54482AB4.6060707@ohio.edu> Message-ID: <28B9BBC5-54B8-46E9-82C3-BFA7F6146951@illinois.edu> > Do we need a vanilla C API in addition to the C++ API offered? Could be > that this was a requirement, so won't argue the point: just making sure > someone asked this question. Yeah, if a goal is to make it easy to instrument existing applications w/ the library, the idea was that some will want/need the C interface. >> - Need/want overload or flow-control mechanisms? >> >> E.g. a simple policy for handling overload is to let a user specify >> a threshold for how many items are allowed in a queue before new >> messages are dropped. > > More general question: are there expectations of reliability on broker > messages? I'm not really familiar enough with the actor model in > general / this library to know. Depending on the protocol being used, > it could also be that the transport is going to provide flow-control of > its own. The message delivery is going to be reliable and taken care of for us. I was more referring to situations such as receiving remote logs at a rate faster than one can process. What happens currently in Bro and Broker is messages pile up until memory is exhausted and you crash. I don?t know how great the expectation is to handle that gracefully right away. We could probably easily put in a way to let a user specify ?it?s ok to drop messages for this queue if it gets overloaded?. Matthias has been talking some about getting flow control mechanisms (e.g. overloaded actor tells senders ?I?m currently overloaded, please slow down?) in CAF on their mailing list. I?m not sure the degree to which something like that would be helpful in Broker ? it may just push the overload problem to the sender(s) and begs the question of what they do if they can?t artificially slow down? > Also, one thing that might concern me a little is that, based on a very > limited understanding of CAF, messages appear to be garbage collected > [1]. How are the messages GC'd? Depending on how the GC works, my > concern there would be that a poorly-timed GC cycle could lead to drops. My impression was that messages are reference counted, copy-on-write tuple values that get reclaimed automatically when the ref count reaches zero. - Jon From seth at icir.org Thu Oct 23 09:04:47 2014 From: seth at icir.org (Seth Hall) Date: Thu, 23 Oct 2014 12:04:47 -0400 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <28B9BBC5-54B8-46E9-82C3-BFA7F6146951@illinois.edu> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <54482AB4.6060707@ohio.edu> <28B9BBC5-54B8-46E9-82C3-BFA7F6146951@illinois.edu> Message-ID: <88A81CD4-3DCE-4E59-A4B1-89BC982280F7@icir.org> On Oct 23, 2014, at 11:41 AM, Siwek, Jon wrote: > it may just push the overload problem to the sender(s) and begs the question of what they do if they can?t artificially slow down? With regards to logging, I think this is one area where you can cheat a bit and just push back at scriptland to give script developers a chance to know if a logging queue is getting backed up. There are a number of ways we could deal with overload situations there. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From jsiwek at illinois.edu Thu Oct 23 09:48:54 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 23 Oct 2014 16:48:54 +0000 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <88A81CD4-3DCE-4E59-A4B1-89BC982280F7@icir.org> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <54482AB4.6060707@ohio.edu> <28B9BBC5-54B8-46E9-82C3-BFA7F6146951@illinois.edu> <88A81CD4-3DCE-4E59-A4B1-89BC982280F7@icir.org> Message-ID: > On Oct 23, 2014, at 11:04 AM, Seth Hall wrote: > > On Oct 23, 2014, at 11:41 AM, Siwek, Jon wrote: > >> it may just push the overload problem to the sender(s) and begs the question of what they do if they can?t artificially slow down? > > With regards to logging, I think this is one area where you can cheat a bit and just push back at scriptland to give script developers a chance to know if a logging queue is getting backed up. There are a number of ways we could deal with overload situations there. Yeah, leaving things up to application may be reasonable here. For most of the messaging patterns in Broker, I expect it to be lightweight to hand off pending messages from Broker?s queues to the application, so essentially the application is already going to be left to manage its own resources. For Bro that could mean tying in to the script-layer like you suggested and shuffle around or even disable some logging/events (on either sender or receiver side). We could probably already be doing something like that in Bro without Broker, so maybe that is a hint that it may not need to be addressed directly in the library. - Jon From robin at icir.org Thu Oct 23 12:47:04 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 23 Oct 2014 12:47:04 -0700 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> Message-ID: <20141023194704.GH97490@icir.org> On Wed, Oct 22, 2014 at 19:39 +0000, you wrote: > If anyone has time/interest, I feel like the main components of Broker > are established now and deserving feedback/critique. Very cool. I'll look through the repository. Some toughts regarding your TODO list, in terms of priorities and your question: > - C API Yeah, I would like to have this early on. In principle, we could postpone to later, but it looks like one of these things that if we don't get it into place right away, it will be even harder to do later. Maybe others can help you with this; once an initial structure is in place it's probably getting quite mechanic. Another question here would be how we ensure that the C API stays in sync going forward. Is there some testing we can put in place? > - Python bindings That I can see skipping for the first version. While it would of cource be great to have, it should be pretty straight forward to do once the C bindings are there. > - Persistent storage backend My guess is that storage will end up being the primary initial use case (because that's a capability we don't have right now; vs. replacing exisitng stuff), so yeah, a good target. > - SSL/IPv6 (dependent on actor-framework support) Can do later, though I don't rememeber what the conclusion was if/how the actor-framework supports these? > - Need/want overload or flow-control mechanisms? Punting for now sounds good, per the other mails. Maybe CAF will get some support eventually that we can leverage, and/or we can add the script-level hooks. > - In-place data store value modifications > > Plan to support increment/decrement on integral values. > Need any other operations? Set operations (insert element, remove element). > What to do when applying an operation to invalid data type? > Plan to just send error message back to sender and leave further > decisions up to them. Sounds like a more general question: what to do in terms of semantic errors? There are probably more like that, like writing to a store that doesn't exist. Error messages sound right, with hooks to report them to the application. In Bro they can show up as events in script land. > - Data store support for optional expiry model > > What are the desired mechanims? Options: > > (1) Inserter may specify "expire this entry at time X" ? > (2) Inserter may specify "expire this entry based on > create/read/modification access time" ? How about providing on-create and on-modification, but not on-read. In those two cases there's already communication necessary anyways, and the expiration time could be piggy-bagged on that. > (3) Other hooks to make expiry conditional? Hmm, maybe, but unclear how it would look like. Could work like Bro's expire func (i.e., potentially delay expiration), but I think this hook could really only run only at the storage node directly. I'd skip for now. > - Data typing model > (1) Data holds additional type tag to suggest how to interpret > (2) Fully implement separate Bro-types. > > Planning to try integrating w/ Bro as it is and see what specific > problems arise. I think (1) may end up being helpful, but maybe not > required and I'd like to avoid (2) if possible. I'm not fully sure what you mean by (2) but I believe I agree. :) (1) would be good; maybe Bro doesn't strictly need it, but (a) it would allow it to double-check input at least; and (b) for independent applications it will be quite helpful, as otherwise it's hard to work with data of different input types dynamically (in other words: the applications would need a way to define what to expect, forcing them to replicate the typing that exists in Bro already). A more general question in this context: what's the trust model? Are we expecting that a client taking part in the communication will play by the rules of the protocol? That's what current Bro does, and I think it's a reasonable assumption. On the other hand, maybe Broker could do a bit better, as it's data model isn't as complex as Bro's native Val structure. Asked differently: to which degree can a receiver validate that incoming data makes sense (with some appropiate definition of "makes sense"; there are differen ones, like having the right binary layout, or semantically sending valid information) > - Bro integration > Is Broker the default in Bro 2.4 ? That implies requiring C++11. > Also I'm requiring CMake 2.8.12+ and may be hard to go below 2.8. > Bro is still happy with 2.6.3. We should definitly intregrate it. It could be an optional dependency, or a mandatory component. I'm leaning towards the latter, to pave the way for the future. But yeah, that then means requiring C++11 (and a current cmake). Did we ever come to a conclusion on C++11 for Bro? We did the survey, but I don't recall if we settled on whether it's ok to switch now? Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Thu Oct 23 14:56:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 23 Oct 2014 16:56:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1274) Moving GeoIP Code to Plugin In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18510#comment-18510 ] Robin Sommer commented on BIT-1274: ----------------------------------- I've just pushed a small change to master that's necessary to get this running; there was an ordering issue in terms of how scripts get loaded. I've tweaked your code a bit more, including the namespacing Seth suggested; patch attached. With that it now works for me: {code} # cat a.bro @load Bro/GeoIP event bro_init() { print GeoIP::lookup_location(131.159.14.1); } # bro ./a.bro [country_code=DE, region=02, city=Munich, latitude=48.150002, longitude=11.5833] {code} > Moving GeoIP Code to Plugin > --------------------------- > > Key: BIT-1274 > URL: https://bro-tracker.atlassian.net/browse/BIT-1274 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: AK > > I've started moving the GeoIP code to a plugin. The branch of Bro I'm working from is here: https://github.com/anthonykasza/bro/tree/topic/akasza/geoplugin. > The source for the plugin is here: https://github.com/anthonykasza/Bro_GeoIP. > Any pointers would be appreciated. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 23 14:57:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 23 Oct 2014 16:57:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1274) Moving GeoIP Code to Plugin In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1274: ------------------------------ Attachment: ak.patch > Moving GeoIP Code to Plugin > --------------------------- > > Key: BIT-1274 > URL: https://bro-tracker.atlassian.net/browse/BIT-1274 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: AK > Attachments: ak.patch > > > I've started moving the GeoIP code to a plugin. The branch of Bro I'm working from is here: https://github.com/anthonykasza/bro/tree/topic/akasza/geoplugin. > The source for the plugin is here: https://github.com/anthonykasza/Bro_GeoIP. > Any pointers would be appreciated. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jsiwek at illinois.edu Thu Oct 23 14:59:13 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 23 Oct 2014 21:59:13 +0000 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <20141023194704.GH97490@icir.org> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <20141023194704.GH97490@icir.org> Message-ID: <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> > On Oct 23, 2014, at 2:47 PM, Robin Sommer wrote: > >> - C API > > Yeah, I would like to have this early on. In principle, we could > postpone to later, but it looks like one of these things that if we > don't get it into place right away, it will be even harder to do > later. Maybe others can help you with this; once an initial structure > is in place it's probably getting quite mechanic. I agree it will probably be a pretty mechanical process once there?s a better foundation that tackles some of the harder aspects and common patterns. But I wonder if prioritizing Bro integration over this may be more helpful ? if we learn significant parts of the C++ interface need to change, then that may mean some of the effort making the C interface goes to waste. > Another question here would be how we ensure that the C API stays in > sync going forward. Is there some testing we can put in place? There?s already some unit tests for the C++ API that could be replicated in C. >> - Python bindings > > That I can see skipping for the first version. While it would of > cource be great to have, it should be pretty straight forward to do > once the C bindings are there. Won?t BroControl require this? >> - Persistent storage backend > > My guess is that storage will end up being the primary initial use > case (because that's a capability we don't have right now; vs. > replacing exisitng stuff), so yeah, a good target. I started looking in to this a little and I?m thinking either LevelDB or RocksDB may be good default choices to use here. >> - SSL/IPv6 (dependent on actor-framework support) > > Can do later, though I don't rememeber what the conclusion was if/how the > actor-framework supports these? IIRC, the idea was that it could, but doesn?t yet. I also think it?s not critical to have these in the initial version. >> - In-place data store value modifications >> >> Plan to support increment/decrement on integral values. >> Need any other operations? > > Set operations (insert element, remove element). Ack. >> What to do when applying an operation to invalid data type? >> Plan to just send error message back to sender and leave further >> decisions up to them. > > Sounds like a more general question: what to do in terms of semantic > errors? There are probably more like that, like writing to a store > that doesn't exist. Error messages sound right, with hooks to report > them to the application. In Bro they can show up as events in script > land. Yeah, probably does warrant a general solution for application to get at errors that can?t be reported synchronously ? I can recall a few other unrelated places in the code I?ve written something like ?TODO: log an error?. >> - Data store support for optional expiry model >> >> What are the desired mechanims? Options: >> >> (1) Inserter may specify "expire this entry at time X" ? >> (2) Inserter may specify "expire this entry based on >> create/read/modification access time" ? > > How about providing on-create and on-modification, but not on-read. In > those two cases there's already communication necessary anyways, and > the expiration time could be piggy-bagged on that. May be reasonable, I?ll aim to support that and see if I hit any issues. >> (3) Other hooks to make expiry conditional? > > Hmm, maybe, but unclear how it would look like. Could work like Bro's > expire func (i.e., potentially delay expiration), but I think this > hook could really only run only at the storage node directly. I'd skip > for now. Ack. Was hoping skipping this would be ok :) > A more general question in this context: what's the trust model? Are > we expecting that a client taking part in the communication will play > by the rules of the protocol? That's what current Bro does, and I > think it's a reasonable assumption. This was also my expectation. > On the other hand, maybe Broker > could do a bit better, as it's data model isn't as complex as Bro's > native Val structure. Asked differently: to which degree can a > receiver validate that incoming data makes sense (with some appropiate > definition of "makes sense"; there are differen ones, like having the > right binary layout, or semantically sending valid information) If ?receiver" here means the application I think it can validate pretty well. For example, if the application is Bro and it?s receiving logs, I expect it currently has enough information from Broker to be able to tell if what it got can actually be converted in to the types it needs to log. If ?receiver? means Broker versus The Internet, I?m not actually sure the extent it would hold up -- that seems dependent on CAF being able to hold up. >> - Bro integration >> Is Broker the default in Bro 2.4 ? That implies requiring C++11. >> Also I'm requiring CMake 2.8.12+ and may be hard to go below 2.8. >> Bro is still happy with 2.6.3. > > We should definitly intregrate it. It could be an optional dependency, > or a mandatory component. I'm leaning towards the latter, to pave the > way for the future. But yeah, that then means requiring C++11 (and a > current cmake). > > Did we ever come to a conclusion on C++11 for Bro? We did the survey, > but I don't recall if we settled on whether it's ok to switch now? We didn?t reach a conclusion. I can take a closer look, but I recall supporting EL6 may be important. On RHEL6, looks like GCC 4.4.7 is the default, but all major C++11 features I think appear in 4.8 and later (I?m not sure it?s worth it to try and figure out what specific C++11 features we can ?get away with? and thus require older compiler version). - Jon From robin at icir.org Thu Oct 23 15:20:35 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 23 Oct 2014 15:20:35 -0700 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <20141023194704.GH97490@icir.org> <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> Message-ID: <20141023222035.GB63601@icir.org> On Thu, Oct 23, 2014 at 21:59 +0000, you wrote: > patterns. But I wonder if prioritizing Bro integration over this may > be more helpful ? if we learn significant parts of the C++ interface > need to change, I can see that either way actually: doing the C interface could turn up problems with the C++ structure as well. But I agree in terms of priority: Bro over C. > There?s already some unit tests for the C++ API that could be > replicated in C. What I meant was ensuring the two stay in sync. Say if we added a new capability to C++, can we trigger somehow a test failure if we forget to add it to C? > Won?t BroControl require this? I was imagining for 2.4 we'd leave the BroControl parts in place as they are now, i.e., using the old comm framework. Were you planing to replace that already? > I started looking in to this a little and I?m thinking either LevelDB > or RocksDB may be good default choices to use here. (No experience with either, will take a look) > If ?receiver? means Broker versus The Internet I was thinking about this case (the other current Bro does already as well), but yeah, CAF certainly plays a part there as well. > On RHEL6, looks like GCC 4.4.7 is the default, but all major C++11 > features I think appear in 4.8 and later So maybe that means we'll need to wait another release at least before making it mandatory, and giving people a heads-up that we plan to do the switch. On the other hand, I would prefer to have it fully in there right away, so that scripts can start to rely on it as soon as possible. Tough call ... > (I?m not sure it?s worth it to try and figure out what specific C++11 > features we can ?get away with? and thus require older compiler > version). Agree. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From noreply at bro.org Fri Oct 24 00:00:21 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 24 Oct 2014 00:00:21 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410240700.s9O70L4K014109@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------------- ---------------- ---------- ------------- ---------- ---------------------------------------------------------------------- BIT-1277 [1] Bro Christian Struck Christian Struck 2014-10-22 - Normal activeHTTP's input read crashes when the returning content-length is 0 [1] BIT-1277 https://bro-tracker.atlassian.net/browse/BIT-1277 From jsiwek at illinois.edu Fri Oct 24 08:02:57 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Fri, 24 Oct 2014 15:02:57 +0000 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <20141023222035.GB63601@icir.org> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <20141023194704.GH97490@icir.org> <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> <20141023222035.GB63601@icir.org> Message-ID: <8A92C1FE-71DA-493F-9620-353451BD9061@illinois.edu> > On Oct 23, 2014, at 5:20 PM, Robin Sommer wrote: > > What I meant was ensuring the two stay in sync. Say if we added a new > capability to C++, can we trigger somehow a test failure if we forget > to add it to C? Don?t have any ideas for how to do that at the moment, but, yes, it would be nice to have that type of coverage test. > I was imagining for 2.4 we'd leave the BroControl parts in place as > they are now, i.e., using the old comm framework. Were you planing to > replace that already? Yeah, I was thinking the user would be given a binary choice: either everything uses new comm or everything uses old comm. But I guess it also works to say old comm is still available for everything it used to be, but additionally/concurrently there?s the option of trying the new comm for tasks related to A, B, and C. I?m not sure what approach I like best, but may make sense to go in to the integration process with the intent of just making incremental additions and then see how far we get. - Jon From jira at bro-tracker.atlassian.net Fri Oct 24 11:18:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 24 Oct 2014 13:18:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1277) activeHTTP's input read crashes when the returning content-length is 0 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1277: --------------------------------- Assignee: Robin Sommer (was: Christian Struck) > activeHTTP's input read crashes when the returning content-length is 0 > ---------------------------------------------------------------------- > > Key: BIT-1277 > URL: https://bro-tracker.atlassian.net/browse/BIT-1277 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 with bro/master and curl 7.26.0 > Reporter: Christian Struck > Assignee: Robin Sommer > Labels: language > > curl does not write an bodyfile if the returned content length is 0. therefore the fileread of the bodyfile will fail. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 24 11:59:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 24 Oct 2014 13:59:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1277) activeHTTP's input read crashes when the returning content-length is 0 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1277: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > activeHTTP's input read crashes when the returning content-length is 0 > ---------------------------------------------------------------------- > > Key: BIT-1277 > URL: https://bro-tracker.atlassian.net/browse/BIT-1277 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 with bro/master and curl 7.26.0 > Reporter: Christian Struck > Assignee: Robin Sommer > Labels: language > > curl does not write an bodyfile if the returned content length is 0. therefore the fileread of the bodyfile will fail. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From robin at icir.org Fri Oct 24 14:24:52 2014 From: robin at icir.org (Robin Sommer) Date: Fri, 24 Oct 2014 14:24:52 -0700 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <8A92C1FE-71DA-493F-9620-353451BD9061@illinois.edu> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <20141023194704.GH97490@icir.org> <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> <20141023222035.GB63601@icir.org> <8A92C1FE-71DA-493F-9620-353451BD9061@illinois.edu> Message-ID: <20141024212452.GT96754@icir.org> On Fri, Oct 24, 2014 at 15:02 +0000, you wrote: > But I guess it also works to say old comm is still available for > everything it used to be, but additionally/concurrently there?s the > option of trying the new comm for tasks related to A, B, and C. This would be my inclination, as it limits the scope of what we need to get in shape for the first release. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From noreply at bro.org Sun Oct 26 00:00:24 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 26 Oct 2014 00:00:24 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410260700.s9Q70O5J031095@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------------- ---------- -------------------- #2 [1] cmake genesislive2007 [2] 2014-10-25 Docfix: missed ) [3] [1] Pull Request #2 https://github.com/bro/cmake/pull/2 [2] genesislive2007 https://github.com/genesislive2007 [3] Merge Pull Request #2 with git pull --no-ff --no-commit https://github.com/genesislive2007/cmake.git patch-1 From noreply at bro.org Mon Oct 27 00:00:27 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 27 Oct 2014 00:00:27 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410270700.s9R70RD4032244@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------------- ---------- ----------------------------------------- #16 [1] bro vice [2] 2014-10-26 Wrong port in scripting documentation [3] #2 [4] cmake genesislive2007 [5] 2014-10-25 Docfix: missed ) [6] [1] Pull Request #16 https://github.com/bro/bro/pull/16 [2] vice https://github.com/vice [3] Merge Pull Request #16 with git pull --no-ff --no-commit https://github.com/vice/bro.git patch-1 [4] Pull Request #2 https://github.com/bro/cmake/pull/2 [5] genesislive2007 https://github.com/genesislive2007 [6] Merge Pull Request #2 with git pull --no-ff --no-commit https://github.com/genesislive2007/cmake.git patch-1 From jira at bro-tracker.atlassian.net Tue Oct 28 07:19:07 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Tue, 28 Oct 2014 09:19:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1280) Checking index in vectors is broken In-Reply-To: References: Message-ID: Seth Hall created BIT-1280: ------------------------------ Summary: Checking index in vectors is broken Key: BIT-1280 URL: https://bro-tracker.atlassian.net/browse/BIT-1280 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.4 Reporter: Seth Hall If you try to check an index in a vector for existence, you get an error... {quote} event bro_init() { local vec: vector of count = vector(); if ( 2 in vec ) print vec[2]; print vec; } {quote} Error: {quote} fatal error in bool: BroType::AsVectorType (bool/vector) (bool) {quote} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 28 07:29:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 28 Oct 2014 09:29:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1280) Checking index in vectors is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18600#comment-18600 ] Jon Siwek commented on BIT-1280: -------------------------------- My first instinct is to make that {noformat} if ( 2 in vec ) {noformat} condition the equivalent of {noformat} if ( |vec| > 2 ) {noformat} Make sense or no? > Checking index in vectors is broken > ----------------------------------- > > Key: BIT-1280 > URL: https://bro-tracker.atlassian.net/browse/BIT-1280 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > > If you try to check an index in a vector for existence, you get an error... > {quote} > event bro_init() > { > local vec: vector of count = vector(); > if ( 2 in vec ) > print vec[2]; > print vec; > } > {quote} > Error: > {quote} > fatal error in bool: BroType::AsVectorType (bool/vector) (bool) > {quote} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 28 07:30:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 28 Oct 2014 09:30:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1280) Checking index in vectors is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1280: --------------------------- Description: If you try to check an index in a vector for existence, you get an error... {noformat} event bro_init() { local vec: vector of count = vector(); if ( 2 in vec ) print vec[2]; print vec; } {noformat} Error: {quote} fatal error in bool: BroType::AsVectorType (bool/vector) (bool) {quote} was: If you try to check an index in a vector for existence, you get an error... {quote} event bro_init() { local vec: vector of count = vector(); if ( 2 in vec ) print vec[2]; print vec; } {quote} Error: {quote} fatal error in bool: BroType::AsVectorType (bool/vector) (bool) {quote} > Checking index in vectors is broken > ----------------------------------- > > Key: BIT-1280 > URL: https://bro-tracker.atlassian.net/browse/BIT-1280 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > > If you try to check an index in a vector for existence, you get an error... > {noformat} > event bro_init() > { > local vec: vector of count = vector(); > if ( 2 in vec ) > print vec[2]; > print vec; > } > {noformat} > Error: > {quote} > fatal error in bool: BroType::AsVectorType (bool/vector) (bool) > {quote} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 28 07:32:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Tue, 28 Oct 2014 09:32:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1280) Checking index in vectors is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18601#comment-18601 ] Johanna Amann commented on BIT-1280: ------------------------------------ I actually thought about exactly that when I read the bug report - and I am not sure. Just because the length of a vector is greater than a specific element does not mean that all smaller elements are set at the moment (unless I am very much mistaken). > Checking index in vectors is broken > ----------------------------------- > > Key: BIT-1280 > URL: https://bro-tracker.atlassian.net/browse/BIT-1280 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > > If you try to check an index in a vector for existence, you get an error... > {noformat} > event bro_init() > { > local vec: vector of count = vector(); > if ( 2 in vec ) > print vec[2]; > print vec; > } > {noformat} > Error: > {quote} > fatal error in bool: BroType::AsVectorType (bool/vector) (bool) > {quote} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 28 07:50:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 28 Oct 2014 09:50:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1280) Checking index in vectors is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18602#comment-18602 ] Jon Siwek commented on BIT-1280: -------------------------------- {quote} Just because the length of a vector is greater than a specific element does not mean that all smaller elements are set at the moment (unless I am very much mistaken). {quote} Yeah, I think I remember it working like that also, but not sure if it's actually intended to be used in that way. Probably should be easy to implement the "in" operator so it just checks that. Unless there's some larger functionality Seth wanted? > Checking index in vectors is broken > ----------------------------------- > > Key: BIT-1280 > URL: https://bro-tracker.atlassian.net/browse/BIT-1280 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > > If you try to check an index in a vector for existence, you get an error... > {noformat} > event bro_init() > { > local vec: vector of count = vector(); > if ( 2 in vec ) > print vec[2]; > print vec; > } > {noformat} > Error: > {quote} > fatal error in bool: BroType::AsVectorType (bool/vector) (bool) > {quote} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 28 12:32:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 28 Oct 2014 14:32:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1280) Checking index in vectors is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1280: --------------------------- Fix Version/s: 2.4 > Checking index in vectors is broken > ----------------------------------- > > Key: BIT-1280 > URL: https://bro-tracker.atlassian.net/browse/BIT-1280 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Fix For: 2.4 > > > If you try to check an index in a vector for existence, you get an error... > {noformat} > event bro_init() > { > local vec: vector of count = vector(); > if ( 2 in vec ) > print vec[2]; > print vec; > } > {noformat} > Error: > {quote} > fatal error in bool: BroType::AsVectorType (bool/vector) (bool) > {quote} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 28 12:33:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 28 Oct 2014 14:33:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1280) Checking index in vectors is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18603#comment-18603 ] Jon Siwek commented on BIT-1280: -------------------------------- Fix in topic/jsiwek/bit-1280 > Checking index in vectors is broken > ----------------------------------- > > Key: BIT-1280 > URL: https://bro-tracker.atlassian.net/browse/BIT-1280 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Fix For: 2.4 > > > If you try to check an index in a vector for existence, you get an error... > {noformat} > event bro_init() > { > local vec: vector of count = vector(); > if ( 2 in vec ) > print vec[2]; > print vec; > } > {noformat} > Error: > {quote} > fatal error in bool: BroType::AsVectorType (bool/vector) (bool) > {quote} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Tue Oct 28 12:33:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 28 Oct 2014 14:33:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1280) Checking index in vectors is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1280: --------------------------- Status: Merge Request (was: Open) > Checking index in vectors is broken > ----------------------------------- > > Key: BIT-1280 > URL: https://bro-tracker.atlassian.net/browse/BIT-1280 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Fix For: 2.4 > > > If you try to check an index in a vector for existence, you get an error... > {noformat} > event bro_init() > { > local vec: vector of count = vector(); > if ( 2 in vec ) > print vec[2]; > print vec; > } > {noformat} > Error: > {quote} > fatal error in bool: BroType::AsVectorType (bool/vector) (bool) > {quote} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From johanna at icir.org Tue Oct 28 13:25:00 2014 From: johanna at icir.org (Johanna Amann) Date: Tue, 28 Oct 2014 13:25:00 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/fastpath' (432744f) In-Reply-To: <201410282018.s9SKI9n4019685@bro-ids.icir.org> References: <201410282018.s9SKI9n4019685@bro-ids.icir.org> Message-ID: <20141028202455.GA84346@user149.sys.icsi.berkeley.edu> On Tue, Oct 28, 2014 at 01:18:09PM -0700, Jonathan Siwek wrote: > Some didn't look quite right so fixed while merging: the return value of > fwrite is in terms of number of objects written, not number of bytes > written and some calls still mixed those up. Thank you, I totally managed to miss that. Johanna From noreply at bro.org Wed Oct 29 00:00:29 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 29 Oct 2014 00:00:29 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410290700.s9T70TF2002208@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ----------------------------------- BIT-1280 [1] Bro Seth Hall - 2014-10-28 2.4 Normal Checking index in vectors is broken [1] BIT-1280 https://bro-tracker.atlassian.net/browse/BIT-1280 From jira at bro-tracker.atlassian.net Wed Oct 29 12:28:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Wed, 29 Oct 2014 14:28:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1281) Bro crashes with Segfault when declaring a function but never defining it. In-Reply-To: References: Message-ID: Christian Struck created BIT-1281: ------------------------------------- Summary: Bro crashes with Segfault when declaring a function but never defining it. Key: BIT-1281 URL: https://bro-tracker.atlassian.net/browse/BIT-1281 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Environment: Debian 7 Reporter: Christian Struck 1414607518.791169 error in **** value used but not set (****::function) Segmentation fault Bro already knows whats happening but does still segfault, I think this should not happen. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 29 13:07:08 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Wed, 29 Oct 2014 15:07:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1281) Bro crashes with Segfault when declaring a function but never defining it. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18604#comment-18604 ] Johanna Amann commented on BIT-1281: ------------------------------------ Could you perhaps add a minimal script to reproduce this? :) > Bro crashes with Segfault when declaring a function but never defining it. > -------------------------------------------------------------------------- > > Key: BIT-1281 > URL: https://bro-tracker.atlassian.net/browse/BIT-1281 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > > 1414607518.791169 error in **** value used but not set (****::function) > Segmentation fault > Bro already knows whats happening but does still segfault, I think this should not happen. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 29 13:44:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Wed, 29 Oct 2014 15:44:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1281) Bro crashes with Segfault when declaring a function but never defining it. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Struck updated BIT-1281: ---------------------------------- Attachment: test.bro yup, seems I was too unspecific. So I tried to use the result of an undefined function. > Bro crashes with Segfault when declaring a function but never defining it. > -------------------------------------------------------------------------- > > Key: BIT-1281 > URL: https://bro-tracker.atlassian.net/browse/BIT-1281 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Attachments: test.bro > > > 1414607518.791169 error in **** value used but not set (****::function) > Segmentation fault > Bro already knows whats happening but does still segfault, I think this should not happen. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From vlad at grigorescu.org Wed Oct 29 13:48:02 2014 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Wed, 29 Oct 2014 16:48:02 -0400 Subject: [Bro-Dev] Help Troubleshooting a Perftools Memleak Message-ID: The MySQL analyzer is ready to go, apart from one issue: a memleak btest that I wrote is failing on some of Bro's regex code. > # @TEST-EXEC: HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local btest-bg-run bro bro -b -m -r $TRACES/mysql/mysql.trace %INPUT > > @load base/protocols/mysql This results in: > Leak check net_run detected leaks of 203 bytes in 4 objects > > The 4 largest leaks: > Leak of 72 bytes in 1 objects allocated from: > @ 53e92d > > Leak of 56 bytes in 1 objects allocated from: > @ 52fb66 > @ 83f663 > > Leak of 56 bytes in 1 objects allocated from: > @ 52fd52 > @ 20014 > > Leak of 19 bytes in 1 objects allocated from: > @ 53e9b6 Digging a little deeper shows that two of these leaks are in RE_parse (re-parse.y:110 and re-parse.y:133), one is in re__scan_buffer (re-scan.cc:2035), and one is in re__scan_bytes (re-scan.cc::2084). The only regular expression that I have in the analyzer is: type NUL_String = RE/[^\0]*/; I'm pretty sure that this isn't really an issue, but can anyone help with figuring out how to get the btest to pass? I'd really like to have a memleak test for this. Thanks, --Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141029/b60ad458/attachment.html From jira at bro-tracker.atlassian.net Wed Oct 29 13:53:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Wed, 29 Oct 2014 15:53:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1282) when statement crashes bro with segfault when using undefined input. In-Reply-To: References: Message-ID: Christian Struck created BIT-1282: ------------------------------------- Summary: when statement crashes bro with segfault when using undefined input. Key: BIT-1282 URL: https://bro-tracker.atlassian.net/browse/BIT-1282 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Environment: Debian 7 Reporter: Christian Struck Attachments: test.bro when trying to use a when statement where the right value is undefined and trying to access the left value in the when statement bro crashes with a segfault. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 29 13:54:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Wed, 29 Oct 2014 15:54:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1281) Bro crashes with Segfault when declaring a function but never defining it. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18606#comment-18606 ] Christian Struck commented on BIT-1281: --------------------------------------- ok this does not seem to be a problem with undefined functions, it's a problem of the when statement. closing this bug and opened BIT-1282 with the correct error description > Bro crashes with Segfault when declaring a function but never defining it. > -------------------------------------------------------------------------- > > Key: BIT-1281 > URL: https://bro-tracker.atlassian.net/browse/BIT-1281 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Attachments: test.bro > > > 1414607518.791169 error in **** value used but not set (****::function) > Segmentation fault > Bro already knows whats happening but does still segfault, I think this should not happen. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 29 13:55:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Wed, 29 Oct 2014 15:55:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1281) Bro crashes with Segfault when declaring a function but never defining it. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Struck updated BIT-1281: ---------------------------------- Resolution: Won't Fix Status: Closed (was: Open) BIT-1282 > Bro crashes with Segfault when declaring a function but never defining it. > -------------------------------------------------------------------------- > > Key: BIT-1281 > URL: https://bro-tracker.atlassian.net/browse/BIT-1281 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Attachments: test.bro > > > 1414607518.791169 error in **** value used but not set (****::function) > Segmentation fault > Bro already knows whats happening but does still segfault, I think this should not happen. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 29 14:33:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Wed, 29 Oct 2014 16:33:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1282) when statement crashes bro with segfault when using undefined input. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1282: ------------------------------- Resolution: Duplicate Status: Closed (was: Open) > when statement crashes bro with segfault when using undefined input. > -------------------------------------------------------------------- > > Key: BIT-1282 > URL: https://bro-tracker.atlassian.net/browse/BIT-1282 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Labels: language > Attachments: test.bro > > > when trying to use a when statement where the right value is undefined and trying to access the left value in the when statement bro crashes with a segfault. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 29 14:33:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Wed, 29 Oct 2014 16:33:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1282) when statement crashes bro with segfault when using undefined input. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18608#comment-18608 ] Johanna Amann commented on BIT-1282: ------------------------------------ This is a duplicate of BIT-1176 (sorry that I did not notice that earlier). > when statement crashes bro with segfault when using undefined input. > -------------------------------------------------------------------- > > Key: BIT-1282 > URL: https://bro-tracker.atlassian.net/browse/BIT-1282 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Labels: language > Attachments: test.bro > > > when trying to use a when statement where the right value is undefined and trying to access the left value in the when statement bro crashes with a segfault. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 29 14:34:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Wed, 29 Oct 2014 16:34:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1176) Using an undefined function in a when statement causes a segfault In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18609#comment-18609 ] Johanna Amann commented on BIT-1176: ------------------------------------ As Christian described in BIT-1282 with another testcase, this does not only work with functions but with all kinds of variables. > Using an undefined function in a when statement causes a segfault > ----------------------------------------------------------------- > > Key: BIT-1176 > URL: https://bro-tracker.atlassian.net/browse/BIT-1176 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Johanna Amann > Fix For: 2.4 > > Attachments: crashme.bro > > > Running the following script crashes bro with a null-pointer exception: > {code:title=crashMe.bro} > global crashMe: function():string; > when( local result = crashMe() ) { > print result; > } > {code} > Backtrace: > {code} > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > 320 > 321 int Val::IsZero() const > 322 { > -> 323 switch ( type->InternalType() ) { > 324 case TYPE_INTERNAL_INT: return val.int_val == 0; > 325 case TYPE_INTERNAL_UNSIGNED: return val.uint_val == 0; > 326 case TYPE_INTERNAL_DOUBLE: return val.double_val == 0.0; > (lldb) bt > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > * frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > frame #1: 0x000000010020b452 bro`Trigger::Eval(this=0x0000000105d45d60) + 578 at Trigger.cc:209 > frame #2: 0x000000010020ae95 bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 1285 at Trigger.cc:140 > frame #3: 0x000000010020a98a bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 106 at Trigger.cc:147 > frame #4: 0x000000010020566f bro`WhenStmt::Exec(this=0x0000000104a00900, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 239 at Stmt.cc:2041 > frame #5: 0x0000000100203204 bro`StmtList::Exec(this=0x00000001049fbe80, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 228 at Stmt.cc:1639 > frame #6: 0x000000010003d244 bro`main(argc=2, argv=0x00007fff5fbffa40) + 15476 at main.cc:1116 > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 29 15:19:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Wed, 29 Oct 2014 17:19:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1253) Bro 2.3 - 2.3.1 manager dieing on Bivio hardware In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1253: ------------------------------- Status: Reopened (was: Closed) > Bro 2.3 - 2.3.1 manager dieing on Bivio hardware > ------------------------------------------------ > > Key: BIT-1253 > URL: https://bro-tracker.atlassian.net/browse/BIT-1253 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Bro 2.3 and Bro 2.3.1 > bivio hardwareLinux CPU.2.6.31-45 has curl 7.36 gperftools 2.2 flex 2.5.39 bison 3.0.2 libpcap 1.1 swig 2.0.8 > Reporter: Larry Leviton > Assignee: Johanna Amann > Fix For: 2.4 > > > After starting bro up, the bro manager crashes in less than 60 seconds. > Thanks for any help you can give. > Sent stack trace to vendor (at bottom), and here was their response: > Comment(s): Hello Larry, > We have duplicated a crash in our lab setup that seems to be identical to that experienced by you. The code has changed quite a bit from 2.1 to 2.3.1, and we suspect a bug was introduced. > What is going on, seems to be that a writer thread is being terminated, and the destructor for the Ascii writer is called eventually. However, the destructor code does some checks and finds out that proper cleanup has not been done, so it aborts. This does not seem to be due to any library incompatibility, and looks more like maybe a race condition was introduced. > Since you knows the Bro developers, can you please ask them to take a look this and get back to us? We think it requires their expertise at this point. > Thank You, > Hassan. > > Bivio Case Information: > Bivio Case #: 4566243 > Date Created: 9/02/2014 08:02 AM PDT > Stack trace below: > GNU gdb (GDB) Fedora (6.8.50.20090302-40.fc11) Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "ppc-redhat-linux-gnu". > For bug reporting instructions, please see: > ... > backtrace > [New Thread 25501] > [New Thread 25328] > [New Thread 25378] > [New Thread 25379] > [New Thread 25380] > [New Thread 25381] > [New Thread 25382] > [New Thread 25383] > [New Thread 25384] > [New Thread 25385] > [New Thread 25386] > [New Thread 25389] > [New Thread 25442] > warning: Can't read pathname for load map: Input/output error. > Missing separate debuginfo for /usr/local/lib/libz.so.1 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/a2/0a0d1fc0d48c2a303af1417ccc03308b9de04a > Missing separate debuginfo for /usr/local/lib/libtcmalloc.so.4 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/27/eaf56bc64810920d55b9530156c1e8ffbfd43e > Missing separate debuginfo for /usr/local/lib/libcurl.so.4 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/a7/9a2cebb4abc156495ec0806b1c18015c8eba01 > Reading symbols from /usr/lib/libpcap.so.1...done. > Loaded symbols for /usr/lib/libpcap.so.1 Reading symbols from /usr/lib/libssl.so.10...done. > Loaded symbols for /usr/lib/libssl.so.10 Reading symbols from /usr/lib/libcrypto.so.10...done. > Loaded symbols for /usr/lib/libcrypto.so.10 Reading symbols from /usr/lib/libbind.so.4...done. > Loaded symbols for /usr/lib/libbind.so.4 Reading symbols from /usr/local/lib/libz.so.1...done. > Loaded symbols for /usr/local/lib/libz.so.1 Reading symbols from /usr/local/lib/libtcmalloc.so.4...done. > Loaded symbols for /usr/local/lib/libtcmalloc.so.4 Reading symbols from /usr/local/lib/libcurl.so.4...done. > Loaded symbols for /usr/local/lib/libcurl.so.4 Reading symbols from /lib/libpthread.so.0...done. > Loaded symbols for /lib/libpthread.so.0 > Reading symbols from /lib/libdl.so.2...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /usr/lib/libstdc++.so.6...done. > Loaded symbols for /usr/lib/libstdc++.so.6 Reading symbols from /lib/libm.so.6...done. > Loaded symbols for /lib/libm.so.6 > Reading symbols from /lib/libgcc_s.so.1...done. > Loaded symbols for /lib/libgcc_s.so.1 > Reading symbols from /lib/libc.so.6...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libzcp.so...done. > Loaded symbols for /usr/lib/libzcp.so > Reading symbols from /lib/libgssapi_krb5.so.2...done. > Loaded symbols for /lib/libgssapi_krb5.so.2 Reading symbols from /lib/libkrb5.so.3...done. > Loaded symbols for /lib/libkrb5.so.3 > Reading symbols from /lib/libcom_err.so.2...done. > Loaded symbols for /lib/libcom_err.so.2 > Reading symbols from /lib/libk5crypto.so.3...done. > Loaded symbols for /lib/libk5crypto.so.3 Reading symbols from /lib/libresolv.so.2...done. > Loaded symbols for /lib/libresolv.so.2 > Reading symbols from /lib/librt.so.1...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/ld.so.1...done. > Loaded symbols for /lib/ld.so.1 > Reading symbols from /lib/libbvsp.so...done. > Loaded symbols for /lib/libbvsp.so > Reading symbols from /lib/libbcon.so...done. > Loaded symbols for /lib/libbcon.so > Reading symbols from /lib/libkrb5support.so.0...done. > Loaded symbols for /lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. > Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /usr/lib/libxml2.so.2...done. > Loaded symbols for /usr/lib/libxml2.so.2 Reading symbols from /lib/libhmlibs.so...done. > Loaded symbols for /lib/libhmlibs.so > Reading symbols from /lib/libhmolddb.so...done. > Loaded symbols for /lib/libhmolddb.so > Reading symbols from /lib/libcf.so...done. > Loaded symbols for /lib/libcf.so > Reading symbols from /lib/libbvsep.so...done. > Loaded symbols for /lib/libbvsep.so > Reading symbols from /usr/lib/libnrddi.so...done. > Loaded symbols for /usr/lib/libnrddi.so > Reading symbols from /lib/libselinux.so.1...done. > Loaded symbols for /lib/libselinux.so.1 > Core was generated by `/var/tmp/bro/spool/tmp/bro -U .status -p broctl -p broctl-live -p local -p mana'. > Program terminated with signal 6, Aborted. > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. > in ../nptl/sysdeps/unix/sysv/linux/raise.c > Missing separate debuginfos, use: debuginfo-install e2fsprogs-libs-1.41.9-2.fc11.ppc glibc-2.17-4.fc11.ppc keyutils-libs-1.2-5.fc11.ppc krb5-libs-1.9.3-1.fc11.ppc libbind-6.0-1.fc11.ppc libgcc-4.4.1-2.fc11.ppc libselinux-2.0.80-1.fc11.ppc libstdc++-4.4.1-2.fc11.ppc libxml2-2.7.6-1.fc11.ppc openssl-libs-1.0.1e-37.fc11.1.ppc > (gdb) backtrace > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > #1 0x0f6d0de0 in *__GI_abort () at abort.c:90 > #2 0x1024be70 in logging::writer::Ascii::~Ascii (this=0x11a87200, __in_chrg=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/logging/writers/Ascii.cc:186 > #3 0x10236b70 in threading::Manager::Process (this=0x10dae180) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/threading/Manager.cc:171 > #4 0x101a5400 in net_run () at /bivio/scsi/b/levitonl/bro-2.3.1/src/Net.cc:389 > #5 0x100f7554 in main (argc=, argv=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/main.cc:1165 > Current language: auto; currently minimal > (gdb) > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > #1 0x0f6d0de0 in *__GI_abort () at abort.c:90 > #2 0x1024be70 in logging::writer::Ascii::~Ascii (this=0x11a87200, __in_chrg=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/logging/writers/Ascii.cc:186 > #3 0x10236b70 in threading::Manager::Process (this=0x10dae180) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/threading/Manager.cc:171 > #4 0x101a5400 in net_run () at /bivio/scsi/b/levitonl/bro-2.3.1/src/Net.cc:389 > #5 0x100f7554 in main (argc=, argv=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/main.cc:1165 > (gdb) quit -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Wed Oct 29 15:19:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Wed, 29 Oct 2014 17:19:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1253) Bro 2.3 - 2.3.1 manager dieing on Bivio hardware In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1253: ------------------------------- Resolution: Fixed Status: Closed (was: Open) > Bro 2.3 - 2.3.1 manager dieing on Bivio hardware > ------------------------------------------------ > > Key: BIT-1253 > URL: https://bro-tracker.atlassian.net/browse/BIT-1253 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Bro 2.3 and Bro 2.3.1 > bivio hardwareLinux CPU.2.6.31-45 has curl 7.36 gperftools 2.2 flex 2.5.39 bison 3.0.2 libpcap 1.1 swig 2.0.8 > Reporter: Larry Leviton > Assignee: Johanna Amann > Fix For: 2.4 > > > After starting bro up, the bro manager crashes in less than 60 seconds. > Thanks for any help you can give. > Sent stack trace to vendor (at bottom), and here was their response: > Comment(s): Hello Larry, > We have duplicated a crash in our lab setup that seems to be identical to that experienced by you. The code has changed quite a bit from 2.1 to 2.3.1, and we suspect a bug was introduced. > What is going on, seems to be that a writer thread is being terminated, and the destructor for the Ascii writer is called eventually. However, the destructor code does some checks and finds out that proper cleanup has not been done, so it aborts. This does not seem to be due to any library incompatibility, and looks more like maybe a race condition was introduced. > Since you knows the Bro developers, can you please ask them to take a look this and get back to us? We think it requires their expertise at this point. > Thank You, > Hassan. > > Bivio Case Information: > Bivio Case #: 4566243 > Date Created: 9/02/2014 08:02 AM PDT > Stack trace below: > GNU gdb (GDB) Fedora (6.8.50.20090302-40.fc11) Copyright (C) 2009 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "ppc-redhat-linux-gnu". > For bug reporting instructions, please see: > ... > backtrace > [New Thread 25501] > [New Thread 25328] > [New Thread 25378] > [New Thread 25379] > [New Thread 25380] > [New Thread 25381] > [New Thread 25382] > [New Thread 25383] > [New Thread 25384] > [New Thread 25385] > [New Thread 25386] > [New Thread 25389] > [New Thread 25442] > warning: Can't read pathname for load map: Input/output error. > Missing separate debuginfo for /usr/local/lib/libz.so.1 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/a2/0a0d1fc0d48c2a303af1417ccc03308b9de04a > Missing separate debuginfo for /usr/local/lib/libtcmalloc.so.4 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/27/eaf56bc64810920d55b9530156c1e8ffbfd43e > Missing separate debuginfo for /usr/local/lib/libcurl.so.4 > Try: yum --enablerepo='*-debuginfo' install /usr/lib/debug/.build-id/a7/9a2cebb4abc156495ec0806b1c18015c8eba01 > Reading symbols from /usr/lib/libpcap.so.1...done. > Loaded symbols for /usr/lib/libpcap.so.1 Reading symbols from /usr/lib/libssl.so.10...done. > Loaded symbols for /usr/lib/libssl.so.10 Reading symbols from /usr/lib/libcrypto.so.10...done. > Loaded symbols for /usr/lib/libcrypto.so.10 Reading symbols from /usr/lib/libbind.so.4...done. > Loaded symbols for /usr/lib/libbind.so.4 Reading symbols from /usr/local/lib/libz.so.1...done. > Loaded symbols for /usr/local/lib/libz.so.1 Reading symbols from /usr/local/lib/libtcmalloc.so.4...done. > Loaded symbols for /usr/local/lib/libtcmalloc.so.4 Reading symbols from /usr/local/lib/libcurl.so.4...done. > Loaded symbols for /usr/local/lib/libcurl.so.4 Reading symbols from /lib/libpthread.so.0...done. > Loaded symbols for /lib/libpthread.so.0 > Reading symbols from /lib/libdl.so.2...done. > Loaded symbols for /lib/libdl.so.2 > Reading symbols from /usr/lib/libstdc++.so.6...done. > Loaded symbols for /usr/lib/libstdc++.so.6 Reading symbols from /lib/libm.so.6...done. > Loaded symbols for /lib/libm.so.6 > Reading symbols from /lib/libgcc_s.so.1...done. > Loaded symbols for /lib/libgcc_s.so.1 > Reading symbols from /lib/libc.so.6...done. > Loaded symbols for /lib/libc.so.6 > Reading symbols from /usr/lib/libzcp.so...done. > Loaded symbols for /usr/lib/libzcp.so > Reading symbols from /lib/libgssapi_krb5.so.2...done. > Loaded symbols for /lib/libgssapi_krb5.so.2 Reading symbols from /lib/libkrb5.so.3...done. > Loaded symbols for /lib/libkrb5.so.3 > Reading symbols from /lib/libcom_err.so.2...done. > Loaded symbols for /lib/libcom_err.so.2 > Reading symbols from /lib/libk5crypto.so.3...done. > Loaded symbols for /lib/libk5crypto.so.3 Reading symbols from /lib/libresolv.so.2...done. > Loaded symbols for /lib/libresolv.so.2 > Reading symbols from /lib/librt.so.1...done. > Loaded symbols for /lib/librt.so.1 > Reading symbols from /lib/ld.so.1...done. > Loaded symbols for /lib/ld.so.1 > Reading symbols from /lib/libbvsp.so...done. > Loaded symbols for /lib/libbvsp.so > Reading symbols from /lib/libbcon.so...done. > Loaded symbols for /lib/libbcon.so > Reading symbols from /lib/libkrb5support.so.0...done. > Loaded symbols for /lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...done. > Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /usr/lib/libxml2.so.2...done. > Loaded symbols for /usr/lib/libxml2.so.2 Reading symbols from /lib/libhmlibs.so...done. > Loaded symbols for /lib/libhmlibs.so > Reading symbols from /lib/libhmolddb.so...done. > Loaded symbols for /lib/libhmolddb.so > Reading symbols from /lib/libcf.so...done. > Loaded symbols for /lib/libcf.so > Reading symbols from /lib/libbvsep.so...done. > Loaded symbols for /lib/libbvsep.so > Reading symbols from /usr/lib/libnrddi.so...done. > Loaded symbols for /usr/lib/libnrddi.so > Reading symbols from /lib/libselinux.so.1...done. > Loaded symbols for /lib/libselinux.so.1 > Core was generated by `/var/tmp/bro/spool/tmp/bro -U .status -p broctl -p broctl-live -p local -p mana'. > Program terminated with signal 6, Aborted. > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. > in ../nptl/sysdeps/unix/sysv/linux/raise.c > Missing separate debuginfos, use: debuginfo-install e2fsprogs-libs-1.41.9-2.fc11.ppc glibc-2.17-4.fc11.ppc keyutils-libs-1.2-5.fc11.ppc krb5-libs-1.9.3-1.fc11.ppc libbind-6.0-1.fc11.ppc libgcc-4.4.1-2.fc11.ppc libselinux-2.0.80-1.fc11.ppc libstdc++-4.4.1-2.fc11.ppc libxml2-2.7.6-1.fc11.ppc openssl-libs-1.0.1e-37.fc11.1.ppc > (gdb) backtrace > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > #1 0x0f6d0de0 in *__GI_abort () at abort.c:90 > #2 0x1024be70 in logging::writer::Ascii::~Ascii (this=0x11a87200, __in_chrg=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/logging/writers/Ascii.cc:186 > #3 0x10236b70 in threading::Manager::Process (this=0x10dae180) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/threading/Manager.cc:171 > #4 0x101a5400 in net_run () at /bivio/scsi/b/levitonl/bro-2.3.1/src/Net.cc:389 > #5 0x100f7554 in main (argc=, argv=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/main.cc:1165 > Current language: auto; currently minimal > (gdb) > #0 0x0f6cf01c in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 > #1 0x0f6d0de0 in *__GI_abort () at abort.c:90 > #2 0x1024be70 in logging::writer::Ascii::~Ascii (this=0x11a87200, __in_chrg=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/logging/writers/Ascii.cc:186 > #3 0x10236b70 in threading::Manager::Process (this=0x10dae180) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/threading/Manager.cc:171 > #4 0x101a5400 in net_run () at /bivio/scsi/b/levitonl/bro-2.3.1/src/Net.cc:389 > #5 0x100f7554 in main (argc=, argv=) > at /bivio/scsi/b/levitonl/bro-2.3.1/src/main.cc:1165 > (gdb) quit -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From robin at icir.org Wed Oct 29 20:35:12 2014 From: robin at icir.org (Robin Sommer) Date: Wed, 29 Oct 2014 20:35:12 -0700 Subject: [Bro-Dev] Help Troubleshooting a Perftools Memleak In-Reply-To: References: Message-ID: <20141030033512.GA1488@icir.org> On Wed, Oct 29, 2014 at 16:48 -0400, you wrote: > Digging a little deeper shows that two of these leaks are in RE_parse > (re-parse.y:110 and re-parse.y:133), one is in re__scan_buffer > (re-scan.cc:2035), and one is in re__scan_bytes (re-scan.cc::2084). I'm pretty sure I know where that's coming from: normally regexp compilation is happening before Bro's main loop, and hence not accounted for by the leak checking. BinPAC however delays initialization until the first time it tries to match something. Unfortunately the obvious fix doesn't work. I'm pasting it in below, but that change lets Bro crash because it now depends on the order in which global constructors run. If anybody has a better idea, let me know. Robin diff --git a/lib/binpac_regex.h b/lib/binpac_regex.h index b41e6db..7acc2c6 100644 --- a/lib/binpac_regex.h +++ b/lib/binpac_regex.h @@ -14,7 +14,8 @@ public: RegExMatcher(const char *pattern) : pattern_(pattern) { - re_matcher_ = 0; + re_matcher_ = new RE_Matcher(pattern_.c_str()); + re_matcher_->Compile(); } ~RegExMatcher() @@ -25,11 +26,6 @@ public: // Returns the length of longest match, or -1 on mismatch. int MatchPrefix(const_byteptr data, int len) { - if ( ! re_matcher_ ) - { - re_matcher_ = new RE_Matcher(pattern_.c_str()); - re_matcher_->Compile(); - } return re_matcher_->MatchPrefix(data, len); } -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From noreply at bro.org Thu Oct 30 00:00:23 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 30 Oct 2014 00:00:23 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410300700.s9U70NSe016276@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ----------------------------------- BIT-1280 [1] Bro Seth Hall - 2014-10-28 2.4 Normal Checking index in vectors is broken [1] BIT-1280 https://bro-tracker.atlassian.net/browse/BIT-1280 From jsiwek at illinois.edu Thu Oct 30 09:09:25 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 30 Oct 2014 16:09:25 +0000 Subject: [Bro-Dev] Help Troubleshooting a Perftools Memleak In-Reply-To: <20141030033512.GA1488@icir.org> References: <20141030033512.GA1488@icir.org> Message-ID: <4A7F30E3-42FD-4B42-B36A-8D0BFA16A0E6@illinois.edu> > On Oct 29, 2014, at 10:35 PM, Robin Sommer wrote: > > Unfortunately the obvious fix doesn't work. I'm pasting it in below, > but that change lets Bro crash because it now depends on the order in > which global constructors run. If anybody has a better idea, let me > know. Extending your idea a bit and probably hacking it into a place originally intended for something else. It may be a breaking change for other applications besides Bro, depending on how they define their RE_Matcher::Compile(), but not hard to fix. diff --git a/lib/binpac_regex.h b/lib/binpac_regex.h index b41e6db..ce5e7e3 100644 --- a/lib/binpac_regex.h +++ b/lib/binpac_regex.h @@ -14,7 +14,8 @@ public: RegExMatcher(const char *pattern) : pattern_(pattern) { - re_matcher_ = 0; + re_matcher_ = new RE_Matcher(pattern_.c_str()); + re_matcher_->Compile(1); } ~RegExMatcher() @@ -25,11 +26,6 @@ public: // Returns the length of longest match, or -1 on mismatch. int MatchPrefix(const_byteptr data, int len) { - if ( ! re_matcher_ ) - { - re_matcher_ = new RE_Matcher(pattern_.c_str()); - re_matcher_->Compile(); - } return re_matcher_->MatchPrefix(data, len); } diff --git a/src/RE.cc b/src/RE.cc index 4855b0e..8aecbed 100644 --- a/src/RE.cc +++ b/src/RE.cc @@ -426,8 +426,27 @@ void RE_Matcher::AddPat(const char* new_pat) re_exact->AddPat(new_pat); } +std::vector uncompiled_re_matchers; + +int init_uncompiled_re_matchers() + { + int rval = 1; + + for ( size_t i = 0; i < uncompiled_re_matchers.size(); ++i ) + if ( ! uncompiled_re_matchers[i]->Compile() ) + rval = 0; + + return rval; + } + int RE_Matcher::Compile(int lazy) { + if ( lazy ) + { + uncompiled_re_matchers.push_back(this); + return 1; + } + return re_anywhere->Compile(lazy) && re_exact->Compile(lazy); } diff --git a/src/RE.h b/src/RE.h index 7437dbb..dc9f881 100644 --- a/src/RE.h +++ b/src/RE.h @@ -11,6 +11,7 @@ #include #include +#include #include typedef int (*cce_func)(int); @@ -224,6 +225,10 @@ protected: Specific_RE_Matcher* re_exact; }; +extern std::vector uncompiled_re_matchers; + +int init_uncompiled_re_matchers(); + declare(PList, RE_Matcher); typedef PList(RE_Matcher) re_matcher_list; diff --git a/src/main.cc b/src/main.cc index 63949c5..a2be76d 100644 --- a/src/main.cc +++ b/src/main.cc @@ -775,6 +775,9 @@ int main(int argc, char** argv) // DEBUG_MSG("HMAC key: %s\n", md5_digest_print(shared_hmac_md5_key)); init_hash_function(); + if ( ! init_uncompiled_re_matchers() ) + reporter->Error("Failed to lazy-compile regex matchers."); + ERR_load_crypto_strings(); OPENSSL_add_all_algorithms_conf(); SSL_library_init(); From jira at bro-tracker.atlassian.net Thu Oct 30 10:27:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 30 Oct 2014 12:27:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1176) Using an undefined function in a when statement causes a segfault In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18610#comment-18610 ] Jon Siwek commented on BIT-1176: -------------------------------- What's the expected behavior? It doesn't look like something easily detected at parse-time, so ideas: (1) fatal error message at run-time with a core dump should the code ever be executed (which may be infrequently if it's buried within complicated logic). (2) nonfatal error message at run-time, but the when body can still be triggered as normal if the RHS is ever assigned a value. topic/jsiwek/bit-1176 does (2). If that seems ok, please flip ticket to a merge request. > Using an undefined function in a when statement causes a segfault > ----------------------------------------------------------------- > > Key: BIT-1176 > URL: https://bro-tracker.atlassian.net/browse/BIT-1176 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Johanna Amann > Fix For: 2.4 > > Attachments: crashme.bro > > > Running the following script crashes bro with a null-pointer exception: > {code:title=crashMe.bro} > global crashMe: function():string; > when( local result = crashMe() ) { > print result; > } > {code} > Backtrace: > {code} > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > 320 > 321 int Val::IsZero() const > 322 { > -> 323 switch ( type->InternalType() ) { > 324 case TYPE_INTERNAL_INT: return val.int_val == 0; > 325 case TYPE_INTERNAL_UNSIGNED: return val.uint_val == 0; > 326 case TYPE_INTERNAL_DOUBLE: return val.double_val == 0.0; > (lldb) bt > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > * frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > frame #1: 0x000000010020b452 bro`Trigger::Eval(this=0x0000000105d45d60) + 578 at Trigger.cc:209 > frame #2: 0x000000010020ae95 bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 1285 at Trigger.cc:140 > frame #3: 0x000000010020a98a bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 106 at Trigger.cc:147 > frame #4: 0x000000010020566f bro`WhenStmt::Exec(this=0x0000000104a00900, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 239 at Stmt.cc:2041 > frame #5: 0x0000000100203204 bro`StmtList::Exec(this=0x00000001049fbe80, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 228 at Stmt.cc:1639 > frame #6: 0x000000010003d244 bro`main(argc=2, argv=0x00007fff5fbffa40) + 15476 at main.cc:1116 > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 30 10:51:08 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Thu, 30 Oct 2014 12:51:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1176) Using an undefined function in a when statement causes a segfault In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1176: ------------------------------- Status: Merge Request (was: Open) > Using an undefined function in a when statement causes a segfault > ----------------------------------------------------------------- > > Key: BIT-1176 > URL: https://bro-tracker.atlassian.net/browse/BIT-1176 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Johanna Amann > Fix For: 2.4 > > Attachments: crashme.bro > > > Running the following script crashes bro with a null-pointer exception: > {code:title=crashMe.bro} > global crashMe: function():string; > when( local result = crashMe() ) { > print result; > } > {code} > Backtrace: > {code} > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > 320 > 321 int Val::IsZero() const > 322 { > -> 323 switch ( type->InternalType() ) { > 324 case TYPE_INTERNAL_INT: return val.int_val == 0; > 325 case TYPE_INTERNAL_UNSIGNED: return val.uint_val == 0; > 326 case TYPE_INTERNAL_DOUBLE: return val.double_val == 0.0; > (lldb) bt > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > * frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > frame #1: 0x000000010020b452 bro`Trigger::Eval(this=0x0000000105d45d60) + 578 at Trigger.cc:209 > frame #2: 0x000000010020ae95 bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 1285 at Trigger.cc:140 > frame #3: 0x000000010020a98a bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 106 at Trigger.cc:147 > frame #4: 0x000000010020566f bro`WhenStmt::Exec(this=0x0000000104a00900, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 239 at Stmt.cc:2041 > frame #5: 0x0000000100203204 bro`StmtList::Exec(this=0x00000001049fbe80, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 228 at Stmt.cc:1639 > frame #6: 0x000000010003d244 bro`main(argc=2, argv=0x00007fff5fbffa40) + 15476 at main.cc:1116 > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 30 10:51:08 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Thu, 30 Oct 2014 12:51:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1176) Using an undefined function in a when statement causes a segfault In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18611#comment-18611 ] Johanna Amann commented on BIT-1176: ------------------------------------ (2) is fine from my point of view, flipping to merge request. Thank you. > Using an undefined function in a when statement causes a segfault > ----------------------------------------------------------------- > > Key: BIT-1176 > URL: https://bro-tracker.atlassian.net/browse/BIT-1176 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Johanna Amann > Fix For: 2.4 > > Attachments: crashme.bro > > > Running the following script crashes bro with a null-pointer exception: > {code:title=crashMe.bro} > global crashMe: function():string; > when( local result = crashMe() ) { > print result; > } > {code} > Backtrace: > {code} > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > 320 > 321 int Val::IsZero() const > 322 { > -> 323 switch ( type->InternalType() ) { > 324 case TYPE_INTERNAL_INT: return val.int_val == 0; > 325 case TYPE_INTERNAL_UNSIGNED: return val.uint_val == 0; > 326 case TYPE_INTERNAL_DOUBLE: return val.double_val == 0.0; > (lldb) bt > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > * frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > frame #1: 0x000000010020b452 bro`Trigger::Eval(this=0x0000000105d45d60) + 578 at Trigger.cc:209 > frame #2: 0x000000010020ae95 bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 1285 at Trigger.cc:140 > frame #3: 0x000000010020a98a bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 106 at Trigger.cc:147 > frame #4: 0x000000010020566f bro`WhenStmt::Exec(this=0x0000000104a00900, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 239 at Stmt.cc:2041 > frame #5: 0x0000000100203204 bro`StmtList::Exec(this=0x00000001049fbe80, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 228 at Stmt.cc:1639 > frame #6: 0x000000010003d244 bro`main(argc=2, argv=0x00007fff5fbffa40) + 15476 at main.cc:1116 > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 30 10:52:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Thu, 30 Oct 2014 12:52:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1176) Using an undefined function in a when statement causes a segfault In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1176: ------------------------------- Assignee: Robin Sommer > Using an undefined function in a when statement causes a segfault > ----------------------------------------------------------------- > > Key: BIT-1176 > URL: https://bro-tracker.atlassian.net/browse/BIT-1176 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Johanna Amann > Assignee: Robin Sommer > Fix For: 2.4 > > Attachments: crashme.bro > > > Running the following script crashes bro with a null-pointer exception: > {code:title=crashMe.bro} > global crashMe: function():string; > when( local result = crashMe() ) { > print result; > } > {code} > Backtrace: > {code} > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > 320 > 321 int Val::IsZero() const > 322 { > -> 323 switch ( type->InternalType() ) { > 324 case TYPE_INTERNAL_INT: return val.int_val == 0; > 325 case TYPE_INTERNAL_UNSIGNED: return val.uint_val == 0; > 326 case TYPE_INTERNAL_DOUBLE: return val.double_val == 0.0; > (lldb) bt > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > * frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > frame #1: 0x000000010020b452 bro`Trigger::Eval(this=0x0000000105d45d60) + 578 at Trigger.cc:209 > frame #2: 0x000000010020ae95 bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 1285 at Trigger.cc:140 > frame #3: 0x000000010020a98a bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 106 at Trigger.cc:147 > frame #4: 0x000000010020566f bro`WhenStmt::Exec(this=0x0000000104a00900, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 239 at Stmt.cc:2041 > frame #5: 0x0000000100203204 bro`StmtList::Exec(this=0x00000001049fbe80, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 228 at Stmt.cc:1639 > frame #6: 0x000000010003d244 bro`main(argc=2, argv=0x00007fff5fbffa40) + 15476 at main.cc:1116 > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 30 11:41:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 30 Oct 2014 13:41:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-949) High CPU from polling loop on low traffic links In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18612#comment-18612 ] Jon Siwek commented on BIT-949: ------------------------------- I think the main problem people were seeing here is resolved w/ the merging of BIT-1260. If not, please re-open. > High CPU from polling loop on low traffic links > ----------------------------------------------- > > Key: BIT-949 > URL: https://bro-tracker.atlassian.net/browse/BIT-949 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: liamrandall > Priority: Low > Fix For: 2.4 > > > In low or no traffic links Bro consumes a large amount of the CPU. Bro has a core processing loop that it needs to do regularly which involves checking for packets from interfaces, checking for communication traffic (in the case of a cluster of Bro processes), processing scheduled events, etc. It's this processing loop that is causing the CPU utilization when there is no traffic. > Moving Bro to standalone mode will reduce the CPU load on sensors with out traffic. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 30 11:42:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 30 Oct 2014 13:42:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-949) High CPU from polling loop on low traffic links In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-949: -------------------------- Resolution: Fixed Status: Closed (was: Open) > High CPU from polling loop on low traffic links > ----------------------------------------------- > > Key: BIT-949 > URL: https://bro-tracker.atlassian.net/browse/BIT-949 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: liamrandall > Priority: Low > Fix For: 2.4 > > > In low or no traffic links Bro consumes a large amount of the CPU. Bro has a core processing loop that it needs to do regularly which involves checking for packets from interfaces, checking for communication traffic (in the case of a cluster of Bro processes), processing scheduled events, etc. It's this processing loop that is causing the CPU utilization when there is no traffic. > Moving Bro to standalone mode will reduce the CPU load on sensors with out traffic. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 30 12:07:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 30 Oct 2014 14:07:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-316) Bro integer type cleanup meta ticket In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-316: -------------------------- Resolution: Fixed Status: Closed (was: Open) Shouldn't need a meta ticket to track this anymore -- file separate tickets for specific concerns if ones don't already exist. > Bro integer type cleanup meta ticket > ------------------------------------ > > Key: BIT-316 > URL: https://bro-tracker.atlassian.net/browse/BIT-316 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro > Reporter: gregor > Labels: inttypes > Fix For: 2.4 > > > {noformat} > #!rst > This is a meta-ticket for the integer type cleanup. I.e., > All thickets related to this should have the keyword ``inttypes``! > These tickets can be found through Report `{13}`:trac: > * Changing to 64 bit integeres by default were it makes sense. > * Find places that might overflow and change to wider types > * Use standard integer types (``inttypes.h``) > * Find and change type inconsitencies > * ... > Setting milestone to Bro 1.6 but can also be pushed to 1.7 for some of the issues. > {noformat} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 30 12:19:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 30 Oct 2014 14:19:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-924) String BIFs Return 1-indexed string_arrays In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18614#comment-18614 ] Jon Siwek commented on BIT-924: ------------------------------- Would be nice to change index to start at 0 for consistency, but maybe not worth suddenly breaking existing code that relies on the 1 based indexing. Maybe a safe approach is to just make sure these are well-documented to warn that it's 1 based, then implement alternate versions under slightly different names (e.g. suffix with a '0') and return 0 based values. Could also think about removing the old versions, but just changing them in-place seems hard to warn users about. > String BIFs Return 1-indexed string_arrays > ------------------------------------------ > > Key: BIT-924 > URL: https://bro-tracker.atlassian.net/browse/BIT-924 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Fix For: 2.4 > > > The following BIFs return 1-indexed string_arrays: > * sort_string_array > * split > * split1 > * split_all > * split_n -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 30 20:01:07 2014 From: jira at bro-tracker.atlassian.net (AK (JIRA)) Date: Thu, 30 Oct 2014 22:01:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1283) Bro crashes when using &encrypt In-Reply-To: References: Message-ID: AK created BIT-1283: ----------------------- Summary: Bro crashes when using &encrypt Key: BIT-1283 URL: https://bro-tracker.atlassian.net/browse/BIT-1283 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Environment: bro version 2.3-263-debug Reporter: AK Bro crashes when applying the &encrypt attribute when opening a file. bro -Ci eth0 -e 'global f1: file = open("f.out") &encrypt;' -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Thu Oct 30 20:21:07 2014 From: jira at bro-tracker.atlassian.net (AK (JIRA)) Date: Thu, 30 Oct 2014 22:21:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1284) Cannot use variables as timeout intervals In-Reply-To: References: Message-ID: AK created BIT-1284: ----------------------- Summary: Cannot use variables as timeout intervals Key: BIT-1284 URL: https://bro-tracker.atlassian.net/browse/BIT-1284 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Reporter: AK Intuitively, scriptland should allow variables as timeout intervals. Compare the following: bro -e 'global t: interval = 2secs; when(1==1){print "1 is 1";} timeout 2secs {print "timed out";}' bro -e 'global t: interval = 2secs; when(1==1){print "1 is 1";} timeout t {print "timed out";}' -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From noreply at bro.org Fri Oct 31 00:00:33 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 31 Oct 2014 00:00:33 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201410310700.s9V70XgE022961@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ------------ ---------- ------------- ---------- ----------------------------------------------------------------- BIT-1280 [1] Bro Seth Hall - 2014-10-28 2.4 Normal Checking index in vectors is broken BIT-1176 [2] Bro Johanna Amann Robin Sommer 2014-10-30 2.4 Normal Using an undefined function in a when statement causes a segfault [1] BIT-1280 https://bro-tracker.atlassian.net/browse/BIT-1280 [2] BIT-1176 https://bro-tracker.atlassian.net/browse/BIT-1176 From jira at bro-tracker.atlassian.net Fri Oct 31 07:24:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 31 Oct 2014 09:24:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1166) installation does not take place in given prefix entirely In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1166: --------------------------- Status: Closed (was: Reopened) > installation does not take place in given prefix entirely > --------------------------------------------------------- > > Key: BIT-1166 > URL: https://bro-tracker.atlassian.net/browse/BIT-1166 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, BroControl > Affects Versions: git/master > Reporter: Matthias Vallentin > Labels: build > Fix For: 2.4 > > > When configuring Bro to remain in a given prefix, say {{/opt/bro}}, the installation of BroControl still attempts to create a spool directory outside of the prefix: > {code} > ./configure --prefix=/opt/bro > make > make install > [...] > CMake Error at aux/broctl/cmake_install.cmake:200 (FILE): > file cannot create directory: /var/opt/bro/spool. Maybe need > administrative privileges. > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 07:59:07 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Fri, 31 Oct 2014 09:59:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-924) String BIFs Return 1-indexed string_arrays In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18615#comment-18615 ] grigorescu commented on BIT-924: -------------------------------- I think more generally, we want a good way to be able to make breaking changes to BIFs and the base scripts, which can be thought of as an API. As more and more people are writing custom scripts and depending on certain behavior, this will be harder to change. At the same time, we don't want to handcuff ourselves from making these changes if needed. A couple of things we could do are: * Have a special section in NEWS about changes that break current functionality * We could have depreciation warnings in reporter.log, but people don't often check this. Maybe we need a deprecated.log that scripts/BIFs that are deprecated will write to? We could also elevated this by printing something in stdout/stderr (e.g. "Warning - your Bro instance is using deprecated features that will be removed in a future release. Please check deprecated.log for details.") With this kind of notification, I could see a staged approach to "fix" these string functions (using split as an example, obviously extended to all the functions above): # Create split_0 and deprecate split # Rename split_0 to split, create a deprecated split_1 that implements the old functionality # Remove split_1 Of course, if we're comfortable with the amount of notification users would get, we could shorten this as needed. > String BIFs Return 1-indexed string_arrays > ------------------------------------------ > > Key: BIT-924 > URL: https://bro-tracker.atlassian.net/browse/BIT-924 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Fix For: 2.4 > > > The following BIFs return 1-indexed string_arrays: > * sort_string_array > * split > * split1 > * split_all > * split_n -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 08:34:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 31 Oct 2014 10:34:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-924) String BIFs Return 1-indexed string_arrays In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18616#comment-18616 ] Jon Siwek commented on BIT-924: ------------------------------- {quote} I think more generally, we want a good way to be able to make breaking changes to BIFs and the base scripts, which can be thought of as an API. {quote} Agreed. {quote} With this kind of notification, I could see a staged approach to "fix" these string functions (using split as an example, obviously extended to all the functions above): Create split_0 and deprecate split Rename split_0 to split, create a deprecated split_1 that implements the old functionality Remove split_1 {quote} Is it reasonable to expect a user to never "skip a release version" ? Because if they did, they may miss a deprecation warning and just be silently using the breaking change. But yeah, maybe the best that can be done here to clearly indicate in NEWS which breaking changes a user will need to audit their code for and also provide helpful hints in the form of deprecated usage warnings for a limited time. > String BIFs Return 1-indexed string_arrays > ------------------------------------------ > > Key: BIT-924 > URL: https://bro-tracker.atlassian.net/browse/BIT-924 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Fix For: 2.4 > > > The following BIFs return 1-indexed string_arrays: > * sort_string_array > * split > * split1 > * split_all > * split_n -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 08:41:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 31 Oct 2014 10:41:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1284) Cannot use variables as timeout intervals In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1284: --------------------------- Fix Version/s: 2.4 > Cannot use variables as timeout intervals > ----------------------------------------- > > Key: BIT-1284 > URL: https://bro-tracker.atlassian.net/browse/BIT-1284 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: AK > Fix For: 2.4 > > > Intuitively, scriptland should allow variables as timeout intervals. Compare the following: > bro -e 'global t: interval = 2secs; when(1==1){print "1 is 1";} timeout 2secs {print "timed out";}' > bro -e 'global t: interval = 2secs; when(1==1){print "1 is 1";} timeout t {print "timed out";}' -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 08:42:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 31 Oct 2014 10:42:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1284) Cannot use variables as timeout intervals In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1284: --------------------------- Resolution: Fixed Status: Closed (was: Open) Thanks, fixed in master. > Cannot use variables as timeout intervals > ----------------------------------------- > > Key: BIT-1284 > URL: https://bro-tracker.atlassian.net/browse/BIT-1284 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: AK > Fix For: 2.4 > > > Intuitively, scriptland should allow variables as timeout intervals. Compare the following: > bro -e 'global t: interval = 2secs; when(1==1){print "1 is 1";} timeout 2secs {print "timed out";}' > bro -e 'global t: interval = 2secs; when(1==1){print "1 is 1";} timeout t {print "timed out";}' -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 08:46:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 31 Oct 2014 10:46:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1166) installation does not take place in given prefix entirely In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1166: --------------------------- Status: Reopened (was: Closed) > installation does not take place in given prefix entirely > --------------------------------------------------------- > > Key: BIT-1166 > URL: https://bro-tracker.atlassian.net/browse/BIT-1166 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, BroControl > Affects Versions: git/master > Reporter: Matthias Vallentin > Labels: build > Fix For: 2.4 > > > When configuring Bro to remain in a given prefix, say {{/opt/bro}}, the installation of BroControl still attempts to create a spool directory outside of the prefix: > {code} > ./configure --prefix=/opt/bro > make > make install > [...] > CMake Error at aux/broctl/cmake_install.cmake:200 (FILE): > file cannot create directory: /var/opt/bro/spool. Maybe need > administrative privileges. > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 08:46:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 31 Oct 2014 10:46:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1166) installation does not take place in given prefix entirely In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1166: --------------------------- Resolution: Fixed (was: Cannot Reproduce) Status: Closed (was: Reopened) > installation does not take place in given prefix entirely > --------------------------------------------------------- > > Key: BIT-1166 > URL: https://bro-tracker.atlassian.net/browse/BIT-1166 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, BroControl > Affects Versions: git/master > Reporter: Matthias Vallentin > Labels: build > Fix For: 2.4 > > > When configuring Bro to remain in a given prefix, say {{/opt/bro}}, the installation of BroControl still attempts to create a spool directory outside of the prefix: > {code} > ./configure --prefix=/opt/bro > make > make install > [...] > CMake Error at aux/broctl/cmake_install.cmake:200 (FILE): > file cannot create directory: /var/opt/bro/spool. Maybe need > administrative privileges. > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 10:17:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 31 Oct 2014 12:17:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1283) Bro crashes when using &encrypt In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1283: --------------------------- Resolution: Fixed Fix Version/s: 2.4 Status: Closed (was: Open) Fixed in master. But missing test case since I couldn't quickly come up with a way to reverse the operation via command line or script: seems like one may need to write a small program to parse the file header and then use the openssl envelope api to do the asymmetric decryption. > Bro crashes when using &encrypt > ------------------------------- > > Key: BIT-1283 > URL: https://bro-tracker.atlassian.net/browse/BIT-1283 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: bro version 2.3-263-debug > Reporter: AK > Fix For: 2.4 > > > Bro crashes when applying the &encrypt attribute when opening a file. > bro -Ci eth0 -e 'global f1: file = open("f.out") &encrypt;' -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 13:52:07 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Fri, 31 Oct 2014 15:52:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1285) MySQL Protocol Analyzer In-Reply-To: References: Message-ID: grigorescu created BIT-1285: ------------------------------- Summary: MySQL Protocol Analyzer Key: BIT-1285 URL: https://bro-tracker.atlassian.net/browse/BIT-1285 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: git/master Reporter: grigorescu topic/vladg/mysql is ready to be merged. Note: memleak btest core.leaks.mysql is currently failing due to an issue with how regexes are initialized. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 14:02:07 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Fri, 31 Oct 2014 16:02:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1285) MySQL Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] grigorescu updated BIT-1285: ---------------------------- Status: Merge Request (was: Open) > MySQL Protocol Analyzer > ----------------------- > > Key: BIT-1285 > URL: https://bro-tracker.atlassian.net/browse/BIT-1285 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > > topic/vladg/mysql is ready to be merged. > Note: memleak btest core.leaks.mysql is currently failing due to an issue with how regexes are initialized. -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 16:24:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Oct 2014 18:24:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1280) Checking index in vectors is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1280: --------------------------------- Assignee: Robin Sommer > Checking index in vectors is broken > ----------------------------------- > > Key: BIT-1280 > URL: https://bro-tracker.atlassian.net/browse/BIT-1280 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Assignee: Robin Sommer > Fix For: 2.4 > > > If you try to check an index in a vector for existence, you get an error... > {noformat} > event bro_init() > { > local vec: vector of count = vector(); > if ( 2 in vec ) > print vec[2]; > print vec; > } > {noformat} > Error: > {quote} > fatal error in bool: BroType::AsVectorType (bool/vector) (bool) > {quote} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 16:28:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Oct 2014 18:28:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1280) Checking index in vectors is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18619#comment-18619 ] Robin Sommer commented on BIT-1280: ----------------------------------- Merging. Generally, I would actually prefer vector elements to be automatically initialized with null values corresponding to the vector's type; then the "in" would always return true. However, we don't have the concept of a type-specific null/default values. > Checking index in vectors is broken > ----------------------------------- > > Key: BIT-1280 > URL: https://bro-tracker.atlassian.net/browse/BIT-1280 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Assignee: Robin Sommer > Fix For: 2.4 > > > If you try to check an index in a vector for existence, you get an error... > {noformat} > event bro_init() > { > local vec: vector of count = vector(); > if ( 2 in vec ) > print vec[2]; > print vec; > } > {noformat} > Error: > {quote} > fatal error in bool: BroType::AsVectorType (bool/vector) (bool) > {quote} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 16:35:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Oct 2014 18:35:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1283) Bro crashes when using &encrypt In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18620#comment-18620 ] Robin Sommer commented on BIT-1283: ----------------------------------- Bro 1.5 came with a tool bdcat that decrypts these files. I'm reopening the ticket to see if we want to bring that back. > Bro crashes when using &encrypt > ------------------------------- > > Key: BIT-1283 > URL: https://bro-tracker.atlassian.net/browse/BIT-1283 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: bro version 2.3-263-debug > Reporter: AK > Fix For: 2.4 > > > Bro crashes when applying the &encrypt attribute when opening a file. > bro -Ci eth0 -e 'global f1: file = open("f.out") &encrypt;' -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 16:35:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Oct 2014 18:35:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1283) Bro crashes when using &encrypt In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1283: ------------------------------ Status: Reopened (was: Closed) > Bro crashes when using &encrypt > ------------------------------- > > Key: BIT-1283 > URL: https://bro-tracker.atlassian.net/browse/BIT-1283 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: bro version 2.3-263-debug > Reporter: AK > Fix For: 2.4 > > > Bro crashes when applying the &encrypt attribute when opening a file. > bro -Ci eth0 -e 'global f1: file = open("f.out") &encrypt;' -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From robin at icir.org Fri Oct 31 16:43:38 2014 From: robin at icir.org (Robin Sommer) Date: Fri, 31 Oct 2014 16:43:38 -0700 Subject: [Bro-Dev] [JIRA] (BIT-924) String BIFs Return 1-indexed string_arrays In-Reply-To: References: Message-ID: <20141031234338.GA67664@icir.org> Yeah, that sounds good. We could also add a command line flag that statically checks for use of deprecated functionality so that people don't only see it during runtime. A slightly alternative approach would be CMake's policies, where one can set (and eventually switch) a default value. But I've always found that confusing because it's not just the cmake's version that determines what's acceptable, but also the policy setting. From jira at bro-tracker.atlassian.net Fri Oct 31 16:45:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Oct 2014 18:45:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-924) String BIFs Return 1-indexed string_arrays In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18621#comment-18621 ] Robin Sommer commented on BIT-924: ---------------------------------- Yeah, that sounds good. We could also add a command line flag that statically checks for use of deprecated functionality so that people don't only see it during runtime. A slightly alternative approach would be CMake's policies, where one can set (and eventually switch) a default value. But I've always found that confusing because it's not just the cmake's version that determines what's acceptable, but also the policy setting. > String BIFs Return 1-indexed string_arrays > ------------------------------------------ > > Key: BIT-924 > URL: https://bro-tracker.atlassian.net/browse/BIT-924 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Fix For: 2.4 > > > The following BIFs return 1-indexed string_arrays: > * sort_string_array > * split > * split1 > * split_all > * split_n -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From robin at icir.org Fri Oct 31 17:48:11 2014 From: robin at icir.org (Robin Sommer) Date: Fri, 31 Oct 2014 17:48:11 -0700 Subject: [Bro-Dev] Help Troubleshooting a Perftools Memleak In-Reply-To: <4A7F30E3-42FD-4B42-B36A-8D0BFA16A0E6@illinois.edu> References: <20141030033512.GA1488@icir.org> <4A7F30E3-42FD-4B42-B36A-8D0BFA16A0E6@illinois.edu> Message-ID: <20141101004811.GB67664@icir.org> On Thu, Oct 30, 2014 at 16:09 +0000, you wrote: > Extending your idea a bit and probably hacking it into a place > originally intended for something else. It may be a breaking change > for other applications besides Bro, depending on how they define their > RE_Matcher::Compile(), but not hard to fix. I'm using your code but calling it slightly different via a new binpac::init() function. Not sure it's much better, but a tiny bit maybe. :) I've pushed the change into master but have actually not tried yet if it indeed fixes the reported memleak. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Fri Oct 31 18:32:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Oct 2014 20:32:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1280) Checking index in vectors is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1280: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) . > Checking index in vectors is broken > ----------------------------------- > > Key: BIT-1280 > URL: https://bro-tracker.atlassian.net/browse/BIT-1280 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Assignee: Robin Sommer > Fix For: 2.4 > > > If you try to check an index in a vector for existence, you get an error... > {noformat} > event bro_init() > { > local vec: vector of count = vector(); > if ( 2 in vec ) > print vec[2]; > print vec; > } > {noformat} > Error: > {quote} > fatal error in bool: BroType::AsVectorType (bool/vector) (bool) > {quote} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 18:32:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Oct 2014 20:32:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1176) Using an undefined function in a when statement causes a segfault In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1176: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Using an undefined function in a when statement causes a segfault > ----------------------------------------------------------------- > > Key: BIT-1176 > URL: https://bro-tracker.atlassian.net/browse/BIT-1176 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Johanna Amann > Assignee: Robin Sommer > Fix For: 2.4 > > Attachments: crashme.bro > > > Running the following script crashes bro with a null-pointer exception: > {code:title=crashMe.bro} > global crashMe: function():string; > when( local result = crashMe() ) { > print result; > } > {code} > Backtrace: > {code} > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > 320 > 321 int Val::IsZero() const > 322 { > -> 323 switch ( type->InternalType() ) { > 324 case TYPE_INTERNAL_INT: return val.int_val == 0; > 325 case TYPE_INTERNAL_UNSIGNED: return val.uint_val == 0; > 326 case TYPE_INTERNAL_DOUBLE: return val.double_val == 0.0; > (lldb) bt > * thread #1: tid = 0x226111, 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x30) > * frame #0: 0x000000010022bddf bro`Val::IsZero(this=0x0000000000000000) const + 15 at Val.cc:323 > frame #1: 0x000000010020b452 bro`Trigger::Eval(this=0x0000000105d45d60) + 578 at Trigger.cc:209 > frame #2: 0x000000010020ae95 bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 1285 at Trigger.cc:140 > frame #3: 0x000000010020a98a bro`Trigger(this=0x0000000105d45d60, arg_cond=0x0000000104a00390, arg_body=0x0000000104a00500, arg_timeout_stmts=0x0000000000000000, arg_timeout=0x0000000000000000, arg_frame=0x00007fff5fbfec80, arg_is_return=false, arg_location=0x00000001049fb7a0) + 106 at Trigger.cc:147 > frame #4: 0x000000010020566f bro`WhenStmt::Exec(this=0x0000000104a00900, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 239 at Stmt.cc:2041 > frame #5: 0x0000000100203204 bro`StmtList::Exec(this=0x00000001049fbe80, f=0x00007fff5fbfec80, flow=0x00007fff5fbfece8) const + 228 at Stmt.cc:1639 > frame #6: 0x000000010003d244 bro`main(argc=2, argv=0x00007fff5fbffa40) + 15476 at main.cc:1116 > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From vlad at grigorescu.org Fri Oct 31 18:54:19 2014 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Fri, 31 Oct 2014 21:54:19 -0400 Subject: [Bro-Dev] Help Troubleshooting a Perftools Memleak In-Reply-To: <20141101004811.GB67664@icir.org> References: <20141030033512.GA1488@icir.org> <4A7F30E3-42FD-4B42-B36A-8D0BFA16A0E6@illinois.edu> <20141101004811.GB67664@icir.org> Message-ID: On Fri, Oct 31, 2014 at 8:48 PM, Robin Sommer wrote: > > I've pushed the change into master but have actually not > tried yet if it indeed fixes the reported memleak. > Thanks, Robin. I tested it and perftools is no longer reporting any leaks. --Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141031/10189c6a/attachment.html From jira at bro-tracker.atlassian.net Fri Oct 31 20:34:08 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Fri, 31 Oct 2014 22:34:08 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-924) String BIFs Return 1-indexed string_arrays In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18623#comment-18623 ] Seth Hall commented on BIT-924: ------------------------------- Agreed. The only issue in this 1-indexed string issue is that we couldn't detect it. Sounds great for other stuff though. > String BIFs Return 1-indexed string_arrays > ------------------------------------------ > > Key: BIT-924 > URL: https://bro-tracker.atlassian.net/browse/BIT-924 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Fix For: 2.4 > > > The following BIFs return 1-indexed string_arrays: > * sort_string_array > * split > * split1 > * split_all > * split_n -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005) From jira at bro-tracker.atlassian.net Fri Oct 31 20:36:07 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Fri, 31 Oct 2014 22:36:07 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1283) Bro crashes when using &encrypt In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18624#comment-18624 ] Seth Hall commented on BIT-1283: -------------------------------- Bro used to ship with this tool. It was named bdcat. > Bro crashes when using &encrypt > ------------------------------- > > Key: BIT-1283 > URL: https://bro-tracker.atlassian.net/browse/BIT-1283 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: bro version 2.3-263-debug > Reporter: AK > Fix For: 2.4 > > > Bro crashes when applying the &encrypt attribute when opening a file. > bro -Ci eth0 -e 'global f1: file = open("f.out") &encrypt;' -- This message was sent by Atlassian JIRA (v6.4-OD-07-004#64005)