From noreply at bro.org Thu Jan 1 00:00:50 2015 From: noreply at bro.org (Merge Tracker) Date: Thu, 1 Jan 2015 00:00:50 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501010800.t0180oCs013872@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ------------- ------------- ---------- ---------- ------------- ---------- ------------------------- BIT-1297 [1] trace-summary Daniel Thayer - 2014-12-16 2.4 Normal trace-summary needs tests [1] BIT-1297 https://bro-tracker.atlassian.net/browse/BIT-1297 From noreply at bro.org Fri Jan 2 00:00:23 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 2 Jan 2015 00:00:23 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501020800.t0280N4U020757@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ------------- ------------- ---------- ---------- ------------- ---------- ------------------------- BIT-1297 [1] trace-summary Daniel Thayer - 2014-12-16 2.4 Normal trace-summary needs tests [1] BIT-1297 https://bro-tracker.atlassian.net/browse/BIT-1297 From jira at bro-tracker.atlassian.net Fri Jan 2 10:50:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 2 Jan 2015 12:50:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1297) trace-summary needs tests In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1297: ------------------------------ Status: Closed (was: Merge Request) > trace-summary needs tests > ------------------------- > > Key: BIT-1297 > URL: https://bro-tracker.atlassian.net/browse/BIT-1297 > Project: Bro Issue Tracker > Issue Type: Problem > Components: trace-summary > Reporter: Daniel Thayer > Fix For: 2.4 > > > There are no tests in the trace-summary repo. -- This message was sent by Atlassian JIRA (v6.4-OD-12-026#64007) From jsiwek at illinois.edu Tue Jan 6 14:15:07 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Tue, 6 Jan 2015 22:15:07 +0000 Subject: [Bro-Dev] switch to requiring CMake 2.8+ for Bro 2.4 ? Message-ID: <2ABE9CFC-5E7C-4AB7-A7E4-E840152D371F@illinois.edu> Doing a quick survey of what the major platforms offer, I found EL 6.6: CMake 2.8.12 Ubuntu 14.04 LTS: CMake 2.8.7 Debian 7.0: CMake 2.8.9 FreeBSD 9.3: CMake 2.8.12 Bumping up to requiring 2.8 would allow use of generator expressions. I?d be able to use that to properly fix the CMake policy warnings in bro-aux about using the LOCATION target property. Bumping up to 2.8.8 would allow use of OBJECT libraries. That would be a nice-to-have that I think would simplify CMake logic regarding how shared and static versions of libraries get built (e.g. would simplify broccoli, broker, and plugin support). Not aware of anything in versions beyond 2.8.8 that make them compelling enough to consider. It seems reasonable to now require 2.8.0, so I want to at least do that. And if Ubuntu 14.04 gets 2.8.8 before the next Bro release, I say switch to requiring 2.8.8. Other thoughts/opinions? - Jon From dnthayer at illinois.edu Tue Jan 6 16:37:42 2015 From: dnthayer at illinois.edu (Daniel Thayer) Date: Tue, 6 Jan 2015 18:37:42 -0600 Subject: [Bro-Dev] switch to requiring CMake 2.8+ for Bro 2.4 ? In-Reply-To: <2ABE9CFC-5E7C-4AB7-A7E4-E840152D371F@illinois.edu> References: <2ABE9CFC-5E7C-4AB7-A7E4-E840152D371F@illinois.edu> Message-ID: <54AC7FD6.8030002@illinois.edu> On 01/06/2015 04:15 PM, Siwek, Jon wrote: > Doing a quick survey of what the major platforms offer, I found > > EL 6.6: CMake 2.8.12 > Ubuntu 14.04 LTS: CMake 2.8.7 > Debian 7.0: CMake 2.8.9 > FreeBSD 9.3: CMake 2.8.12 > > Bumping up to requiring 2.8 would allow use of generator expressions. I?d be able to use that to properly fix the CMake policy warnings in bro-aux about using the LOCATION target property. > > Bumping up to 2.8.8 would allow use of OBJECT libraries. That would be a nice-to-have that I think would simplify CMake logic regarding how shared and static versions of libraries get built (e.g. would simplify broccoli, broker, and plugin support). > > Not aware of anything in versions beyond 2.8.8 that make them compelling enough to consider. > > It seems reasonable to now require 2.8.0, so I want to at least do that. And if Ubuntu 14.04 gets 2.8.8 before the next Bro release, I say switch to requiring 2.8.8. Other thoughts/opinions? > > - Jon Something looks wrong to me (debian 7 is older than ubuntu 14.04, but has a newer version of cmake?). Might want to double-check what's included with ubuntu 14.04 (I have debian 7, and it does indeed have cmake 2.8.9). From noreply at bro.org Wed Jan 7 00:00:23 2015 From: noreply at bro.org (Merge Tracker) Date: Wed, 7 Jan 2015 00:00:23 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501070800.t0780NVA027953@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- --------------------------------------------------- b5e9433 [1] bro Daniel Thayer 2015-01-07 Improve documentation of the Intelligence Framework [1] b5e9433 https://github.com/bro/bro/commit/b5e9433b043bd354024b9945d00de22f50c26cad From jsiwek at illinois.edu Wed Jan 7 07:54:57 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Wed, 7 Jan 2015 15:54:57 +0000 Subject: [Bro-Dev] switch to requiring CMake 2.8+ for Bro 2.4 ? In-Reply-To: References: <2ABE9CFC-5E7C-4AB7-A7E4-E840152D371F@illinois.edu> <54AC7FD6.8030002@illinois.edu> Message-ID: <76838E6B-FAEF-4E2E-B937-C9A6B7BA5CBD@illinois.edu> > On Jan 7, 2015, at 8:00 AM, Damian Gerow wrote: > > Something looks wrong to me (debian 7 is older than ubuntu 14.04, but > has a newer version of cmake?). Might want to double-check what's > included with ubuntu 14.04 (I have debian 7, and it does indeed > have cmake 2.8.9). > > I have an Ubuntu 14.04.1 system, and it's got 2.8.12 installed (for Bro). It's Ubuntu 12.04 systems that have 2.8.7 available. Thanks, that's right: I was actually looking at 12.04 LTS, but mistakenly wrote 14.04. Support for 12.04 is supposed to go through April 2017, so my suggestions don?t change ? switching to require 2.8 is ?safe?, but 2.8.8 may be premature. - Jon From robin at icir.org Wed Jan 7 08:04:01 2015 From: robin at icir.org (Robin Sommer) Date: Wed, 7 Jan 2015 08:04:01 -0800 Subject: [Bro-Dev] switch to requiring CMake 2.8+ for Bro 2.4 ? In-Reply-To: <76838E6B-FAEF-4E2E-B937-C9A6B7BA5CBD@illinois.edu> References: <2ABE9CFC-5E7C-4AB7-A7E4-E840152D371F@illinois.edu> <54AC7FD6.8030002@illinois.edu> <76838E6B-FAEF-4E2E-B937-C9A6B7BA5CBD@illinois.edu> Message-ID: <20150107160401.GA25947@icir.org> On Wed, Jan 07, 2015 at 15:54 +0000, you wrote: > switching to require 2.8 is ?safe?, but 2.8.8 may be premature. Sounds good. (I'm looking forward to getting object libraries eventually though!) Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Wed Jan 7 13:10:00 2015 From: jira at bro-tracker.atlassian.net (hui (JIRA)) Date: Wed, 7 Jan 2015 15:10:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1231) DNP3 Analyzer Supports for DNP3-over-UDP In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19303#comment-19303 ] hui commented on BIT-1231: -------------------------- I just added several test cases for DNP3 over UDP in topic/robin/dnp3-merge-v4. Ready to merge now. > DNP3 Analyzer Supports for DNP3-over-UDP > ---------------------------------------- > > Key: BIT-1231 > URL: https://bro-tracker.atlassian.net/browse/BIT-1231 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.3 > Reporter: hui > Assignee: hui > Labels: DNP3, analyzer > > Two major changes are made for the DNP3 analyzer > 1. Make the analyzer support both the DNP3-over-UDP and the DNP3-over-TCP. > The changes are made in DNP3.cc, DNP3.h and dpd.sig > 2. Fix a bug in the binpac codes of the DNP3 analyzer > The changes are made in dnp3-protocol.pac. The changes results in different baseline results of testing/btest/Baseline/scripts.base.protocols.dnp3.dnp3_link_only > -- This message was sent by Atlassian JIRA (v6.4-OD-12-026#64007) From jira at bro-tracker.atlassian.net Wed Jan 7 13:10:00 2015 From: jira at bro-tracker.atlassian.net (hui (JIRA)) Date: Wed, 7 Jan 2015 15:10:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1231) DNP3 Analyzer Supports for DNP3-over-UDP In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hui updated BIT-1231: --------------------- Status: Merge Request (was: Open) > DNP3 Analyzer Supports for DNP3-over-UDP > ---------------------------------------- > > Key: BIT-1231 > URL: https://bro-tracker.atlassian.net/browse/BIT-1231 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.3 > Reporter: hui > Assignee: hui > Labels: DNP3, analyzer > > Two major changes are made for the DNP3 analyzer > 1. Make the analyzer support both the DNP3-over-UDP and the DNP3-over-TCP. > The changes are made in DNP3.cc, DNP3.h and dpd.sig > 2. Fix a bug in the binpac codes of the DNP3 analyzer > The changes are made in dnp3-protocol.pac. The changes results in different baseline results of testing/btest/Baseline/scripts.base.protocols.dnp3.dnp3_link_only > -- This message was sent by Atlassian JIRA (v6.4-OD-12-026#64007) From noreply at bro.org Thu Jan 8 00:00:26 2015 From: noreply at bro.org (Merge Tracker) Date: Thu, 8 Jan 2015 00:00:26 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501080800.t0880QVb011254@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ---------------------------------------- BIT-1231 [1] Bro hui hui 2015-01-07 - Normal DNP3 Analyzer Supports for DNP3-over-UDP [1] BIT-1231 https://bro-tracker.atlassian.net/browse/BIT-1231 From jira at bro-tracker.atlassian.net Thu Jan 8 12:23:00 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 8 Jan 2015 14:23:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1302) configuration of dynamic Bro plugin easily desynchronizes with Bro's configuration In-Reply-To: References: Message-ID: Jon Siwek created BIT-1302: ------------------------------ Summary: configuration of dynamic Bro plugin easily desynchronizes with Bro's configuration Key: BIT-1302 URL: https://bro-tracker.atlassian.net/browse/BIT-1302 Project: Bro Issue Tracker Issue Type: Problem Components: Bro, bro-aux Reporter: Jon Siwek Priority: Low Any way for a dynamic plugin to automatically detect Bro's CMakeCache.txt has been changed since the last time it did a "load_cache" so that it can re-run the CMake configuration process? Maybe a hacky way would be to force the top-level/skeleton Makefile of the plugin to always do a `./configure` or a `touch build/CMakeCache.txt`. The specific problem I ran in to was 1) do a plain `./configure` of Bro 2) configure/build a plugin (e.g. I was using btest/plugins/file-plugin) 3) change my mind and do a `./configure --enable-debug` of Bro. 4) (re)building the plugin still uses the original compiler flags inherited from Bro's CMakeCache, but it's really important that it be using the same debug flags. In this case not too bad to realize that ABI of the Val class depends on -DDEBUG, but was still pretty unique/subtle to trace the resulting crashes back to the difference in compile flags between Bro and the plugin. -- This message was sent by Atlassian JIRA (v6.4-OD-12-026#64007) From jira at bro-tracker.atlassian.net Thu Jan 8 12:38:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 8 Jan 2015 14:38:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1302) configuration of dynamic Bro plugin easily desynchronizes with Bro's configuration In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19304#comment-19304 ] Robin Sommer commented on BIT-1302: ----------------------------------- Good point. I'll put this one my todo list for cleaning up the plugin code, I need to work more on that Makefile skeleton anyways. > configuration of dynamic Bro plugin easily desynchronizes with Bro's configuration > ---------------------------------------------------------------------------------- > > Key: BIT-1302 > URL: https://bro-tracker.atlassian.net/browse/BIT-1302 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, bro-aux > Reporter: Jon Siwek > Priority: Low > > Any way for a dynamic plugin to automatically detect Bro's CMakeCache.txt has been changed since the last time it did a "load_cache" so that it can re-run the CMake configuration process? > Maybe a hacky way would be to force the top-level/skeleton Makefile of the plugin to always do a `./configure` or a `touch build/CMakeCache.txt`. > The specific problem I ran in to was > 1) do a plain `./configure` of Bro > 2) configure/build a plugin (e.g. I was using btest/plugins/file-plugin) > 3) change my mind and do a `./configure --enable-debug` of Bro. > 4) (re)building the plugin still uses the original compiler flags inherited from Bro's CMakeCache, but it's really important that it be using the same debug flags. In this case not too bad to realize that ABI of the Val class depends on -DDEBUG, but was still pretty unique/subtle to trace the resulting crashes back to the difference in compile flags between Bro and the plugin. -- This message was sent by Atlassian JIRA (v6.4-OD-12-026#64007) From jira at bro-tracker.atlassian.net Thu Jan 8 12:39:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 8 Jan 2015 14:39:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1302) configuration of dynamic Bro plugin easily desynchronizes with Bro's configuration In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1302: --------------------------------- Assignee: Robin Sommer > configuration of dynamic Bro plugin easily desynchronizes with Bro's configuration > ---------------------------------------------------------------------------------- > > Key: BIT-1302 > URL: https://bro-tracker.atlassian.net/browse/BIT-1302 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, bro-aux > Reporter: Jon Siwek > Assignee: Robin Sommer > Priority: Low > > Any way for a dynamic plugin to automatically detect Bro's CMakeCache.txt has been changed since the last time it did a "load_cache" so that it can re-run the CMake configuration process? > Maybe a hacky way would be to force the top-level/skeleton Makefile of the plugin to always do a `./configure` or a `touch build/CMakeCache.txt`. > The specific problem I ran in to was > 1) do a plain `./configure` of Bro > 2) configure/build a plugin (e.g. I was using btest/plugins/file-plugin) > 3) change my mind and do a `./configure --enable-debug` of Bro. > 4) (re)building the plugin still uses the original compiler flags inherited from Bro's CMakeCache, but it's really important that it be using the same debug flags. In this case not too bad to realize that ABI of the Val class depends on -DDEBUG, but was still pretty unique/subtle to trace the resulting crashes back to the difference in compile flags between Bro and the plugin. -- This message was sent by Atlassian JIRA (v6.4-OD-12-026#64007) From noreply at bro.org Fri Jan 9 00:00:22 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 9 Jan 2015 00:00:22 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501090800.t0980MJb028718@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ---------------------------------------- BIT-1231 [1] Bro hui hui 2015-01-07 - Normal DNP3 Analyzer Supports for DNP3-over-UDP [1] BIT-1231 https://bro-tracker.atlassian.net/browse/BIT-1231 From robin at icir.org Fri Jan 9 07:55:22 2015 From: robin at icir.org (Robin Sommer) Date: Fri, 9 Jan 2015 07:55:22 -0800 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/broker: Add support for building/linking broker within bro (7120098) In-Reply-To: <201501082246.t08MktTO018494@bro-ids.icir.org> References: <201501082246.t08MktTO018494@bro-ids.icir.org> Message-ID: <20150109155522.GG73339@icir.org> On Thu, Jan 08, 2015 at 14:46 -0800, you wrote: > Author: Jon Siwek > > Add support for building/linking broker within bro > > The new --enable-broker flag can be used to toggle the use of Broker, > which also implies building with -std=c++11, though nothing makes > use of these features at the moment. We should probably make the C++11 dependency more explicit, in particular given that we want to prepare people for requiring it after 2.4. One idea would be an explicit --enable-C++11 configure switch, which --enable-broker would then either require, or activate automatically along with itself. That would then also allow us to generally test Bro compilation in C++11 mode. In addition, it would be good to check at configure time that the compiler indeed supports C++11; and if not, give an explicit erorr message stating so (rather than failing compiling later). Maybe even do that check without --enable-C++11 and warn people with older compilers that Bro in the future won't compile for them anymore. Nothing to do immediately, but to keep in mind as we get closer to the next release. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Fri Jan 9 08:49:01 2015 From: jira at bro-tracker.atlassian.net (Steve Egbert (JIRA)) Date: Fri, 9 Jan 2015 10:49:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Egbert updated BIT-1238: ------------------------------ Status: Merge Request (was: Open) > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > Attachments: test.tar.gz > > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-12-026#64007) From jira at bro-tracker.atlassian.net Fri Jan 9 08:50:00 2015 From: jira at bro-tracker.atlassian.net (Steve Egbert (JIRA)) Date: Fri, 9 Jan 2015 10:50:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Egbert updated BIT-1238: ------------------------------ Status: Open (was: Merge Request) > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > Attachments: test.tar.gz > > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-12-026#64007) From jira at bro-tracker.atlassian.net Fri Jan 9 12:17:00 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Fri, 9 Jan 2015 14:17:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19305#comment-19305 ] Seth Hall commented on BIT-1238: -------------------------------- Could you check master now to see if your problem is fixed? The branch that was fixing this problem has been merged. > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > Attachments: test.tar.gz > > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-12-026#64007) From jira at bro-tracker.atlassian.net Fri Jan 9 13:49:00 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Fri, 9 Jan 2015 15:49:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1238: --------------------------- Resolution: Fixed Fix Version/s: 2.4 git/master Status: Closed (was: Open) I'm going to go ahead and close this since things are at least significantly better now. > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > Fix For: git/master, 2.4 > > Attachments: test.tar.gz > > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-12-026#64007) From noreply at bro.org Sat Jan 10 00:00:22 2015 From: noreply at bro.org (Merge Tracker) Date: Sat, 10 Jan 2015 00:00:22 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501100800.t0A80Mlo014442@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ---------------------------------------- BIT-1231 [1] Bro hui hui 2015-01-07 - Normal DNP3 Analyzer Supports for DNP3-over-UDP [1] BIT-1231 https://bro-tracker.atlassian.net/browse/BIT-1231 From noreply at bro.org Sun Jan 11 00:00:23 2015 From: noreply at bro.org (Merge Tracker) Date: Sun, 11 Jan 2015 00:00:23 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501110800.t0B80NvC002624@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ---------------------------------------- BIT-1231 [1] Bro hui hui 2015-01-07 - Normal DNP3 Analyzer Supports for DNP3-over-UDP [1] BIT-1231 https://bro-tracker.atlassian.net/browse/BIT-1231 From noreply at bro.org Mon Jan 12 00:00:24 2015 From: noreply at bro.org (Merge Tracker) Date: Mon, 12 Jan 2015 00:00:24 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501120800.t0C80OQO011645@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ---------------------------------------- BIT-1231 [1] Bro hui hui 2015-01-07 - Normal DNP3 Analyzer Supports for DNP3-over-UDP [1] BIT-1231 https://bro-tracker.atlassian.net/browse/BIT-1231 From jira at bro-tracker.atlassian.net Mon Jan 12 21:03:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Mon, 12 Jan 2015 23:03:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1303) pysubnettree tests should be changed to use btest In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1303: ---------------------------------- Summary: pysubnettree tests should be changed to use btest Key: BIT-1303 URL: https://bro-tracker.atlassian.net/browse/BIT-1303 Project: Bro Issue Tracker Issue Type: Problem Components: pysubnettree Reporter: Daniel Thayer Fix For: 2.4 The test cases in pysubnettree should be changed to use btest so that the tests are easier to run and can be better organized by splitting them into multiple test files. -- This message was sent by Atlassian JIRA (v6.4-OD-13-024#64009) From noreply at bro.org Tue Jan 13 00:00:29 2015 From: noreply at bro.org (Merge Tracker) Date: Tue, 13 Jan 2015 00:00:29 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501130800.t0D80Tf5025487@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ---------------------------------------- BIT-1231 [1] Bro hui hui 2015-01-07 - Normal DNP3 Analyzer Supports for DNP3-over-UDP [1] BIT-1231 https://bro-tracker.atlassian.net/browse/BIT-1231 From jira at bro-tracker.atlassian.net Tue Jan 13 08:21:02 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 13 Jan 2015 10:21:02 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1303) pysubnettree tests should be changed to use btest In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19400#comment-19400 ] Jon Siwek commented on BIT-1303: -------------------------------- pysubnettree is used by some as a standalone python module. From the perspective of those users, does adding an additional dependency on btest make it easier for them to run the test suite? And I'm not sure the tests require much comparison of output to established baselines. Instead, correctness can be checked programmatically in a more direct way, so is btest being that helpful for those types of problems? > pysubnettree tests should be changed to use btest > ------------------------------------------------- > > Key: BIT-1303 > URL: https://bro-tracker.atlassian.net/browse/BIT-1303 > Project: Bro Issue Tracker > Issue Type: Problem > Components: pysubnettree > Reporter: Daniel Thayer > Fix For: 2.4 > > > The test cases in pysubnettree should be changed to use btest > so that the tests are easier to run and can be better organized > by splitting them into multiple test files. -- This message was sent by Atlassian JIRA (v6.4-OD-13-024#64009) From jira at bro-tracker.atlassian.net Tue Jan 13 11:46:00 2015 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Tue, 13 Jan 2015 13:46:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1285) MySQL Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] grigorescu updated BIT-1285: ---------------------------- Status: Merge Request (was: Open) > MySQL Protocol Analyzer > ----------------------- > > Key: BIT-1285 > URL: https://bro-tracker.atlassian.net/browse/BIT-1285 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Vlad Grigorescu > > topic/vladg/mysql is ready to be merged. > Note: memleak btest core.leaks.mysql is currently failing due to an issue with how regexes are initialized. -- This message was sent by Atlassian JIRA (v6.4-OD-13-024#64009) From jira at bro-tracker.atlassian.net Tue Jan 13 11:46:00 2015 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Tue, 13 Jan 2015 13:46:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1285) MySQL Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19401#comment-19401 ] grigorescu commented on BIT-1285: --------------------------------- Thanks for the suggestions, Robin. I think all of those make sense. I made the changes in topic/vladg/mysql, tested, and updated the btests. > MySQL Protocol Analyzer > ----------------------- > > Key: BIT-1285 > URL: https://bro-tracker.atlassian.net/browse/BIT-1285 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Vlad Grigorescu > > topic/vladg/mysql is ready to be merged. > Note: memleak btest core.leaks.mysql is currently failing due to an issue with how regexes are initialized. -- This message was sent by Atlassian JIRA (v6.4-OD-13-024#64009) From robin at icir.org Tue Jan 13 15:48:02 2015 From: robin at icir.org (Robin Sommer) Date: Tue, 13 Jan 2015 15:48:02 -0800 Subject: [Bro-Dev] [JIRA] (BIT-1303) pysubnettree tests should be changed to use btest In-Reply-To: References: Message-ID: <20150113234802.GP91308@icir.org> I can see arguments either way here. Using btest would be consistent with the other repositories; and adding a new dependency for testing is probably not too problematic. But yeah, btest isn't the natural approach here. So I'm torn and am fine either way. :) From jira at bro-tracker.atlassian.net Tue Jan 13 15:49:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 13 Jan 2015 17:49:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1303) pysubnettree tests should be changed to use btest In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1303: ------------------------------ I can see arguments either way here. Using btest would be consistent with the other repositories; and adding a new dependency for testing is probably not too problematic. But yeah, btest isn't the natural approach here. So I'm torn and am fine either way. :) > pysubnettree tests should be changed to use btest > ------------------------------------------------- > > Key: BIT-1303 > URL: https://bro-tracker.atlassian.net/browse/BIT-1303 > Project: Bro Issue Tracker > Issue Type: Problem > Components: pysubnettree > Reporter: Daniel Thayer > Fix For: 2.4 > > > The test cases in pysubnettree should be changed to use btest > so that the tests are easier to run and can be better organized > by splitting them into multiple test files. -- This message was sent by Atlassian JIRA (v6.4-OD-13-024#64009) From noreply at bro.org Wed Jan 14 00:00:25 2015 From: noreply at bro.org (Merge Tracker) Date: Wed, 14 Jan 2015 00:00:25 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501140800.t0E80P9K005159@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- --------------- ---------- ------------- ---------- ---------------------------------------- BIT-1285 [1] Bro grigorescu Vlad Grigorescu 2015-01-13 - Normal MySQL Protocol Analyzer BIT-1231 [2] Bro hui hui 2015-01-07 - Normal DNP3 Analyzer Supports for DNP3-over-UDP Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------------------------ 0480f0d [3] bro Johanna Amann 2015-01-13 small changes to ec curve names in a newer draft [1] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 [2] BIT-1231 https://bro-tracker.atlassian.net/browse/BIT-1231 [3] 0480f0d https://github.com/bro/bro/commit/0480f0d81160e19f17f4107608c0f2fafdb15ef9 From jira at bro-tracker.atlassian.net Wed Jan 14 07:19:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 14 Jan 2015 09:19:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1285) MySQL Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1285: --------------------------------- Assignee: Robin Sommer (was: Vlad Grigorescu) > MySQL Protocol Analyzer > ----------------------- > > Key: BIT-1285 > URL: https://bro-tracker.atlassian.net/browse/BIT-1285 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Robin Sommer > > topic/vladg/mysql is ready to be merged. > Note: memleak btest core.leaks.mysql is currently failing due to an issue with how regexes are initialized. -- This message was sent by Atlassian JIRA (v6.4-OD-13-024#64009) From jira at bro-tracker.atlassian.net Wed Jan 14 07:20:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 14 Jan 2015 09:20:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1231) DNP3 Analyzer Supports for DNP3-over-UDP In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1231: --------------------------------- Assignee: Robin Sommer (was: hui) > DNP3 Analyzer Supports for DNP3-over-UDP > ---------------------------------------- > > Key: BIT-1231 > URL: https://bro-tracker.atlassian.net/browse/BIT-1231 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.3 > Reporter: hui > Assignee: Robin Sommer > Labels: DNP3, analyzer > > Two major changes are made for the DNP3 analyzer > 1. Make the analyzer support both the DNP3-over-UDP and the DNP3-over-TCP. > The changes are made in DNP3.cc, DNP3.h and dpd.sig > 2. Fix a bug in the binpac codes of the DNP3 analyzer > The changes are made in dnp3-protocol.pac. The changes results in different baseline results of testing/btest/Baseline/scripts.base.protocols.dnp3.dnp3_link_only > -- This message was sent by Atlassian JIRA (v6.4-OD-13-024#64009) From jira at bro-tracker.atlassian.net Wed Jan 14 15:24:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 14 Jan 2015 17:24:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1285) MySQL Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1285: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > MySQL Protocol Analyzer > ----------------------- > > Key: BIT-1285 > URL: https://bro-tracker.atlassian.net/browse/BIT-1285 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Robin Sommer > > topic/vladg/mysql is ready to be merged. > Note: memleak btest core.leaks.mysql is currently failing due to an issue with how regexes are initialized. -- This message was sent by Atlassian JIRA (v6.4-OD-13-024#64009) From jira at bro-tracker.atlassian.net Wed Jan 14 15:24:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 14 Jan 2015 17:24:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1231) DNP3 Analyzer Supports for DNP3-over-UDP In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1231: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > DNP3 Analyzer Supports for DNP3-over-UDP > ---------------------------------------- > > Key: BIT-1231 > URL: https://bro-tracker.atlassian.net/browse/BIT-1231 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.3 > Reporter: hui > Assignee: Robin Sommer > Labels: DNP3, analyzer > > Two major changes are made for the DNP3 analyzer > 1. Make the analyzer support both the DNP3-over-UDP and the DNP3-over-TCP. > The changes are made in DNP3.cc, DNP3.h and dpd.sig > 2. Fix a bug in the binpac codes of the DNP3 analyzer > The changes are made in dnp3-protocol.pac. The changes results in different baseline results of testing/btest/Baseline/scripts.base.protocols.dnp3.dnp3_link_only > -- This message was sent by Atlassian JIRA (v6.4-OD-13-024#64009) From noreply at bro.org Thu Jan 15 00:00:26 2015 From: noreply at bro.org (Merge Tracker) Date: Thu, 15 Jan 2015 00:00:26 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501150800.t0F80Qk0011461@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------------------------ 0480f0d [1] bro Johanna Amann 2015-01-13 small changes to ec curve names in a newer draft [1] 0480f0d https://github.com/bro/bro/commit/0480f0d81160e19f17f4107608c0f2fafdb15ef9 From noreply at bro.org Fri Jan 16 00:00:26 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 16 Jan 2015 00:00:26 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501160800.t0G80QJ8030085@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------------------------ 0480f0d [1] bro Johanna Amann 2015-01-13 small changes to ec curve names in a newer draft [1] 0480f0d https://github.com/bro/bro/commit/0480f0d81160e19f17f4107608c0f2fafdb15ef9 From robin at icir.org Fri Jan 16 07:32:57 2015 From: robin at icir.org (Robin Sommer) Date: Fri, 16 Jan 2015 07:32:57 -0800 Subject: [Bro-Dev] Runtime increases Message-ID: <20150116153257.GA14049@icir.org> With latest master, I'm seeing larger-than-usual runtimes on the parts of the test-suite. Not quite sure when that started, any idea? [ 71%] tests.ipv6 ... failed (+2.5%) [ 85%] tests.m57-long ... failed (+1.2%) (Sometimes other external tests trigger the threshold too, it's a bit inconsistent; but normally, none does). Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jsiwek at illinois.edu Fri Jan 16 10:09:13 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Fri, 16 Jan 2015 18:09:13 +0000 Subject: [Bro-Dev] Runtime increases In-Reply-To: <20150116153257.GA14049@icir.org> References: <20150116153257.GA14049@icir.org> Message-ID: <8A754CC1-B6FD-4818-B906-83C9752A1EB5@illinois.edu> > On Jan 16, 2015, at 9:32 AM, Robin Sommer wrote: > > With latest master, I'm seeing larger-than-usual runtimes on the parts > of the test-suite. Not quite sure when that started, any idea? > > [ 71%] tests.ipv6 ... failed (+2.5%) > [ 85%] tests.m57-long ... failed (+1.2%) When I measured timing differences caused by adding file reassembly, it was usually around +1%. - Jon From jira at bro-tracker.atlassian.net Fri Jan 16 11:59:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Fri, 16 Jan 2015 13:59:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1304) trace-summary should be updated to support newer versions of Python In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1304: ---------------------------------- Summary: trace-summary should be updated to support newer versions of Python Key: BIT-1304 URL: https://bro-tracker.atlassian.net/browse/BIT-1304 Project: Bro Issue Tracker Issue Type: Problem Components: trace-summary Reporter: Daniel Thayer Fix For: 2.4 Some of the code in trace-summary is not valid syntax on Python version >= 3. It should be updated to work on any Python version >= 2.6. -- This message was sent by Atlassian JIRA (v6.4-OD-13-024#64009) From robin at icir.org Fri Jan 16 15:39:05 2015 From: robin at icir.org (Robin Sommer) Date: Fri, 16 Jan 2015 15:39:05 -0800 Subject: [Bro-Dev] Runtime increases In-Reply-To: <8A754CC1-B6FD-4818-B906-83C9752A1EB5@illinois.edu> References: <20150116153257.GA14049@icir.org> <8A754CC1-B6FD-4818-B906-83C9752A1EB5@illinois.edu> Message-ID: <20150116233905.GE73150@icir.org> On Fri, Jan 16, 2015 at 18:09 +0000, you wrote: > When I measured timing differences caused by adding file reassembly, > it was usually around +1%. Do you understand where that increase is coming from? Is it indeed because Bro is doing additional reassembly work now? In other words, it's not overhead incurred on traffic that does't require reassembly? Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jsiwek at illinois.edu Sun Jan 18 07:41:20 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Sun, 18 Jan 2015 15:41:20 +0000 Subject: [Bro-Dev] Runtime increases In-Reply-To: <20150116233905.GE73150@icir.org> References: <20150116153257.GA14049@icir.org> <8A754CC1-B6FD-4818-B906-83C9752A1EB5@illinois.edu> <20150116233905.GE73150@icir.org> Message-ID: <4187F65D-DF6F-4D41-B6FF-3E16D4E22F23@illinois.edu> > On Jan 16, 2015, at 5:39 PM, Robin Sommer wrote: > >> When I measured timing differences caused by adding file reassembly, >> it was usually around +1%. > > Do you understand where that increase is coming from? Is it indeed > because Bro is doing additional reassembly work now? In other words, > it's not overhead incurred on traffic that does't require reassembly? Roughly: the increase of ?default_file_bof_buffer_size? from 1024 to 4096 bytes is significant. That affects all file analysis, not just what needs reassembling. This setting changes how much data is copied in to a buffer for use with mime type signature matching. IIRC, signature matching is a large portion of file analysis cost. Average timings for 5 runs of `time bro -r ipv6.trace local "Site::local_nets={192.168.0.0/16}?`: bro/master, default_file_bof_buffer_size=4096 avg real is 9.9484 seconds avg sys is 0.718 seconds avg user is 11.3786 seconds bro/master, default_file_bof_buffer_size=1024 avg real is 9.356 seconds avg sys is 0.6782 seconds avg user is 10.9312 seconds bro/6f2b8cb, default_file_bof_buffer_size=4096 avg real is 10.018 seconds avg sys is 0.691 seconds avg user is 11.4358 seconds bro/6f2b8cb, default_file_bof_buffer_size=1024 avg real is 9.4856 seconds avg sys is 0.7148 seconds avg user is 11.1298 seconds Interesting that for the same default_file_bof_buffer_size, the new version of Bro w/ file reassembly is actually better. Does that help, or want me to look more in to it? - Jon From seth at icir.org Mon Jan 19 07:25:17 2015 From: seth at icir.org (Seth Hall) Date: Mon, 19 Jan 2015 10:25:17 -0500 Subject: [Bro-Dev] Runtime increases In-Reply-To: <4187F65D-DF6F-4D41-B6FF-3E16D4E22F23@illinois.edu> References: <20150116153257.GA14049@icir.org> <8A754CC1-B6FD-4818-B906-83C9752A1EB5@illinois.edu> <20150116233905.GE73150@icir.org> <4187F65D-DF6F-4D41-B6FF-3E16D4E22F23@illinois.edu> Message-ID: > On Jan 18, 2015, at 10:41 AM, Siwek, Jon wrote: > > Interesting that for the same default_file_bof_buffer_size, the new version of Bro w/ file reassembly is actually better. I suspect it?s because I modified the low level handling of files. The flow of chunks when they first enter the file analysis framework is quite different now. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From jira at bro-tracker.atlassian.net Mon Jan 19 09:40:01 2015 From: jira at bro-tracker.atlassian.net (scampbell (JIRA)) Date: Mon, 19 Jan 2015 11:40:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19500#comment-19500 ] scampbell commented on BIT-757: ------------------------------- Just wanted to upvote this ticket as the returned list from split() cause not being able to sort the results of a split so that they are in the same order as the original object is kinda maddening.... Perhaps a vsplit() bif which will return a vector rather than a list? > Change split* to return a string_vec rather string_array > -------------------------------------------------------- > > Key: BIT-757 > URL: https://bro-tracker.atlassian.net/browse/BIT-757 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Labels: language > > Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a }}{{table[count] of string}}{{. However, these BiFs should return a }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential iteration over the result. The problem with the current approach is not only that it is wrong modeled (the associative container does not make sense), but also that iteration over the elements, which are obviously ordered, is neither deterministic nor sequential. Presumably this mismatch exists because vectors were not available when the }}{{split*}}` functions have been created. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Mon Jan 19 12:11:00 2015 From: jira at bro-tracker.atlassian.net (Matthias Vallentin (JIRA)) Date: Mon, 19 Jan 2015 14:11:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19501#comment-19501 ] Matthias Vallentin commented on BIT-757: ---------------------------------------- While I'd like to see this feature getting added, I think a new function {{vsplit}} would bloat the API, unless there is good use case for having {{string_set}} as well. I don't see that use case though. Anyone else? > Change split* to return a string_vec rather string_array > -------------------------------------------------------- > > Key: BIT-757 > URL: https://bro-tracker.atlassian.net/browse/BIT-757 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Labels: language > > Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a }}{{table[count] of string}}{{. However, these BiFs should return a }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential iteration over the result. The problem with the current approach is not only that it is wrong modeled (the associative container does not make sense), but also that iteration over the elements, which are obviously ordered, is neither deterministic nor sequential. Presumably this mismatch exists because vectors were not available when the }}{{split*}}` functions have been created. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From noreply at bro.org Tue Jan 20 00:00:20 2015 From: noreply at bro.org (Merge Tracker) Date: Tue, 20 Jan 2015 00:00:20 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501200800.t0K80KcF014252@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ---------------------------- f7085cb [1] btest Daniel Thayer 2015-01-19 Fix some typos in the README [1] f7085cb https://github.com/bro/btest/commit/f7085cb940eda911b55a97949b80b71a13d98c92 From jira at bro-tracker.atlassian.net Tue Jan 20 04:05:00 2015 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Tue, 20 Jan 2015 06:05:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19502#comment-19502 ] Brian O'Berry commented on BIT-1238: ------------------------------------ We installed the file signatures from master (base/frameworks/files/magic) on a 2.3.1 system, which eliminated the false positives we were experiencing. This brought in unrelated signature changes, so we're in the process of verifying signatures for other file types that are important to us. l'll let you know if we find any discrepancies, but so far things look solid. Thank you! > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > Fix For: git/master, 2.4 > > Attachments: test.tar.gz > > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Tue Jan 20 06:07:00 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Tue, 20 Jan 2015 08:07:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1238: --------------------------- We?ve been meaning to write a test suite for our file signature matching because right now it?s hard to trust that we?re doing things correctly as we continue moving forward, but I never got around to it when I was making this set of changes unfortunately. > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > Fix For: git/master, 2.4 > > Attachments: test.tar.gz > > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Tue Jan 20 08:10:00 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 20 Jan 2015 10:10:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-757: -------------------------- Fix Version/s: 2.4 > Change split* to return a string_vec rather string_array > -------------------------------------------------------- > > Key: BIT-757 > URL: https://bro-tracker.atlassian.net/browse/BIT-757 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Labels: language > Fix For: 2.4 > > > Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a }}{{table[count] of string}}{{. However, these BiFs should return a }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential iteration over the result. The problem with the current approach is not only that it is wrong modeled (the associative container does not make sense), but also that iteration over the elements, which are obviously ordered, is neither deterministic nor sequential. Presumably this mismatch exists because vectors were not available when the }}{{split*}}` functions have been created. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Tue Jan 20 08:14:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 20 Jan 2015 10:14:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek reassigned BIT-757: ----------------------------- Assignee: Jon Siwek > Change split* to return a string_vec rather string_array > -------------------------------------------------------- > > Key: BIT-757 > URL: https://bro-tracker.atlassian.net/browse/BIT-757 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Assignee: Jon Siwek > Labels: language > Fix For: 2.4 > > > Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a }}{{table[count] of string}}{{. However, these BiFs should return a }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential iteration over the result. The problem with the current approach is not only that it is wrong modeled (the associative container does not make sense), but also that iteration over the elements, which are obviously ordered, is neither deterministic nor sequential. Presumably this mismatch exists because vectors were not available when the }}{{split*}}` functions have been created. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Tue Jan 20 08:14:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 20 Jan 2015 10:14:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19504#comment-19504 ] Jon Siwek commented on BIT-757: ------------------------------- My initial reaction is also to just change those functions directly to return {{vector of string}}. I'll take a look. > Change split* to return a string_vec rather string_array > -------------------------------------------------------- > > Key: BIT-757 > URL: https://bro-tracker.atlassian.net/browse/BIT-757 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Labels: language > Fix For: 2.4 > > > Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a }}{{table[count] of string}}{{. However, these BiFs should return a }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential iteration over the result. The problem with the current approach is not only that it is wrong modeled (the associative container does not make sense), but also that iteration over the elements, which are obviously ordered, is neither deterministic nor sequential. Presumably this mismatch exists because vectors were not available when the }}{{split*}}` functions have been created. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Tue Jan 20 09:20:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 20 Jan 2015 11:20:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-924) String BIFs Return 1-indexed string_arrays In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek reassigned BIT-924: ----------------------------- Assignee: Jon Siwek > String BIFs Return 1-indexed string_arrays > ------------------------------------------ > > Key: BIT-924 > URL: https://bro-tracker.atlassian.net/browse/BIT-924 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Jon Siwek > Fix For: 2.4 > > > The following BIFs return 1-indexed string_arrays: > * sort_string_array > * split > * split1 > * split_all > * split_n -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Tue Jan 20 09:58:01 2015 From: jira at bro-tracker.atlassian.net (Brian O'Berry (JIRA)) Date: Tue, 20 Jan 2015 11:58:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19505#comment-19505 ] Brian O'Berry commented on BIT-1238: ------------------------------------ We'd love to contribute a test suite, if you're interested. Would you care to discuss your ideas and/or Bro requirements? We already have a process that we're using to compare file type identification between the Bro 2.2 magic db and the signatures from master (but running on 2.3.1). > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > Fix For: git/master, 2.4 > > Attachments: test.tar.gz > > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Tue Jan 20 09:59:00 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 20 Jan 2015 11:59:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19506#comment-19506 ] Jon Siwek commented on BIT-757: ------------------------------- Bah, this is related to BIT-924: these functions are using 1-based indexing. So changing them to return a vector also begs to treat them like vectors commonly are w/ 0-based indexing. And changing the indexing scheme deserves a method of deprecating or ability to switch between a 0-based versus 1-based indexing "policy", so that we don't silently break code that people have written which depends on the original 1-based indexing. I am thinking the easiest/quickest way is to add new functions, name them appropriately w/ intention that they'll stick around for the long haul, add a sort of &deprecated attribute to split() and friends, and then later remove those deprecated functions. Let me know if there's other opinions. > Change split* to return a string_vec rather string_array > -------------------------------------------------------- > > Key: BIT-757 > URL: https://bro-tracker.atlassian.net/browse/BIT-757 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Assignee: Jon Siwek > Labels: language > Fix For: 2.4 > > > Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a }}{{table[count] of string}}{{. However, these BiFs should return a }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential iteration over the result. The problem with the current approach is not only that it is wrong modeled (the associative container does not make sense), but also that iteration over the elements, which are obviously ordered, is neither deterministic nor sequential. Presumably this mismatch exists because vectors were not available when the }}{{split*}}` functions have been created. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Tue Jan 20 12:01:03 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Tue, 20 Jan 2015 14:01:03 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-757: -------------------------- Your proposal sounds great to me. > Change split* to return a string_vec rather string_array > -------------------------------------------------------- > > Key: BIT-757 > URL: https://bro-tracker.atlassian.net/browse/BIT-757 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Assignee: Jon Siwek > Labels: language > Fix For: 2.4 > > > Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a }}{{table[count] of string}}{{. However, these BiFs should return a }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential iteration over the result. The problem with the current approach is not only that it is wrong modeled (the associative container does not make sense), but also that iteration over the elements, which are obviously ordered, is neither deterministic nor sequential. Presumably this mismatch exists because vectors were not available when the }}{{split*}}` functions have been created. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From noreply at bro.org Wed Jan 21 00:00:27 2015 From: noreply at bro.org (Merge Tracker) Date: Wed, 21 Jan 2015 00:00:27 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501210800.t0L80RSQ010969@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ---------------------------- f7085cb [1] btest Daniel Thayer 2015-01-19 Fix some typos in the README [1] f7085cb https://github.com/bro/btest/commit/f7085cb940eda911b55a97949b80b71a13d98c92 From jira at bro-tracker.atlassian.net Wed Jan 21 14:47:02 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 21 Jan 2015 16:47:02 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek reassigned BIT-757: ----------------------------- Assignee: Robin Sommer (was: Jon Siwek) > Change split* to return a string_vec rather string_array > -------------------------------------------------------- > > Key: BIT-757 > URL: https://bro-tracker.atlassian.net/browse/BIT-757 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Assignee: Robin Sommer > Labels: language > Fix For: 2.4 > > > Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a }}{{table[count] of string}}{{. However, these BiFs should return a }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential iteration over the result. The problem with the current approach is not only that it is wrong modeled (the associative container does not make sense), but also that iteration over the elements, which are obviously ordered, is neither deterministic nor sequential. Presumably this mismatch exists because vectors were not available when the }}{{split*}}` functions have been created. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 21 14:47:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 21 Jan 2015 16:47:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-924) String BIFs Return 1-indexed string_arrays In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-924: -------------------------- Resolution: Duplicate Status: Closed (was: Open) See BIT-757. > String BIFs Return 1-indexed string_arrays > ------------------------------------------ > > Key: BIT-924 > URL: https://bro-tracker.atlassian.net/browse/BIT-924 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Jon Siwek > Fix For: 2.4 > > > The following BIFs return 1-indexed string_arrays: > * sort_string_array > * split > * split1 > * split_all > * split_n -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 21 14:47:02 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 21 Jan 2015 16:47:02 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-757: -------------------------- Status: Merge Request (was: Open) > Change split* to return a string_vec rather string_array > -------------------------------------------------------- > > Key: BIT-757 > URL: https://bro-tracker.atlassian.net/browse/BIT-757 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Assignee: Robin Sommer > Labels: language > Fix For: 2.4 > > > Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a }}{{table[count] of string}}{{. However, these BiFs should return a }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential iteration over the result. The problem with the current approach is not only that it is wrong modeled (the associative container does not make sense), but also that iteration over the elements, which are obviously ordered, is neither deterministic nor sequential. Presumably this mismatch exists because vectors were not available when the }}{{split*}}` functions have been created. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 21 14:49:00 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 21 Jan 2015 16:49:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19509#comment-19509 ] Jon Siwek commented on BIT-757: ------------------------------- See topic/jsiwek/deprecation in bro, bro-testing, and bro-testing-private. It adds the deprecation mechanism and deprecates split* and related functions in favor of alternatives that use a string_vec. > Change split* to return a string_vec rather string_array > -------------------------------------------------------- > > Key: BIT-757 > URL: https://bro-tracker.atlassian.net/browse/BIT-757 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Assignee: Robin Sommer > Labels: language > Fix For: 2.4 > > > Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a }}{{table[count] of string}}{{. However, these BiFs should return a }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential iteration over the result. The problem with the current approach is not only that it is wrong modeled (the associative container does not make sense), but also that iteration over the elements, which are obviously ordered, is neither deterministic nor sequential. Presumably this mismatch exists because vectors were not available when the }}{{split*}}` functions have been created. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From noreply at bro.org Thu Jan 22 00:00:20 2015 From: noreply at bro.org (Merge Tracker) Date: Thu, 22 Jan 2015 00:00:20 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501220800.t0M80K2c007832@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ------------------ ------------ ---------- ------------- ---------- -------------------------------------------------------- BIT-757 [1] Bro Matthias Vallentin Robin Sommer 2015-01-21 2.4 Normal Change split* to return a string_vec rather string_array Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ---------------------------- f7085cb [2] btest Daniel Thayer 2015-01-19 Fix some typos in the README [1] BIT-757 https://bro-tracker.atlassian.net/browse/BIT-757 [2] f7085cb https://github.com/bro/btest/commit/f7085cb940eda911b55a97949b80b71a13d98c92 From robin at icir.org Thu Jan 22 08:21:53 2015 From: robin at icir.org (Robin Sommer) Date: Thu, 22 Jan 2015 08:21:53 -0800 Subject: [Bro-Dev] Runtime increases In-Reply-To: <4187F65D-DF6F-4D41-B6FF-3E16D4E22F23@illinois.edu> References: <20150116153257.GA14049@icir.org> <8A754CC1-B6FD-4818-B906-83C9752A1EB5@illinois.edu> <20150116233905.GE73150@icir.org> <4187F65D-DF6F-4D41-B6FF-3E16D4E22F23@illinois.edu> Message-ID: <20150122162153.GS85972@icir.org> On Sun, Jan 18, 2015 at 15:41 +0000, you wrote: > Roughly: the increase of ?default_file_bof_buffer_size? from 1024 to > 4096 bytes is significant. I can confirm that: if I switch back to 1024, things actually get faster than before for me, too. That is great, not only do we understand what happened, but we actually improved things. :) Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Thu Jan 22 14:49:00 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 22 Jan 2015 16:49:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1305) Consider marking some attributes as deprecated In-Reply-To: References: Message-ID: Jon Siwek created BIT-1305: ------------------------------ Summary: Consider marking some attributes as deprecated Key: BIT-1305 URL: https://bro-tracker.atlassian.net/browse/BIT-1305 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Jon Siwek Fix For: 2.4 Likely candidates for deprecation: &rotate_interval &rotate_size &encrypt &mergeable &synchronize &persistent &group While the mechanism I added in BIT-757 can't be used to mark attributes as deprecated, I'm thinking it's not difficult to just hard code the scanner to emit a warning when encountering certain attributes. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Thu Jan 22 14:57:00 2015 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Thu, 22 Jan 2015 16:57:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1306) bro process would get stuck/freeze with myricom drivers In-Reply-To: References: Message-ID: Aashish Sharma created BIT-1306: ----------------------------------- Summary: bro process would get stuck/freeze with myricom drivers Key: BIT-1306 URL: https://bro-tracker.atlassian.net/browse/BIT-1306 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Environment: OS: FreeBSD 9.3-RELEASE-p5 OS bro version 2.3-328 git log -1 --format="%H" 379593c7fded0f9791ae71a52dd78a4c9d5a2c1f Reporter: Aashish Sharma When I stop bro (in cluster mode), one of the bro worker process (random) would get stuck and wouldn't shutdown, stop or even be killed using kill -s 9. System has to be ultimately rebooted to remove stuck bro process. On running myri_start_stop I see: # /usr/local/opt/snf/sbin/myri_start_stop stop Removing myri_snf.ko kldunload: can't unload file: Device busy It appears that the myri_snf.ko driver cannot be unloaded because of the stuck bro process. That process still has an open descriptor on the Sniffer device/driver and bro process freezes More details: The bro process is stuck in RNE state R Marks a runnable process. N The process has reduced CPU scheduling priority (see setpriority(2)). E The process is trying to exit. Here is an example: ### stuck process: [bro at 01 ~]$ ps auxwww | fgrep 1616 bro 1616 100.0 0.0 758040 60480 ?? RNE 2:57PM 53:50.04 /usr/local/bro-git/bin/bro -i myri0 -U .status -p broctl -p broctl-live -p local -p worker-1-1 mgr.bro broctl base/frameworks/cluster local-worker.bro broctl/auto ####when checking for process in proc: [bro at c ~]$ ls -l /proc/1616 ls: /proc/1616: No such file or directory -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From noreply at bro.org Fri Jan 23 00:00:19 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 23 Jan 2015 00:00:19 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501230800.t0N80JP2003274@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ------------------ ------------ ---------- ------------- ---------- -------------------------------------------------------- BIT-757 [1] Bro Matthias Vallentin Robin Sommer 2015-01-21 2.4 Normal Change split* to return a string_vec rather string_array [1] BIT-757 https://bro-tracker.atlassian.net/browse/BIT-757 From noreply at bro.org Sat Jan 24 00:00:32 2015 From: noreply at bro.org (Merge Tracker) Date: Sat, 24 Jan 2015 00:00:32 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501240800.t0O80WQc030359@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ------------------ ------------ ---------- ------------- ---------- -------------------------------------------------------- BIT-757 [1] Bro Matthias Vallentin Robin Sommer 2015-01-21 2.4 Normal Change split* to return a string_vec rather string_array [1] BIT-757 https://bro-tracker.atlassian.net/browse/BIT-757 From noreply at bro.org Sun Jan 25 00:00:23 2015 From: noreply at bro.org (Merge Tracker) Date: Sun, 25 Jan 2015 00:00:23 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501250800.t0P80NXK008456@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ------------------ ------------ ---------- ------------- ---------- -------------------------------------------------------- BIT-757 [1] Bro Matthias Vallentin Robin Sommer 2015-01-21 2.4 Normal Change split* to return a string_vec rather string_array [1] BIT-757 https://bro-tracker.atlassian.net/browse/BIT-757 From noreply at bro.org Mon Jan 26 00:00:43 2015 From: noreply at bro.org (Merge Tracker) Date: Mon, 26 Jan 2015 00:00:43 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501260800.t0Q80hnv018367@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ------------------ ------------ ---------- ------------- ---------- -------------------------------------------------------- BIT-757 [1] Bro Matthias Vallentin Robin Sommer 2015-01-21 2.4 Normal Change split* to return a string_vec rather string_array [1] BIT-757 https://bro-tracker.atlassian.net/browse/BIT-757 From noreply at bro.org Tue Jan 27 00:00:27 2015 From: noreply at bro.org (Merge Tracker) Date: Tue, 27 Jan 2015 00:00:27 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501270800.t0R80RjJ006696@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ------------------ ------------ ---------- ------------- ---------- -------------------------------------------------------- BIT-757 [1] Bro Matthias Vallentin Robin Sommer 2015-01-21 2.4 Normal Change split* to return a string_vec rather string_array [1] BIT-757 https://bro-tracker.atlassian.net/browse/BIT-757 From noreply at bro.org Wed Jan 28 00:00:23 2015 From: noreply at bro.org (Merge Tracker) Date: Wed, 28 Jan 2015 00:00:23 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501280800.t0S80NQG012143@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ------------------ ------------ ---------- ------------- ---------- -------------------------------------------------------- BIT-757 [1] Bro Matthias Vallentin Robin Sommer 2015-01-21 2.4 Normal Change split* to return a string_vec rather string_array [1] BIT-757 https://bro-tracker.atlassian.net/browse/BIT-757 From jira at bro-tracker.atlassian.net Wed Jan 28 06:23:00 2015 From: jira at bro-tracker.atlassian.net (jdonnelly (JIRA)) Date: Wed, 28 Jan 2015 08:23:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro 2.3.2 build fails when pulled from git In-Reply-To: References: Message-ID: jdonnelly created BIT-1307: ------------------------------ Summary: bro 2.3.2 build fails when pulled from git Key: BIT-1307 URL: https://bro-tracker.atlassian.net/browse/BIT-1307 Project: Bro Issue Tracker Issue Type: Problem Components: BinPAC Reporter: jdonnelly If a package is missing the configure step should find it. I am getting this error on a fresh checkout: git clone --recursive cd bro ./configure --enable-debug make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 07:49:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 28 Jan 2015 09:49:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1307: --------------------------- Summary: bro build fails when pulled from git (was: bro 2.3.2 build fails when pulled from git) > bro build fails when pulled from git > ------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 07:49:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 28 Jan 2015 09:49:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1307: --------------------------- Component/s: (was: BinPAC) Bro > bro build fails when pulled from git > ------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 07:49:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 28 Jan 2015 09:49:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1307: --------------------------- Fix Version/s: 2.4 > bro build fails when pulled from git > ------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 07:49:00 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 28 Jan 2015 09:49:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro 2.3.2 build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19510#comment-19510 ] Jon Siwek commented on BIT-1307: -------------------------------- What version of libpcap do you have? It looks like Bro isn't checking a minimum required version, but may now need at least 1.1.0 (only a guess from quick googling around). > bro 2.3.2 build fails when pulled from git > ------------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 07:58:00 2015 From: jira at bro-tracker.atlassian.net (jdonnelly (JIRA)) Date: Wed, 28 Jan 2015 09:58:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19511#comment-19511 ] jdonnelly commented on BIT-1307: -------------------------------- root at dyn-x64-01:/work/jpd/dyn/src/bro-fork/bro# dpkg -l | grep pcap ii libpcap-dev 1.5.3-2 all development library for libpcap (transitional package) ii libpcap0.8:amd64 1.5.3-2 amd64 system interface for user-level packet capture ii libpcap0.8-dev 1.5.3-2 amd64 development library and header files for libpcap0.8 root at dyn-x64-01:/work/jpd/dyn/src/bro-fork/bro# > bro build fails when pulled from git > ------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 09:40:00 2015 From: jira at bro-tracker.atlassian.net (jdonnelly (JIRA)) Date: Wed, 28 Jan 2015 11:40:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19512#comment-19512 ] jdonnelly commented on BIT-1307: -------------------------------- I went back to a older version of bro: root at dyn-x64-01:/work/jpd/dyn/src/brotest# cat VERSION 2.3-238 And it fails too with the same PCAP error . ;-/ > bro build fails when pulled from git > ------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 10:20:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 28 Jan 2015 12:20:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19513#comment-19513 ] Jon Siwek commented on BIT-1307: -------------------------------- After `./configure`, what's the output of `grep PCAP build/CMakeCache.txt` ? And does that actually point to the expected pcap library/headers? i.e. do you have more than one libpcap version installed and is it picking up the right one? > bro build fails when pulled from git > ------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 10:33:00 2015 From: jira at bro-tracker.atlassian.net (jdonnelly (JIRA)) Date: Wed, 28 Jan 2015 12:33:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19514#comment-19514 ] jdonnelly commented on BIT-1307: -------------------------------- root at dyn-x64-01:/work/jpd/dyn/src/bro-fork/bro# grep PCAP build/CMakeCache.txt PCAP_INCLUDE_DIR:PATH=/usr/local/include PCAP_LIBRARY:FILEPATH=/usr/local/lib/libpcap.a PCAP_ROOT_DIR:PATH=/usr/local //Details about finding PCAP FIND_PACKAGE_MESSAGE_DETAILS_PCAP:INTERNAL=[/usr/local/lib/libpcap.a][/usr/local/include][v()] HAVE_LIBPCAP_PCAP_FREECODE:INTERNAL=1 HAVE_PCAP_INT_H:INTERNAL= //Test LIBPCAP_PCAP_COMPILE_NOPCAP_HAS_ERROR_PARAMETER LIBPCAP_PCAP_COMPILE_NOPCAP_HAS_ERROR_PARAMETER:INTERNAL= //Test LIBPCAP_PCAP_COMPILE_NOPCAP_NO_ERROR_PARAMETER LIBPCAP_PCAP_COMPILE_NOPCAP_NO_ERROR_PARAMETER:INTERNAL=1 //ADVANCED property for variable: PCAP_INCLUDE_DIR PCAP_INCLUDE_DIR-ADVANCED:INTERNAL=1 //ADVANCED property for variable: PCAP_LIBRARY PCAP_LIBRARY-ADVANCED:INTERNAL=1 //Test PCAP_LINKS_SOLO PCAP_LINKS_SOLO:INTERNAL=1 //ADVANCED property for variable: PCAP_ROOT_DIR PCAP_ROOT_DIR-ADVANCED:INTERNAL=1 > bro build fails when pulled from git > ------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 10:39:00 2015 From: jira at bro-tracker.atlassian.net (jdonnelly (JIRA)) Date: Wed, 28 Jan 2015 12:39:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19515#comment-19515 ] jdonnelly commented on BIT-1307: -------------------------------- Looks like it is picking up /usr/local > bro build fails when pulled from git > ------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 10:46:00 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 28 Jan 2015 12:46:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19516#comment-19516 ] Jon Siwek commented on BIT-1307: -------------------------------- What version of libpcap is installed in /usr/local? I don't expect that came from the OS. If it's actually the one you want, `./configure --with-pcap=/usr` should get it to use the standard one coming from the OS. > bro build fails when pulled from git > ------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 11:09:00 2015 From: jira at bro-tracker.atlassian.net (jdonnelly (JIRA)) Date: Wed, 28 Jan 2015 13:09:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19517#comment-19517 ] jdonnelly commented on BIT-1307: -------------------------------- removing /usr/local pcap files fixed it ! > bro build fails when pulled from git > ------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From jira at bro-tracker.atlassian.net Wed Jan 28 11:10:00 2015 From: jira at bro-tracker.atlassian.net (jdonnelly (JIRA)) Date: Wed, 28 Jan 2015 13:10:00 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1307) bro build fails when pulled from git In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jdonnelly updated BIT-1307: --------------------------- Resolution: Fixed Status: Closed (was: Open) using pcap installed from source ( in /usr/local/ ) was the problem. Oops > bro build fails when pulled from git > ------------------------------------ > > Key: BIT-1307 > URL: https://bro-tracker.atlassian.net/browse/BIT-1307 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: jdonnelly > Fix For: 2.4 > > > If a package is missing the configure step should find it. > I am getting this error on a fresh checkout: > git clone --recursive > cd bro > ./configure --enable-debug > make[3]: Entering directory `/work/jpd/dyn/src/bro-fork/bro/build' > [ 20%] Building CXX object src/analyzer/protocol/bittorrent/CMakeFs/plugin-Bro-BitTorrent.dir/BitTorrent.cc.o > In file included from /work/jpd/dyn/src/bro-fork/bro/src/Net.h:12:0, > from /work/jpd/dyn/src/bro-fork/bro/src/RuleMatcher.h:15, > from /work/jpd/dyn/src/bro-fork/bro/src/Conn.h:13, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/tcp/TCP.h:11, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.h:6, > from /work/jpd/dyn/src/bro-fork/bro/src/analyzer/protocol/bittorrent/BitTorrent.cc:3: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h: In constructor ?iosource::PktSrc::Properties::Properties()?: > /work/jpd/dyn/src/bro-fork/bro/src/iosource/PktSrc.h:272:14: error: ?PCAP_NETMASK_UNKNOWN? was not declared in this scope > netmask = PCAP_NETMASK_UNKNOWN; -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011) From noreply at bro.org Thu Jan 29 00:00:41 2015 From: noreply at bro.org (Merge Tracker) Date: Thu, 29 Jan 2015 00:00:41 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501290800.t0T80fb6001729@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ------------------ ------------ ---------- ------------- ---------- -------------------------------------------------------- BIT-757 [1] Bro Matthias Vallentin Robin Sommer 2015-01-21 2.4 Normal Change split* to return a string_vec rather string_array [1] BIT-757 https://bro-tracker.atlassian.net/browse/BIT-757 From noreply at bro.org Fri Jan 30 00:00:24 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 30 Jan 2015 00:00:24 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201501300800.t0U80O1p032221@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ------------------ ------------ ---------- ------------- ---------- -------------------------------------------------------- BIT-757 [1] Bro Matthias Vallentin Robin Sommer 2015-01-21 2.4 Normal Change split* to return a string_vec rather string_array [1] BIT-757 https://bro-tracker.atlassian.net/browse/BIT-757 From jira at bro-tracker.atlassian.net Fri Jan 30 14:41:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 30 Jan 2015 16:41:01 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-757) Change split* to return a string_vec rather string_array In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-757: ----------------------------- Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Change split* to return a string_vec rather string_array > -------------------------------------------------------- > > Key: BIT-757 > URL: https://bro-tracker.atlassian.net/browse/BIT-757 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Assignee: Robin Sommer > Labels: language > Fix For: 2.4 > > > Currently, `{{split}}{{ and friends return a }}{{string_array}}{{, which is a }}{{table[count] of string}}{{. However, these BiFs should return a }}{{string_vec}}{{ or }}{{vector of string}}{{ to allow for sequential iteration over the result. The problem with the current approach is not only that it is wrong modeled (the associative container does not make sense), but also that iteration over the elements, which are obviously ordered, is neither deterministic nor sequential. Presumably this mismatch exists because vectors were not available when the }}{{split*}}` functions have been created. -- This message was sent by Atlassian JIRA (v6.4-OD-13-026#64011)