From leres at ee.lbl.gov Mon Jan 6 15:08:08 2014 From: leres at ee.lbl.gov (Craig Leres) Date: Mon, 06 Jan 2014 15:08:08 -0800 Subject: [Bro-Dev] Fwd: [REL - 10amd64-default][security/bro] Failed for bro-2.2 in build In-Reply-To: <20131203160954.GG52843@icir.org> References: <201311300808.rAU88xFv086973@beefy2.isc.freebsd.org> <529A961A.4090809@ee.lbl.gov> <20131203160954.GG52843@icir.org> Message-ID: <52CB3758.50003@ee.lbl.gov> On 12/03/13 08:09, Robin Sommer wrote: > Which clang version is this? I've tried it with a recent version of > the clang 3.4 release branch, and that works fine for me. > > But based on the error message, I'm attaching a patch; does that help > by any chance? I submitted a PR with your patch and it worked (thanks!). Here's some feedback: > - The SQLite.cc change also needs to be applied to > src/input/readers/Raw.cc and src/input/readers/SQLite.cc. Please > upstream that as you see fit. > - You didn't manage to reproduce the build failures locally because > it's an issue caused by libc++, not clang. If you build with clang > but still use libstdc++, everything works as expected. Craig From jira at bro-tracker.atlassian.net Wed Jan 8 14:07:30 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 8 Jan 2014 16:07:30 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1108) Add broctl option to set PF_RING cluster type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1108: ------------------------------- Fix Version/s: 2.3 > Add broctl option to set PF_RING cluster type > --------------------------------------------- > > Key: BIT-1108 > URL: https://bro-tracker.atlassian.net/browse/BIT-1108 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.3 > > > Currently, when using PF_RING, broctl chooses the PF_RING > cluster type by setting the environment variable > PCAP_PF_RING_USE_CLUSTER_PER_FLOW. In order to use a > different cluster type, we would need to set a different > environment variable (the PF_RING-aware libpcap does not > look at the actual value of the environment variable, > just whether the variable is defined or not), but there is > no option in broctl to do this. > To address this issue, a new broctl option PFRINGClusterType > can be added, then a user could change the value of this > option to choose a different PF_RING cluster type (and the > broctl pf_ring plugin would set the appropriate env. variable). > The allowed values of this new broctl option would be: > "2-tuple", "4-tuple", "5-tuple", "tcp-5-tuple", "round-robin", > or "6-tuple" (this one corresponds to the current > cluster type used by broctl). By default, PFRINGClusterType > would be set to "6-tuple". -- This message was sent by Atlassian JIRA (v6.2-OD-06-43#6210) From jira at bro-tracker.atlassian.net Wed Jan 8 14:07:30 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 8 Jan 2014 16:07:30 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1108) Add broctl option to set PF_RING cluster type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15100#comment-15100 ] Daniel Thayer commented on BIT-1108: ------------------------------------ In branch topic/dnthayer/ticket1108, I've added a new broctl option "PFRINGClusterType", changed the default cluster type to "4-tuple", added tests, and updated existing test baselines. > Add broctl option to set PF_RING cluster type > --------------------------------------------- > > Key: BIT-1108 > URL: https://bro-tracker.atlassian.net/browse/BIT-1108 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.3 > > > Currently, when using PF_RING, broctl chooses the PF_RING > cluster type by setting the environment variable > PCAP_PF_RING_USE_CLUSTER_PER_FLOW. In order to use a > different cluster type, we would need to set a different > environment variable (the PF_RING-aware libpcap does not > look at the actual value of the environment variable, > just whether the variable is defined or not), but there is > no option in broctl to do this. > To address this issue, a new broctl option PFRINGClusterType > can be added, then a user could change the value of this > option to choose a different PF_RING cluster type (and the > broctl pf_ring plugin would set the appropriate env. variable). > The allowed values of this new broctl option would be: > "2-tuple", "4-tuple", "5-tuple", "tcp-5-tuple", "round-robin", > or "6-tuple" (this one corresponds to the current > cluster type used by broctl). By default, PFRINGClusterType > would be set to "6-tuple". -- This message was sent by Atlassian JIRA (v6.2-OD-06-43#6210) From jira at bro-tracker.atlassian.net Wed Jan 8 14:07:30 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 8 Jan 2014 16:07:30 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1108) Add broctl option to set PF_RING cluster type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1108: ------------------------------- Status: Merge Request (was: In Progress) > Add broctl option to set PF_RING cluster type > --------------------------------------------- > > Key: BIT-1108 > URL: https://bro-tracker.atlassian.net/browse/BIT-1108 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.3 > > > Currently, when using PF_RING, broctl chooses the PF_RING > cluster type by setting the environment variable > PCAP_PF_RING_USE_CLUSTER_PER_FLOW. In order to use a > different cluster type, we would need to set a different > environment variable (the PF_RING-aware libpcap does not > look at the actual value of the environment variable, > just whether the variable is defined or not), but there is > no option in broctl to do this. > To address this issue, a new broctl option PFRINGClusterType > can be added, then a user could change the value of this > option to choose a different PF_RING cluster type (and the > broctl pf_ring plugin would set the appropriate env. variable). > The allowed values of this new broctl option would be: > "2-tuple", "4-tuple", "5-tuple", "tcp-5-tuple", "round-robin", > or "6-tuple" (this one corresponds to the current > cluster type used by broctl). By default, PFRINGClusterType > would be set to "6-tuple". -- This message was sent by Atlassian JIRA (v6.2-OD-06-43#6210) From noreply at bro.org Thu Jan 9 00:00:24 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 9 Jan 2014 00:00:24 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401090800.s0980OtN022940@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- --------------------------------------------- BIT-1108 [1] BroControl Daniel Thayer - 2014-01-08 2.3 Normal Add broctl option to set PF_RING cluster type Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------- ---------- ------------------------------------------------------------ 22f8bb9 [2] bro Seth Hall 2014-01-08 Fix for packet writing to make it use the global snaplength. 28673bd [3] bro Seth Hall 2014-01-08 Fix for traffic with TCP segmentation offloading with IP hea [1] BIT-1108 https://bro-tracker.atlassian.net/browse/BIT-1108 [2] 22f8bb9 https://github.com/bro/bro/commit/22f8bb9dd8a189e6a042041e76bca5bcbea0ad7a [3] 28673bd https://github.com/bro/bro/commit/28673bd198ae6911cb137be956a7cc421210b98b From jira at bro-tracker.atlassian.net Thu Jan 9 18:21:30 2014 From: jira at bro-tracker.atlassian.net (scampbell (JIRA)) Date: Thu, 9 Jan 2014 20:21:30 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1110) BRO_DISABLE_BROXYGEN env variable not working In-Reply-To: References: Message-ID: scampbell created BIT-1110: ------------------------------ Summary: BRO_DISABLE_BROXYGEN env variable not working Key: BIT-1110 URL: https://bro-tracker.atlassian.net/browse/BIT-1110 Project: Bro Issue Tracker Issue Type: Patch Components: Bro Affects Versions: 2.2 Environment: Scientific linux v6.2 Reporter: scampbell Priority: High Attachments: Manager.cc.diff When running bro from the command line - for example: [scottc at sigma-n SSHD_BRO]$ bin/bro test you get the error: internal error: Broxygen can't get mtime of bro binary : No such file or directory Aborted even when the bro binary is in the runtime path. Setting: [scottc at sigma-n SSHD_BRO]$ env | grep BRO BRO_DISABLE_BROXYGEN=T does not help - see patch below. As well, running with -X or --broxygen did not change anything. A quick hack "fixed" the problem, but the problem should probably be looked over more closely as I stopped with the bandaid. -- This message was sent by Atlassian JIRA (v6.2-OD-06-43#6210) From noreply at bro.org Fri Jan 10 00:00:14 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 10 Jan 2014 00:00:14 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401100800.s0A80EYs000824@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- --------------------------------------------- BIT-1108 [1] BroControl Daniel Thayer - 2014-01-08 2.3 Normal Add broctl option to set PF_RING cluster type Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------- ---------- ------------------------------------------------------------ 22f8bb9 [2] bro Seth Hall 2014-01-08 Fix for packet writing to make it use the global snaplength. 28673bd [3] bro Seth Hall 2014-01-08 Fix for traffic with TCP segmentation offloading with IP hea [1] BIT-1108 https://bro-tracker.atlassian.net/browse/BIT-1108 [2] 22f8bb9 https://github.com/bro/bro/commit/22f8bb9dd8a189e6a042041e76bca5bcbea0ad7a [3] 28673bd https://github.com/bro/bro/commit/28673bd198ae6911cb137be956a7cc421210b98b From noreply at bro.org Sat Jan 11 00:00:13 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 11 Jan 2014 00:00:13 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401110800.s0B80DV2009355@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- --------------------------------------------- BIT-1108 [1] BroControl Daniel Thayer - 2014-01-08 2.3 Normal Add broctl option to set PF_RING cluster type Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------- ---------- ------------------------------------------------------------ e0082e6 [2] bro Jon Siwek 2014-01-10 Improve GeoIP City database support. beea92c [3] bro Jon Siwek 2014-01-10 Broxygen init fixes, addresses BIT-1110. 22f8bb9 [4] bro Seth Hall 2014-01-08 Fix for packet writing to make it use the global snaplength. 28673bd [5] bro Seth Hall 2014-01-08 Fix for traffic with TCP segmentation offloading with IP hea [1] BIT-1108 https://bro-tracker.atlassian.net/browse/BIT-1108 [2] e0082e6 https://github.com/bro/bro/commit/e0082e6bcb7b346b87d9ad02a35dcab0209a351f [3] beea92c https://github.com/bro/bro/commit/beea92ce6ceb03c2d81f58635723bc1ec02646b9 [4] 22f8bb9 https://github.com/bro/bro/commit/22f8bb9dd8a189e6a042041e76bca5bcbea0ad7a [5] 28673bd https://github.com/bro/bro/commit/28673bd198ae6911cb137be956a7cc421210b98b From noreply at bro.org Sun Jan 12 00:00:12 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 12 Jan 2014 00:00:12 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401120800.s0C80Cf9017592@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- --------------------------------------------- BIT-1108 [1] BroControl Daniel Thayer - 2014-01-08 2.3 Normal Add broctl option to set PF_RING cluster type Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------- ---------- ------------------------------------------------------------ e0082e6 [2] bro Jon Siwek 2014-01-10 Improve GeoIP City database support. beea92c [3] bro Jon Siwek 2014-01-10 Broxygen init fixes, addresses BIT-1110. 22f8bb9 [4] bro Seth Hall 2014-01-08 Fix for packet writing to make it use the global snaplength. 28673bd [5] bro Seth Hall 2014-01-08 Fix for traffic with TCP segmentation offloading with IP hea [1] BIT-1108 https://bro-tracker.atlassian.net/browse/BIT-1108 [2] e0082e6 https://github.com/bro/bro/commit/e0082e6bcb7b346b87d9ad02a35dcab0209a351f [3] beea92c https://github.com/bro/bro/commit/beea92ce6ceb03c2d81f58635723bc1ec02646b9 [4] 22f8bb9 https://github.com/bro/bro/commit/22f8bb9dd8a189e6a042041e76bca5bcbea0ad7a [5] 28673bd https://github.com/bro/bro/commit/28673bd198ae6911cb137be956a7cc421210b98b From noreply at bro.org Mon Jan 13 00:00:16 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 13 Jan 2014 00:00:16 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401130800.s0D80GsN026404@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- --------------------------------------------- BIT-1108 [1] BroControl Daniel Thayer - 2014-01-08 2.3 Normal Add broctl option to set PF_RING cluster type Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------- ---------- ------------------------------------------------------------ e0082e6 [2] bro Jon Siwek 2014-01-10 Improve GeoIP City database support. beea92c [3] bro Jon Siwek 2014-01-10 Broxygen init fixes, addresses BIT-1110. 22f8bb9 [4] bro Seth Hall 2014-01-08 Fix for packet writing to make it use the global snaplength. 28673bd [5] bro Seth Hall 2014-01-08 Fix for traffic with TCP segmentation offloading with IP hea [1] BIT-1108 https://bro-tracker.atlassian.net/browse/BIT-1108 [2] e0082e6 https://github.com/bro/bro/commit/e0082e6bcb7b346b87d9ad02a35dcab0209a351f [3] beea92c https://github.com/bro/bro/commit/beea92ce6ceb03c2d81f58635723bc1ec02646b9 [4] 22f8bb9 https://github.com/bro/bro/commit/22f8bb9dd8a189e6a042041e76bca5bcbea0ad7a [5] 28673bd https://github.com/bro/bro/commit/28673bd198ae6911cb137be956a7cc421210b98b From jira at bro-tracker.atlassian.net Mon Jan 13 02:36:25 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 13 Jan 2014 04:36:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1108) Add broctl option to set PF_RING cluster type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1108: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Add broctl option to set PF_RING cluster type > --------------------------------------------- > > Key: BIT-1108 > URL: https://bro-tracker.atlassian.net/browse/BIT-1108 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.3 > > > Currently, when using PF_RING, broctl chooses the PF_RING > cluster type by setting the environment variable > PCAP_PF_RING_USE_CLUSTER_PER_FLOW. In order to use a > different cluster type, we would need to set a different > environment variable (the PF_RING-aware libpcap does not > look at the actual value of the environment variable, > just whether the variable is defined or not), but there is > no option in broctl to do this. > To address this issue, a new broctl option PFRINGClusterType > can be added, then a user could change the value of this > option to choose a different PF_RING cluster type (and the > broctl pf_ring plugin would set the appropriate env. variable). > The allowed values of this new broctl option would be: > "2-tuple", "4-tuple", "5-tuple", "tcp-5-tuple", "round-robin", > or "6-tuple" (this one corresponds to the current > cluster type used by broctl). By default, PFRINGClusterType > would be set to "6-tuple". -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Mon Jan 13 11:14:25 2014 From: jira at bro-tracker.atlassian.net (Chas DiFatta (JIRA)) Date: Mon, 13 Jan 2014 13:14:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1111) will not start In-Reply-To: References: Message-ID: Chas DiFatta created BIT-1111: --------------------------------- Summary: will not start Key: BIT-1111 URL: https://bro-tracker.atlassian.net/browse/BIT-1111 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.2 Environment: Mac OSX 10.9 Reporter: Chas DiFatta Priority: High there is a problem when starting as i get the following error internal error: can't load magic file : no magic files loaded /opt/bro/share/broctl/scripts/run-bro: line 82: 1175 Abort trap: 6 (core dumped) nohup $mybro "$@" have a new mac running 10.9. downloaded 2.2 for OSX, it installed fine. followed the recommended changes to the config files. ran the following command as root. sh-3.2# ./broctl Welcome to BroControl 1.2 Type "help" for help. [BroControl] > install removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/site ... done. removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/auto ... done. creating policy directories ... done. installing site policies ... done. generating standalone-layout.bro ... done. generating local-networks.bro ... done. generating broctl-config.bro ... done. updating nodes ... done. [BroControl] > start starting bro ... . bro terminated immediately after starting; check output with "diag" [BroControl] > diag [bro] Bro 2.2 Darwin 13.0.0 No gdb installed. ==== No reporter.log ==== stderr.log internal error: can't load magic file : no magic files loaded /opt/bro/share/broctl/scripts/run-bro: line 82: 1175 Abort trap: 6 (core dumped) nohup $mybro "$@" ==== stdout.log unlimited unlimited unlimited ==== .cmdline -i eh0 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto ==== .env_vars PATH=/opt/bro/bin:/opt/bro/share/broctl/scripts:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin BROPATH=/opt/bro/spool/installed-scripts-do-not-touch/site::/opt/bro/spool/installed-scripts-do-not-touch/auto:/opt/bro/share/bro:/opt/bro/share/bro/policy:/opt/bro/share/bro/site CLUSTER_NODE= ==== .status TERMINATED [internal_error] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Mon Jan 13 12:12:25 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 13 Jan 2014 14:12:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1111) will not start In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200#comment-15200 ] Jon Siwek commented on BIT-1111: -------------------------------- Three options to fix your install: 1) Just install libmagic from MacPorts. 2) Compile/install Bro from source. 3) Install libmagic from source or a different package manager, but make sure when running Bro that the MAGIC environment variable points to the magic database file that comes with it. I'd suggest going w/ option 2. Explanation: OS X doesn't ship with libmagic, so the binary package for Bro on OS X was compiled against a MacPorts' libmagic statically. Unfortunately, that's not enough to make the binary package completely independent as Bro still expects the "default" database shipping w/ libmagic to be present, which it may not always be on the system where it's installed. The way to fix this in Bro itself would be to consolidate libmagic usage to rely on database(s) that ship w/ Bro and possibly even redistribute a version of libmagic w/ Bro so it can be compiled against a consistent version guaranteed to work w/ the database(s) (I'm in favor of both). > will not start > -------------- > > Key: BIT-1111 > URL: https://bro-tracker.atlassian.net/browse/BIT-1111 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Mac OSX 10.9 > Reporter: Chas DiFatta > Priority: High > Labels: broctl > > there is a problem when starting as i get the following error > internal error: can't load magic file : no magic files loaded > /opt/bro/share/broctl/scripts/run-bro: line 82: 1175 Abort trap: 6 (core dumped) nohup $mybro "$@" > have a new mac running 10.9. downloaded 2.2 for OSX, it installed fine. followed the recommended changes to the config files. ran the following command as root. > sh-3.2# ./broctl > Welcome to BroControl 1.2 > Type "help" for help. > [BroControl] > install > removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/site ... done. > removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/auto ... done. > creating policy directories ... done. > installing site policies ... done. > generating standalone-layout.bro ... done. > generating local-networks.bro ... done. > generating broctl-config.bro ... done. > updating nodes ... done. > [BroControl] > start > starting bro ... > . > bro terminated immediately after starting; check output with "diag" > [BroControl] > diag > [bro] > Bro 2.2 > Darwin 13.0.0 > No gdb installed. > ==== No reporter.log > ==== stderr.log > internal error: can't load magic file : no magic files loaded > /opt/bro/share/broctl/scripts/run-bro: line 82: 1175 Abort trap: 6 (core dumped) nohup $mybro "$@" > ==== stdout.log > unlimited > unlimited > unlimited > ==== .cmdline > -i eh0 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto > ==== .env_vars > PATH=/opt/bro/bin:/opt/bro/share/broctl/scripts:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin > BROPATH=/opt/bro/spool/installed-scripts-do-not-touch/site::/opt/bro/spool/installed-scripts-do-not-touch/auto:/opt/bro/share/bro:/opt/bro/share/bro/policy:/opt/bro/share/bro/site > CLUSTER_NODE= > ==== .status > TERMINATED [internal_error] > ==== No prof.log > ==== No packet_filter.log > ==== No loaded_scripts.log -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 08:09:25 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 15 Jan 2014 10:09:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1089) Please install sample/example broctl .cfg files In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1089: ------------------------------- Fix Version/s: 2.2 > Please install sample/example broctl .cfg files > ----------------------------------------------- > > Key: BIT-1089 > URL: https://bro-tracker.atlassian.net/browse/BIT-1089 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: BroControl > Reporter: leres > Priority: Low > Fix For: 2.2 > > -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 08:22:25 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 15 Jan 2014 10:22:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1112) topic/dnthayer/misc-improvements In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1112: ---------------------------------- Summary: topic/dnthayer/misc-improvements Key: BIT-1112 URL: https://bro-tracker.atlassian.net/browse/BIT-1112 Project: Bro Issue Tracker Issue Type: Problem Components: BroControl Reporter: Daniel Thayer Fix For: 2.3 The branch topic/dnthayer/misc-improvements contains some small fixes/improvements: improve broctl output formatting, fix "top" output on OS X Mavericks, fix minor issue with plugin init() return values. Also included are some changes from Justin Azoff: plugin code cleanup (remove redundant plugin initialization, and use getattr for lookup of plugin methods), and enable dead host caching in cron mode. -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 08:24:25 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 15 Jan 2014 10:24:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1112) topic/dnthayer/misc-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1112: ------------------------------- Status: Merge Request (was: Open) > topic/dnthayer/misc-improvements > -------------------------------- > > Key: BIT-1112 > URL: https://bro-tracker.atlassian.net/browse/BIT-1112 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.3 > > > The branch topic/dnthayer/misc-improvements contains some small > fixes/improvements: improve broctl output formatting, fix "top" output > on OS X Mavericks, fix minor issue with plugin init() return values. > Also included are some changes from Justin Azoff: plugin > code cleanup (remove redundant plugin initialization, and use > getattr for lookup of plugin methods), and enable dead host > caching in cron mode. -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 13:12:25 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Wed, 15 Jan 2014 15:12:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1113) topic/jazoff/notice_file_info In-Reply-To: References: Message-ID: Justin Azoff created BIT-1113: --------------------------------- Summary: topic/jazoff/notice_file_info Key: BIT-1113 URL: https://bro-tracker.atlassian.net/browse/BIT-1113 Project: Bro Issue Tracker Issue Type: Patch Components: Bro Affects Versions: 2.2 Reporter: Justin Azoff -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 13:15:25 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Wed, 15 Jan 2014 15:15:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1113) topic/jazoff/notice_file_info In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Azoff updated BIT-1113: ------------------------------ Status: Merge Request (was: Open) > topic/jazoff/notice_file_info > ----------------------------- > > Key: BIT-1113 > URL: https://bro-tracker.atlassian.net/browse/BIT-1113 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 13:15:25 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Wed, 15 Jan 2014 15:15:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1113) topic/jazoff/notice_file_info In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201#comment-15201 ] Justin Azoff commented on BIT-1113: ----------------------------------- This branch contains a single change that adds the new file information to notice emails. The resulting notices look like the following: {code} Message: Malware Hash Registry Detection rate: 11% Last seen: 2014-01-07 12:38:05 Sub-message: https://www.virustotal.com/en/search/?query=c2937b7e2619af42c1cfa13e061c6a0f9133b2bb File Description: http://staticwajam-wajam.netdna-ssl.com/static/update/wajam_update.exe?v0.016 File Mime Type: application/x-dosexec Connection: ... Connection uid: ... ... {code} > topic/jazoff/notice_file_info > ----------------------------- > > Key: BIT-1113 > URL: https://bro-tracker.atlassian.net/browse/BIT-1113 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 13:17:25 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Wed, 15 Jan 2014 15:17:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1114) topic/jazoff/ssl-validation-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Azoff updated BIT-1114: ------------------------------ Status: Merge Request (was: Open) > topic/jazoff/ssl-validation-fix > ------------------------------- > > Key: BIT-1114 > URL: https://bro-tracker.atlassian.net/browse/BIT-1114 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 13:17:25 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Wed, 15 Jan 2014 15:17:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1114) topic/jazoff/ssl-validation-fix In-Reply-To: References: Message-ID: Justin Azoff created BIT-1114: --------------------------------- Summary: topic/jazoff/ssl-validation-fix Key: BIT-1114 URL: https://bro-tracker.atlassian.net/browse/BIT-1114 Project: Bro Issue Tracker Issue Type: Patch Components: Bro Affects Versions: 2.2 Reporter: Justin Azoff -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 13:17:25 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Wed, 15 Jan 2014 15:17:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1114) topic/jazoff/ssl-validation-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202#comment-15202 ] Justin Azoff commented on BIT-1114: ----------------------------------- This branch contains a single commit that fixes the use of the recently_validated_certs table. It was being checked for the presence of a cached validation result, but on a cache miss, the validation result was not being added. > topic/jazoff/ssl-validation-fix > ------------------------------- > > Key: BIT-1114 > URL: https://bro-tracker.atlassian.net/browse/BIT-1114 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 13:19:25 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Wed, 15 Jan 2014 15:19:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1115) topic/jazoff/suppression In-Reply-To: References: Message-ID: Justin Azoff created BIT-1115: --------------------------------- Summary: topic/jazoff/suppression Key: BIT-1115 URL: https://bro-tracker.atlassian.net/browse/BIT-1115 Project: Bro Issue Tracker Issue Type: Patch Components: Bro Affects Versions: 2.2 Reporter: Justin Azoff -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 13:33:25 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Wed, 15 Jan 2014 15:33:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1115) topic/jazoff/suppression In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Azoff updated BIT-1115: ------------------------------ Status: Merge Request (was: Open) > topic/jazoff/suppression > ------------------------ > > Key: BIT-1115 > URL: https://bro-tracker.atlassian.net/browse/BIT-1115 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 13:33:25 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Wed, 15 Jan 2014 15:33:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1115) topic/jazoff/suppression In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15203#comment-15203 ] Justin Azoff commented on BIT-1115: ----------------------------------- Instead of storing the entire notice in Notice::suppressing, just store the time the notice should be suppressed until. This has the same functionality, except that end_suppression can no longer be generated. This has the effect of greatly reducing the memory usage on a bro cluster that is raising a lot of suppressed notices. This can happen if suppression is enabled, but the suppression id is too specific and multiple notices are raised anyway. This problem is exacerbated on cluster nodes that are running 10 workers, since the suppression information is duplicated across all workers ( and then across all nodes ) For a stress test of a pcap that raises 38609 notices: | Without the patch | 147255296 maximum resident set size| | With the patch | 49586176 maximum resident set size| | Difference | 93 MB | On the real cluster, I was seeing memory usage growing at the rate of 2 megabytes/second or so. Even with 24G of ram the nodes were OOMing after a few hours. Bro workers would crash, eventually resync the data, and crash again. > topic/jazoff/suppression > ------------------------ > > Key: BIT-1115 > URL: https://bro-tracker.atlassian.net/browse/BIT-1115 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 14:50:25 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 15 Jan 2014 16:50:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1116) topic/jsiwek/libmagic-integration In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1116: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/libmagic-integration > --------------------------------- > > Key: BIT-1116 > URL: https://bro-tracker.atlassian.net/browse/BIT-1116 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch is in bro, 3rdparty, bromagic, bro-testing, and bro-testing-private repos. It integrates libmagic 5.16 into Bro as a CMake ExternalProject, which requires CMake >= 2.8.0, so that one does not have to install libmagic to build bro. > Resolves BIT-1111, BIT-1096. -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Wed Jan 15 14:50:25 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 15 Jan 2014 16:50:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1116) topic/jsiwek/libmagic-integration In-Reply-To: References: Message-ID: Jon Siwek created BIT-1116: ------------------------------ Summary: topic/jsiwek/libmagic-integration Key: BIT-1116 URL: https://bro-tracker.atlassian.net/browse/BIT-1116 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Reporter: Jon Siwek Fix For: 2.3 This branch is in bro, 3rdparty, bromagic, bro-testing, and bro-testing-private repos. It integrates libmagic 5.16 into Bro as a CMake ExternalProject, which requires CMake >= 2.8.0, so that one does not have to install libmagic to build bro. Resolves BIT-1111, BIT-1096. -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From noreply at bro.org Thu Jan 16 00:00:12 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 16 Jan 2014 00:00:12 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401160800.s0G80C1U004758@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- ------------------------------------- BIT-1116 [1] Bro Jon Siwek - 2014-01-15 2.3 Normal topic/jsiwek/libmagic-integration [2] BIT-1115 [3] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/suppression [4] BIT-1114 [5] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/ssl-validation-fix [6] BIT-1113 [7] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/notice_file_info [8] BIT-1112 [9] BroControl Daniel Thayer - 2014-01-15 2.3 Normal topic/dnthayer/misc-improvements [10] [1] BIT-1116 https://bro-tracker.atlassian.net/browse/BIT-1116 [2] libmagic-integration https://github.com/bro/bro/tree/topic/jsiwek/libmagic-integration [3] BIT-1115 https://bro-tracker.atlassian.net/browse/BIT-1115 [4] suppression https://github.com/bro/bro/tree/topic/jazoff/suppression [5] BIT-1114 https://bro-tracker.atlassian.net/browse/BIT-1114 [6] ssl-validation-fix https://github.com/bro/bro/tree/topic/jazoff/ssl-validation-fix [7] BIT-1113 https://bro-tracker.atlassian.net/browse/BIT-1113 [8] notice_file_info https://github.com/bro/bro/tree/topic/jazoff/notice_file_info [9] BIT-1112 https://bro-tracker.atlassian.net/browse/BIT-1112 [10] misc-improvements https://github.com/bro/brocontrol/tree/topic/dnthayer/misc-improvements From jira at bro-tracker.atlassian.net Thu Jan 16 08:40:25 2014 From: jira at bro-tracker.atlassian.net (Chas DiFatta (JIRA)) Date: Thu, 16 Jan 2014 10:40:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1111) will not start In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204#comment-15204 ] Chas DiFatta commented on BIT-1111: ----------------------------------- thanks jon for getting back to me as i?ll give it a try. a number of us are at flocon and were hoping to get bro up natively on the apple platform as the VM demo was kind of rough. all the best, ?cd > will not start > -------------- > > Key: BIT-1111 > URL: https://bro-tracker.atlassian.net/browse/BIT-1111 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Mac OSX 10.9 > Reporter: Chas DiFatta > Priority: High > Labels: broctl > > there is a problem when starting as i get the following error > internal error: can't load magic file : no magic files loaded > /opt/bro/share/broctl/scripts/run-bro: line 82: 1175 Abort trap: 6 (core dumped) nohup $mybro "$@" > have a new mac running 10.9. downloaded 2.2 for OSX, it installed fine. followed the recommended changes to the config files. ran the following command as root. > sh-3.2# ./broctl > Welcome to BroControl 1.2 > Type "help" for help. > [BroControl] > install > removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/site ... done. > removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/auto ... done. > creating policy directories ... done. > installing site policies ... done. > generating standalone-layout.bro ... done. > generating local-networks.bro ... done. > generating broctl-config.bro ... done. > updating nodes ... done. > [BroControl] > start > starting bro ... > . > bro terminated immediately after starting; check output with "diag" > [BroControl] > diag > [bro] > Bro 2.2 > Darwin 13.0.0 > No gdb installed. > ==== No reporter.log > ==== stderr.log > internal error: can't load magic file : no magic files loaded > /opt/bro/share/broctl/scripts/run-bro: line 82: 1175 Abort trap: 6 (core dumped) nohup $mybro "$@" > ==== stdout.log > unlimited > unlimited > unlimited > ==== .cmdline > -i eh0 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto > ==== .env_vars > PATH=/opt/bro/bin:/opt/bro/share/broctl/scripts:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin > BROPATH=/opt/bro/spool/installed-scripts-do-not-touch/site::/opt/bro/spool/installed-scripts-do-not-touch/auto:/opt/bro/share/bro:/opt/bro/share/bro/policy:/opt/bro/share/bro/site > CLUSTER_NODE= > ==== .status > TERMINATED [internal_error] > ==== No prof.log > ==== No packet_filter.log > ==== No loaded_scripts.log -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Thu Jan 16 13:40:25 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Thu, 16 Jan 2014 15:40:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1117) Broctl base commucation port should be configurable In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205#comment-15205 ] Justin Azoff commented on BIT-1117: ----------------------------------- Currently this is hardcoded to 47759 and it can't be changed without editing install.py > Broctl base commucation port should be configurable > --------------------------------------------------- > > Key: BIT-1117 > URL: https://bro-tracker.atlassian.net/browse/BIT-1117 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Reporter: Justin Azoff > -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Thu Jan 16 13:40:25 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Thu, 16 Jan 2014 15:40:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1117) Broctl base commucation port should be configurable In-Reply-To: References: Message-ID: Justin Azoff created BIT-1117: --------------------------------- Summary: Broctl base commucation port should be configurable Key: BIT-1117 URL: https://bro-tracker.atlassian.net/browse/BIT-1117 Project: Bro Issue Tracker Issue Type: New Feature Components: BroControl Reporter: Justin Azoff -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Thu Jan 16 14:58:25 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 16 Jan 2014 16:58:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-867) GRE support In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15206#comment-15206 ] Jon Siwek commented on BIT-867: ------------------------------- This is implemented in topic/jsiwek/gre in just the bro repo. > GRE support > ----------- > > Key: BIT-867 > URL: https://bro-tracker.atlassian.net/browse/BIT-867 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Fix For: 2.3 > > > Should be rather easy to add support for GRE tunnels now. -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Thu Jan 16 14:58:25 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 16 Jan 2014 16:58:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-867) GRE support In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-867: -------------------------- Status: Merge Request (was: Open) > GRE support > ----------- > > Key: BIT-867 > URL: https://bro-tracker.atlassian.net/browse/BIT-867 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Fix For: 2.3 > > > Should be rather easy to add support for GRE tunnels now. -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Thu Jan 16 19:54:25 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 16 Jan 2014 21:54:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1117) Broctl base communication port should be configurable In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1117: ------------------------------- Description: Broctl automatically assigns ports for Bro to listen on, starting with port number 47760. There is no config option to change this. Summary: Broctl base communication port should be configurable (was: Broctl base commucation port should be configurable) > Broctl base communication port should be configurable > ----------------------------------------------------- > > Key: BIT-1117 > URL: https://bro-tracker.atlassian.net/browse/BIT-1117 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Reporter: Justin Azoff > > Broctl automatically assigns ports for Bro to listen on, starting with port number 47760. There is no config option to change this. -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From jira at bro-tracker.atlassian.net Thu Jan 16 20:21:25 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 16 Jan 2014 22:21:25 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1117) Broctl base communication port should be configurable In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207#comment-15207 ] Daniel Thayer commented on BIT-1117: ------------------------------------ I've created branch topic/dnthayer/ticket1117 which adds a new option "BroPort" with default value 47760 (this is the same starting port number that is used currently). > Broctl base communication port should be configurable > ----------------------------------------------------- > > Key: BIT-1117 > URL: https://bro-tracker.atlassian.net/browse/BIT-1117 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Reporter: Justin Azoff > > Broctl automatically assigns ports for Bro to listen on, starting with port number 47760. There is no config option to change this. -- This message was sent by Atlassian JIRA (v6.2-OD-07-027#6211) From noreply at bro.org Fri Jan 17 00:02:38 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 17 Jan 2014 00:02:38 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401170802.s0H82c9h028675@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- ------------------------------------- BIT-1116 [1] Bro Jon Siwek - 2014-01-15 2.3 Normal topic/jsiwek/libmagic-integration [2] BIT-1115 [3] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/suppression [4] BIT-1114 [5] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/ssl-validation-fix [6] BIT-1113 [7] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/notice_file_info [8] BIT-1112 [9] BroControl Daniel Thayer - 2014-01-15 2.3 Normal topic/dnthayer/misc-improvements [10] BIT-867 [11] Bro Robin Sommer - 2014-01-16 2.3 Normal GRE support [1] BIT-1116 https://bro-tracker.atlassian.net/browse/BIT-1116 [2] libmagic-integration https://github.com/bro/bro/tree/topic/jsiwek/libmagic-integration [3] BIT-1115 https://bro-tracker.atlassian.net/browse/BIT-1115 [4] suppression https://github.com/bro/bro/tree/topic/jazoff/suppression [5] BIT-1114 https://bro-tracker.atlassian.net/browse/BIT-1114 [6] ssl-validation-fix https://github.com/bro/bro/tree/topic/jazoff/ssl-validation-fix [7] BIT-1113 https://bro-tracker.atlassian.net/browse/BIT-1113 [8] notice_file_info https://github.com/bro/bro/tree/topic/jazoff/notice_file_info [9] BIT-1112 https://bro-tracker.atlassian.net/browse/BIT-1112 [10] misc-improvements https://github.com/bro/brocontrol/tree/topic/dnthayer/misc-improvements [11] BIT-867 https://bro-tracker.atlassian.net/browse/BIT-867 From noreply at bro.org Sat Jan 18 00:00:17 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 18 Jan 2014 00:00:17 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401180800.s0I80HDV017921@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- ------------------------------------- BIT-1116 [1] Bro Jon Siwek - 2014-01-15 2.3 Normal topic/jsiwek/libmagic-integration [2] BIT-1115 [3] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/suppression [4] BIT-1114 [5] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/ssl-validation-fix [6] BIT-1113 [7] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/notice_file_info [8] BIT-1112 [9] BroControl Daniel Thayer - 2014-01-15 2.3 Normal topic/dnthayer/misc-improvements [10] BIT-867 [11] Bro Robin Sommer - 2014-01-16 2.3 Normal GRE support Open Fastpath Commits ====================== Commit Component Author Date Summary ------------ ----------- ------------- ---------- ----------------------------------- d472eee [12] broctl Daniel Thayer 2014-01-17 Fix bug with IPv6Comm broctl option [1] BIT-1116 https://bro-tracker.atlassian.net/browse/BIT-1116 [2] libmagic-integration https://github.com/bro/bro/tree/topic/jsiwek/libmagic-integration [3] BIT-1115 https://bro-tracker.atlassian.net/browse/BIT-1115 [4] suppression https://github.com/bro/bro/tree/topic/jazoff/suppression [5] BIT-1114 https://bro-tracker.atlassian.net/browse/BIT-1114 [6] ssl-validation-fix https://github.com/bro/bro/tree/topic/jazoff/ssl-validation-fix [7] BIT-1113 https://bro-tracker.atlassian.net/browse/BIT-1113 [8] notice_file_info https://github.com/bro/bro/tree/topic/jazoff/notice_file_info [9] BIT-1112 https://bro-tracker.atlassian.net/browse/BIT-1112 [10] misc-improvements https://github.com/bro/brocontrol/tree/topic/dnthayer/misc-improvements [11] BIT-867 https://bro-tracker.atlassian.net/browse/BIT-867 [12] d472eee https://github.com/bro/broctl/commit/d472eee62386b3d02613946233ecf66c569a97a8 From noreply at bro.org Sun Jan 19 00:00:14 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 19 Jan 2014 00:00:14 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401190800.s0J80EeY002487@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- ------------------------------------- BIT-1116 [1] Bro Jon Siwek - 2014-01-15 2.3 Normal topic/jsiwek/libmagic-integration [2] BIT-1115 [3] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/suppression [4] BIT-1114 [5] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/ssl-validation-fix [6] BIT-1113 [7] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/notice_file_info [8] BIT-1112 [9] BroControl Daniel Thayer - 2014-01-15 2.3 Normal topic/dnthayer/misc-improvements [10] BIT-867 [11] Bro Robin Sommer - 2014-01-16 2.3 Normal GRE support Open Fastpath Commits ====================== Commit Component Author Date Summary ------------ ----------- ------------- ---------- ----------------------------------- d472eee [12] broctl Daniel Thayer 2014-01-17 Fix bug with IPv6Comm broctl option [1] BIT-1116 https://bro-tracker.atlassian.net/browse/BIT-1116 [2] libmagic-integration https://github.com/bro/bro/tree/topic/jsiwek/libmagic-integration [3] BIT-1115 https://bro-tracker.atlassian.net/browse/BIT-1115 [4] suppression https://github.com/bro/bro/tree/topic/jazoff/suppression [5] BIT-1114 https://bro-tracker.atlassian.net/browse/BIT-1114 [6] ssl-validation-fix https://github.com/bro/bro/tree/topic/jazoff/ssl-validation-fix [7] BIT-1113 https://bro-tracker.atlassian.net/browse/BIT-1113 [8] notice_file_info https://github.com/bro/bro/tree/topic/jazoff/notice_file_info [9] BIT-1112 https://bro-tracker.atlassian.net/browse/BIT-1112 [10] misc-improvements https://github.com/bro/brocontrol/tree/topic/dnthayer/misc-improvements [11] BIT-867 https://bro-tracker.atlassian.net/browse/BIT-867 [12] d472eee https://github.com/bro/broctl/commit/d472eee62386b3d02613946233ecf66c569a97a8 From noreply at bro.org Mon Jan 20 00:00:14 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 20 Jan 2014 00:00:14 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401200800.s0K80E2m012124@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ---------- ---------- ------------- ---------- ------------------------------------- BIT-1116 [1] Bro Jon Siwek - 2014-01-15 2.3 Normal topic/jsiwek/libmagic-integration [2] BIT-1115 [3] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/suppression [4] BIT-1114 [5] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/ssl-validation-fix [6] BIT-1113 [7] Bro Justin Azoff - 2014-01-15 - Normal topic/jazoff/notice_file_info [8] BIT-1112 [9] BroControl Daniel Thayer - 2014-01-15 2.3 Normal topic/dnthayer/misc-improvements [10] BIT-867 [11] Bro Robin Sommer - 2014-01-16 2.3 Normal GRE support Open Fastpath Commits ====================== Commit Component Author Date Summary ------------ ----------- ------------- ---------- ----------------------------------- d472eee [12] broctl Daniel Thayer 2014-01-17 Fix bug with IPv6Comm broctl option [1] BIT-1116 https://bro-tracker.atlassian.net/browse/BIT-1116 [2] libmagic-integration https://github.com/bro/bro/tree/topic/jsiwek/libmagic-integration [3] BIT-1115 https://bro-tracker.atlassian.net/browse/BIT-1115 [4] suppression https://github.com/bro/bro/tree/topic/jazoff/suppression [5] BIT-1114 https://bro-tracker.atlassian.net/browse/BIT-1114 [6] ssl-validation-fix https://github.com/bro/bro/tree/topic/jazoff/ssl-validation-fix [7] BIT-1113 https://bro-tracker.atlassian.net/browse/BIT-1113 [8] notice_file_info https://github.com/bro/bro/tree/topic/jazoff/notice_file_info [9] BIT-1112 https://bro-tracker.atlassian.net/browse/BIT-1112 [10] misc-improvements https://github.com/bro/brocontrol/tree/topic/dnthayer/misc-improvements [11] BIT-867 https://bro-tracker.atlassian.net/browse/BIT-867 [12] d472eee https://github.com/bro/broctl/commit/d472eee62386b3d02613946233ecf66c569a97a8 From jira at bro-tracker.atlassian.net Mon Jan 20 12:28:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 20 Jan 2014 14:28:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-867) GRE support In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300#comment-15300 ] Robin Sommer commented on BIT-867: ---------------------------------- {noformat} // Not considering routing presence bit since it's deprecated... {noformat} Would it hurt to add that? Looks like it's just another length adjustment if present? > GRE support > ----------- > > Key: BIT-867 > URL: https://bro-tracker.atlassian.net/browse/BIT-867 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Fix For: 2.3 > > > Should be rather easy to add support for GRE tunnels now. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Mon Jan 20 12:49:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 20 Jan 2014 14:49:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1115) topic/jazoff/suppression In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1115: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/jazoff/suppression > ------------------------ > > Key: BIT-1115 > URL: https://bro-tracker.atlassian.net/browse/BIT-1115 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Mon Jan 20 12:49:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 20 Jan 2014 14:49:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1116) topic/jsiwek/libmagic-integration In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1116: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/jsiwek/libmagic-integration > --------------------------------- > > Key: BIT-1116 > URL: https://bro-tracker.atlassian.net/browse/BIT-1116 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch is in bro, 3rdparty, bromagic, bro-testing, and bro-testing-private repos. It integrates libmagic 5.16 into Bro as a CMake ExternalProject, which requires CMake >= 2.8.0, so that one does not have to install libmagic to build bro. > Resolves BIT-1111, BIT-1096. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Mon Jan 20 12:49:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 20 Jan 2014 14:49:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1113) topic/jazoff/notice_file_info In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1113: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/jazoff/notice_file_info > ----------------------------- > > Key: BIT-1113 > URL: https://bro-tracker.atlassian.net/browse/BIT-1113 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Mon Jan 20 12:49:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 20 Jan 2014 14:49:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-867) GRE support In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-867: ----------------------------- Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > GRE support > ----------- > > Key: BIT-867 > URL: https://bro-tracker.atlassian.net/browse/BIT-867 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Fix For: 2.3 > > > Should be rather easy to add support for GRE tunnels now. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Mon Jan 20 12:49:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 20 Jan 2014 14:49:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1114) topic/jazoff/ssl-validation-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1114: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/jazoff/ssl-validation-fix > ------------------------------- > > Key: BIT-1114 > URL: https://bro-tracker.atlassian.net/browse/BIT-1114 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Mon Jan 20 12:51:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 20 Jan 2014 14:51:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1112) topic/dnthayer/misc-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1112: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/dnthayer/misc-improvements > -------------------------------- > > Key: BIT-1112 > URL: https://bro-tracker.atlassian.net/browse/BIT-1112 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.3 > > > The branch topic/dnthayer/misc-improvements contains some small > fixes/improvements: improve broctl output formatting, fix "top" output > on OS X Mavericks, fix minor issue with plugin init() return values. > Also included are some changes from Justin Azoff: plugin > code cleanup (remove redundant plugin initialization, and use > getattr for lookup of plugin methods), and enable dead host > caching in cron mode. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From noreply at bro.org Tue Jan 21 00:00:11 2014 From: noreply at bro.org (Merge Tracker) Date: Tue, 21 Jan 2014 00:00:11 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401210800.s0L80BrZ000700@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------------------- 4d6d601 [1] broctl Daniel Thayer 2014-01-20 Correct typo in CHANGES and update the docs [1] 4d6d601 https://github.com/bro/broctl/commit/4d6d6014c96d230ca64150eddd5e13f61b19c7f3 From jira at bro-tracker.atlassian.net Tue Jan 21 07:43:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 21 Jan 2014 09:43:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-867) GRE support In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301#comment-15301 ] Jon Siwek commented on BIT-867: ------------------------------- It involves parsing a list of variable length fields, which should be easy, but I'd rather not add that code unless a pcap to test against is found (which may be hard since the use of it is deprecated). > GRE support > ----------- > > Key: BIT-867 > URL: https://bro-tracker.atlassian.net/browse/BIT-867 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Fix For: 2.3 > > > Should be rather easy to add support for GRE tunnels now. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Tue Jan 21 14:23:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 21 Jan 2014 16:23:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1118) topic/jsiwek/review-rafael-bro-manual-changes In-Reply-To: References: Message-ID: Jon Siwek created BIT-1118: ------------------------------ Summary: topic/jsiwek/review-rafael-bro-manual-changes Key: BIT-1118 URL: https://bro-tracker.atlassian.net/browse/BIT-1118 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Jon Siwek Fix For: 2.3 This branch has Rafael's changes to the Bro Manual with some cleanup and added unit tests by me. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Tue Jan 21 14:23:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 21 Jan 2014 16:23:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1118) topic/jsiwek/review-rafael-bro-manual-changes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1118: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/review-rafael-bro-manual-changes > --------------------------------------------- > > Key: BIT-1118 > URL: https://bro-tracker.atlassian.net/browse/BIT-1118 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch has Rafael's changes to the Bro Manual with some cleanup and added unit tests by me. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From noreply at bro.org Wed Jan 22 00:00:13 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 22 Jan 2014 00:00:13 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401220800.s0M80DK4005965@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------------------------------- BIT-1118 [1] Bro Jon Siwek - 2014-01-21 2.3 Normal topic/jsiwek/review-rafael-bro-manual-changes [2] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------------------ 3992022 [3] broctl Daniel Thayer 2014-01-21 Fix bug with timemachineport broctl option [1] BIT-1118 https://bro-tracker.atlassian.net/browse/BIT-1118 [2] review-rafael-bro-manual-changes https://github.com/bro/bro/tree/topic/jsiwek/review-rafael-bro-manual-changes [3] 3992022 https://github.com/bro/broctl/commit/3992022db32d1f1869ab2076692152c629b7f887 From jira at bro-tracker.atlassian.net Wed Jan 22 12:14:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 22 Jan 2014 14:14:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1118) topic/jsiwek/review-rafael-bro-manual-changes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1118: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/jsiwek/review-rafael-bro-manual-changes > --------------------------------------------- > > Key: BIT-1118 > URL: https://bro-tracker.atlassian.net/browse/BIT-1118 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch has Rafael's changes to the Bro Manual with some cleanup and added unit tests by me. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From noreply at bro.org Thu Jan 23 00:00:16 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 23 Jan 2014 00:00:16 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401230800.s0N80GBV018799@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- --------------- ------------ ---------- --------------------------------------------------------------------- #1 [1] broccoli-python pohjo03x [2] 2014-01-22 Supply Bro "connection alive" and "connection delete" mechanisms. [3] [1] Pull Request #1 https://github.com/bro/broccoli-python/pull/1 [2] pohjo03x https://github.com/pohjo03x [3] Merge Pull Request #1 with git pull https://github.com/pohjo03x/broccoli-python.git master From jira at bro-tracker.atlassian.net Fri Jan 24 15:23:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 24 Jan 2014 17:23:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements In-Reply-To: References: Message-ID: Jon Siwek created BIT-1119: ------------------------------ Summary: topic/jsiwek/tcp-improvements Key: BIT-1119 URL: https://bro-tracker.atlassian.net/browse/BIT-1119 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Jon Siwek Fix For: 2.3 This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail). The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Fri Jan 24 15:25:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 24 Jan 2014 17:25:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1119: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/tcp-improvements > ----------------------------- > > Key: BIT-1119 > URL: https://bro-tracker.atlassian.net/browse/BIT-1119 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail). > The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From noreply at bro.org Sat Jan 25 00:00:13 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 25 Jan 2014 00:00:13 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401250800.s0P80DNL025824@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- --------------------------------- BIT-1119 [1] Bro Jon Siwek - 2014-01-24 2.3 Normal topic/jsiwek/tcp-improvements [2] [1] BIT-1119 https://bro-tracker.atlassian.net/browse/BIT-1119 [2] tcp-improvements https://github.com/bro/bro/tree/topic/jsiwek/tcp-improvements From noreply at bro.org Sun Jan 26 00:00:11 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 26 Jan 2014 00:00:11 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401260800.s0Q80BIA005053@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- --------------------------------- BIT-1119 [1] Bro Jon Siwek - 2014-01-24 2.3 Normal topic/jsiwek/tcp-improvements [2] [1] BIT-1119 https://bro-tracker.atlassian.net/browse/BIT-1119 [2] tcp-improvements https://github.com/bro/bro/tree/topic/jsiwek/tcp-improvements From noreply at bro.org Mon Jan 27 00:00:14 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 27 Jan 2014 00:00:14 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401270800.s0R80ENI020129@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- --------------------------------- BIT-1119 [1] Bro Jon Siwek - 2014-01-24 2.3 Normal topic/jsiwek/tcp-improvements [2] [1] BIT-1119 https://bro-tracker.atlassian.net/browse/BIT-1119 [2] tcp-improvements https://github.com/bro/bro/tree/topic/jsiwek/tcp-improvements From jira at bro-tracker.atlassian.net Mon Jan 27 10:34:58 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Mon, 27 Jan 2014 12:34:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1120) Fix & extend x509_extension event In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann updated BIT-1120: -------------------------------- Status: Merge Request (was: Open) > Fix & extend x509_extension event > --------------------------------- > > Key: BIT-1120 > URL: https://bro-tracker.atlassian.net/browse/BIT-1120 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.2 > Reporter: Bernhard Amann > Fix For: 2.3 > > > Please merge topic/bernhard/fix-x509-extension. > This branch fixes and extends the x509_extension event, which was never called in the previous implementation. The event now parses the extension into a bro data structure. If supports printing it, it is converted into the openssl ascii output, otherwise a raw hex-dump is output. > New event syntax: > event x509_extension(c: connection, is_orig: bool, cert:X509, extension: X509_extension_info) > Example output for extension: > [name=X509v3 Extended Key Usage, > short_name=extendedKeyUsage, > oid=2.5.29.37, > critical=F, > value=TLS Web Server Authentication, TLS Web Client Authentication] > [name=X509v3 Certificate Policies, > short_name=certificatePolicies, > oid=2.5.29.32, > critical=F, > value=Policy: 1.3.6.1.4.1.6449.1.2.1.3.4^J CPS: https://secure.comodo.com/CPS^J] -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Mon Jan 27 10:34:58 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Mon, 27 Jan 2014 12:34:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1120) Fix & extend x509_extension event In-Reply-To: References: Message-ID: Bernhard Amann created BIT-1120: ----------------------------------- Summary: Fix & extend x509_extension event Key: BIT-1120 URL: https://bro-tracker.atlassian.net/browse/BIT-1120 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.2, git/master Reporter: Bernhard Amann Fix For: 2.3 Please merge topic/bernhard/fix-x509-extension. This branch fixes and extends the x509_extension event, which was never called in the previous implementation. The event now parses the extension into a bro data structure. If supports printing it, it is converted into the openssl ascii output, otherwise a raw hex-dump is output. New event syntax: event x509_extension(c: connection, is_orig: bool, cert:X509, extension: X509_extension_info) Example output for extension: [name=X509v3 Extended Key Usage, short_name=extendedKeyUsage, oid=2.5.29.37, critical=F, value=TLS Web Server Authentication, TLS Web Client Authentication] [name=X509v3 Certificate Policies, short_name=certificatePolicies, oid=2.5.29.32, critical=F, value=Policy: 1.3.6.1.4.1.6449.1.2.1.3.4^J CPS: https://secure.comodo.com/CPS^J] -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Mon Jan 27 10:53:59 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Mon, 27 Jan 2014 12:53:59 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-760) Lift Server Alternative Name (SAN) field to scripting layer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302#comment-15302 ] Bernhard Amann commented on BIT-760: ------------------------------------ BIT-1120 now allows access to the SAN field. However, field access is only provided in the OpenSSL ascii formatted syntax. Still not quite where it should be, but this might already be enough for the purpose of some people.. > Lift Server Alternative Name (SAN) field to scripting layer > ----------------------------------------------------------- > > Key: BIT-760 > URL: https://bro-tracker.atlassian.net/browse/BIT-760 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Matthias Vallentin > Assignee: Bernhard Amann > Labels: analyzer > Fix For: 2.3 > > > It would be nice to have the *Subject Alternative Name (SAN)* field of an X.509 certificate available at the scripting layer. It contains a list of domains that should be used in addition to the CN field of the subject to verify that a domain matches the certificate. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From noreply at bro.org Tue Jan 28 00:00:14 2014 From: noreply at bro.org (Merge Tracker) Date: Tue, 28 Jan 2014 00:00:14 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401280800.s0S80EUf017471@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- -------------- ---------- ---------- ------------- ---------- --------------------------------- BIT-1120 [1] Bro Bernhard Amann - 2014-01-27 2.3 Normal Fix & extend x509_extension event BIT-1119 [2] Bro Jon Siwek - 2014-01-24 2.3 Normal topic/jsiwek/tcp-improvements [3] [1] BIT-1120 https://bro-tracker.atlassian.net/browse/BIT-1120 [2] BIT-1119 https://bro-tracker.atlassian.net/browse/BIT-1119 [3] tcp-improvements https://github.com/bro/bro/tree/topic/jsiwek/tcp-improvements From jira at bro-tracker.atlassian.net Tue Jan 28 10:15:58 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 28 Jan 2014 12:15:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1121) topic/dnthayer/test-improvements In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1121: ---------------------------------- Summary: topic/dnthayer/test-improvements Key: BIT-1121 URL: https://bro-tracker.atlassian.net/browse/BIT-1121 Project: Bro Issue Tracker Issue Type: Problem Components: BroControl Reporter: Daniel Thayer Fix For: 2.3 Various improvements to the test build scripts to address some error scenarios and to provide convenience features (added a new makefile target "rerun" to more easily re-run failed tests, and scripts now recognize two new env. vars. to enable doing a non-standard build). Improved the test diff canonifiers to do more thorough checking, and to workaround an issue in btest-diff which was causing some failed tests to not be reported as failed. Added lots of new tests (there are now 50% more test cases) to fill in gaps in the test coverage. Also improved many existing tests. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Tue Jan 28 10:15:58 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 28 Jan 2014 12:15:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1121) topic/dnthayer/test-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1121: ------------------------------- Status: Merge Request (was: Open) > topic/dnthayer/test-improvements > -------------------------------- > > Key: BIT-1121 > URL: https://bro-tracker.atlassian.net/browse/BIT-1121 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.3 > > > Various improvements to the test build scripts to address some > error scenarios and to provide convenience features (added a > new makefile target "rerun" to more easily re-run failed tests, > and scripts now recognize two new env. vars. to enable doing a > non-standard build). Improved the test diff canonifiers > to do more thorough checking, and to workaround an issue in btest-diff > which was causing some failed tests to not be reported as failed. > Added lots of new tests (there are now 50% more test cases) to > fill in gaps in the test coverage. Also improved many existing > tests. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Tue Jan 28 12:15:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 28 Jan 2014 14:15:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1122) topic/jsiwek/dns-improvements In-Reply-To: References: Message-ID: Jon Siwek created BIT-1122: ------------------------------ Summary: topic/jsiwek/dns-improvements Key: BIT-1122 URL: https://bro-tracker.atlassian.net/browse/BIT-1122 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Reporter: Jon Siwek Fix For: 2.3 This branch is in bro, bro-testing, and bro-testing-private repos. - Fixes incorrect parsing of DNS message format for messages with empty question sections. - Changes dns.log to only include standard queries (opcode == 1). - Adds "dns_unknown_reply" event for RR types that Bro doesn't know how to parse, which improves accuracy of request-reply pair matching performed by the default DNS scripts. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Tue Jan 28 12:15:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 28 Jan 2014 14:15:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1122) topic/jsiwek/dns-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1122: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/dns-improvements > ----------------------------- > > Key: BIT-1122 > URL: https://bro-tracker.atlassian.net/browse/BIT-1122 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch is in bro, bro-testing, and bro-testing-private repos. > - Fixes incorrect parsing of DNS message format for messages with empty question sections. > - Changes dns.log to only include standard queries (opcode == 1). > - Adds "dns_unknown_reply" event for RR types that Bro doesn't know how to parse, which improves accuracy of request-reply pair matching performed by the default DNS scripts. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Tue Jan 28 12:17:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 28 Jan 2014 14:17:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1122) topic/jsiwek/dns-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1122: --------------------------- Issue Type: Improvement (was: Problem) > topic/jsiwek/dns-improvements > ----------------------------- > > Key: BIT-1122 > URL: https://bro-tracker.atlassian.net/browse/BIT-1122 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch is in bro, bro-testing, and bro-testing-private repos. > - Fixes incorrect parsing of DNS message format for messages with empty question sections. > - Changes dns.log to only include standard queries (opcode == 1). > - Adds "dns_unknown_reply" event for RR types that Bro doesn't know how to parse, which improves accuracy of request-reply pair matching performed by the default DNS scripts. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Tue Jan 28 15:34:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 28 Jan 2014 17:34:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303#comment-15303 ] Robin Sommer commented on BIT-1119: ----------------------------------- I'm going ahead merging this but I'm wondering about the new {{detect_filtered_trace}} flag. It's pretty common (in the research world, anyways :) to run Bro on a SYN/FIN/RST trace and I imagine having this by default off can add a lot for warnings in that case. Can we add some other heuristic to detect such a trace (i.e., guess whether {{detect_filtered_trace}} should be on) ? A (very) coarse approach would simply be a global variable recording if we've ever seen anything else than a TCP control packet. Thoughts? > topic/jsiwek/tcp-improvements > ----------------------------- > > Key: BIT-1119 > URL: https://bro-tracker.atlassian.net/browse/BIT-1119 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail). > The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Tue Jan 28 15:47:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 28 Jan 2014 17:47:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1122) topic/jsiwek/dns-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1122: --------------------------------- Assignee: Seth Hall > topic/jsiwek/dns-improvements > ----------------------------- > > Key: BIT-1122 > URL: https://bro-tracker.atlassian.net/browse/BIT-1122 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Seth Hall > Fix For: 2.3 > > > This branch is in bro, bro-testing, and bro-testing-private repos. > - Fixes incorrect parsing of DNS message format for messages with empty question sections. > - Changes dns.log to only include standard queries (opcode == 1). > - Adds "dns_unknown_reply" event for RR types that Bro doesn't know how to parse, which improves accuracy of request-reply pair matching performed by the default DNS scripts. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From noreply at bro.org Wed Jan 29 00:00:15 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 29 Jan 2014 00:00:15 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401290800.s0T80FNF032361@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- -------------- ---------- ---------- ------------- ---------- ------------------------------------ BIT-1122 [1] Bro Jon Siwek Seth Hall 2014-01-28 2.3 Normal topic/jsiwek/dns-improvements [2] BIT-1121 [3] BroControl Daniel Thayer - 2014-01-28 2.3 Normal topic/dnthayer/test-improvements [4] BIT-1120 [5] Bro Bernhard Amann - 2014-01-27 2.3 Normal Fix & extend x509_extension event BIT-1119 [6] Bro Jon Siwek - 2014-01-28 2.3 Normal topic/jsiwek/tcp-improvements [7] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- --------------------------------------------- 62b3cb0 [8] bro Bernhard Amann 2014-01-28 Also use exec-module test to check for leaks. [1] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [2] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements [3] BIT-1121 https://bro-tracker.atlassian.net/browse/BIT-1121 [4] test-improvements https://github.com/bro/brocontrol/tree/topic/dnthayer/test-improvements [5] BIT-1120 https://bro-tracker.atlassian.net/browse/BIT-1120 [6] BIT-1119 https://bro-tracker.atlassian.net/browse/BIT-1119 [7] tcp-improvements https://github.com/bro/bro/tree/topic/jsiwek/tcp-improvements [8] 62b3cb0 https://github.com/bro/bro/commit/62b3cb0a5b7bdd8fed1d7d0dae3337115b2feae7 From jira at bro-tracker.atlassian.net Wed Jan 29 08:15:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 29 Jan 2014 10:15:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15304#comment-15304 ] Jon Siwek commented on BIT-1119: -------------------------------- {quote} I'm going ahead merging this but I'm wondering about the new detect_filtered_trace flag. It's pretty common (in the research world, anyways to run Bro on a SYN/FIN/RST trace and I imagine having this by default off can add a lot for warnings in that case. Can we add some other heuristic to detect such a trace (i.e., guess whether detect_filtered_trace should be on) ? A (very) coarse approach would simply be a global variable recording if we've ever seen anything else than a TCP control packet. Thoughts? {quote} If a person found out that Bro automatically switched modes part way through the trace, they will probably just re-run after manually toggling the option, right? Maybe treat it in a similar way to checksums -- have a FAQ and/or have some script warn if all TCP connections are missing 100% of content and suggest toggling {{detect_filtered_trace}} if the person would like to trade off correctness for minimized output. But if it's actually not that important for a person using filtered traces to minimize output, I think it's fine enough as is? > topic/jsiwek/tcp-improvements > ----------------------------- > > Key: BIT-1119 > URL: https://bro-tracker.atlassian.net/browse/BIT-1119 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail). > The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Wed Jan 29 08:24:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 29 Jan 2014 10:24:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15305#comment-15305 ] Robin Sommer commented on BIT-1119: ----------------------------------- {quote} have some script warn if all TCP connections are missing 100% of content and suggest toggling detect_filtered_trace {quote} I like that, is that something we can do efficiently? {quote} But if it's actually not that important for a person using filtered traces to minimize output, I think it's fine enough as is? {quote} it's less the volume of output but the potential for confusion: one sees it and starts wondering what's wrong. It's easy to forget that TCP analysis gets confused because the trace is filtered. So if there was some way to point that out, that's all it would need. It's not a biggie but it's indeed in the same category like the checksums: something easy to get wrong without realizing what's going on, in particular because we're changing the default here. > topic/jsiwek/tcp-improvements > ----------------------------- > > Key: BIT-1119 > URL: https://bro-tracker.atlassian.net/browse/BIT-1119 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail). > The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Wed Jan 29 08:44:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 29 Jan 2014 10:44:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1121) topic/dnthayer/test-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1121: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/dnthayer/test-improvements > -------------------------------- > > Key: BIT-1121 > URL: https://bro-tracker.atlassian.net/browse/BIT-1121 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.3 > > > Various improvements to the test build scripts to address some > error scenarios and to provide convenience features (added a > new makefile target "rerun" to more easily re-run failed tests, > and scripts now recognize two new env. vars. to enable doing a > non-standard build). Improved the test diff canonifiers > to do more thorough checking, and to workaround an issue in btest-diff > which was causing some failed tests to not be reported as failed. > Added lots of new tests (there are now 50% more test cases) to > fill in gaps in the test coverage. Also improved many existing > tests. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Wed Jan 29 08:44:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 29 Jan 2014 10:44:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1120) Fix & extend x509_extension event In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1120: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Fix & extend x509_extension event > --------------------------------- > > Key: BIT-1120 > URL: https://bro-tracker.atlassian.net/browse/BIT-1120 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.2 > Reporter: Bernhard Amann > Fix For: 2.3 > > > Please merge topic/bernhard/fix-x509-extension. > This branch fixes and extends the x509_extension event, which was never called in the previous implementation. The event now parses the extension into a bro data structure. If supports printing it, it is converted into the openssl ascii output, otherwise a raw hex-dump is output. > New event syntax: > event x509_extension(c: connection, is_orig: bool, cert:X509, extension: X509_extension_info) > Example output for extension: > [name=X509v3 Extended Key Usage, > short_name=extendedKeyUsage, > oid=2.5.29.37, > critical=F, > value=TLS Web Server Authentication, TLS Web Client Authentication] > [name=X509v3 Certificate Policies, > short_name=certificatePolicies, > oid=2.5.29.32, > critical=F, > value=Policy: 1.3.6.1.4.1.6449.1.2.1.3.4^J CPS: https://secure.comodo.com/CPS^J] -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Wed Jan 29 09:02:58 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 29 Jan 2014 11:02:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1119: --------------------------- Attachment: signature.asc We could probably do it similarly to how we're doing the detection of invalid checksums by sampling weirds for a little bit. I also like this approach a lot. I think that keeping the default settings of Bro working "correctly" in the normal case is good, but it's awesome to be able to notify people when things are failing and how they could fix it. > topic/jsiwek/tcp-improvements > ----------------------------- > > Key: BIT-1119 > URL: https://bro-tracker.atlassian.net/browse/BIT-1119 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc > > > This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail). > The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Wed Jan 29 09:47:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 29 Jan 2014 11:47:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307#comment-15307 ] Jon Siwek commented on BIT-1119: -------------------------------- {quote} it's less the volume of output but the potential for confusion: one sees it and starts wondering what's wrong. It's easy to forget that TCP analysis gets confused because the trace is filtered. {quote} I might be misremembering (or repressed the details of the TCP code), but isn't the TCP analysis *less* confused in the face of filtered traces with the change? i.e. things are now most correct and it actually reports content gaps so e.g. missing_bytes fields for connections can be populated. {quote} but it's awesome to be able to notify people when things are failing and how they could fix it. {quote} I wouldn't say filtered traces fail due to the change, you just get more, possibly unexpected but not incorrect, output. (I'm just trying to clarify perspective, not really against idea of sampling weirds to issue suggestion/warning) > topic/jsiwek/tcp-improvements > ----------------------------- > > Key: BIT-1119 > URL: https://bro-tracker.atlassian.net/browse/BIT-1119 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc > > > This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail). > The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Wed Jan 29 10:53:58 2014 From: jira at bro-tracker.atlassian.net (Jeannette Dopheide (JIRA)) Date: Wed, 29 Jan 2014 12:53:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1123) topic/jdopheid/bro/edits_to_installation_and_getting_started In-Reply-To: References: Message-ID: Jeannette Dopheide created BIT-1123: --------------------------------------- Summary: topic/jdopheid/bro/edits_to_installation_and_getting_started Key: BIT-1123 URL: https://bro-tracker.atlassian.net/browse/BIT-1123 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Jeannette Dopheide Minor grammar edits to Installation and Quick Start pages Also, please let me know if I need to modify future JIRA tickets. Thanks, Jeannette ************************ Repository : ssh://git at bro-ids.icir.org/bro On branch : topic/jdopheid/bro/edits_to_installation_and_getting_started Link : https://github.com/bro/bro/commit/4c52c378d5873abb052d688251f0ec7f5aa1c514 And: Repository : ssh://git at bro-ids.icir.org/bro On branch : topic/jdopheid/bro/edits_to_installation_and_getting_started Link : https://github.com/bro/bro/commit/af95026348688e0df8c867f67d2a53a3d440cf41 -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Wed Jan 29 11:28:58 2014 From: jira at bro-tracker.atlassian.net (Jeannette Dopheide (JIRA)) Date: Wed, 29 Jan 2014 13:28:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1123) topic/jdopheid/bro/edits_to_installation_and_getting_started In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeannette Dopheide updated BIT-1123: ------------------------------------ Status: Merge Request (was: Open) > topic/jdopheid/bro/edits_to_installation_and_getting_started > ------------------------------------------------------------ > > Key: BIT-1123 > URL: https://bro-tracker.atlassian.net/browse/BIT-1123 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jeannette Dopheide > > Minor grammar edits to Installation and Quick Start pages > Also, please let me know if I need to modify future JIRA tickets. > Thanks, > Jeannette > ************************ > Repository : ssh://git at bro-ids.icir.org/bro > On branch : topic/jdopheid/bro/edits_to_installation_and_getting_started > Link : https://github.com/bro/bro/commit/4c52c378d5873abb052d688251f0ec7f5aa1c514 > And: > Repository : ssh://git at bro-ids.icir.org/bro > On branch : topic/jdopheid/bro/edits_to_installation_and_getting_started > Link : https://github.com/bro/bro/commit/af95026348688e0df8c867f67d2a53a3d440cf41 -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Wed Jan 29 11:47:58 2014 From: jira at bro-tracker.atlassian.net (Jeannette Dopheide (JIRA)) Date: Wed, 29 Jan 2014 13:47:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1123) topic/jdopheid/bro/edits_to_installation_and_getting_started In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeannette Dopheide updated BIT-1123: ------------------------------------ Fix Version/s: 2.3 > topic/jdopheid/bro/edits_to_installation_and_getting_started > ------------------------------------------------------------ > > Key: BIT-1123 > URL: https://bro-tracker.atlassian.net/browse/BIT-1123 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jeannette Dopheide > Fix For: 2.3 > > > Minor grammar edits to Installation and Quick Start pages > Also, please let me know if I need to modify future JIRA tickets. > Thanks, > Jeannette > ************************ > Repository : ssh://git at bro-ids.icir.org/bro > On branch : topic/jdopheid/bro/edits_to_installation_and_getting_started > Link : https://github.com/bro/bro/commit/4c52c378d5873abb052d688251f0ec7f5aa1c514 > And: > Repository : ssh://git at bro-ids.icir.org/bro > On branch : topic/jdopheid/bro/edits_to_installation_and_getting_started > Link : https://github.com/bro/bro/commit/af95026348688e0df8c867f67d2a53a3d440cf41 -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Wed Jan 29 15:27:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 29 Jan 2014 17:27:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1125) topic/jsiwek/http-file-id-caching In-Reply-To: References: Message-ID: Jon Siwek created BIT-1125: ------------------------------ Summary: topic/jsiwek/http-file-id-caching Key: BIT-1125 URL: https://bro-tracker.atlassian.net/browse/BIT-1125 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Jon Siwek Fix For: 2.3 This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Wed Jan 29 15:27:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 29 Jan 2014 17:27:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1125) topic/jsiwek/http-file-id-caching In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1125: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/http-file-id-caching > --------------------------------- > > Key: BIT-1125 > URL: https://bro-tracker.atlassian.net/browse/BIT-1125 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Wed Jan 29 15:27:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 29 Jan 2014 17:27:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1124) process command misplaces custom scripts In-Reply-To: References: Message-ID: Robin Sommer created BIT-1124: --------------------------------- Summary: process command misplaces custom scripts Key: BIT-1124 URL: https://bro-tracker.atlassian.net/browse/BIT-1124 Project: Bro Issue Tracker Issue Type: Problem Components: BroControl Affects Versions: 2.2 Reporter: Robin Sommer {noformat} # cat test.bro @load base/utils/site print Site::local_nets; {noformat} {{broctl process trace.pcap test.bro}} gives: {noformat} error in /usr/local/bro-2.2/share/bro/policy/misc/loaded-scripts.bro, line 4: syntax error, at or near ?module" {noformat} I believe it's due to test.bro being placed in the middle of the command line that {{process}} builds. If I move it to the end, it works fine. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From noreply at bro.org Thu Jan 30 00:00:15 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 30 Jan 2014 00:00:15 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401300800.s0U80Fas010347@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------------ ---------- ---------- ------------- ---------- ---------------------------------------------------------------- BIT-1125 [1] Bro Jon Siwek - 2014-01-29 2.3 Normal topic/jsiwek/http-file-id-caching [2] BIT-1123 [3] Bro Jeannette Dopheide - 2014-01-29 2.3 Normal topic/jdopheid/bro/edits_to_installation_and_getting_started [4] BIT-1122 [5] Bro Jon Siwek Seth Hall 2014-01-28 2.3 Normal topic/jsiwek/dns-improvements [6] BIT-1119 [7] Bro Jon Siwek - 2014-01-29 2.3 Normal topic/jsiwek/tcp-improvements [8] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- --------------------------------------------- 62b3cb0 [9] bro Bernhard Amann 2014-01-28 Also use exec-module test to check for leaks. [1] BIT-1125 https://bro-tracker.atlassian.net/browse/BIT-1125 [2] http-file-id-caching https://github.com/bro/bro/tree/topic/jsiwek/http-file-id-caching [3] BIT-1123 https://bro-tracker.atlassian.net/browse/BIT-1123 [4] edits_to_installation_and_getting_started https://github.com/bro/bro/tree/topic/jdopheid/bro/edits_to_installation_and_getting_started [5] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [6] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements [7] BIT-1119 https://bro-tracker.atlassian.net/browse/BIT-1119 [8] tcp-improvements https://github.com/bro/bro/tree/topic/jsiwek/tcp-improvements [9] 62b3cb0 https://github.com/bro/bro/commit/62b3cb0a5b7bdd8fed1d7d0dae3337115b2feae7 From robin at icir.org Thu Jan 30 08:50:33 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 30 Jan 2014 08:50:33 -0800 Subject: [Bro-Dev] Dot release? Message-ID: <20140130165033.GI98333@icir.org> Folks, making a 2.2.1 release has been coming up a few times and I'm thinking we should just snapshot current master for that. We've been fixing quite a number of things since 2.2, yet there aren't any larger new features yet (GRE tunnel decapsulation being the only one I can think of right now). I'd wait for two more things though: - Merging, and some testing, of Jon's recent file analysis framework API changes that make the file handle management more efficient. - Figuring out the exec and/or sumstats problems (it looks certain at this point that exec isn't cleaning up fully; and sumstats may have a larger than expected CPU impact, but that's not clear yet I believe). Once 2.2.1 is out, I'd then next work on merging my dynamic plugin code, which is mostly ready but needs cleanup, review, documentation, testing. How does that sound? If good, now would also be the time to finalize any other minor fixes that people might want to see in 2.2.1. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From slagell at illinois.edu Thu Jan 30 08:57:22 2014 From: slagell at illinois.edu (Slagell, Adam J) Date: Thu, 30 Jan 2014 16:57:22 +0000 Subject: [Bro-Dev] Dot release? In-Reply-To: <20140130165033.GI98333@icir.org> References: <20140130165033.GI98333@icir.org> Message-ID: <68A854A8-C032-4E86-9DBD-923AF6117A0D@illinois.edu> I like that plan. I think there are some minor Maverick's issues too that Daniel found. So we might want to get those in there as well. On Jan 30, 2014, at 10:50 AM, Robin Sommer wrote: > Folks, > > making a 2.2.1 release has been coming up a few times and I'm thinking > we should just snapshot current master for that. We've been fixing > quite a number of things since 2.2, yet there aren't any larger new > features yet (GRE tunnel decapsulation being the only one I can think > of right now). > > I'd wait for two more things though: > > - Merging, and some testing, of Jon's recent file analysis > framework API changes that make the file handle management more > efficient. > > - Figuring out the exec and/or sumstats problems (it looks certain > at this point that exec isn't cleaning up fully; and sumstats may > have a larger than expected CPU impact, but that's not clear yet I > believe). > > Once 2.2.1 is out, I'd then next work on merging my dynamic plugin > code, which is mostly ready but needs cleanup, review, documentation, > testing. > > How does that sound? If good, now would also be the time to finalize > any other minor fixes that people might want to see in 2.2.1. > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev ------ Adam J. Slagell Chief Information Security Officer Assistant Director, Cybersecurity National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.ncsa.illinois.edu/~slagell/ "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From bernhard at ICSI.Berkeley.EDU Thu Jan 30 10:17:16 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Thu, 30 Jan 2014 10:17:16 -0800 Subject: [Bro-Dev] Dot release? In-Reply-To: <68A854A8-C032-4E86-9DBD-923AF6117A0D@illinois.edu> References: <20140130165033.GI98333@icir.org> <68A854A8-C032-4E86-9DBD-923AF6117A0D@illinois.edu> Message-ID: I already told Robin - but just for the record, I think it is a good idea/plan. Bernhard On Jan 30, 2014, at 8:57 AM, Slagell, Adam J wrote: > I like that plan. I think there are some minor Maverick's issues too that Daniel found. So we might want to get those in there as well. > > On Jan 30, 2014, at 10:50 AM, Robin Sommer wrote: > >> Folks, >> >> making a 2.2.1 release has been coming up a few times and I'm thinking >> we should just snapshot current master for that. We've been fixing >> quite a number of things since 2.2, yet there aren't any larger new >> features yet (GRE tunnel decapsulation being the only one I can think >> of right now). >> >> I'd wait for two more things though: >> >> - Merging, and some testing, of Jon's recent file analysis >> framework API changes that make the file handle management more >> efficient. >> >> - Figuring out the exec and/or sumstats problems (it looks certain >> at this point that exec isn't cleaning up fully; and sumstats may >> have a larger than expected CPU impact, but that's not clear yet I >> believe). >> >> Once 2.2.1 is out, I'd then next work on merging my dynamic plugin >> code, which is mostly ready but needs cleanup, review, documentation, >> testing. >> >> How does that sound? If good, now would also be the time to finalize >> any other minor fixes that people might want to see in 2.2.1. >> >> Robin >> >> -- >> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org >> ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin >> _______________________________________________ >> bro-dev mailing list >> bro-dev at bro.org >> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > ------ > > Adam J. Slagell > Chief Information Security Officer > Assistant Director, Cybersecurity > National Center for Supercomputing Applications > University of Illinois at Urbana-Champaign > www.ncsa.illinois.edu/~slagell/ > > "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From jira at bro-tracker.atlassian.net Thu Jan 30 10:25:58 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 30 Jan 2014 12:25:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1124) process command misplaces custom scripts In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308#comment-15308 ] Daniel Thayer commented on BIT-1124: ------------------------------------ If you run "broctl process" with a custom script, then you need to specify "--" so that broctl knows where the Bro options end and where the custom scripts begin. So your example should be: broctl process trace.pcap -- test.bro Doing so moves test.bro farther down towards the end of the Bro command line. However, in this example we still get a different error, because broctl adds its own script (process-trace.bro, which contains a couple of redefs) at the very end of the Bro command. I will change the order so that the process-trace.bro is before the custom scripts (which solves the syntax error). > process command misplaces custom scripts > ---------------------------------------- > > Key: BIT-1124 > URL: https://bro-tracker.atlassian.net/browse/BIT-1124 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Robin Sommer > > {noformat} > # cat test.bro > @load base/utils/site > print Site::local_nets; > {noformat} > {{broctl process trace.pcap test.bro}} gives: > {noformat} > error in /usr/local/bro-2.2/share/bro/policy/misc/loaded-scripts.bro, line 4: syntax error, at or near ?module" > {noformat} > I believe it's due to test.bro being placed in the middle of the command line that {{process}} builds. If I move it to the end, it works fine. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From liam at broala.com Thu Jan 30 10:32:46 2014 From: liam at broala.com (Liam Randall) Date: Thu, 30 Jan 2014 13:32:46 -0500 Subject: [Bro-Dev] Dot release? In-Reply-To: References: <20140130165033.GI98333@icir.org> <68A854A8-C032-4E86-9DBD-923AF6117A0D@illinois.edu> Message-ID: Yes, the current master is WAY more stable on busy production sensors that 2.2. For sites really leaning on the intel framework master is the only way to go. Thanks, Liam Randall On Thu, Jan 30, 2014 at 1:17 PM, Bernhard Amann wrote: > I already told Robin - but just for the record, I think it is a good > idea/plan. > > Bernhard > > On Jan 30, 2014, at 8:57 AM, Slagell, Adam J wrote: > > > I like that plan. I think there are some minor Maverick's issues too > that Daniel found. So we might want to get those in there as well. > > > > On Jan 30, 2014, at 10:50 AM, Robin Sommer wrote: > > > >> Folks, > >> > >> making a 2.2.1 release has been coming up a few times and I'm thinking > >> we should just snapshot current master for that. We've been fixing > >> quite a number of things since 2.2, yet there aren't any larger new > >> features yet (GRE tunnel decapsulation being the only one I can think > >> of right now). > >> > >> I'd wait for two more things though: > >> > >> - Merging, and some testing, of Jon's recent file analysis > >> framework API changes that make the file handle management more > >> efficient. > >> > >> - Figuring out the exec and/or sumstats problems (it looks certain > >> at this point that exec isn't cleaning up fully; and sumstats may > >> have a larger than expected CPU impact, but that's not clear yet I > >> believe). > >> > >> Once 2.2.1 is out, I'd then next work on merging my dynamic plugin > >> code, which is mostly ready but needs cleanup, review, documentation, > >> testing. > >> > >> How does that sound? If good, now would also be the time to finalize > >> any other minor fixes that people might want to see in 2.2.1. > >> > >> Robin > >> > >> -- > >> Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > >> ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > >> _______________________________________________ > >> bro-dev mailing list > >> bro-dev at bro.org > >> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > > > ------ > > > > Adam J. Slagell > > Chief Information Security Officer > > Assistant Director, Cybersecurity > > National Center for Supercomputing Applications > > University of Illinois at Urbana-Champaign > > www.ncsa.illinois.edu/~slagell/ > > > > "Under the Illinois Freedom of Information Act (FOIA), any written > communication to or from University employees regarding University business > is a public record and may be subject to public disclosure." > > > > > > _______________________________________________ > > bro-dev mailing list > > bro-dev at bro.org > > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -- Liam Randall Managing Partner 510-281-0760 www.Broala.com >From the creators of Bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20140130/69030534/attachment.html From vallentin at icir.org Thu Jan 30 11:22:01 2014 From: vallentin at icir.org (Matthias Vallentin) Date: Thu, 30 Jan 2014 11:22:01 -0800 Subject: [Bro-Dev] Dot release? In-Reply-To: <20140130165033.GI98333@icir.org> References: <20140130165033.GI98333@icir.org> Message-ID: > I'd wait for two more things though: Aashish also raised some potential bugs with Bro's hashing. It appears that the Bloom filters fill up too quickly, i.e., do not meet their false positive requirements. My hunch is that this has to do with the construction of hash functions, perhaps they are not pairwise independent unless parametrized in a certain way, or perhaps there's just some other smaller bug in place. In any case, it needs to be fixed and I wonder whether 2.2.1 is the right target for that. Matthias From robin at icir.org Thu Jan 30 11:34:41 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 30 Jan 2014 11:34:41 -0800 Subject: [Bro-Dev] Dot release? In-Reply-To: References: <20140130165033.GI98333@icir.org> Message-ID: <20140130193441.GJ98333@icir.org> On Thu, Jan 30, 2014 at 11:22 -0800, you wrote: > fixed and I wonder whether 2.2.1 is the right target for that. Yes, that would be good to get in there too if we can figure out what's going on. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Thu Jan 30 15:37:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 30 Jan 2014 17:37:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1122) topic/jsiwek/dns-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15309#comment-15309 ] Jon Siwek commented on BIT-1122: -------------------------------- I just pushed another commit on this branch containing a rewrite of the query-reply state tracking and matching logic. It now relies on "dns_end" event to pair messages and log them. The old way of tracking the number of resource records seen versus the total number declared in the reply message is too unreliable in many cases. > topic/jsiwek/dns-improvements > ----------------------------- > > Key: BIT-1122 > URL: https://bro-tracker.atlassian.net/browse/BIT-1122 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Seth Hall > Fix For: 2.3 > > > This branch is in bro, bro-testing, and bro-testing-private repos. > - Fixes incorrect parsing of DNS message format for messages with empty question sections. > - Changes dns.log to only include standard queries (opcode == 1). > - Adds "dns_unknown_reply" event for RR types that Bro doesn't know how to parse, which improves accuracy of request-reply pair matching performed by the default DNS scripts. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Thu Jan 30 16:04:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 30 Jan 2014 18:04:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1119: ------------------------------ Status: Open (was: Merge Request) > topic/jsiwek/tcp-improvements > ----------------------------- > > Key: BIT-1119 > URL: https://bro-tracker.atlassian.net/browse/BIT-1119 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc > > > This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail). > The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Thu Jan 30 16:12:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 30 Jan 2014 18:12:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1125) topic/jsiwek/http-file-id-caching In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310#comment-15310 ] Robin Sommer commented on BIT-1125: ----------------------------------- For the case that the core can compute the file id itself without needing the script-land, is the idea that it then just passes it in as the {{cached_id}}? > topic/jsiwek/http-file-id-caching > --------------------------------- > > Key: BIT-1125 > URL: https://bro-tracker.atlassian.net/browse/BIT-1125 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > > This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Thu Jan 30 17:35:58 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 30 Jan 2014 19:35:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1124) process command misplaces custom scripts In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311#comment-15311 ] Daniel Thayer commented on BIT-1124: ------------------------------------ In branch topic/dnthayer/ticket1124, I've changed the order of scripts so that user-specified scripts are always at the end of the Bro command, and I've improved the broctl help message to show how the process command should be used. > process command misplaces custom scripts > ---------------------------------------- > > Key: BIT-1124 > URL: https://bro-tracker.atlassian.net/browse/BIT-1124 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Robin Sommer > Fix For: 2.3 > > > {noformat} > # cat test.bro > @load base/utils/site > print Site::local_nets; > {noformat} > {{broctl process trace.pcap test.bro}} gives: > {noformat} > error in /usr/local/bro-2.2/share/bro/policy/misc/loaded-scripts.bro, line 4: syntax error, at or near ?module" > {noformat} > I believe it's due to test.bro being placed in the middle of the command line that {{process}} builds. If I move it to the end, it works fine. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Thu Jan 30 17:35:58 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 30 Jan 2014 19:35:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1124) process command misplaces custom scripts In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1124: ------------------------------- Status: Merge Request (was: Open) > process command misplaces custom scripts > ---------------------------------------- > > Key: BIT-1124 > URL: https://bro-tracker.atlassian.net/browse/BIT-1124 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Robin Sommer > Fix For: 2.3 > > > {noformat} > # cat test.bro > @load base/utils/site > print Site::local_nets; > {noformat} > {{broctl process trace.pcap test.bro}} gives: > {noformat} > error in /usr/local/bro-2.2/share/bro/policy/misc/loaded-scripts.bro, line 4: syntax error, at or near ?module" > {noformat} > I believe it's due to test.bro being placed in the middle of the command line that {{process}} builds. If I move it to the end, it works fine. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Thu Jan 30 17:35:58 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 30 Jan 2014 19:35:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1124) process command misplaces custom scripts In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1124: ------------------------------- Fix Version/s: 2.3 > process command misplaces custom scripts > ---------------------------------------- > > Key: BIT-1124 > URL: https://bro-tracker.atlassian.net/browse/BIT-1124 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Robin Sommer > Fix For: 2.3 > > > {noformat} > # cat test.bro > @load base/utils/site > print Site::local_nets; > {noformat} > {{broctl process trace.pcap test.bro}} gives: > {noformat} > error in /usr/local/bro-2.2/share/bro/policy/misc/loaded-scripts.bro, line 4: syntax error, at or near ?module" > {noformat} > I believe it's due to test.bro being placed in the middle of the command line that {{process}} builds. If I move it to the end, it works fine. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From seth at icir.org Thu Jan 30 17:46:50 2014 From: seth at icir.org (Seth Hall) Date: Thu, 30 Jan 2014 20:46:50 -0500 Subject: [Bro-Dev] Dot release? In-Reply-To: References: <20140130165033.GI98333@icir.org> <68A854A8-C032-4E86-9DBD-923AF6117A0D@illinois.edu> Message-ID: <6FF63268-4757-46A2-B343-BFAEEA0580FA@icir.org> On Jan 30, 2014, at 1:17 PM, Bernhard Amann wrote: > I already told Robin - but just for the record, I think it is a good idea/plan. I'm in the same boat as Bernhard here. Looking forward to the 2.2.1 release. ;) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20140130/32080bfe/attachment.bin From jira at bro-tracker.atlassian.net Thu Jan 30 17:52:58 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 30 Jan 2014 19:52:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1125) topic/jsiwek/http-file-id-caching In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1125: --------------------------- Attachment: signature.asc I've been thinking about this and I'm not sure how I feel about analyzers computing their own identifiers. That actually causes inconsistent behavior because a user would have to know that a certain analyzer does that or that it does that in certain cases. i.e. the user would have no control over how file chunks are tied together to form complete files. Is this something that is already implemented? > topic/jsiwek/http-file-id-caching > --------------------------------- > > Key: BIT-1125 > URL: https://bro-tracker.atlassian.net/browse/BIT-1125 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc > > > This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From noreply at bro.org Fri Jan 31 00:00:11 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 31 Jan 2014 00:00:11 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201401310800.s0V80BAL023823@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------------ ---------- ---------- ------------- ---------- ---------------------------------------------------------------- BIT-1125 [1] Bro Jon Siwek - 2014-01-30 2.3 Normal topic/jsiwek/http-file-id-caching [2] BIT-1124 [3] BroControl Robin Sommer - 2014-01-30 2.3 Normal process command misplaces custom scripts BIT-1123 [4] Bro Jeannette Dopheide - 2014-01-29 2.3 Normal topic/jdopheid/bro/edits_to_installation_and_getting_started [5] BIT-1122 [6] Bro Jon Siwek Seth Hall 2014-01-30 2.3 Normal topic/jsiwek/dns-improvements [7] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- --------------------------------------------- 62b3cb0 [8] bro Bernhard Amann 2014-01-28 Also use exec-module test to check for leaks. [1] BIT-1125 https://bro-tracker.atlassian.net/browse/BIT-1125 [2] http-file-id-caching https://github.com/bro/bro/tree/topic/jsiwek/http-file-id-caching [3] BIT-1124 https://bro-tracker.atlassian.net/browse/BIT-1124 [4] BIT-1123 https://bro-tracker.atlassian.net/browse/BIT-1123 [5] edits_to_installation_and_getting_started https://github.com/bro/bro/tree/topic/jdopheid/bro/edits_to_installation_and_getting_started [6] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [7] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements [8] 62b3cb0 https://github.com/bro/bro/commit/62b3cb0a5b7bdd8fed1d7d0dae3337115b2feae7 From jira at bro-tracker.atlassian.net Fri Jan 31 07:41:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 31 Jan 2014 09:41:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1125) topic/jsiwek/http-file-id-caching In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313#comment-15313 ] Jon Siwek commented on BIT-1125: -------------------------------- {quote} For the case that the core can compute the file id itself without needing the script-land, is the idea that it then just passes it in as the cached_id? {quote} Yes, and it can ignore the return value from those methods and just always supply its own file ID if that's what it wants to do. {quote} I've been thinking about this and I'm not sure how I feel about analyzers computing their own identifiers. That actually causes inconsistent behavior because a user would have to know that a certain analyzer does that or that it does that in certain cases. i.e. the user would have no control over how file chunks are tied together to form complete files. {quote} Probably few users are going to want change how file IDs are calculated in the first place and the cases where an analyzer directly calculated a file ID are probably going to be the ones where there's not really any other sane way to do it. I do agree it's somewhat inconsistent, though. {quote} Is this something that is already implemented? {quote} Yes, it comes free w/ the new support for caching a file ID returned from script-land due to the way the code is structured (just in this case the return value from file analysis API functions is whatever was passed in instead of something calculated in script-land). > topic/jsiwek/http-file-id-caching > --------------------------------- > > Key: BIT-1125 > URL: https://bro-tracker.atlassian.net/browse/BIT-1125 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc > > > This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Fri Jan 31 08:17:58 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Fri, 31 Jan 2014 10:17:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1125) topic/jsiwek/http-file-id-caching In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1125: --------------------------- Attachment: signature.asc True. I think the cases where there is really only one way to do it are pretty limited. Maybe just the old "File" analyzer that is used for FTP and IRC transfers? Ah, ok. Thanks. > topic/jsiwek/http-file-id-caching > --------------------------------- > > Key: BIT-1125 > URL: https://bro-tracker.atlassian.net/browse/BIT-1125 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc, signature.asc > > > This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Fri Jan 31 08:17:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Jan 2014 10:17:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1125) topic/jsiwek/http-file-id-caching In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315#comment-15315 ] Robin Sommer commented on BIT-1125: ----------------------------------- Agree with Jon, I think we want the option, it just feels unnecessary to pass through script-land in cases where there's really no question on how to compute the handle. I don't think that's actually different from other low-level decisions analyzers sometimes make on how to process something without asking script-land for its opinion. Also, analyzers can document whether they offer any customization. I think I'll rename {{cached_id}} to {{precomputed_id}} then make it cover both cases. (and I would like to have a document eventually that summarizes the options an analyzer have :-) > topic/jsiwek/http-file-id-caching > --------------------------------- > > Key: BIT-1125 > URL: https://bro-tracker.atlassian.net/browse/BIT-1125 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc, signature.asc > > > This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Fri Jan 31 08:26:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Jan 2014 10:26:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1125) topic/jsiwek/http-file-id-caching In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316#comment-15316 ] Robin Sommer commented on BIT-1125: ----------------------------------- {{I think the cases where there is really only one way to do it are pretty limited.} Also recursive content inspection of container formats. > topic/jsiwek/http-file-id-caching > --------------------------------- > > Key: BIT-1125 > URL: https://bro-tracker.atlassian.net/browse/BIT-1125 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc, signature.asc > > > This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Fri Jan 31 08:43:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Jan 2014 10:43:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1125) topic/jsiwek/http-file-id-caching In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1125: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/jsiwek/http-file-id-caching > --------------------------------- > > Key: BIT-1125 > URL: https://bro-tracker.atlassian.net/browse/BIT-1125 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc, signature.asc > > > This branch is in bro and bro-testing repos. It adds a file ID caching / "fast path" mechanism to the file analysis API and adapts HTTP to use it for performance improvement. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From robin at icir.org Fri Jan 31 11:24:28 2014 From: robin at icir.org (Robin Sommer) Date: Fri, 31 Jan 2014 11:24:28 -0800 Subject: [Bro-Dev] Test timing measurements (Re: [Bro-Commits] [git/btest] topic/robin/timing: Adding a timing mode that records test execution times per host. (808fd76)) In-Reply-To: <1D657054C5E2994C98FE68E14DD10A2B3894345B04@EXMAIL1.ohio.edu> References: <201401311701.s0VH1eU7005355@bro-ids.icir.org> <1D657054C5E2994C98FE68E14DD10A2B3894345B04@EXMAIL1.ohio.edu> Message-ID: <20140131192428.GP3813@icir.org> (Moving from bro-commits to bro-dev). On Fri, Jan 31, 2014 at 12:51 -0500, you wrote: > Instruction counts are probably going to have a strong dependency on > the compiler version / options used to generate the code. I believe > these counts could additionally be influenced by e.g. library > upgrades, even when restricted to a single host and using a specific > compiler / options. True, but I'm not sure that's necessarily a bad thing. If the count changes signficantly, it's worth understanding where it's coming from I would say. btest won't complain as long as deviations are within a reasonable range (1% by default, don't know if that's the right value). I'm also not sure if instruction count is the right feature; there are plenty others one could measure, like cycles etc. I was just thinking this might be the most stable. > One alternative approach to tracking IDs for timing baselines might be > to use system tools to gather a list of all libraries bro is linked > against. A problem with this is that btest doesn't know about Bro. :-) The way I'm doing it currently is that instruction count is measured for all BTEST-EXEC commands that are part of a test, which are then summed up for a single number. I'd like to keep it the way that btest can measure arbitrary command lines (which is part of the challenge of finding a stable way of doing so ...). > Additionally, formatting the temporary file in a human-readable way > and keeping it as part of / in addition to the baseline can yield > potentially useful information when looking into timing differences. It's, more or less, human-readable: > cat Baseline/_Timing/2a6b457d90e93b6688f312f87f677c5c tests.m57-long 705347795206 tests.ipv6 104508274160 tests.m57-short 68458131160 What I'm mostly wondering about is if it's worth commiting data that's very specific to a single user/machine to the repos? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Fri Jan 31 11:46:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Jan 2014 13:46:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1124) process command misplaces custom scripts In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1124: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > process command misplaces custom scripts > ---------------------------------------- > > Key: BIT-1124 > URL: https://bro-tracker.atlassian.net/browse/BIT-1124 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Robin Sommer > Fix For: 2.3 > > > {noformat} > # cat test.bro > @load base/utils/site > print Site::local_nets; > {noformat} > {{broctl process trace.pcap test.bro}} gives: > {noformat} > error in /usr/local/bro-2.2/share/bro/policy/misc/loaded-scripts.bro, line 4: syntax error, at or near ?module" > {noformat} > I believe it's due to test.bro being placed in the middle of the command line that {{process}} builds. If I move it to the end, it works fine. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Fri Jan 31 11:46:58 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 31 Jan 2014 13:46:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1123) topic/jdopheid/bro/edits_to_installation_and_getting_started In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1123: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/jdopheid/bro/edits_to_installation_and_getting_started > ------------------------------------------------------------ > > Key: BIT-1123 > URL: https://bro-tracker.atlassian.net/browse/BIT-1123 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jeannette Dopheide > Fix For: 2.3 > > > Minor grammar edits to Installation and Quick Start pages > Also, please let me know if I need to modify future JIRA tickets. > Thanks, > Jeannette > ************************ > Repository : ssh://git at bro-ids.icir.org/bro > On branch : topic/jdopheid/bro/edits_to_installation_and_getting_started > Link : https://github.com/bro/bro/commit/4c52c378d5873abb052d688251f0ec7f5aa1c514 > And: > Repository : ssh://git at bro-ids.icir.org/bro > On branch : topic/jdopheid/bro/edits_to_installation_and_getting_started > Link : https://github.com/bro/bro/commit/af95026348688e0df8c867f67d2a53a3d440cf41 -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Fri Jan 31 14:14:58 2014 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Fri, 31 Jan 2014 16:14:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1126) Logs disappearing after bro termination In-Reply-To: References: Message-ID: Aashish Sharma created BIT-1126: ----------------------------------- Summary: Logs disappearing after bro termination Key: BIT-1126 URL: https://bro-tracker.atlassian.net/browse/BIT-1126 Project: Bro Issue Tracker Issue Type: Problem Components: BroControl Affects Versions: 2.2 Environment: freebsd Reporter: Aashish Sharma Priority: High I have noticed several times that in the event of bro termination after expiration of StopTimeout, bro logs disappear. This is generally seen when log sizes are much bigger (for example after overnight) This issue was present in bro-2.1 and continue to be present in bro-2.2 I see (kill from control.py - kick in often when stopping or restarting bro) because catch-n-release is still trying to flush its tables (which takes long time). Then there is no logs from overnight! I can provide more information if desired (or even a test case). Thanks, Aashish -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Fri Jan 31 15:11:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 31 Jan 2014 17:11:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317#comment-15317 ] Jon Siwek commented on BIT-1119: -------------------------------- Added a new commit on the branch to add a script which auto-detects/warns about running on filtered trace. > topic/jsiwek/tcp-improvements > ----------------------------- > > Key: BIT-1119 > URL: https://bro-tracker.atlassian.net/browse/BIT-1119 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc > > > This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail). > The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211) From jira at bro-tracker.atlassian.net Fri Jan 31 15:11:58 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 31 Jan 2014 17:11:58 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1119: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/tcp-improvements > ----------------------------- > > Key: BIT-1119 > URL: https://bro-tracker.atlassian.net/browse/BIT-1119 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc > > > This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail). > The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-07-028#6211)