From noreply at bro.org Thu Aug 1 00:00:03 2013 From: noreply at bro.org (Merge Tracker) Date: Thu, 1 Aug 2013 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308010700.r71703MK004260@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- -------------- ---------- ---------- ------------- ---------- ------------------------- BIT-1048 [1] Bro Bernhard Amann - 2013-07-31 - Normal merge topic/bernhard/topk [1] BIT-1048 https://bro-tracker.atlassian.net/browse/BIT-1048 From jira at bro-tracker.atlassian.net Thu Aug 1 03:49:06 2013 From: jira at bro-tracker.atlassian.net (Anonymous (JIRA)) Date: Thu, 1 Aug 2013 05:49:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1047) Delete old scripts before installing new ones In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anonymous updated BIT-1047: --------------------------- Status: In Progress (was: Open) > Delete old scripts before installing new ones > --------------------------------------------- > > Key: BIT-1047 > URL: https://bro-tracker.atlassian.net/browse/BIT-1047 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: Robin Sommer > Fix For: 2.2 > > > People keep having problems when they install a new Bro version > over the installation of an old one because scripts that have disappeared in the new version will keep sticking around from the previous installation. > We should simply remove the old scripts/base and scripts/policy before installing anything new. People aren't supposed to edit in there so that should be safe. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 07:31:06 2013 From: jira at bro-tracker.atlassian.net (Anonymous (JIRA)) Date: Thu, 1 Aug 2013 09:31:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1047) Delete old scripts before installing new ones In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anonymous updated BIT-1047: --------------------------- Status: Open (was: In Progress) > Delete old scripts before installing new ones > --------------------------------------------- > > Key: BIT-1047 > URL: https://bro-tracker.atlassian.net/browse/BIT-1047 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: Robin Sommer > Fix For: 2.2 > > > People keep having problems when they install a new Bro version > over the installation of an old one because scripts that have disappeared in the new version will keep sticking around from the previous installation. > We should simply remove the old scripts/base and scripts/policy before installing anything new. People aren't supposed to edit in there so that should be safe. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 10:15:06 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 1 Aug 2013 12:15:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-951) Error message from SSL delay logging In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-951: -------------------------- Resolution: Fixed Status: Closed (was: Open) > Error message from SSL delay logging > ------------------------------------ > > Key: BIT-951 > URL: https://bro-tracker.atlassian.net/browse/BIT-951 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Seth Hall > Assignee: Seth Hall > Priority: Low > Fix For: 2.2 > > > This is mostly a reminder for me.. > {noformat} > 1362104865.826919 ./scripts/base/protocols/ssl/./main.bro, lines 193-194: SSL delay tokens not released in time ({ > notary > }) > {noformat} > I got that message in my console when I ran Bro with the local.bro script. I think that we should just go ahead and log the line without waiting if Bro is terminating. I'd rather get the log line even without any extended information and just skip the error message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 10:15:06 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 1 Aug 2013 12:15:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-951) Error message from SSL delay logging In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416#comment-13416 ] Seth Hall commented on BIT-951: ------------------------------- I fixed this with a commit directly to master this morning. > Error message from SSL delay logging > ------------------------------------ > > Key: BIT-951 > URL: https://bro-tracker.atlassian.net/browse/BIT-951 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Seth Hall > Assignee: Seth Hall > Priority: Low > Fix For: 2.2 > > > This is mostly a reminder for me.. > {noformat} > 1362104865.826919 ./scripts/base/protocols/ssl/./main.bro, lines 193-194: SSL delay tokens not released in time ({ > notary > }) > {noformat} > I got that message in my console when I ran Bro with the local.bro script. I think that we should just go ahead and log the line without waiting if Bro is terminating. I'd rather get the log line even without any extended information and just skip the error message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 10:17:06 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 1 Aug 2013 12:17:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-927) topic/seth/metrics-merge: Metrics framework updates In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-927: -------------------------- Resolution: Fixed Status: Closed (was: Open) Closing. This ticket is wildly outdated. > topic/seth/metrics-merge: Metrics framework updates > --------------------------------------------------- > > Key: BIT-927 > URL: https://bro-tracker.atlassian.net/browse/BIT-927 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro > Affects Versions: git/master > Reporter: Seth Hall > Assignee: Seth Hall > Fix For: 2.2 > > > This branch is in a workable state and basically ready to be merged, but I'd appreciate a more detailed API/sanity review from anyone willing to take a look before it gets merged. This code is starting to get more and more important and I don't think we can afford to get it wrong for a release. New scripts include policy/misc/scan.bro, policy/misc/detect-traceroute, and various metrics test scripts that you can find by searching for "base/frameworks/metrics". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 10:25:06 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 1 Aug 2013 12:25:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-176) Adding names to mails from tracker In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-176: -------------------------- Resolution: Fixed Status: Closed (was: Open) This one definitely doesn't apply anymore. > Adding names to mails from tracker > ---------------------------------- > > Key: BIT-176 > URL: https://bro-tracker.atlassian.net/browse/BIT-176 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: TicketTracker > Reporter: Robin Sommer > Assignee: Seth Hall > > Would be cool if the mails generated by the tracker came with the real-name of the person triggering the change in the From line (rather than "Bro Tracker"). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 10:25:06 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 1 Aug 2013 12:25:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-333) Add workflow transitions to Trac's email interface. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-333: -------------------------- Resolution: Fixed Status: Closed (was: Open) Again, doesn't apply anymore. > Add workflow transitions to Trac's email interface. > --------------------------------------------------- > > Key: BIT-333 > URL: https://bro-tracker.atlassian.net/browse/BIT-333 > Project: Bro Issue Tracker > Issue Type: Task > Components: TicketTracker > Reporter: Robin Sommer > Assignee: Seth Hall > > Currently one can't simply close a ticket via mail while setting the resolution correctly. > See https://subtrac.sara.nl/oss/email2trac/wiki/Email2tracConfiguration#Workflow > Note that this should probably wait until we have updated the workflow we use in Trac (like with getting rid of the distinction between new/seen). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 10:52:06 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 1 Aug 2013 12:52:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1049) topic/jsiwek/faf-updates In-Reply-To: References: Message-ID: Jon Siwek created BIT-1049: ------------------------------ Summary: topic/jsiwek/faf-updates Key: BIT-1049 URL: https://bro-tracker.atlassian.net/browse/BIT-1049 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Jon Siwek Assignee: Robin Sommer Fix For: 2.2 This branch has some remaining cleanups related to the FAF: * Fix outdated docs. * Had to reorganize layout of some scripts to make "bare mode" loading work (Broxygen currently relies on that) * Internal code refactoring ** Distinct tag type for file analyzers, separate from protocol analyzer ** File analyzer tag no longer lumped in with AnalyzerArgs ** Removed some duplicated tag/component management code by giving it separate class templates. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 10:52:06 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 1 Aug 2013 12:52:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1047) Delete old scripts before installing new ones In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1047: --------------------------- Status: Merge Request (was: Open) > Delete old scripts before installing new ones > --------------------------------------------- > > Key: BIT-1047 > URL: https://bro-tracker.atlassian.net/browse/BIT-1047 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: Robin Sommer > Fix For: 2.2 > > > People keep having problems when they install a new Bro version > over the installation of an old one because scripts that have disappeared in the new version will keep sticking around from the previous installation. > We should simply remove the old scripts/base and scripts/policy before installing anything new. People aren't supposed to edit in there so that should be safe. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 10:52:06 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 1 Aug 2013 12:52:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1047) Delete old scripts before installing new ones In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1047: --------------------------- Status: Open (was: Merge Request) > Delete old scripts before installing new ones > --------------------------------------------- > > Key: BIT-1047 > URL: https://bro-tracker.atlassian.net/browse/BIT-1047 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: Robin Sommer > Fix For: 2.2 > > > People keep having problems when they install a new Bro version > over the installation of an old one because scripts that have disappeared in the new version will keep sticking around from the previous installation. > We should simply remove the old scripts/base and scripts/policy before installing anything new. People aren't supposed to edit in there so that should be safe. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 10:52:06 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 1 Aug 2013 12:52:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1049) topic/jsiwek/faf-updates In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1049: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/faf-updates > ------------------------ > > Key: BIT-1049 > URL: https://bro-tracker.atlassian.net/browse/BIT-1049 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.2 > > > This branch has some remaining cleanups related to the FAF: > * Fix outdated docs. > * Had to reorganize layout of some scripts to make "bare mode" loading work (Broxygen currently relies on that) > * Internal code refactoring > ** Distinct tag type for file analyzers, separate from protocol analyzer > ** File analyzer tag no longer lumped in with AnalyzerArgs > ** Removed some duplicated tag/component management code by giving it separate class templates. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 12:29:06 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 1 Aug 2013 14:29:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1049) topic/jsiwek/faf-updates In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420#comment-13420 ] Jon Siwek commented on BIT-1049: -------------------------------- Might have introduced a ref-counting bug, need to look more in to it. > topic/jsiwek/faf-updates > ------------------------ > > Key: BIT-1049 > URL: https://bro-tracker.atlassian.net/browse/BIT-1049 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.2 > > > This branch has some remaining cleanups related to the FAF: > * Fix outdated docs. > * Had to reorganize layout of some scripts to make "bare mode" loading work (Broxygen currently relies on that) > * Internal code refactoring > ** Distinct tag type for file analyzers, separate from protocol analyzer > ** File analyzer tag no longer lumped in with AnalyzerArgs > ** Removed some duplicated tag/component management code by giving it separate class templates. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 12:29:06 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 1 Aug 2013 14:29:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1049) topic/jsiwek/faf-updates In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1049: --------------------------- Status: Open (was: Merge Request) > topic/jsiwek/faf-updates > ------------------------ > > Key: BIT-1049 > URL: https://bro-tracker.atlassian.net/browse/BIT-1049 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.2 > > > This branch has some remaining cleanups related to the FAF: > * Fix outdated docs. > * Had to reorganize layout of some scripts to make "bare mode" loading work (Broxygen currently relies on that) > * Internal code refactoring > ** Distinct tag type for file analyzers, separate from protocol analyzer > ** File analyzer tag no longer lumped in with AnalyzerArgs > ** Removed some duplicated tag/component management code by giving it separate class templates. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 12:42:06 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 1 Aug 2013 14:42:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1049) topic/jsiwek/faf-updates In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1049: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/faf-updates > ------------------------ > > Key: BIT-1049 > URL: https://bro-tracker.atlassian.net/browse/BIT-1049 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.2 > > > This branch has some remaining cleanups related to the FAF: > * Fix outdated docs. > * Had to reorganize layout of some scripts to make "bare mode" loading work (Broxygen currently relies on that) > * Internal code refactoring > ** Distinct tag type for file analyzers, separate from protocol analyzer > ** File analyzer tag no longer lumped in with AnalyzerArgs > ** Removed some duplicated tag/component management code by giving it separate class templates. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 15:43:06 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 1 Aug 2013 17:43:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1010) BroControl plugin for adding environment variables In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421#comment-13421 ] Daniel Thayer commented on BIT-1010: ------------------------------------ In the case of "env:VAR1=1", the parser recognizes "env" as the key. > BroControl plugin for adding environment variables > -------------------------------------------------- > > Key: BIT-1010 > URL: https://bro-tracker.atlassian.net/browse/BIT-1010 > Project: Bro Issue Tracker > Issue Type: Task > Components: BroControl > Affects Versions: git/master > Reporter: Seth Hall > Assignee: Daniel Thayer > Fix For: 2.2 > > Attachments: env_vars.py > > > We should have the ability to add environment variables to Bro at start up time. The option should be available globally in broctl.cfg and per-node in node.cfg. The environments variables should be applied to the process with priority based on how specific the variable is applied (per-node variables defined after global variables so that the per-node variable is used). > As a name suggestion for the configuration option: env_vars (same name in node.cfg and broctl.cfg). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 15:48:06 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 1 Aug 2013 17:48:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1010) BroControl plugin for adding environment variables In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1010: ------------------------------- Status: Merge Request (was: Open) > BroControl plugin for adding environment variables > -------------------------------------------------- > > Key: BIT-1010 > URL: https://bro-tracker.atlassian.net/browse/BIT-1010 > Project: Bro Issue Tracker > Issue Type: Task > Components: BroControl > Affects Versions: git/master > Reporter: Seth Hall > Assignee: Daniel Thayer > Fix For: 2.2 > > Attachments: env_vars.py > > > We should have the ability to add environment variables to Bro at start up time. The option should be available globally in broctl.cfg and per-node in node.cfg. The environments variables should be applied to the process with priority based on how specific the variable is applied (per-node variables defined after global variables so that the per-node variable is used). > As a name suggestion for the configuration option: env_vars (same name in node.cfg and broctl.cfg). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 17:49:06 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 1 Aug 2013 19:49:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1048) merge topic/bernhard/topk In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1048: ------------------------------ Resolution: Merged Status: Closed (was: Merge Request) > merge topic/bernhard/topk > ------------------------- > > Key: BIT-1048 > URL: https://bro-tracker.atlassian.net/browse/BIT-1048 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.2 > Reporter: Bernhard Amann > Labels: sumstats, topk > > merge topic/bernhard/topk. The branch should work and is integrated with sumstats. There is a small upcoming interface change - however this will just add new functionality and just change the bifs. For users of sumstats, the integration should not change at all, for others it might take minimal changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 17:49:06 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 1 Aug 2013 19:49:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1049) topic/jsiwek/faf-updates In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1049: ------------------------------ Resolution: Merged Status: Closed (was: Merge Request) > topic/jsiwek/faf-updates > ------------------------ > > Key: BIT-1049 > URL: https://bro-tracker.atlassian.net/browse/BIT-1049 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.2 > > > This branch has some remaining cleanups related to the FAF: > * Fix outdated docs. > * Had to reorganize layout of some scripts to make "bare mode" loading work (Broxygen currently relies on that) > * Internal code refactoring > ** Distinct tag type for file analyzers, separate from protocol analyzer > ** File analyzer tag no longer lumped in with AnalyzerArgs > ** Removed some duplicated tag/component management code by giving it separate class templates. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 1 17:49:06 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 1 Aug 2013 19:49:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1039) Merge request for Bloom filters In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1039: ------------------------------ Resolution: Merged Status: Closed (was: Open) > Merge request for Bloom filters > ------------------------------- > > Key: BIT-1039 > URL: https://bro-tracker.atlassian.net/browse/BIT-1039 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Reporter: Matthias Vallentin > Fix For: 2.2 > > > The Bloom filter implementation in `topic/matthias/bloom-filter` is ready to merge into master. Have a look at the very end of `bro.bif` for the script-land interface. > Internally, we have a new `BloomFilterVal`, which is serializable and mergeable and thus ready for cluster use. This `Val` contains a polymorphic Bloom filter instance, which hides the concrete Bloom filter type (currently only basic and counting). Moreover, this branch introduces the notion of ''hashers'', which are parameterizable (i.e., seedable) structures for hashing values ''k'' times. I recall that Bernhard waits for this feature. See `Hasher.h` for the documented interface. > In the future, we need to rethink how to construct hash functions which only depend on a seed given at script land. This will be important when sharing Bloom filters across organizational boundaries. At this point, the implementation relies on `CompHash` (at least for composite values, such as records) which itself depends on the initial Bro seed generated at startup time or when the user specifies the environment variable `$BRO_SEED`. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Fri Aug 2 00:00:03 2013 From: noreply at bro.org (Merge Tracker) Date: Fri, 2 Aug 2013 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308020700.r72703sa009905@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------- ---------- ------------- ---------- -------------------------------------------------- BIT-1010 [1] BroControl Seth Hall Daniel Thayer 2013-08-01 2.2 Normal BroControl plugin for adding environment variables [1] BIT-1010 https://bro-tracker.atlassian.net/browse/BIT-1010 From jira at bro-tracker.atlassian.net Fri Aug 2 07:58:06 2013 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Fri, 2 Aug 2013 09:58:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1050) Merge request for DHCP analyzer In-Reply-To: References: Message-ID: Vlad Grigorescu created BIT-1050: ------------------------------------ Summary: Merge request for DHCP analyzer Key: BIT-1050 URL: https://bro-tracker.atlassian.net/browse/BIT-1050 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: 2.2 Reporter: Vlad Grigorescu topic/vladg/dhcp is ready to go. I've been running it in prod with no problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 2 07:58:06 2013 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Fri, 2 Aug 2013 09:58:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1050) Merge request for DHCP analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Grigorescu updated BIT-1050: --------------------------------- Status: Merge Request (was: Open) > Merge request for DHCP analyzer > ------------------------------- > > Key: BIT-1050 > URL: https://bro-tracker.atlassian.net/browse/BIT-1050 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.2 > Reporter: Vlad Grigorescu > Labels: analyzer > > topic/vladg/dhcp is ready to go. I've been running it in prod with no problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Sat Aug 3 00:00:05 2013 From: noreply at bro.org (Merge Tracker) Date: Sat, 3 Aug 2013 00:00:05 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308030700.r73705Rb003751@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- --------------- ------------- ---------- ------------- ---------- -------------------------------------------------- BIT-1050 [1] Bro Vlad Grigorescu - 2013-08-02 - Normal Merge request for DHCP analyzer BIT-1010 [2] BroControl Seth Hall Daniel Thayer 2013-08-01 2.2 Normal BroControl plugin for adding environment variables [1] BIT-1050 https://bro-tracker.atlassian.net/browse/BIT-1050 [2] BIT-1010 https://bro-tracker.atlassian.net/browse/BIT-1010 From jira at bro-tracker.atlassian.net Sat Aug 3 08:53:06 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Sat, 3 Aug 2013 10:53:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1010) BroControl plugin for adding environment variables In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1010: ------------------------------ Resolution: Merged Status: Closed (was: Merge Request) > BroControl plugin for adding environment variables > -------------------------------------------------- > > Key: BIT-1010 > URL: https://bro-tracker.atlassian.net/browse/BIT-1010 > Project: Bro Issue Tracker > Issue Type: Task > Components: BroControl > Affects Versions: git/master > Reporter: Seth Hall > Assignee: Daniel Thayer > Fix For: 2.2 > > Attachments: env_vars.py > > > We should have the ability to add environment variables to Bro at start up time. The option should be available globally in broctl.cfg and per-node in node.cfg. The environments variables should be applied to the process with priority based on how specific the variable is applied (per-node variables defined after global variables so that the per-node variable is used). > As a name suggestion for the configuration option: env_vars (same name in node.cfg and broctl.cfg). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Sat Aug 3 21:15:06 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Sat, 3 Aug 2013 23:15:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1050) Merge request for DHCP analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1050: --------------------------------- Assignee: Seth Hall Please take a look at the policy scripts. > Merge request for DHCP analyzer > ------------------------------- > > Key: BIT-1050 > URL: https://bro-tracker.atlassian.net/browse/BIT-1050 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.2 > Reporter: Vlad Grigorescu > Assignee: Seth Hall > Labels: analyzer > > topic/vladg/dhcp is ready to go. I've been running it in prod with no problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Sat Aug 3 21:17:06 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Sat, 3 Aug 2013 23:17:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1050) Merge request for DHCP analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423#comment-13423 ] Robin Sommer commented on BIT-1050: ----------------------------------- Nice. I've merged this with some small changes. Notes: - I'm not sure I like the structure with the two policy scripts. As we don't have anything else extracting MAC addresses currently, would it make sense to just move them into one? If not, I'd at least move the one out of misc, and also change the interface so that (1) users don't manipulate the table directly (but call functions instead), and (2) the logging stays internal to the script (the same functions could take care of that). But I'll let Seth take a look at this (or has he already?), assigning ticket to him. - format_mac(): The comment says "Supports both EUI-48 and EUI-64. If it's neither, returns an empty string.". That didn't seem to match the code, which always passed back something. I've changed the function to take a length parameter, just assuming its long enough makes me uneasy. So now if less bytes are passed in, it returns the empty string the comment promises. :) Also, moved to net_utils as fmt_mac(). - A note for the future: you changed quite a bit of white space/formatting in existing code, which makes reading the diff very hard. Please try to keep things unmodified where you don't make changes. > Merge request for DHCP analyzer > ------------------------------- > > Key: BIT-1050 > URL: https://bro-tracker.atlassian.net/browse/BIT-1050 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.2 > Reporter: Vlad Grigorescu > Assignee: Seth Hall > Labels: analyzer > > topic/vladg/dhcp is ready to go. I've been running it in prod with no problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Sat Aug 3 21:17:06 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Sat, 3 Aug 2013 23:17:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1050) Merge request for DHCP analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1050: ------------------------------ Status: Reopened (was: Closed) > Merge request for DHCP analyzer > ------------------------------- > > Key: BIT-1050 > URL: https://bro-tracker.atlassian.net/browse/BIT-1050 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.2 > Reporter: Vlad Grigorescu > Assignee: Seth Hall > Labels: analyzer > > topic/vladg/dhcp is ready to go. I've been running it in prod with no problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Sat Aug 3 21:17:06 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Sat, 3 Aug 2013 23:17:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1050) Merge request for DHCP analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1050: ------------------------------ Resolution: Merged Status: Closed (was: Merge Request) > Merge request for DHCP analyzer > ------------------------------- > > Key: BIT-1050 > URL: https://bro-tracker.atlassian.net/browse/BIT-1050 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.2 > Reporter: Vlad Grigorescu > Assignee: Seth Hall > Labels: analyzer > > topic/vladg/dhcp is ready to go. I've been running it in prod with no problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Sun Aug 4 06:39:06 2013 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Sun, 4 Aug 2013 08:39:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1050) Merge request for DHCP analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424#comment-13424 ] Vlad Grigorescu commented on BIT-1050: -------------------------------------- Thanks, Robin. Sorry it didn't merge cleanly. {quote} I'm not sure I like the structure with the two policy scripts. As we don't have anything else extracting MAC addresses currently, would it make sense to just move them into one? If not, I'd at least move the one out of misc, and also change the interface so that (1) users don't manipulate the table directly (but call functions instead), and (2) the logging stays internal to the script (the same functions could take care of that). But I'll let Seth take a look at this (or has he already?), assigning ticket to him. {quote} Seth and I have talked about it a bit. My original intention was to also add a script for the ARP analyzer. However, in thinking about it some more, that means that if a host is seen by ARP before DHCP, the DHCP hostname won't be logged. Maybe known_devices logs unique (Analyzer, MAC) pairs instead of unique MACs? {quote} format_mac(): The comment says "Supports both EUI-48 and EUI-64. If it's neither, returns an empty string.". That didn't seem to match the code, which always passed back something. I've changed the function to take a length parameter, just assuming its long enough makes me uneasy. So now if less bytes are passed in, it returns the empty string the comment promises. Also, moved to net_utils as fmt_mac(). {quote} Ah, yes. I changed the function and forgot to update the comment. {quote} A note for the future: you changed quite a bit of white space/formatting in existing code, which makes reading the diff very hard. Please try to keep things unmodified where you don't make changes. {quote} Sorry about that. There seem to be several indentation styles floating around the code, and I've been trying to standardize on the style that new code seems to be written in (hard tabbed [Whitesmiths|https://en.wikipedia.org/wiki/Indent_style#Whitesmiths_style]). I can definitely see how that makes diffs harder, though. Would it be ok for me to have a single commit with just the whitespace changes, which could then be checked with {{git diff --ignore-all-space}}? Or should I just leave it alone? > Merge request for DHCP analyzer > ------------------------------- > > Key: BIT-1050 > URL: https://bro-tracker.atlassian.net/browse/BIT-1050 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.2 > Reporter: Vlad Grigorescu > Assignee: Seth Hall > Labels: analyzer > > topic/vladg/dhcp is ready to go. I've been running it in prod with no problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Sun Aug 4 13:50:06 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Sun, 4 Aug 2013 15:50:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-920) Have broctl return useful exit codes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-920: ------------------------------ Status: Merge Request (was: Open) > Have broctl return useful exit codes > ------------------------------------ > > Key: BIT-920 > URL: https://bro-tracker.atlassian.net/browse/BIT-920 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BroControl > Affects Versions: git/master > Reporter: grigorescu > Assignee: Daniel Thayer > Fix For: 2.2 > > > I've got a broctl branch here: https://github.com/grigorescu/broctl which aims to have it return a 0 or 1 exit code for most execution paths. My dive down this particular rabbit hole started when I wanted to have status return a non-zero exit code if a node had failed, but I tried to cover everything else while I was at it. > If someone could double-check it, to make sure that I didn't miss anything, it'd be much appreciated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Mon Aug 5 00:00:03 2013 From: noreply at bro.org (Merge Tracker) Date: Mon, 5 Aug 2013 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308050700.r75703k6025512@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ---------- ------------- ---------- ------------- ---------- ------------------------------------ BIT-920 [1] BroControl grigorescu Daniel Thayer 2013-08-04 2.2 Normal Have broctl return useful exit codes [1] BIT-920 https://bro-tracker.atlassian.net/browse/BIT-920 From noreply at bro.org Tue Aug 6 00:00:06 2013 From: noreply at bro.org (Merge Tracker) Date: Tue, 6 Aug 2013 00:00:06 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308060700.r76706q2022568@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ---------- ------------- ---------- ------------- ---------- ------------------------------------ BIT-920 [1] BroControl grigorescu Daniel Thayer 2013-08-04 2.2 Normal Have broctl return useful exit codes [1] BIT-920 https://bro-tracker.atlassian.net/browse/BIT-920 From jira at bro-tracker.atlassian.net Tue Aug 6 03:22:06 2013 From: jira at bro-tracker.atlassian.net (Brian Little (JIRA)) Date: Tue, 6 Aug 2013 05:22:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1051) smtp-url-extraction.bro misses/truncates urls between data chunks In-Reply-To: References: Message-ID: Brian Little created BIT-1051: --------------------------------- Summary: smtp-url-extraction.bro misses/truncates urls between data chunks Key: BIT-1051 URL: https://bro-tracker.atlassian.net/browse/BIT-1051 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Reporter: Brian Little Priority: Low Files::add_analyzer(f, Files::ANALYZER_DATA_EVENT, [$stream_event=intel_mime_data]); event intel_mime_data(f: fa_file, data: string) {} I think the file analysis framework sends the data through to the intel_mime_data event in sections (appears that way from adding print debugging). The cutting point between the data sections can fall in the middle of an url, causing the regex to miss the url, or truncate it. What would be the recommended way around for this? (and other usage of file analysis framework) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Wed Aug 7 00:00:05 2013 From: noreply at bro.org (Merge Tracker) Date: Wed, 7 Aug 2013 00:00:05 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308070700.r77705Yl002342@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ---------- ------------- ---------- ------------- ---------- ------------------------------------ BIT-920 [1] BroControl grigorescu Daniel Thayer 2013-08-04 2.2 Normal Have broctl return useful exit codes [1] BIT-920 https://bro-tracker.atlassian.net/browse/BIT-920 From hlin33 at illinois.edu Wed Aug 7 20:19:35 2013 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Wed, 7 Aug 2013 20:19:35 -0700 Subject: [Bro-Dev] Hui Lin_Errors from included Bro policy Message-ID: Hi I am upgrading my OS to Ubuntu 12.04 and try to test my branch of work. However, I got some errors from the master branch when I run bro binary loading just default policy files. error in /usr/local/bro/share/bro/base/frameworks/files/./main.bro, line 317: unknown identifier Site::local_nets, at or near "Site::local_nets" Any idea? Best, Hui Lin -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130807/0b48cfa8/attachment.html From noreply at bro.org Thu Aug 8 00:00:04 2013 From: noreply at bro.org (Merge Tracker) Date: Thu, 8 Aug 2013 00:00:04 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308080700.r78704cv019394@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ---------- ------------- ---------- ------------- ---------- ------------------------------------ BIT-920 [1] BroControl grigorescu Daniel Thayer 2013-08-04 2.2 Normal Have broctl return useful exit codes [1] BIT-920 https://bro-tracker.atlassian.net/browse/BIT-920 From jira at bro-tracker.atlassian.net Thu Aug 8 07:36:06 2013 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Thu, 8 Aug 2013 09:36:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-198) Segmentation fault on last bro svn with dhcp.bro and --use-binpac In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425#comment-13425 ] Vlad Grigorescu commented on BIT-198: ------------------------------------- This PCAP is handled properly in the DHCP analyzer that was recently merged. > Segmentation fault on last bro svn with dhcp.bro and --use-binpac > ----------------------------------------------------------------- > > Key: BIT-198 > URL: https://bro-tracker.atlassian.net/browse/BIT-198 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 1.5.2 > Reporter: rmkml > Labels: 140, all, and, fault, include, last, seg, version > Attachments: bro150pre10oct2009_crashdhcp2nov2009.pcap > > > Hi, > When I run bro150pre (compiled with ipv6) with \--use-binpac and \-r joignedfile.pcap and dhcp(.bro) option, I have a segmentation fault. > Same pb with bro v1.4.0. > Program received signal SIGSEGV, Segmentation fault. > Connection::Weird (this=0x201, name=0x82565ee "DHCP_no_type_option") > at Conn.cc:636 > 636 weird = 1; > (gdb) bt full > #0 Connection::Weird (this=0x201, name=0x82565ee "DHCP_no_type_option") > at Conn.cc:636 > No locals. > BIT-1 0x081e9711 in binpac::DHCP::DHCP_Flow::get_dhcp_msgtype (this=0x9dc0e78, > options=0x9dc10f8) at ../src/Analyzer.h:245 > type = > BIT-2 0x081eb303 in binpac::DHCP::DHCP_Message::Parse (this=0x9dc2980, > t_begin_of_data=0x9daff0c "\001\001\006", t_end_of_data=0x9db0038 "", > t_context=0x9dc1758) at dhcp_pac.cc:559 > t_options+elem+size = > t_options+elem+dataptr = (const_byteptr) 0x9dafffd "" > t_DHCP_Message+size = 241 > t_dataptr_after_options = > +PRETTY_FUNCTION+ = "int binpac::DHCP::DHCP_Message::Parse(const > binpac::uint8*, const binpac::uint8*, binpac::DHCP::ContextDHCP*)" > BIT-3 0x081eb48c in binpac::DHCP::DHCP_Flow::NewData (this=0x9dc0e78, > t_begin_of_data=0x9daff0c "\001\001\006", t_end_of_data=0x9db0038 "") > at dhcp_pac.cc:635 > No locals. > BIT-4 0x08071c8d in Analyzer::NextPacket (this=0x9dc0e20, len=300, > data=0x9daff0c "\001\001\006", is_orig=true, seq=-1, ip=0xbfaaec7c, > caplen=136668654) at Analyzer.cc:334 > No locals. > \---Type to continue, or q to quit--\- > BIT-5 0x08071e9a in Analyzer::ForwardPacket (this=0x9dc1d10, len=300, > data=0x9daff0c "\001\001\006", is_orig=true, seq=-1, ip=0xbfaaec7c, > caplen=300) at Analyzer.cc:426 > current = (Analyzer *) 0x9dc0e20 > i = > BIT-6 0x081b3842 in UDP_Analyzer::DeliverPacket (this=0x9dc1d10, len=300, > data=0x9daff04 "", is_orig=true, seq=-1, ip=0xbfaaec7c, > caplen=) at UDP.cc:166 > vl = (val_list *) 0x9dc1cfc > port_val = \{ = \{ = \{ = \{ > _vptr.SerialObj = 0xb88120, static NEVER = 0, static ALWAYS = 1, > static factories = 0x9b4e930, static names = 0x9b4e950, > static time_counter = 483}, in_ser_cache = 8, location = 0xbfaaea98, > ref_cnt = 11144894, static suppress_runtime = 0}, > static register_type = \{}, tid = \{id = 171810783520, > static counter = 44910}, val = \{int_val = 165422456, > uint_val = 165422456, addr_val = 0x9dc2578, subnet_val = \{net = \{ > 165422456, 165412884, 8, 3215649464}, width = 135472165}, > double_val = 3.5568581552422788e-261, string_val = 0x9dc2578, > func_val = 0x9dc2578, file_val = 0x9dc2578, re_val = 0x9dc2578, > table_val = 0x9dc2578, val_list_val = 0x9dc2578, > vector_val = 0x9dc2578}, type = 0x28, attribs = 0x9dc1cfc}, > bq. static register_type = \{}, tid = \{id = 710444731003305986, > \---Type to continue, or q to quit--\- > static counter = 44910}} > result = > ulen = 300 > +PRETTY_FUNCTION__ = "virtual void UDP_Analyzer::DeliverPacket(int, > const u_char*, bool, int, const IP_Hdr*, int)" > BIT-7 0x08071c8d in Analyzer::NextPacket (this=0x9dc1d10, len=308, > data=0x9daff04 "", is_orig=true, seq=-1, ip=0xbfaaec7c, caplen=136668654) > at Analyzer.cc:334 > No locals. > BIT-8 0x080858e5 in Connection::NextPacket (this=0x9dc1c6c, t=1257158012.610261, > is_orig=1, ip=0xbfaaec7c, len=308, caplen=308, data=@0xbfaaebdc, > record_packet=@0xbfaaebd8, record_content=@0xbfaaebd4, hdr=0x9dafa40, > pkt=0x9dafee2 "", hdr_size=14) at Conn.cc:247 > No locals. > BIT-9 0x08183a8d in NetSessions::DoNextPacket (this=0x9dbfee8, > t=1257158012.610261, hdr=0x9dafa40, ip_hdr=0xbfaaec7c, > pkt=0x9dafee2 "", hdr_size=14) at Sessions.cc:663 > ih = > caplen = 308 > ip4 = (const ip *) 0x9dafef0 > len = > proto = 17 > f = (class FragReassembler *) 0x0 > \---Type to continue, or q to quit--\- > frag_field = > min_hdr_len = > data = (const u_char *) 0x9daff04 "" > id = \{src_addr = 0xbfaaec84, dst_addr = 0xbfaaec94, src_port = 17408, > bq. dst_port = 17152, is_one_way = false} > d = (class Dictionary *) 0x9dc0008 > pass_to_conn_compressor = > h = (HashKey *) 0x9d2eb58 > conn = (class Connection *) 0x9dc1c6c > record_packet = 1 > record_content = 1 > BIT-10 0x081841ed in NetSessions::NextPacket (this=0x9dbfee8, > t=1257158012.610261, hdr=0x9dafa40, pkt=0x9dafee2 "", hdr_size=14, > pkt_elem=0x0) at Sessions.cc:305 > ip_hdr = \{ip4 = 0x9dafef0, ip6 = 0x0, src_addr = \{0, 0, 0, 0}, > bq. dst_addr = \{0, 0, 0, 4294967295}, del = 0} > BIT-11 0x0813f2a1 in net_packet_dispatch (t=1257158012.610261, hdr=0x9dafa40, > pkt=0x9dafee2 "", hdr_size=14, src_ps=0x9dafa08, pkt_elem=0x0) > at Net.cc:435 > tmgr = > sp = > load_freq = 0 > BIT-12 0x0813f7a9 in net_packet_arrival (t=1257158012.610261, hdr=0x9dafa40, > \---Type to continue, or q to quit--\- > pkt=0x9dafee2 "", hdr_size=14, src_ps=0x9dafa08) at Net.cc:498 > No locals. > BIT-13 0x0814e5bf in PktSrc::Process (this=0x9dafa08) at PktSrc.cc:199 > No locals. > BIT-14 0x0813f527 in net_run () at Net.cc:528 > ts = 1257158012.610261 > src = (IOSource *) 0x201 > BIT-15 0x0804f80f in main (argc=1346586692, argv=0xbfaaf144) at main.cc:999 > flow = FLOW_NEXT > f = \{ = \{ = \{_vptr.SerialObj = 0x8249f28, > static NEVER = 0, static ALWAYS = 1, static factories = 0x9b4e930, > static names = 0x9b4e950, static time_counter = 483}, > in_ser_cache = false, location = 0x0, ref_cnt = 1, > static suppress_runtime = 0}, frame = 0x9dc0478, size = 1194, > bq. function = 0x0, func_args = 0x0, next_stmt = 0x0, > bq. break_before_next_stmt = false, break_on_return = false, trigger = 0x0, > bq. call = 0x0, delayed = false} > interfaces = \{ = \{entry = 0x9b52538, chunk_size = 10, > max_entries = 10, num_entries = 0}, } > read_files = \{ = \{entry = 0x9b52568, chunk_size = 10, > max_entries = 10, num_entries = 1}, } > netflows = \{ = \{entry = 0x9b52598, chunk_size = 10, > max_entries = 10, num_entries = 0}, } > \---Type to continue, or q to quit--\- > flow_files = \{ = \{entry = 0x9b525c8, chunk_size = 10, > max_entries = 10, num_entries = 0}, } > rule_files = \{ = \{entry = 0x9b525f8, chunk_size = 10, > max_entries = 10, num_entries = 1}, } > transformed_writefile = 0x0 > bst_file = 0x0 > id_name = 0x0 > events_file = 0x0 > seed_load_file = 0x0 > seed_save_file = 0x0 > seed = 0 > dump_cfg = 0 > do_watchdog = 0 > override_ignore_checksums = 0 > rule_debug = 0 > RE_level = 4 > dns_type = DNS_FAKE > oldhandler = > p = > long_optsind = 35 > opts = "A:a:B:D:e:f:I:i:K:n:p:R:r:s:T:t:U:w:x:X:y:Y:z:CFGHLOPSWdghlv", > '\0' > op = > \---Type to continue, or q to quit--\- > script_rule_files = > tmp = 0x0 > s = > bro_alarm_file = > bro_init = \{handler = 0x9b6d138} > dead_handlers = > alive_handlers = > long_opts = {{name = 0x82251d9 "debug-policy", has_arg = 0, > flag = 0x0, val = 100}, \{name = 0x82251e6 "dump-config", has_arg = 0, > flag = 0x0, val = 103}, \{name = 0x82251f2 "exec", has_arg = 1, flag = 0x0, > val = 101}, \{name = 0x823bc9d "filter", has_arg = 1, flag = 0x0, > val = 102}, \{name = 0x82251f7 "help", has_arg = 0, flag = 0x0, val = 104}, > bq. \{name = 0x82251fc "iface", has_arg = 1, flag = 0x0, val = 105}, \{ > name = 0x8225202 "print-scripts", has_arg = 0, flag = 0x0, val = 108}, \{ > name = 0x82507d3 "prefix", has_arg = 1, flag = 0x0, val = 112}, \{ > name = 0x8225210 "readfile", has_arg = 1, flag = 0x0, val = 114}, \{ > name = 0x8225219 "flowfile", has_arg = 1, flag = 0x0, val = 121}, \{ > name = 0x8225222 "netflow", has_arg = 1, flag = 0x0, val = 89}, \{ > name = 0x822522a "rulefile", has_arg = 1, flag = 0x0, val = 115}, \{ > name = 0x8225233 "tracefile", has_arg = 1, flag = 0x0, val = 116}, \{ > name = 0x822523d "writefile", has_arg = 1, flag = 0x0, val = 119}, \{ > name = 0x824698f "version", has_arg = 0, flag = 0x0, val = 118}, \{ > name = 0x8225247 "print-state", has_arg = 1, flag = 0x0, val = 120}, \{ > \---Type to continue, or q to quit--\- > name = 0x8225253 "analyze", has_arg = 1, flag = 0x0, val = 122}, \{ > name = 0x822525b "transfile", has_arg = 1, flag = 0x0, val = 65}, \{ > name = 0x8225265 "no-checksums", has_arg = 0, flag = 0x0, val = 67}, \{ > name = 0x8225272 "dfa-cache", has_arg = 1, flag = 0x0, val = 68}, \{ > name = 0x822527c "force-dns", has_arg = 0, flag = 0x0, val = 70}, \{ > name = 0x8225286 "load-seeds", has_arg = 1, flag = 0x0, val = 71}, \{ > name = 0x8225291 "save-seeds", has_arg = 1, flag = 0x0, val = 72}, \{ > name = 0x822529c "set-seed", has_arg = 1, flag = 0x0, val = 74}, \{ > name = 0x82252a5 "md5-hashkey", has_arg = 1, flag = 0x0, val = 75}, \{ > name = 0x82252b1 "rule-benchmark", has_arg = 0, flag = 0x0, val = 76}, \{ > name = 0x82252c0 "optimize", has_arg = 0, flag = 0x0, val = 79}, \{ > name = 0x82252c9 "prime-dns", has_arg = 0, flag = 0x0, val = 80}, \{ > name = 0x82252d3 "replay", has_arg = 1, flag = 0x0, val = 82}, \{ > name = 0x82252da "debug-rules", has_arg = 0, flag = 0x0, val = 83}, \{ > name = 0x82252e6 "re-level", has_arg = 1, flag = 0x0, val = 82}, \{ > name = 0x82252ef "watchdog", has_arg = 0, flag = 0x0, val = 87}, \{ > name = 0x82252f8 "print-id", has_arg = 1, flag = 0x0, val = 73}, \{ > name = 0x8225301 "status-file", has_arg = 1, flag = 0x0, val = 85}, \{ > name = 0x822530d "pseudo-realtime", has_arg = 2, flag = 0x0, val = 69}, \{ > name = 0x822531d "use-binpac", has_arg = 0, flag = 0x82b3d48, val = 1}, \{ > name = 0x0, has_arg = 0, flag = 0x0, val = 0}} > Regards > Rmkml > Crusoe-Researches.com -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 8 07:38:06 2013 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Thu, 8 Aug 2013 09:38:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-198) Segmentation fault on last bro svn with dhcp.bro and --use-binpac In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Grigorescu updated BIT-198: -------------------------------- Resolution: Fixed Status: Closed (was: Open) > Segmentation fault on last bro svn with dhcp.bro and --use-binpac > ----------------------------------------------------------------- > > Key: BIT-198 > URL: https://bro-tracker.atlassian.net/browse/BIT-198 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 1.5.2 > Reporter: rmkml > Labels: 140, all, and, fault, include, last, seg, version > Attachments: bro150pre10oct2009_crashdhcp2nov2009.pcap > > > Hi, > When I run bro150pre (compiled with ipv6) with \--use-binpac and \-r joignedfile.pcap and dhcp(.bro) option, I have a segmentation fault. > Same pb with bro v1.4.0. > Program received signal SIGSEGV, Segmentation fault. > Connection::Weird (this=0x201, name=0x82565ee "DHCP_no_type_option") > at Conn.cc:636 > 636 weird = 1; > (gdb) bt full > #0 Connection::Weird (this=0x201, name=0x82565ee "DHCP_no_type_option") > at Conn.cc:636 > No locals. > BIT-1 0x081e9711 in binpac::DHCP::DHCP_Flow::get_dhcp_msgtype (this=0x9dc0e78, > options=0x9dc10f8) at ../src/Analyzer.h:245 > type = > BIT-2 0x081eb303 in binpac::DHCP::DHCP_Message::Parse (this=0x9dc2980, > t_begin_of_data=0x9daff0c "\001\001\006", t_end_of_data=0x9db0038 "", > t_context=0x9dc1758) at dhcp_pac.cc:559 > t_options+elem+size = > t_options+elem+dataptr = (const_byteptr) 0x9dafffd "" > t_DHCP_Message+size = 241 > t_dataptr_after_options = > +PRETTY_FUNCTION+ = "int binpac::DHCP::DHCP_Message::Parse(const > binpac::uint8*, const binpac::uint8*, binpac::DHCP::ContextDHCP*)" > BIT-3 0x081eb48c in binpac::DHCP::DHCP_Flow::NewData (this=0x9dc0e78, > t_begin_of_data=0x9daff0c "\001\001\006", t_end_of_data=0x9db0038 "") > at dhcp_pac.cc:635 > No locals. > BIT-4 0x08071c8d in Analyzer::NextPacket (this=0x9dc0e20, len=300, > data=0x9daff0c "\001\001\006", is_orig=true, seq=-1, ip=0xbfaaec7c, > caplen=136668654) at Analyzer.cc:334 > No locals. > \---Type to continue, or q to quit--\- > BIT-5 0x08071e9a in Analyzer::ForwardPacket (this=0x9dc1d10, len=300, > data=0x9daff0c "\001\001\006", is_orig=true, seq=-1, ip=0xbfaaec7c, > caplen=300) at Analyzer.cc:426 > current = (Analyzer *) 0x9dc0e20 > i = > BIT-6 0x081b3842 in UDP_Analyzer::DeliverPacket (this=0x9dc1d10, len=300, > data=0x9daff04 "", is_orig=true, seq=-1, ip=0xbfaaec7c, > caplen=) at UDP.cc:166 > vl = (val_list *) 0x9dc1cfc > port_val = \{ = \{ = \{ = \{ > _vptr.SerialObj = 0xb88120, static NEVER = 0, static ALWAYS = 1, > static factories = 0x9b4e930, static names = 0x9b4e950, > static time_counter = 483}, in_ser_cache = 8, location = 0xbfaaea98, > ref_cnt = 11144894, static suppress_runtime = 0}, > static register_type = \{}, tid = \{id = 171810783520, > static counter = 44910}, val = \{int_val = 165422456, > uint_val = 165422456, addr_val = 0x9dc2578, subnet_val = \{net = \{ > 165422456, 165412884, 8, 3215649464}, width = 135472165}, > double_val = 3.5568581552422788e-261, string_val = 0x9dc2578, > func_val = 0x9dc2578, file_val = 0x9dc2578, re_val = 0x9dc2578, > table_val = 0x9dc2578, val_list_val = 0x9dc2578, > vector_val = 0x9dc2578}, type = 0x28, attribs = 0x9dc1cfc}, > bq. static register_type = \{}, tid = \{id = 710444731003305986, > \---Type to continue, or q to quit--\- > static counter = 44910}} > result = > ulen = 300 > +PRETTY_FUNCTION__ = "virtual void UDP_Analyzer::DeliverPacket(int, > const u_char*, bool, int, const IP_Hdr*, int)" > BIT-7 0x08071c8d in Analyzer::NextPacket (this=0x9dc1d10, len=308, > data=0x9daff04 "", is_orig=true, seq=-1, ip=0xbfaaec7c, caplen=136668654) > at Analyzer.cc:334 > No locals. > BIT-8 0x080858e5 in Connection::NextPacket (this=0x9dc1c6c, t=1257158012.610261, > is_orig=1, ip=0xbfaaec7c, len=308, caplen=308, data=@0xbfaaebdc, > record_packet=@0xbfaaebd8, record_content=@0xbfaaebd4, hdr=0x9dafa40, > pkt=0x9dafee2 "", hdr_size=14) at Conn.cc:247 > No locals. > BIT-9 0x08183a8d in NetSessions::DoNextPacket (this=0x9dbfee8, > t=1257158012.610261, hdr=0x9dafa40, ip_hdr=0xbfaaec7c, > pkt=0x9dafee2 "", hdr_size=14) at Sessions.cc:663 > ih = > caplen = 308 > ip4 = (const ip *) 0x9dafef0 > len = > proto = 17 > f = (class FragReassembler *) 0x0 > \---Type to continue, or q to quit--\- > frag_field = > min_hdr_len = > data = (const u_char *) 0x9daff04 "" > id = \{src_addr = 0xbfaaec84, dst_addr = 0xbfaaec94, src_port = 17408, > bq. dst_port = 17152, is_one_way = false} > d = (class Dictionary *) 0x9dc0008 > pass_to_conn_compressor = > h = (HashKey *) 0x9d2eb58 > conn = (class Connection *) 0x9dc1c6c > record_packet = 1 > record_content = 1 > BIT-10 0x081841ed in NetSessions::NextPacket (this=0x9dbfee8, > t=1257158012.610261, hdr=0x9dafa40, pkt=0x9dafee2 "", hdr_size=14, > pkt_elem=0x0) at Sessions.cc:305 > ip_hdr = \{ip4 = 0x9dafef0, ip6 = 0x0, src_addr = \{0, 0, 0, 0}, > bq. dst_addr = \{0, 0, 0, 4294967295}, del = 0} > BIT-11 0x0813f2a1 in net_packet_dispatch (t=1257158012.610261, hdr=0x9dafa40, > pkt=0x9dafee2 "", hdr_size=14, src_ps=0x9dafa08, pkt_elem=0x0) > at Net.cc:435 > tmgr = > sp = > load_freq = 0 > BIT-12 0x0813f7a9 in net_packet_arrival (t=1257158012.610261, hdr=0x9dafa40, > \---Type to continue, or q to quit--\- > pkt=0x9dafee2 "", hdr_size=14, src_ps=0x9dafa08) at Net.cc:498 > No locals. > BIT-13 0x0814e5bf in PktSrc::Process (this=0x9dafa08) at PktSrc.cc:199 > No locals. > BIT-14 0x0813f527 in net_run () at Net.cc:528 > ts = 1257158012.610261 > src = (IOSource *) 0x201 > BIT-15 0x0804f80f in main (argc=1346586692, argv=0xbfaaf144) at main.cc:999 > flow = FLOW_NEXT > f = \{ = \{ = \{_vptr.SerialObj = 0x8249f28, > static NEVER = 0, static ALWAYS = 1, static factories = 0x9b4e930, > static names = 0x9b4e950, static time_counter = 483}, > in_ser_cache = false, location = 0x0, ref_cnt = 1, > static suppress_runtime = 0}, frame = 0x9dc0478, size = 1194, > bq. function = 0x0, func_args = 0x0, next_stmt = 0x0, > bq. break_before_next_stmt = false, break_on_return = false, trigger = 0x0, > bq. call = 0x0, delayed = false} > interfaces = \{ = \{entry = 0x9b52538, chunk_size = 10, > max_entries = 10, num_entries = 0}, } > read_files = \{ = \{entry = 0x9b52568, chunk_size = 10, > max_entries = 10, num_entries = 1}, } > netflows = \{ = \{entry = 0x9b52598, chunk_size = 10, > max_entries = 10, num_entries = 0}, } > \---Type to continue, or q to quit--\- > flow_files = \{ = \{entry = 0x9b525c8, chunk_size = 10, > max_entries = 10, num_entries = 0}, } > rule_files = \{ = \{entry = 0x9b525f8, chunk_size = 10, > max_entries = 10, num_entries = 1}, } > transformed_writefile = 0x0 > bst_file = 0x0 > id_name = 0x0 > events_file = 0x0 > seed_load_file = 0x0 > seed_save_file = 0x0 > seed = 0 > dump_cfg = 0 > do_watchdog = 0 > override_ignore_checksums = 0 > rule_debug = 0 > RE_level = 4 > dns_type = DNS_FAKE > oldhandler = > p = > long_optsind = 35 > opts = "A:a:B:D:e:f:I:i:K:n:p:R:r:s:T:t:U:w:x:X:y:Y:z:CFGHLOPSWdghlv", > '\0' > op = > \---Type to continue, or q to quit--\- > script_rule_files = > tmp = 0x0 > s = > bro_alarm_file = > bro_init = \{handler = 0x9b6d138} > dead_handlers = > alive_handlers = > long_opts = {{name = 0x82251d9 "debug-policy", has_arg = 0, > flag = 0x0, val = 100}, \{name = 0x82251e6 "dump-config", has_arg = 0, > flag = 0x0, val = 103}, \{name = 0x82251f2 "exec", has_arg = 1, flag = 0x0, > val = 101}, \{name = 0x823bc9d "filter", has_arg = 1, flag = 0x0, > val = 102}, \{name = 0x82251f7 "help", has_arg = 0, flag = 0x0, val = 104}, > bq. \{name = 0x82251fc "iface", has_arg = 1, flag = 0x0, val = 105}, \{ > name = 0x8225202 "print-scripts", has_arg = 0, flag = 0x0, val = 108}, \{ > name = 0x82507d3 "prefix", has_arg = 1, flag = 0x0, val = 112}, \{ > name = 0x8225210 "readfile", has_arg = 1, flag = 0x0, val = 114}, \{ > name = 0x8225219 "flowfile", has_arg = 1, flag = 0x0, val = 121}, \{ > name = 0x8225222 "netflow", has_arg = 1, flag = 0x0, val = 89}, \{ > name = 0x822522a "rulefile", has_arg = 1, flag = 0x0, val = 115}, \{ > name = 0x8225233 "tracefile", has_arg = 1, flag = 0x0, val = 116}, \{ > name = 0x822523d "writefile", has_arg = 1, flag = 0x0, val = 119}, \{ > name = 0x824698f "version", has_arg = 0, flag = 0x0, val = 118}, \{ > name = 0x8225247 "print-state", has_arg = 1, flag = 0x0, val = 120}, \{ > \---Type to continue, or q to quit--\- > name = 0x8225253 "analyze", has_arg = 1, flag = 0x0, val = 122}, \{ > name = 0x822525b "transfile", has_arg = 1, flag = 0x0, val = 65}, \{ > name = 0x8225265 "no-checksums", has_arg = 0, flag = 0x0, val = 67}, \{ > name = 0x8225272 "dfa-cache", has_arg = 1, flag = 0x0, val = 68}, \{ > name = 0x822527c "force-dns", has_arg = 0, flag = 0x0, val = 70}, \{ > name = 0x8225286 "load-seeds", has_arg = 1, flag = 0x0, val = 71}, \{ > name = 0x8225291 "save-seeds", has_arg = 1, flag = 0x0, val = 72}, \{ > name = 0x822529c "set-seed", has_arg = 1, flag = 0x0, val = 74}, \{ > name = 0x82252a5 "md5-hashkey", has_arg = 1, flag = 0x0, val = 75}, \{ > name = 0x82252b1 "rule-benchmark", has_arg = 0, flag = 0x0, val = 76}, \{ > name = 0x82252c0 "optimize", has_arg = 0, flag = 0x0, val = 79}, \{ > name = 0x82252c9 "prime-dns", has_arg = 0, flag = 0x0, val = 80}, \{ > name = 0x82252d3 "replay", has_arg = 1, flag = 0x0, val = 82}, \{ > name = 0x82252da "debug-rules", has_arg = 0, flag = 0x0, val = 83}, \{ > name = 0x82252e6 "re-level", has_arg = 1, flag = 0x0, val = 82}, \{ > name = 0x82252ef "watchdog", has_arg = 0, flag = 0x0, val = 87}, \{ > name = 0x82252f8 "print-id", has_arg = 1, flag = 0x0, val = 73}, \{ > name = 0x8225301 "status-file", has_arg = 1, flag = 0x0, val = 85}, \{ > name = 0x822530d "pseudo-realtime", has_arg = 2, flag = 0x0, val = 69}, \{ > name = 0x822531d "use-binpac", has_arg = 0, flag = 0x82b3d48, val = 1}, \{ > name = 0x0, has_arg = 0, flag = 0x0, val = 0}} > Regards > Rmkml > Crusoe-Researches.com -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jsiwek at illinois.edu Thu Aug 8 11:59:19 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Thu, 8 Aug 2013 18:59:19 +0000 Subject: [Bro-Dev] Hui Lin_Errors from included Bro policy In-Reply-To: References: Message-ID: > error in /usr/local/bro/share/bro/base/frameworks/files/./main.bro, line 317: unknown identifier Site::local_nets, at or near "Site::local_nets" Maybe check if "local_nets" is actually defined in the version of base/utils/site.bro that you have (and that it's in the export section for module "Site"). If it's not, then you've got a modified/bad/old/wrong version of that script installed. You can also load policy/misc/loaded-scripts.bro, run bro, then check the loaded_scripts.log to see if you're actually loading the scripts from the location you expect. - Jon From hlin33 at illinois.edu Thu Aug 8 13:08:48 2013 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Thu, 8 Aug 2013 13:08:48 -0700 Subject: [Bro-Dev] Hui Lin_Errors from included Bro policy In-Reply-To: <454d46ea47e8442dae8d4895fb9220de@CITESHT1.ad.uillinois.edu> References: <454d46ea47e8442dae8d4895fb9220de@CITESHT1.ad.uillinois.edu> Message-ID: Just realize that yesterday, when I git clone, I actually git from git.bro-ids.org, instead of the git.bro.org. Can that be the case? On Thu, Aug 8, 2013 at 11:59 AM, Siwek, Jonathan Luke wrote: > > error in /usr/local/bro/share/bro/base/frameworks/files/./main.bro, line > 317: unknown identifier Site::local_nets, at or near "Site::local_nets" > > Maybe check if "local_nets" is actually defined in the version of > base/utils/site.bro that you have (and that it's in the export section for > module "Site"). If it's not, then you've got a modified/bad/old/wrong > version of that script installed. You can also load > policy/misc/loaded-scripts.bro, run bro, then check the loaded_scripts.log > to see if you're actually loading the scripts from the location you expect. > > - Jon -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130808/3e7ef9ab/attachment.html From jsiwek at illinois.edu Thu Aug 8 13:13:26 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Thu, 8 Aug 2013 20:13:26 +0000 Subject: [Bro-Dev] Hui Lin_Errors from included Bro policy In-Reply-To: References: <454d46ea47e8442dae8d4895fb9220de@CITESHT1.ad.uillinois.edu> Message-ID: > Just realize that yesterday, when I git clone, I actually git from git.bro-ids.org, instead of the git.bro.org. Can that be the case? Both names point to the same place; shouldn't matter which is used. - Jon From noreply at bro.org Fri Aug 9 00:00:05 2013 From: noreply at bro.org (Merge Tracker) Date: Fri, 9 Aug 2013 00:00:05 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308090700.r79705TU029529@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ---------- ------------- ---------- ------------- ---------- ------------------------------------ BIT-920 [1] BroControl grigorescu Daniel Thayer 2013-08-04 2.2 Normal Have broctl return useful exit codes [1] BIT-920 https://bro-tracker.atlassian.net/browse/BIT-920 From jira at bro-tracker.atlassian.net Fri Aug 9 10:01:06 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 9 Aug 2013 12:01:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1052) topic/jsiwek/load-order-fix In-Reply-To: References: Message-ID: Jon Siwek created BIT-1052: ------------------------------ Summary: topic/jsiwek/load-order-fix Key: BIT-1052 URL: https://bro-tracker.atlassian.net/browse/BIT-1052 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Jon Siwek Assignee: Robin Sommer Fix For: 2.2 This branch is in the {{cmake}} and {{bro}} repo. Hopefully it makes the load order of auto-generated scripts containing BIF function declarations more stable across platforms; unit tests were checking that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 9 10:03:06 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 9 Aug 2013 12:03:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1052) topic/jsiwek/load-order-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1052: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/load-order-fix > --------------------------- > > Key: BIT-1052 > URL: https://bro-tracker.atlassian.net/browse/BIT-1052 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.2 > > > This branch is in the {{cmake}} and {{bro}} repo. Hopefully it makes the load order of auto-generated scripts containing BIF function declarations more stable across platforms; unit tests were checking that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 9 16:07:06 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 9 Aug 2013 18:07:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1050) Merge request for DHCP analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426#comment-13426 ] Robin Sommer commented on BIT-1050: ----------------------------------- It could also log an update when it gets more information than logged last time. However I'd vote for just combining the two scripts into one for now until we have that other script and can flesh out the interface. I think it's a mix of different ones, I call it "Vern style". :) A separate commit that changes just formatting would definitly be better for such changes (it wasn't just whitespace, sometimes braces moved so that git's white-space-ignore still reported them). Generally, I don't think it's worth too much attention for existing code. I'm hoping we'll eventually have a tool that formats things into a consistent style automatically (I have been playing with clang-format a bit, I think that might work). > Merge request for DHCP analyzer > ------------------------------- > > Key: BIT-1050 > URL: https://bro-tracker.atlassian.net/browse/BIT-1050 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: 2.2 > Reporter: Vlad Grigorescu > Assignee: Seth Hall > Labels: analyzer > > topic/vladg/dhcp is ready to go. I've been running it in prod with no problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 9 17:06:06 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 9 Aug 2013 19:06:06 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1052) topic/jsiwek/load-order-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1052: ------------------------------ Resolution: Merged Status: Closed (was: Merge Request) > topic/jsiwek/load-order-fix > --------------------------- > > Key: BIT-1052 > URL: https://bro-tracker.atlassian.net/browse/BIT-1052 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.2 > > > This branch is in the {{cmake}} and {{bro}} repo. Hopefully it makes the load order of auto-generated scripts containing BIF function declarations more stable across platforms; unit tests were checking that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Sat Aug 10 00:00:05 2013 From: noreply at bro.org (Merge Tracker) Date: Sat, 10 Aug 2013 00:00:05 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308100700.r7A705cg015341@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ---------- ------------- ---------- ------------- ---------- ------------------------------------ BIT-920 [1] BroControl grigorescu Daniel Thayer 2013-08-04 2.2 Normal Have broctl return useful exit codes [1] BIT-920 https://bro-tracker.atlassian.net/browse/BIT-920 From noreply at bro.org Sun Aug 11 00:00:04 2013 From: noreply at bro.org (Merge Tracker) Date: Sun, 11 Aug 2013 00:00:04 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308110700.r7B704LW008340@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ---------- ------------- ---------- ------------- ---------- ------------------------------------ BIT-920 [1] BroControl grigorescu Daniel Thayer 2013-08-04 2.2 Normal Have broctl return useful exit codes [1] BIT-920 https://bro-tracker.atlassian.net/browse/BIT-920 From jira at bro-tracker.atlassian.net Sun Aug 11 20:25:21 2013 From: jira at bro-tracker.atlassian.net (Derek Ditch (JIRA)) Date: Sun, 11 Aug 2013 22:25:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1053) Update RPM spec file for Broccoli In-Reply-To: References: Message-ID: Derek Ditch created BIT-1053: -------------------------------- Summary: Update RPM spec file for Broccoli Key: BIT-1053 URL: https://bro-tracker.atlassian.net/browse/BIT-1053 Project: Bro Issue Tracker Issue Type: Patch Components: Broccoli Affects Versions: git/master Environment: EL6 platforms Reporter: Derek Ditch I updated the spec file for the Broccoli module in order to be able to run automated builds with mock. The spec file now passes the rpmlint tests and successfully compiles and packages broccoli (with python) on EL6 platforms. This should also work for Fedora, though I haven't ran that build yet. Things to do: - split out language bindings to separate packages - Fix configure scripts/cmake interface to accept standard variables for building; this enable use of the %cmake or %configure macros See my pull request on GitHub: https://github.com/bro/broccoli/pull/1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Mon Aug 12 00:00:08 2013 From: noreply at bro.org (Merge Tracker) Date: Mon, 12 Aug 2013 00:00:08 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308120700.r7C708Om018312@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ---------- ------------- ---------- ------------- ---------- ------------------------------------ BIT-920 [1] BroControl grigorescu Daniel Thayer 2013-08-04 2.2 Normal Have broctl return useful exit codes Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ---------------------------------------------- #1 [2] bro anthonykasza [3] 2013-08-11 levenshtein distance [4] #1 [5] broccoli dcode [6] 2013-08-12 Updated specfile. Works under mock for EL6 [7] [1] BIT-920 https://bro-tracker.atlassian.net/browse/BIT-920 [2] Pull Request #1 https://github.com/bro/bro/issues/1 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #1 with git pull https://github.com/anthonykasza/bro.git master [5] Pull Request #1 https://github.com/bro/broccoli/issues/1 [6] dcode https://github.com/dcode [7] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From robin at icir.org Mon Aug 12 09:04:42 2013 From: robin at icir.org (Robin Sommer) Date: Mon, 12 Aug 2013 09:04:42 -0700 Subject: [Bro-Dev] Planing for a 2.2 beta Message-ID: <20130812160442.GC1711@icir.org> This is what I have on my list as remaining for a 2.2 beta: - Fix sumstats framework (Seth; or is it done already now?) - HyperLogLog (Bernhard) - DHCP script cleanup (Seth/Vlad; see BIT-1050) - DNP3 finalizing (Robin, Hui) - Windows executable analyzer (Seth; going to happen?) - SIP analyzer (Vlad; going to happen?) - Bloomfilter test failures (Matthias) - Input framework test failures (Bernhard) - X509 extensions (going to happen? can somebody remind we what this is about?) Anything I'm missing? I'd like put a feature freeze in place. Can we aim to have this all in by the end of this week? Then we could target a 2.2 beta by the end of next. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From seth at icir.org Mon Aug 12 09:48:34 2013 From: seth at icir.org (Seth Hall) Date: Mon, 12 Aug 2013 12:48:34 -0400 Subject: [Bro-Dev] Planing for a 2.2 beta In-Reply-To: <20130812160442.GC1711@icir.org> References: <20130812160442.GC1711@icir.org> Message-ID: On Aug 12, 2013, at 12:04 PM, Robin Sommer wrote: > - Fix sumstats framework (Seth; or is it done already now?) Not done yet. It's broken on clusters at the moment, but I'm trying to get it done this week. > - Windows executable analyzer (Seth; going to happen?) Yes, I hope so. I'll have some stuff out on this soon. > Can we aim to have this all in by the end of this week? Then we could > target a 2.2 beta by the end of next. I'll try my best on my stuff. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130812/4494a3b1/attachment.bin From jsiwek at illinois.edu Mon Aug 12 11:23:46 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Mon, 12 Aug 2013 18:23:46 +0000 Subject: [Bro-Dev] Planing for a 2.2 beta In-Reply-To: <20130812160442.GC1711@icir.org> References: <20130812160442.GC1711@icir.org> Message-ID: > - Input framework test failures (Bernhard) I've also started looking in to the executestdin one which is still showing some real problems and was planning to look at others too. > Anything I'm missing? I want to add something to the file extraction analyzer to make it easier to impose limits. Should just be a minor change to do. - Jon From vladg at cmu.edu Mon Aug 12 11:28:57 2013 From: vladg at cmu.edu (Vlad Grigorescu) Date: Mon, 12 Aug 2013 18:28:57 +0000 Subject: [Bro-Dev] Planing for a 2.2 beta In-Reply-To: <6946_1376323487_r7CG4j6g015402_20130812160442.GC1711@icir.org> References: <6946_1376323487_r7CG4j6g015402_20130812160442.GC1711@icir.org> Message-ID: <1202BE242E080642B0CD0AD0A03E8552D83A95@PGH-MSGMB-03.andrew.ad.cmu.edu> On Aug 12, 2013, at 12:04 PM, Robin Sommer wrote: > - DHCP script cleanup (Seth/Vlad; see BIT-1050) Yep, I'll work on this with Seth. > - SIP analyzer (Vlad; going to happen?) I just have one issue to figure out in BinPAC, to implement this correctly. Right now I'm relying on is_orig, but due to some weird SIP proxying, that doesn't work correctly in a small number of cases (as seen in the real world by Aashish). I'm going to try to get this done by the end of the week, but if I don't get to it I'm fine pushing it to 2.3. --Vlad From jira at bro-tracker.atlassian.net Mon Aug 12 11:46:21 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 12 Aug 2013 13:46:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1053) Update RPM spec file for Broccoli In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1053: --------------------------------- Assignee: Jon Siwek > Update RPM spec file for Broccoli > --------------------------------- > > Key: BIT-1053 > URL: https://bro-tracker.atlassian.net/browse/BIT-1053 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Broccoli > Affects Versions: git/master > Environment: EL6 platforms > Reporter: Derek Ditch > Assignee: Jon Siwek > Labels: cleanup, packaging > > I updated the spec file for the Broccoli module in order to be able to run automated builds with mock. The spec file now passes the rpmlint tests and successfully compiles and packages broccoli (with python) on EL6 platforms. This should also work for Fedora, though I haven't ran that build yet. > Things to do: > - split out language bindings to separate packages > - Fix configure scripts/cmake interface to accept standard variables for building; this enable use of the %cmake or %configure macros > See my pull request on GitHub: https://github.com/bro/broccoli/pull/1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 12 11:46:21 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 12 Aug 2013 13:46:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1053) Update RPM spec file for Broccoli In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500#comment-13500 ] Robin Sommer commented on BIT-1053: ----------------------------------- Jon, can you take a look? > Update RPM spec file for Broccoli > --------------------------------- > > Key: BIT-1053 > URL: https://bro-tracker.atlassian.net/browse/BIT-1053 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Broccoli > Affects Versions: git/master > Environment: EL6 platforms > Reporter: Derek Ditch > Assignee: Jon Siwek > Labels: cleanup, packaging > > I updated the spec file for the Broccoli module in order to be able to run automated builds with mock. The spec file now passes the rpmlint tests and successfully compiles and packages broccoli (with python) on EL6 platforms. This should also work for Fedora, though I haven't ran that build yet. > Things to do: > - split out language bindings to separate packages > - Fix configure scripts/cmake interface to accept standard variables for building; this enable use of the %cmake or %configure macros > See my pull request on GitHub: https://github.com/bro/broccoli/pull/1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 12 12:06:21 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 12 Aug 2013 14:06:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-861) Merging DNP3 Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-861: ----------------------------- Resolution: Merged Status: Closed (was: Open) > Merging DNP3 Analyzer > --------------------- > > Key: BIT-861 > URL: https://bro-tracker.atlassian.net/browse/BIT-861 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro > Affects Versions: git/master > Reporter: hui > Assignee: Robin Sommer > Labels: dnp3 > Fix For: 2.2 > > > Merging the branch topic/hui/powergrid3 into Master > The DNP3 analyzer codes in src/ > DNP3.cc > DNP3.h > dnp3.pac > dnp3-protocol.pac > dnp3-analyzer.pac > dnp3-objects.pac > Policy scripts in policy in scripts/policy/protocols/dnp3 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Tue Aug 13 00:00:09 2013 From: noreply at bro.org (Merge Tracker) Date: Tue, 13 Aug 2013 00:00:09 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308130700.r7D709GI031069@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ---------- ------------- ---------- ------------- ---------- ------------------------------------ BIT-920 [1] BroControl grigorescu Daniel Thayer 2013-08-04 2.2 Normal Have broctl return useful exit codes Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ---------------------------------------------- #1 [2] bro anthonykasza [3] 2013-08-13 levenshtein distance [4] #1 [5] broccoli dcode [6] 2013-08-12 Updated specfile. Works under mock for EL6 [7] [1] BIT-920 https://bro-tracker.atlassian.net/browse/BIT-920 [2] Pull Request #1 https://github.com/bro/bro/issues/1 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #1 with git pull https://github.com/anthonykasza/bro.git master [5] Pull Request #1 https://github.com/bro/broccoli/issues/1 [6] dcode https://github.com/dcode [7] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Tue Aug 13 00:25:21 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Tue, 13 Aug 2013 02:25:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer In-Reply-To: References: Message-ID: Seth Hall created BIT-1054: ------------------------------ Summary: Merge unified2 file analyzer Key: BIT-1054 URL: https://bro-tracker.atlassian.net/browse/BIT-1054 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.2 Reporter: Seth Hall The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 00:25:21 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Tue, 13 Aug 2013 02:25:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1054: --------------------------- Status: Merge Request (was: Open) > Merge unified2 file analyzer > ---------------------------- > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 11:39:21 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 13 Aug 2013 13:39:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1055) topic/dnthayer/test-fixes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1055: ------------------------------- Status: Merge Request (was: Open) > topic/dnthayer/test-fixes > ------------------------- > > Key: BIT-1055 > URL: https://bro-tracker.atlassian.net/browse/BIT-1055 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BTest > Reporter: Daniel Thayer > > The branch topic/dnthayer/test-fixes contains fixes to the btest > tests. I've now tested this branch on all of the Jenkins nodes, and > did not see any failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 11:39:21 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 13 Aug 2013 13:39:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1055) topic/dnthayer/test-fixes In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1055: ---------------------------------- Summary: topic/dnthayer/test-fixes Key: BIT-1055 URL: https://bro-tracker.atlassian.net/browse/BIT-1055 Project: Bro Issue Tracker Issue Type: Patch Components: BTest Reporter: Daniel Thayer The branch topic/dnthayer/test-fixes contains fixes to the btest tests. I've now tested this branch on all of the Jenkins nodes, and did not see any failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 11:54:21 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 13 Aug 2013 13:54:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-920) Have broctl return useful exit codes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-920: ----------------------------- Resolution: Merged Status: Closed (was: Merge Request) > Have broctl return useful exit codes > ------------------------------------ > > Key: BIT-920 > URL: https://bro-tracker.atlassian.net/browse/BIT-920 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BroControl > Affects Versions: git/master > Reporter: grigorescu > Assignee: Daniel Thayer > Fix For: 2.2 > > > I've got a broctl branch here: https://github.com/grigorescu/broctl which aims to have it return a 0 or 1 exit code for most execution paths. My dive down this particular rabbit hole started when I wanted to have status return a non-zero exit code if a node had failed, but I tried to cover everything else while I was at it. > If someone could double-check it, to make sure that I didn't miss anything, it'd be much appreciated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 13:57:21 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 13 Aug 2013 15:57:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-275) broctl 'exec' crashes with dead hosts In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-275: ------------------------------ Fix Version/s: 2.2 > broctl 'exec' crashes with dead hosts > ------------------------------------- > > Key: BIT-275 > URL: https://bro-tracker.atlassian.net/browse/BIT-275 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 1.5.2 > Reporter: solomon > Assignee: Robin Sommer > Fix For: 2.2 > > > When performing the 'exec' command via broctl, if there are dead or otherwise offline hosts, broctl (python) will crash with a 'TypeError'. Relaunching broctl and trying again sometimes works, but many times it will continue to crash if the host remains down. See below: > broctl exec killall \-9 bro > Connection timed out during banner exchange > warning: connection to bro3 broke > Traceback (most recent call last): > bq. File "/home/bro/bin/broctl", line 726, in > loop.onecmd(line) > bq. File "/usr/local/lib/python2.5/cmd.py", line 219, in onecmd > return func(arg) > bq. File "/home/bro/bin/broctl", line 561, in do_exec > control.executeCmd(Config.nodes(), args) > bq. File "/bro/lib/broctl/BroControl/control.py", line 1056, in executeCmd > util.output("[%s] %s\n> %s" % (node.host, (success and " " or "error"), "\n> ".join(output))) > TypeError -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 13:59:21 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 13 Aug 2013 15:59:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-275) broctl 'exec' crashes with dead hosts In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501#comment-13501 ] Daniel Thayer commented on BIT-275: ----------------------------------- This issue has been fixed several months ago. > broctl 'exec' crashes with dead hosts > ------------------------------------- > > Key: BIT-275 > URL: https://bro-tracker.atlassian.net/browse/BIT-275 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 1.5.2 > Reporter: solomon > Assignee: Robin Sommer > Fix For: 2.2 > > > When performing the 'exec' command via broctl, if there are dead or otherwise offline hosts, broctl (python) will crash with a 'TypeError'. Relaunching broctl and trying again sometimes works, but many times it will continue to crash if the host remains down. See below: > broctl exec killall \-9 bro > Connection timed out during banner exchange > warning: connection to bro3 broke > Traceback (most recent call last): > bq. File "/home/bro/bin/broctl", line 726, in > loop.onecmd(line) > bq. File "/usr/local/lib/python2.5/cmd.py", line 219, in onecmd > return func(arg) > bq. File "/home/bro/bin/broctl", line 561, in do_exec > control.executeCmd(Config.nodes(), args) > bq. File "/bro/lib/broctl/BroControl/control.py", line 1056, in executeCmd > util.output("[%s] %s\n> %s" % (node.host, (success and " " or "error"), "\n> ".join(output))) > TypeError -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 14:01:21 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 13 Aug 2013 16:01:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-275) broctl 'exec' crashes with dead hosts In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-275: ------------------------------ Resolution: Fixed Status: Closed (was: Open) > broctl 'exec' crashes with dead hosts > ------------------------------------- > > Key: BIT-275 > URL: https://bro-tracker.atlassian.net/browse/BIT-275 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 1.5.2 > Reporter: solomon > Assignee: Robin Sommer > Fix For: 2.2 > > > When performing the 'exec' command via broctl, if there are dead or otherwise offline hosts, broctl (python) will crash with a 'TypeError'. Relaunching broctl and trying again sometimes works, but many times it will continue to crash if the host remains down. See below: > broctl exec killall \-9 bro > Connection timed out during banner exchange > warning: connection to bro3 broke > Traceback (most recent call last): > bq. File "/home/bro/bin/broctl", line 726, in > loop.onecmd(line) > bq. File "/usr/local/lib/python2.5/cmd.py", line 219, in onecmd > return func(arg) > bq. File "/home/bro/bin/broctl", line 561, in do_exec > control.executeCmd(Config.nodes(), args) > bq. File "/bro/lib/broctl/BroControl/control.py", line 1056, in executeCmd > util.output("[%s] %s\n> %s" % (node.host, (success and " " or "error"), "\n> ".join(output))) > TypeError -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 18:43:21 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 13 Aug 2013 20:43:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1054: ------------------------------ Status: Open (was: Merge Request) > Merge unified2 file analyzer > ---------------------------- > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Seth Hall > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 18:43:21 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 13 Aug 2013 20:43:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502#comment-13502 ] Robin Sommer commented on BIT-1054: ----------------------------------- This has a problem I think, with the external test suite I get lots of changes of the kind below (i.e., it looks like the analyzer is always activated). Also, can you add a couple of sentences at the beginning of the main script on what this is doing and how to use it? {code} -XXXXXXXXXX.XXXXXX 6qMP0t3ckOa 192.168.1.103 212.227.97.133 E1pLDCMCsvj HTTP 0 MD5,SHA1 application/octet-stream - 0.000000 - T 984 984 0 0 F - 14d98ec89c27b7acbe81ccf0d8c9fbad f4310cb8f1d9fb79adf0766e1d370dceebe2900d - - +XXXXXXXXXX.XXXXXX 6qMP0t3ckOa 192.168.1.103 212.227.97.133 E1pLDCMCsvj HTTP 0 UNIFIED2,MD5,SHA1 application/octet-stream - 0.000000 - T 984 984 0 0 F - 14d98ec89c27b7acbe81ccf0d8c9fbad f4310cb8f1d9fb79adf0766e1d370dceebe2900d - - -XXXXXXXXXX.XXXXXX 6qMP0t3ckOa 192.168.1.103 212.227.97.133 E1pLDCMCsvj HTTP 0 MD5,SHA1 application/octet-stream - 0.000000 - T 984 984 0 0 F - 14d98ec89c27b7acbe81ccf0d8c9fbad f4310cb8f1d9fb79adf0766e1d370dceebe2900d - - +XXXXXXXXXX.XXXXXX 6qMP0t3ckOa 192.168.1.103 212.227.97.133 E1pLDCMCsvj HTTP 0 UNIFIED2,MD5,SHA1 application/octet-stream - 0.000000 - T 984 984 0 0 F - 14d98ec89c27b7acbe81ccf0d8c9fbad f4310cb8f1d9fb79adf0766e1d370dceebe2900d - - {code} > Merge unified2 file analyzer > ---------------------------- > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Seth Hall > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 18:43:21 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 13 Aug 2013 20:43:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1054: --------------------------------- Assignee: Seth Hall > Merge unified2 file analyzer > ---------------------------- > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Seth Hall > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 18:45:21 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 13 Aug 2013 20:45:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1055) topic/dnthayer/test-fixes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1055: ------------------------------ Resolution: Fixed Status: Closed (was: Merge Request) > topic/dnthayer/test-fixes > ------------------------- > > Key: BIT-1055 > URL: https://bro-tracker.atlassian.net/browse/BIT-1055 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BTest > Reporter: Daniel Thayer > > The branch topic/dnthayer/test-fixes contains fixes to the btest > tests. I've now tested this branch on all of the Jenkins nodes, and > did not see any failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 21:45:21 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Tue, 13 Aug 2013 23:45:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503#comment-13503 ] Seth Hall commented on BIT-1054: -------------------------------- Oops, that definitely wasn't supposed to happen. Will fix. > Merge unified2 file analyzer > ---------------------------- > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Seth Hall > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 22:02:21 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 14 Aug 2013 00:02:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1054: --------------------------- Assignee: Robin Sommer (was: Seth Hall) > Merge unified2 file analyzer > ---------------------------- > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Robin Sommer > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Tue Aug 13 22:02:21 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 14 Aug 2013 00:02:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504#comment-13504 ] Seth Hall commented on BIT-1054: -------------------------------- Fixed it. > Merge unified2 file analyzer > ---------------------------- > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Seth Hall > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Wed Aug 14 00:00:09 2013 From: noreply at bro.org (Merge Tracker) Date: Wed, 14 Aug 2013 00:00:09 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308140700.r7E709bp013606@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ---------------------------------------------- #1 [1] bro anthonykasza [2] 2013-08-13 levenshtein distance [3] #1 [4] broccoli dcode [5] 2013-08-12 Updated specfile. Works under mock for EL6 [6] [1] Pull Request #1 https://github.com/bro/bro/issues/1 [2] anthonykasza https://github.com/anthonykasza [3] Merge Pull Request #1 with git pull https://github.com/anthonykasza/bro.git master [4] Pull Request #1 https://github.com/bro/broccoli/issues/1 [5] dcode https://github.com/dcode [6] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Wed Aug 14 10:26:21 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 14 Aug 2013 12:26:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1054: ------------------------------ Status: Merge Request (was: Open) > Merge unified2 file analyzer > ---------------------------- > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Robin Sommer > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 14 11:08:21 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 14 Aug 2013 13:08:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1054) Merge unified2 file analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1054: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Merge unified2 file analyzer > ---------------------------- > > Key: BIT-1054 > URL: https://bro-tracker.atlassian.net/browse/BIT-1054 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Robin Sommer > > The branch topic/seth/unified2-analyzer is ready for merging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 14 11:10:21 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 14 Aug 2013 13:10:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-714) broctl install copies policy files to the .site folder in incorrect order In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-714: ------------------------------ Fix Version/s: 2.2 > broctl install copies policy files to the .site folder in incorrect order > ------------------------------------------------------------------------- > > Key: BIT-714 > URL: https://bro-tracker.atlassian.net/browse/BIT-714 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: aashish > Fix For: 2.2 > > > I think broctl install copies the files to .site directory in the order specified by the SitePolicyPath This is probably incorrect. It should instead be in the reverse order of the path specified in the SitePolicyPath > example: > SitePolicyPath=/dir1:/dir2:/dir3:/usr/local/bro/share/bro > if a modified version of file with same name (eg. drop.bro) is in dir1 and original is in /usr/local/bro/share/bro; broctl install will overwrite /dir1/drop.bro with /usr/local/bro/share/bro/drop.bro when it creates /usr/local/bro/share/bro/.site folder. > In theory when bro is starting, it should have preference in loading /dir1/drop.bro over /usr/local/bro/share/bro/drop.bro but currently /dir1/drop.bro gets overwritten by /usr/local/bro/share/bro/drop.bro when broctl install creates .site folder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 14 12:42:21 2013 From: jira at bro-tracker.atlassian.net (Mahmut Bulut (JIRA)) Date: Wed, 14 Aug 2013 14:42:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-779) missing values cause bro to crash when used inside of a 'when' statement. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505#comment-13505 ] Mahmut Bulut commented on BIT-779: ---------------------------------- Its reproduced again like following: {panel:title=Field Value Missing} ERROR: 5700.067414 field value missing [c$smtp$current_entity$filename] (./src/./malware/malware.bro, line 20) ERROR: 5700.067414 field value missing [c$smtp$current_entity$filename] (./src/./malware/malware.bro, line 20) {panel} > missing values cause bro to crash when used inside of a 'when' statement. > ------------------------------------------------------------------------- > > Key: BIT-779 > URL: https://bro-tracker.atlassian.net/browse/BIT-779 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: justin > Priority: High > Labels: InterpreterException, when > Fix For: 2.2 > > > Simplest test case: > {noformat} > event bro_init() > { > local loc: geo_location; > when (local hostname = lookup_addr(127.0.0.1)){ > print "Location", loc$country_code; > print "ok"; > terminate(); > } > } > {noformat} > gives: > {noformat} > terminate called after throwing an instance of 'InterpreterException' > {noformat} > outside of the when block, reporter.log would get: > {noformat} > Reporter::ERROR field value missing [loc$country_code] > {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 14 12:47:21 2013 From: jira at bro-tracker.atlassian.net (Mahmut Bulut (JIRA)) Date: Wed, 14 Aug 2013 14:47:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-779) missing values cause bro to crash when used inside of a 'when' statement. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505#comment-13505 ] Mahmut Bulut edited comment on BIT-779 at 8/14/13 2:45 PM: ----------------------------------------------------------- Its reproduced again like following: {panel:title=Field Value Missing} ERROR: 5700.067414 field value missing [c$smtp$current_entity$filename] (./src/./malware/malware.bro, line 20) ERROR: 5700.067414 field value missing [c$smtp$current_entity$filename] (./src/./malware/malware.bro, line 20) {panel} but not in when statement for example: {noformat} event mime_one_header(c: connection, h: mime_header_rec) { if (/\.([eE][xX][eE]|[dD][lL][lL])/ in c$smtp$current_entity$filename){ print "BRO BRO"; } } {noformat} was (Author: vertexclique): Its reproduced again like following: {panel:title=Field Value Missing} ERROR: 5700.067414 field value missing [c$smtp$current_entity$filename] (./src/./malware/malware.bro, line 20) ERROR: 5700.067414 field value missing [c$smtp$current_entity$filename] (./src/./malware/malware.bro, line 20) {panel} > missing values cause bro to crash when used inside of a 'when' statement. > ------------------------------------------------------------------------- > > Key: BIT-779 > URL: https://bro-tracker.atlassian.net/browse/BIT-779 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: justin > Priority: High > Labels: InterpreterException, when > Fix For: 2.2 > > > Simplest test case: > {noformat} > event bro_init() > { > local loc: geo_location; > when (local hostname = lookup_addr(127.0.0.1)){ > print "Location", loc$country_code; > print "ok"; > terminate(); > } > } > {noformat} > gives: > {noformat} > terminate called after throwing an instance of 'InterpreterException' > {noformat} > outside of the when block, reporter.log would get: > {noformat} > Reporter::ERROR field value missing [loc$country_code] > {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 14 13:03:21 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 14 Aug 2013 15:03:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-779) missing values cause bro to crash when used inside of a 'when' statement. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506#comment-13506 ] Seth Hall commented on BIT-779: ------------------------------- This is on git master? Keep in mind that this is *not* fixed in 2.1. > missing values cause bro to crash when used inside of a 'when' statement. > ------------------------------------------------------------------------- > > Key: BIT-779 > URL: https://bro-tracker.atlassian.net/browse/BIT-779 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: justin > Priority: High > Labels: InterpreterException, when > Fix For: 2.2 > > > Simplest test case: > {noformat} > event bro_init() > { > local loc: geo_location; > when (local hostname = lookup_addr(127.0.0.1)){ > print "Location", loc$country_code; > print "ok"; > terminate(); > } > } > {noformat} > gives: > {noformat} > terminate called after throwing an instance of 'InterpreterException' > {noformat} > outside of the when block, reporter.log would get: > {noformat} > Reporter::ERROR field value missing [loc$country_code] > {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 14 19:00:21 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 14 Aug 2013 21:00:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-714) broctl install copies policy files to the .site folder in incorrect order In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507#comment-13507 ] Daniel Thayer commented on BIT-714: ----------------------------------- Branch topic/dnthayer/ticket714 fixes this issue. > broctl install copies policy files to the .site folder in incorrect order > ------------------------------------------------------------------------- > > Key: BIT-714 > URL: https://bro-tracker.atlassian.net/browse/BIT-714 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: aashish > Fix For: 2.2 > > > I think broctl install copies the files to .site directory in the order specified by the SitePolicyPath This is probably incorrect. It should instead be in the reverse order of the path specified in the SitePolicyPath > example: > SitePolicyPath=/dir1:/dir2:/dir3:/usr/local/bro/share/bro > if a modified version of file with same name (eg. drop.bro) is in dir1 and original is in /usr/local/bro/share/bro; broctl install will overwrite /dir1/drop.bro with /usr/local/bro/share/bro/drop.bro when it creates /usr/local/bro/share/bro/.site folder. > In theory when bro is starting, it should have preference in loading /dir1/drop.bro over /usr/local/bro/share/bro/drop.bro but currently /dir1/drop.bro gets overwritten by /usr/local/bro/share/bro/drop.bro when broctl install creates .site folder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 14 19:00:21 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 14 Aug 2013 21:00:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-714) broctl install copies policy files to the .site folder in incorrect order In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-714: ------------------------------ Status: Merge Request (was: Open) > broctl install copies policy files to the .site folder in incorrect order > ------------------------------------------------------------------------- > > Key: BIT-714 > URL: https://bro-tracker.atlassian.net/browse/BIT-714 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: aashish > Fix For: 2.2 > > > I think broctl install copies the files to .site directory in the order specified by the SitePolicyPath This is probably incorrect. It should instead be in the reverse order of the path specified in the SitePolicyPath > example: > SitePolicyPath=/dir1:/dir2:/dir3:/usr/local/bro/share/bro > if a modified version of file with same name (eg. drop.bro) is in dir1 and original is in /usr/local/bro/share/bro; broctl install will overwrite /dir1/drop.bro with /usr/local/bro/share/bro/drop.bro when it creates /usr/local/bro/share/bro/.site folder. > In theory when bro is starting, it should have preference in loading /dir1/drop.bro over /usr/local/bro/share/bro/drop.bro but currently /dir1/drop.bro gets overwritten by /usr/local/bro/share/bro/drop.bro when broctl install creates .site folder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 14 22:32:21 2013 From: jira at bro-tracker.atlassian.net (Jakub Svoboda (JIRA)) Date: Thu, 15 Aug 2013 00:32:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1056) incorrect values in num_pkts and num_bytes_ip In-Reply-To: References: Message-ID: Jakub Svoboda created BIT-1056: ---------------------------------- Summary: incorrect values in num_pkts and num_bytes_ip Key: BIT-1056 URL: https://bro-tracker.atlassian.net/browse/BIT-1056 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.1, git/master Environment: Two separate testing environments: 1. Intel i3, OpenSuse x64, public IP, Bro is on the computer monitoring its own traffic 2. Intel i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine has private IP, Bro is on virtual machine monitoring its own traffic Reporter: Jakub Svoboda Problem: Bro does not see traffic correctly. * conn.log contains invalid information, with values differing from a network monitoring probe. (The probe runs on a different computer, so there's no interference.) * Our own scripts logging connections using the connection_state_remove event report the same bad values as conn.log. * Specifically, all following values are off in the connection_state_remove event handler: c$orig$num_pkts c$resp$num_pkts c$conn$orig_pkts c$conn$resp_pkts c$orig$num_bytes_ip c$resp$num_bytes_ip c$conn$orig_ip_bytes c$conn$resp_ip_bytes. (Yes, we know some of the values are copied one from another in conn/main.bro.) * Problem is the same on both tested machines. * Problem is the same both when capturing an interface and when analyzing a pcap. * Problem is the same on both Bro 2.1 and the current git version (as of 14.08.2013) The problem is exactly the same on two different computers. 1. i3, OpenSuse, public IP 2. i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine has private IP, Bro is on virtual machine monitoring its own traffic Both computers use 64bit OS and Bro 2.1 compiled from source. Notes: The chance of the network monitoring probe reporting bad values is very low as this probe is used by our department of computer science and several people checked it works correctly. Also, the values reported by Bro are obviously wrong even without no comparison with the probe---e.g. an outgoing flow with 0 outgoing packets, 0 outgoing bytes and nonzero incoming packets/bytes. Test 1: Bro log files were read after bro shutdown so that there should be nothing missing. Bro was installed on a computer with IP 147.251.13.65, monitoring its own traffic. A web browser was used to generate connections. One of the contacted addresses was 77.75.77.170. Data from conn.log: id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes 147.251.13.65 44690 77.75.77.170 993 tcp 0 151 0 0 3 307 (this was the only line with 147.251.13.65 and 77.75.77.170) Data from the network monitoring probe: nfdump filter: ip 147.251.13.65 and ip 77.75.77.170 Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows 2013-08-14 09:54:30.355 0.054 TCP 77.75.77.170:993 -> 147.251.13.65:44690 .AP... 0 3 307 55 45481 102 1 2013-08-14 09:54:30.347 0.062 TCP 147.251.13.65:44690 -> 77.75.77.170:993 .AP... 0 5 348 80 44903 69 1 Summary: total flows: 2, total bytes: 655, total packets: 8, avg bps: 84516, avg pps: 129, avg bpp: 81 Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00 Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460 Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5795248.1 Test 2: Bro log files were read after bro shutdown so that there should be nothing missing. Bro was installed on a computer with IP 147.251.13.65, monitoring its own traffic. A web browser was used to generate connections. One of the contacted addresses was 195.113.232.73. Debug output of our own Bro script watching connections using connection_state_remove event handler. The script's code for the event handler is: event connection_state_remove(c: connection) { NOTICE([$note=profile_creation::Test3, $msg=cat("DBG: ", c$id$orig_h, "->", c$id$resp_h, " c$orig$num_pkts=", c$orig$num_pkts, " c$resp$num_pkts=", c$resp$num_pkts, " c$conn$orig_pkts=", c$conn$orig_pkts, " c$conn$resp_pkts=", c$conn$resp_pkts, " c$orig$num_bytes_ip=", c$orig$num_bytes_ip, " c$resp$num_bytes_ip=", c$resp$num_bytes_ip, " c$conn$orig_ip_bytes=", c$conn$orig_ip_bytes, " c$conn$resp_ip_bytes=", c$conn$resp_ip_bytes, " c$id$orig_p=", c$id$orig_p, " c$id$resp_p=", c$id$resp_p )]); } The cut out output: DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49631/tcp c$id$resp_p=80/tcp DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49721/tcp c$id$resp_p=80/tcp DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49723/tcp c$id$resp_p=80/tcp DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49724/tcp c$id$resp_p=80/tcp DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49725/tcp c$id$resp_p=80/tcp DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=17 c$conn$orig_pkts=0 c$conn$resp_pkts=17 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=21613 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=21613 c$id$orig_p=49725/tcp c$id$resp_p=80/tcp DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=5 c$conn$orig_pkts=0 c$conn$resp_pkts=5 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=5032 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=5032 c$id$orig_p=49721/tcp c$id$resp_p=80/tcp DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=5 c$conn$orig_pkts=0 c$conn$resp_pkts=5 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=5056 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=5056 c$id$orig_p=49723/tcp c$id$resp_p=80/tcp DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=4 c$conn$orig_pkts=0 c$conn$resp_pkts=4 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=4110 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=4110 c$id$orig_p=49724/tcp c$id$resp_p=80/tcp All relevant data from conn.log: id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes 147.251.13.65 49631 195.113.232.73 80 tcp - - 0 0 0 0 147.251.13.65 49721 195.113.232.73 80 tcp - - 0 0 0 0 147.251.13.65 49723 195.113.232.73 80 tcp - - 0 0 0 0 147.251.13.65 49724 195.113.232.73 80 tcp - - 0 0 0 0 147.251.13.65 49725 195.113.232.73 80 tcp - - 0 0 0 0 147.251.13.65 49725 195.113.232.73 80 tcp 0 20729 0 0 17 21613 147.251.13.65 49721 195.113.232.73 80 tcp 0 4772 0 0 5 5032 147.251.13.65 49723 195.113.232.73 80 tcp 0 4796 0 0 5 5056 147.251.13.65 49724 195.113.232.73 80 tcp 0 3902 0 0 4 4110 Data from the network monitoring probe: ** nfdump -M /data/nfsen/profiles-data/live/p3006:p3005:p3004:p3002:p3003:p3001:p3000 -T -r 2013/08/14/nfcapd.201308140955 -o extended -c 500 nfdump filter: ip 147.251.13.65 and ip 195.113.232.73 Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows 2013-08-14 09:54:33.929 28.188 TCP 195.113.232.73:80 -> 147.251.13.65:49725 .AP..F 0 15 19873 0 5640 1324 1 2013-08-14 09:54:17.976 44.141 TCP 195.113.232.73:80 -> 147.251.13.65:49631 .AP.SF 0 6 3709 0 672 618 1 2013-08-14 09:54:33.958 28.159 TCP 195.113.232.73:80 -> 147.251.13.65:49723 .AP..F 0 5 5056 0 1436 1011 1 2013-08-14 09:54:33.948 28.169 TCP 195.113.232.73:80 -> 147.251.13.65:49721 .AP..F 0 5 5032 0 1429 1006 1 2013-08-14 09:54:33.950 28.167 TCP 195.113.232.73:80 -> 147.251.13.65:49724 .AP..F 0 4 4110 0 1167 1027 1 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49725 -> 195.113.232.73:80 .AP..F 0 13 1732 0 488 133 1 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49724 -> 195.113.232.73:80 .AP..F 0 6 1356 0 382 226 1 2013-08-14 09:54:17.972 44.145 TCP 147.251.13.65:49631 -> 195.113.232.73:80 .AP.SF 0 8 746 0 135 93 1 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49723 -> 195.113.232.73:80 .AP..F 0 7 1408 0 397 201 1 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49721 -> 195.113.232.73:80 .AP..F 0 5 1304 0 368 260 1 Summary: total flows: 10, total bytes: 44326, total packets: 74, avg bps: 8032, avg pps: 1, avg bpp: 599 Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00 Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460 Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5782205.9 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Thu Aug 15 00:00:10 2013 From: noreply at bro.org (Merge Tracker) Date: Thu, 15 Aug 2013 00:00:10 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308150700.r7F70A8a009474@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ---------- ---------- ---------- ------------- ---------- ------------------------------------------------------------------------- BIT-714 [1] BroControl aashish - 2013-08-14 2.2 Normal broctl install copies policy files to the .site folder in incorrect order Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ---------------------------------------------- #1 [2] bro anthonykasza [3] 2013-08-13 levenshtein distance [4] #1 [5] broccoli dcode [6] 2013-08-12 Updated specfile. Works under mock for EL6 [7] [1] BIT-714 https://bro-tracker.atlassian.net/browse/BIT-714 [2] Pull Request #1 https://github.com/bro/bro/issues/1 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #1 with git pull https://github.com/anthonykasza/bro.git master [5] Pull Request #1 https://github.com/bro/broccoli/issues/1 [6] dcode https://github.com/dcode [7] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Thu Aug 15 05:01:21 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 15 Aug 2013 07:01:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1056) incorrect values in num_pkts and num_bytes_ip In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508#comment-13508 ] Seth Hall commented on BIT-1056: -------------------------------- What do you mean by a "network probe"? In the logs that you sent that show 0 outbound packets and 0 outbound bytes was that all from hosts sniffing their own local interface? If that's the case then you have invalid checksums (due to on-NIC checksum offloading) which would cause exactly what you were showing. > incorrect values in num_pkts and num_bytes_ip > --------------------------------------------- > > Key: BIT-1056 > URL: https://bro-tracker.atlassian.net/browse/BIT-1056 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.1 > Environment: Two separate testing environments: > 1. Intel i3, OpenSuse x64, public IP, Bro is on the computer monitoring its own traffic > 2. Intel i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine has private IP, Bro is on virtual machine monitoring its own traffic > Reporter: Jakub Svoboda > Labels: issue > > Problem: Bro does not see traffic correctly. > * conn.log contains invalid information, with values differing from a network monitoring probe. (The probe runs on a different computer, so there's no interference.) > * Our own scripts logging connections using the connection_state_remove event report the same bad values as conn.log. > * Specifically, all following values are off in the connection_state_remove event handler: c$orig$num_pkts c$resp$num_pkts c$conn$orig_pkts c$conn$resp_pkts c$orig$num_bytes_ip c$resp$num_bytes_ip c$conn$orig_ip_bytes c$conn$resp_ip_bytes. (Yes, we know some of the values are copied one from another in conn/main.bro.) > * Problem is the same on both tested machines. > * Problem is the same both when capturing an interface and when analyzing a pcap. > * Problem is the same on both Bro 2.1 and the current git version (as of 14.08.2013) > The problem is exactly the same on two different computers. > 1. i3, OpenSuse, public IP > 2. i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine has private IP, Bro is on virtual machine monitoring its own traffic > Both computers use 64bit OS and Bro 2.1 compiled from source. > Notes: The chance of the network monitoring probe reporting bad values is very low as this probe is used by our department of computer science and several people checked it works correctly. > Also, the values reported by Bro are obviously wrong even without no comparison with the probe---e.g. an outgoing flow with 0 outgoing packets, 0 outgoing bytes and nonzero incoming packets/bytes. > Test 1: > Bro log files were read after bro shutdown so that there should be nothing missing. > Bro was installed on a computer with IP 147.251.13.65, monitoring its own traffic. A web browser was used to generate connections. One of the contacted addresses was 77.75.77.170. > Data from conn.log: > id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes > 147.251.13.65 44690 77.75.77.170 993 tcp 0 151 0 0 3 307 > (this was the only line with 147.251.13.65 and 77.75.77.170) > Data from the network monitoring probe: > nfdump filter: > ip 147.251.13.65 and ip 77.75.77.170 > Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows > 2013-08-14 09:54:30.355 0.054 TCP 77.75.77.170:993 -> 147.251.13.65:44690 .AP... 0 3 307 55 45481 102 1 > 2013-08-14 09:54:30.347 0.062 TCP 147.251.13.65:44690 -> 77.75.77.170:993 .AP... 0 5 348 80 44903 69 1 > Summary: total flows: 2, total bytes: 655, total packets: 8, avg bps: 84516, avg pps: 129, avg bpp: 81 > Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00 > Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460 > Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5795248.1 > Test 2: > Bro log files were read after bro shutdown so that there should be nothing missing. > Bro was installed on a computer with IP 147.251.13.65, monitoring its own traffic. A web browser was used to generate connections. One of the contacted addresses was 195.113.232.73. > Debug output of our own Bro script watching connections using connection_state_remove event handler. > The script's code for the event handler is: > event connection_state_remove(c: connection) { > NOTICE([$note=profile_creation::Test3, $msg=cat("DBG: ", c$id$orig_h, "->", c$id$resp_h, " c$orig$num_pkts=", c$orig$num_pkts, " c$resp$num_pkts=", c$resp$num_pkts, " c$conn$orig_pkts=", c$conn$orig_pkts, " c$conn$resp_pkts=", c$conn$resp_pkts, " c$orig$num_bytes_ip=", c$orig$num_bytes_ip, " c$resp$num_bytes_ip=", c$resp$num_bytes_ip, " c$conn$orig_ip_bytes=", c$conn$orig_ip_bytes, " c$conn$resp_ip_bytes=", c$conn$resp_ip_bytes, " c$id$orig_p=", c$id$orig_p, " c$id$resp_p=", c$id$resp_p )]); > } > The cut out output: > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49631/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49721/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49723/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49724/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49725/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=17 c$conn$orig_pkts=0 c$conn$resp_pkts=17 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=21613 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=21613 c$id$orig_p=49725/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=5 c$conn$orig_pkts=0 c$conn$resp_pkts=5 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=5032 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=5032 c$id$orig_p=49721/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=5 c$conn$orig_pkts=0 c$conn$resp_pkts=5 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=5056 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=5056 c$id$orig_p=49723/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=4 c$conn$orig_pkts=0 c$conn$resp_pkts=4 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=4110 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=4110 c$id$orig_p=49724/tcp c$id$resp_p=80/tcp > All relevant data from conn.log: > id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes > 147.251.13.65 49631 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49721 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49723 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49724 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49725 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49725 195.113.232.73 80 tcp 0 20729 0 0 17 21613 > 147.251.13.65 49721 195.113.232.73 80 tcp 0 4772 0 0 5 5032 > 147.251.13.65 49723 195.113.232.73 80 tcp 0 4796 0 0 5 5056 > 147.251.13.65 49724 195.113.232.73 80 tcp 0 3902 0 0 4 4110 > Data from the network monitoring probe: > ** nfdump -M /data/nfsen/profiles-data/live/p3006:p3005:p3004:p3002:p3003:p3001:p3000 -T -r 2013/08/14/nfcapd.201308140955 -o extended -c 500 > nfdump filter: > ip 147.251.13.65 and ip 195.113.232.73 > Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows > 2013-08-14 09:54:33.929 28.188 TCP 195.113.232.73:80 -> 147.251.13.65:49725 .AP..F 0 15 19873 0 5640 1324 1 > 2013-08-14 09:54:17.976 44.141 TCP 195.113.232.73:80 -> 147.251.13.65:49631 .AP.SF 0 6 3709 0 672 618 1 > 2013-08-14 09:54:33.958 28.159 TCP 195.113.232.73:80 -> 147.251.13.65:49723 .AP..F 0 5 5056 0 1436 1011 1 > 2013-08-14 09:54:33.948 28.169 TCP 195.113.232.73:80 -> 147.251.13.65:49721 .AP..F 0 5 5032 0 1429 1006 1 > 2013-08-14 09:54:33.950 28.167 TCP 195.113.232.73:80 -> 147.251.13.65:49724 .AP..F 0 4 4110 0 1167 1027 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49725 -> 195.113.232.73:80 .AP..F 0 13 1732 0 488 133 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49724 -> 195.113.232.73:80 .AP..F 0 6 1356 0 382 226 1 > 2013-08-14 09:54:17.972 44.145 TCP 147.251.13.65:49631 -> 195.113.232.73:80 .AP.SF 0 8 746 0 135 93 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49723 -> 195.113.232.73:80 .AP..F 0 7 1408 0 397 201 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49721 -> 195.113.232.73:80 .AP..F 0 5 1304 0 368 260 1 > Summary: total flows: 10, total bytes: 44326, total packets: 74, avg bps: 8032, avg pps: 1, avg bpp: 599 > Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00 > Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460 > Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5782205.9 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 15 05:06:21 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 15 Aug 2013 07:06:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1056) incorrect values in num_pkts and num_bytes_ip In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509#comment-13509 ] Seth Hall commented on BIT-1056: -------------------------------- Nevermind, is re-read and I see that that is the case. You are seeing effects of TCP checksum offloading. Refer to this blog post for more information about the problem: http://securityonion.blogspot.com/2011/10/when-is-full-packet-capture-not-full.html For testing purposes (*not in production!*) you can put the following line into local.bro: bq. redef ignore_checksums = T; > incorrect values in num_pkts and num_bytes_ip > --------------------------------------------- > > Key: BIT-1056 > URL: https://bro-tracker.atlassian.net/browse/BIT-1056 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.1 > Environment: Two separate testing environments: > 1. Intel i3, OpenSuse x64, public IP, Bro is on the computer monitoring its own traffic > 2. Intel i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine has private IP, Bro is on virtual machine monitoring its own traffic > Reporter: Jakub Svoboda > Labels: issue > > Problem: Bro does not see traffic correctly. > * conn.log contains invalid information, with values differing from a network monitoring probe. (The probe runs on a different computer, so there's no interference.) > * Our own scripts logging connections using the connection_state_remove event report the same bad values as conn.log. > * Specifically, all following values are off in the connection_state_remove event handler: c$orig$num_pkts c$resp$num_pkts c$conn$orig_pkts c$conn$resp_pkts c$orig$num_bytes_ip c$resp$num_bytes_ip c$conn$orig_ip_bytes c$conn$resp_ip_bytes. (Yes, we know some of the values are copied one from another in conn/main.bro.) > * Problem is the same on both tested machines. > * Problem is the same both when capturing an interface and when analyzing a pcap. > * Problem is the same on both Bro 2.1 and the current git version (as of 14.08.2013) > The problem is exactly the same on two different computers. > 1. i3, OpenSuse, public IP > 2. i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine has private IP, Bro is on virtual machine monitoring its own traffic > Both computers use 64bit OS and Bro 2.1 compiled from source. > Notes: The chance of the network monitoring probe reporting bad values is very low as this probe is used by our department of computer science and several people checked it works correctly. > Also, the values reported by Bro are obviously wrong even without no comparison with the probe---e.g. an outgoing flow with 0 outgoing packets, 0 outgoing bytes and nonzero incoming packets/bytes. > Test 1: > Bro log files were read after bro shutdown so that there should be nothing missing. > Bro was installed on a computer with IP 147.251.13.65, monitoring its own traffic. A web browser was used to generate connections. One of the contacted addresses was 77.75.77.170. > Data from conn.log: > id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes > 147.251.13.65 44690 77.75.77.170 993 tcp 0 151 0 0 3 307 > (this was the only line with 147.251.13.65 and 77.75.77.170) > Data from the network monitoring probe: > nfdump filter: > ip 147.251.13.65 and ip 77.75.77.170 > Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows > 2013-08-14 09:54:30.355 0.054 TCP 77.75.77.170:993 -> 147.251.13.65:44690 .AP... 0 3 307 55 45481 102 1 > 2013-08-14 09:54:30.347 0.062 TCP 147.251.13.65:44690 -> 77.75.77.170:993 .AP... 0 5 348 80 44903 69 1 > Summary: total flows: 2, total bytes: 655, total packets: 8, avg bps: 84516, avg pps: 129, avg bpp: 81 > Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00 > Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460 > Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5795248.1 > Test 2: > Bro log files were read after bro shutdown so that there should be nothing missing. > Bro was installed on a computer with IP 147.251.13.65, monitoring its own traffic. A web browser was used to generate connections. One of the contacted addresses was 195.113.232.73. > Debug output of our own Bro script watching connections using connection_state_remove event handler. > The script's code for the event handler is: > event connection_state_remove(c: connection) { > NOTICE([$note=profile_creation::Test3, $msg=cat("DBG: ", c$id$orig_h, "->", c$id$resp_h, " c$orig$num_pkts=", c$orig$num_pkts, " c$resp$num_pkts=", c$resp$num_pkts, " c$conn$orig_pkts=", c$conn$orig_pkts, " c$conn$resp_pkts=", c$conn$resp_pkts, " c$orig$num_bytes_ip=", c$orig$num_bytes_ip, " c$resp$num_bytes_ip=", c$resp$num_bytes_ip, " c$conn$orig_ip_bytes=", c$conn$orig_ip_bytes, " c$conn$resp_ip_bytes=", c$conn$resp_ip_bytes, " c$id$orig_p=", c$id$orig_p, " c$id$resp_p=", c$id$resp_p )]); > } > The cut out output: > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49631/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49721/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49723/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49724/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49725/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=17 c$conn$orig_pkts=0 c$conn$resp_pkts=17 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=21613 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=21613 c$id$orig_p=49725/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=5 c$conn$orig_pkts=0 c$conn$resp_pkts=5 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=5032 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=5032 c$id$orig_p=49721/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=5 c$conn$orig_pkts=0 c$conn$resp_pkts=5 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=5056 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=5056 c$id$orig_p=49723/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=4 c$conn$orig_pkts=0 c$conn$resp_pkts=4 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=4110 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=4110 c$id$orig_p=49724/tcp c$id$resp_p=80/tcp > All relevant data from conn.log: > id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes > 147.251.13.65 49631 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49721 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49723 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49724 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49725 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49725 195.113.232.73 80 tcp 0 20729 0 0 17 21613 > 147.251.13.65 49721 195.113.232.73 80 tcp 0 4772 0 0 5 5032 > 147.251.13.65 49723 195.113.232.73 80 tcp 0 4796 0 0 5 5056 > 147.251.13.65 49724 195.113.232.73 80 tcp 0 3902 0 0 4 4110 > Data from the network monitoring probe: > ** nfdump -M /data/nfsen/profiles-data/live/p3006:p3005:p3004:p3002:p3003:p3001:p3000 -T -r 2013/08/14/nfcapd.201308140955 -o extended -c 500 > nfdump filter: > ip 147.251.13.65 and ip 195.113.232.73 > Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows > 2013-08-14 09:54:33.929 28.188 TCP 195.113.232.73:80 -> 147.251.13.65:49725 .AP..F 0 15 19873 0 5640 1324 1 > 2013-08-14 09:54:17.976 44.141 TCP 195.113.232.73:80 -> 147.251.13.65:49631 .AP.SF 0 6 3709 0 672 618 1 > 2013-08-14 09:54:33.958 28.159 TCP 195.113.232.73:80 -> 147.251.13.65:49723 .AP..F 0 5 5056 0 1436 1011 1 > 2013-08-14 09:54:33.948 28.169 TCP 195.113.232.73:80 -> 147.251.13.65:49721 .AP..F 0 5 5032 0 1429 1006 1 > 2013-08-14 09:54:33.950 28.167 TCP 195.113.232.73:80 -> 147.251.13.65:49724 .AP..F 0 4 4110 0 1167 1027 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49725 -> 195.113.232.73:80 .AP..F 0 13 1732 0 488 133 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49724 -> 195.113.232.73:80 .AP..F 0 6 1356 0 382 226 1 > 2013-08-14 09:54:17.972 44.145 TCP 147.251.13.65:49631 -> 195.113.232.73:80 .AP.SF 0 8 746 0 135 93 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49723 -> 195.113.232.73:80 .AP..F 0 7 1408 0 397 201 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49721 -> 195.113.232.73:80 .AP..F 0 5 1304 0 368 260 1 > Summary: total flows: 10, total bytes: 44326, total packets: 74, avg bps: 8032, avg pps: 1, avg bpp: 599 > Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00 > Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460 > Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5782205.9 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 15 05:08:21 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 15 Aug 2013 07:08:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1056) incorrect values in num_pkts and num_bytes_ip In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1056: --------------------------- Resolution: Rejected Status: Closed (was: Open) > incorrect values in num_pkts and num_bytes_ip > --------------------------------------------- > > Key: BIT-1056 > URL: https://bro-tracker.atlassian.net/browse/BIT-1056 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.1 > Environment: Two separate testing environments: > 1. Intel i3, OpenSuse x64, public IP, Bro is on the computer monitoring its own traffic > 2. Intel i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine has private IP, Bro is on virtual machine monitoring its own traffic > Reporter: Jakub Svoboda > Labels: issue > > Problem: Bro does not see traffic correctly. > * conn.log contains invalid information, with values differing from a network monitoring probe. (The probe runs on a different computer, so there's no interference.) > * Our own scripts logging connections using the connection_state_remove event report the same bad values as conn.log. > * Specifically, all following values are off in the connection_state_remove event handler: c$orig$num_pkts c$resp$num_pkts c$conn$orig_pkts c$conn$resp_pkts c$orig$num_bytes_ip c$resp$num_bytes_ip c$conn$orig_ip_bytes c$conn$resp_ip_bytes. (Yes, we know some of the values are copied one from another in conn/main.bro.) > * Problem is the same on both tested machines. > * Problem is the same both when capturing an interface and when analyzing a pcap. > * Problem is the same on both Bro 2.1 and the current git version (as of 14.08.2013) > The problem is exactly the same on two different computers. > 1. i3, OpenSuse, public IP > 2. i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine has private IP, Bro is on virtual machine monitoring its own traffic > Both computers use 64bit OS and Bro 2.1 compiled from source. > Notes: The chance of the network monitoring probe reporting bad values is very low as this probe is used by our department of computer science and several people checked it works correctly. > Also, the values reported by Bro are obviously wrong even without no comparison with the probe---e.g. an outgoing flow with 0 outgoing packets, 0 outgoing bytes and nonzero incoming packets/bytes. > Test 1: > Bro log files were read after bro shutdown so that there should be nothing missing. > Bro was installed on a computer with IP 147.251.13.65, monitoring its own traffic. A web browser was used to generate connections. One of the contacted addresses was 77.75.77.170. > Data from conn.log: > id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes > 147.251.13.65 44690 77.75.77.170 993 tcp 0 151 0 0 3 307 > (this was the only line with 147.251.13.65 and 77.75.77.170) > Data from the network monitoring probe: > nfdump filter: > ip 147.251.13.65 and ip 77.75.77.170 > Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows > 2013-08-14 09:54:30.355 0.054 TCP 77.75.77.170:993 -> 147.251.13.65:44690 .AP... 0 3 307 55 45481 102 1 > 2013-08-14 09:54:30.347 0.062 TCP 147.251.13.65:44690 -> 77.75.77.170:993 .AP... 0 5 348 80 44903 69 1 > Summary: total flows: 2, total bytes: 655, total packets: 8, avg bps: 84516, avg pps: 129, avg bpp: 81 > Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00 > Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460 > Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5795248.1 > Test 2: > Bro log files were read after bro shutdown so that there should be nothing missing. > Bro was installed on a computer with IP 147.251.13.65, monitoring its own traffic. A web browser was used to generate connections. One of the contacted addresses was 195.113.232.73. > Debug output of our own Bro script watching connections using connection_state_remove event handler. > The script's code for the event handler is: > event connection_state_remove(c: connection) { > NOTICE([$note=profile_creation::Test3, $msg=cat("DBG: ", c$id$orig_h, "->", c$id$resp_h, " c$orig$num_pkts=", c$orig$num_pkts, " c$resp$num_pkts=", c$resp$num_pkts, " c$conn$orig_pkts=", c$conn$orig_pkts, " c$conn$resp_pkts=", c$conn$resp_pkts, " c$orig$num_bytes_ip=", c$orig$num_bytes_ip, " c$resp$num_bytes_ip=", c$resp$num_bytes_ip, " c$conn$orig_ip_bytes=", c$conn$orig_ip_bytes, " c$conn$resp_ip_bytes=", c$conn$resp_ip_bytes, " c$id$orig_p=", c$id$orig_p, " c$id$resp_p=", c$id$resp_p )]); > } > The cut out output: > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49631/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49721/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49723/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49724/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49725/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=17 c$conn$orig_pkts=0 c$conn$resp_pkts=17 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=21613 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=21613 c$id$orig_p=49725/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=5 c$conn$orig_pkts=0 c$conn$resp_pkts=5 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=5032 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=5032 c$id$orig_p=49721/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=5 c$conn$orig_pkts=0 c$conn$resp_pkts=5 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=5056 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=5056 c$id$orig_p=49723/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=4 c$conn$orig_pkts=0 c$conn$resp_pkts=4 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=4110 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=4110 c$id$orig_p=49724/tcp c$id$resp_p=80/tcp > All relevant data from conn.log: > id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes > 147.251.13.65 49631 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49721 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49723 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49724 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49725 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49725 195.113.232.73 80 tcp 0 20729 0 0 17 21613 > 147.251.13.65 49721 195.113.232.73 80 tcp 0 4772 0 0 5 5032 > 147.251.13.65 49723 195.113.232.73 80 tcp 0 4796 0 0 5 5056 > 147.251.13.65 49724 195.113.232.73 80 tcp 0 3902 0 0 4 4110 > Data from the network monitoring probe: > ** nfdump -M /data/nfsen/profiles-data/live/p3006:p3005:p3004:p3002:p3003:p3001:p3000 -T -r 2013/08/14/nfcapd.201308140955 -o extended -c 500 > nfdump filter: > ip 147.251.13.65 and ip 195.113.232.73 > Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows > 2013-08-14 09:54:33.929 28.188 TCP 195.113.232.73:80 -> 147.251.13.65:49725 .AP..F 0 15 19873 0 5640 1324 1 > 2013-08-14 09:54:17.976 44.141 TCP 195.113.232.73:80 -> 147.251.13.65:49631 .AP.SF 0 6 3709 0 672 618 1 > 2013-08-14 09:54:33.958 28.159 TCP 195.113.232.73:80 -> 147.251.13.65:49723 .AP..F 0 5 5056 0 1436 1011 1 > 2013-08-14 09:54:33.948 28.169 TCP 195.113.232.73:80 -> 147.251.13.65:49721 .AP..F 0 5 5032 0 1429 1006 1 > 2013-08-14 09:54:33.950 28.167 TCP 195.113.232.73:80 -> 147.251.13.65:49724 .AP..F 0 4 4110 0 1167 1027 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49725 -> 195.113.232.73:80 .AP..F 0 13 1732 0 488 133 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49724 -> 195.113.232.73:80 .AP..F 0 6 1356 0 382 226 1 > 2013-08-14 09:54:17.972 44.145 TCP 147.251.13.65:49631 -> 195.113.232.73:80 .AP.SF 0 8 746 0 135 93 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49723 -> 195.113.232.73:80 .AP..F 0 7 1408 0 397 201 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49721 -> 195.113.232.73:80 .AP..F 0 5 1304 0 368 260 1 > Summary: total flows: 10, total bytes: 44326, total packets: 74, avg bps: 8032, avg pps: 1, avg bpp: 599 > Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00 > Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460 > Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5782205.9 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From bernhard at ICSI.Berkeley.EDU Thu Aug 15 11:26:53 2013 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Thu, 15 Aug 2013 11:26:53 -0700 Subject: [Bro-Dev] Move 3rdparty into a separate submodule In-Reply-To: References: Message-ID: Just wanted to check if really none has an opinion on this besides me :) Bernhard On Jul 31, 2013, at 4:25 PM, Bernhard Amann wrote: > Hi, > > I just wanted to ask what you think about moving the 3rdparty stuff under src (which at > the moment only contains sqlite) into a separate git submodule? > > That way we keep the 3rdparty sources and their change history out of Bro history. SQLite > alone already has more than 140k lines - and I kind of suspect that we might other 3rdparty > sources in the future. > > The disadvantage of this approach is, that the bro git repository does no longer contain > all source-files necessary to compile bro. On the other hand - stuff like cmake already is > in submodules - so even though the bro git contains all source files at the moment, you > still cannot compile it without checking out the submodules. > > Bernhard From robin at icir.org Thu Aug 15 11:33:18 2013 From: robin at icir.org (Robin Sommer) Date: Thu, 15 Aug 2013 11:33:18 -0700 Subject: [Bro-Dev] Move 3rdparty into a separate submodule In-Reply-To: References: Message-ID: <20130815183318.GR34835@icir.org> I think it's a good idea. Robin On Thu, Aug 15, 2013 at 11:26 -0700, you wrote: > Just wanted to check if really none has an opinion on this besides me :) > > Bernhard > > On Jul 31, 2013, at 4:25 PM, Bernhard Amann wrote: > > > Hi, > > > > I just wanted to ask what you think about moving the 3rdparty stuff under src (which at > > the moment only contains sqlite) into a separate git submodule? > > > > That way we keep the 3rdparty sources and their change history out of Bro history. SQLite > > alone already has more than 140k lines - and I kind of suspect that we might other 3rdparty > > sources in the future. > > > > The disadvantage of this approach is, that the bro git repository does no longer contain > > all source-files necessary to compile bro. On the other hand - stuff like cmake already is > > in submodules - so even though the bro git contains all source files at the moment, you > > still cannot compile it without checking out the submodules. > > > > Bernhard > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From seth at icir.org Thu Aug 15 11:37:57 2013 From: seth at icir.org (Seth Hall) Date: Thu, 15 Aug 2013 14:37:57 -0400 Subject: [Bro-Dev] Move 3rdparty into a separate submodule In-Reply-To: <20130815183318.GR34835@icir.org> References: <20130815183318.GR34835@icir.org> Message-ID: <0416A42A-5926-42BC-A158-C4D850A7398E@icir.org> On Aug 15, 2013, at 2:33 PM, Robin Sommer wrote: > I think it's a good idea. Me too. -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130815/9e047bcd/attachment.bin From noreply at bro.org Fri Aug 16 00:00:10 2013 From: noreply at bro.org (Merge Tracker) Date: Fri, 16 Aug 2013 00:00:10 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308160700.r7G70AmU017470@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ---------- ---------- ---------- ------------- ---------- ------------------------------------------------------------------------- BIT-714 [1] BroControl aashish - 2013-08-14 2.2 Normal broctl install copies policy files to the .site folder in incorrect order Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ---------------------------------------------- #1 [2] bro anthonykasza [3] 2013-08-13 levenshtein distance [4] #1 [5] broccoli dcode [6] 2013-08-12 Updated specfile. Works under mock for EL6 [7] [1] BIT-714 https://bro-tracker.atlassian.net/browse/BIT-714 [2] Pull Request #1 https://github.com/bro/bro/issues/1 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #1 with git pull https://github.com/anthonykasza/bro.git master [5] Pull Request #1 https://github.com/bro/broccoli/issues/1 [6] dcode https://github.com/dcode [7] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Fri Aug 16 11:16:21 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 16 Aug 2013 13:16:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1057) topic/jsiwek/bloomfilter-fix In-Reply-To: References: Message-ID: Jon Siwek created BIT-1057: ------------------------------ Summary: topic/jsiwek/bloomfilter-fix Key: BIT-1057 URL: https://bro-tracker.atlassian.net/browse/BIT-1057 Project: Bro Issue Tracker Issue Type: Patch Components: Bro Affects Versions: git/master Reporter: Jon Siwek Assignee: Matthias Vallentin Fix For: 2.2 The change (1) I did on this branch makes the bloomfilter-seed test start passing on 32-bit systems, though I'm not confident how correct it is. Matthias, can you review it and change this to a merge request if it looks sane? (1) https://github.com/bro/bro/commit/774dadfe9aedc9fed98df69abcc83a3068bdf06b -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 16 13:17:21 2013 From: jira at bro-tracker.atlassian.net (Matthias Vallentin (JIRA)) Date: Fri, 16 Aug 2013 15:17:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1057) topic/jsiwek/bloomfilter-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510#comment-13510 ] Matthias Vallentin commented on BIT-1057: ----------------------------------------- Yup, this looks good to me. For the record: the problem was indeed that the platform-specific block size of the bit vector. This could have resulted in different hash values, because the hashing algorithm took the underlying representations (i.e., the bit vector blocks) as opposed to the individual bits themselves. Moreover, the {{fmt}} function casted the 32-bit value into 64-bit one, potentially causing undefined behavior. I will merge your changes into {{topic/matthias/bloom-filter}} and only do cosmetic polishing, such as using Bro's {{uint64}} instead of {{uint64_t}}. > topic/jsiwek/bloomfilter-fix > ---------------------------- > > Key: BIT-1057 > URL: https://bro-tracker.atlassian.net/browse/BIT-1057 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Matthias Vallentin > Fix For: 2.2 > > > The change (1) I did on this branch makes the bloomfilter-seed test start passing on 32-bit systems, though I'm not confident how correct it is. Matthias, can you review it and change this to a merge request if it looks sane? > (1) https://github.com/bro/bro/commit/774dadfe9aedc9fed98df69abcc83a3068bdf06b -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 16 15:54:21 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 16 Aug 2013 17:54:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-714) broctl install copies policy files to the .site folder in incorrect order In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-714: ----------------------------- Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > broctl install copies policy files to the .site folder in incorrect order > ------------------------------------------------------------------------- > > Key: BIT-714 > URL: https://bro-tracker.atlassian.net/browse/BIT-714 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: aashish > Fix For: 2.2 > > > I think broctl install copies the files to .site directory in the order specified by the SitePolicyPath This is probably incorrect. It should instead be in the reverse order of the path specified in the SitePolicyPath > example: > SitePolicyPath=/dir1:/dir2:/dir3:/usr/local/bro/share/bro > if a modified version of file with same name (eg. drop.bro) is in dir1 and original is in /usr/local/bro/share/bro; broctl install will overwrite /dir1/drop.bro with /usr/local/bro/share/bro/drop.bro when it creates /usr/local/bro/share/bro/.site folder. > In theory when bro is starting, it should have preference in loading /dir1/drop.bro over /usr/local/bro/share/bro/drop.bro but currently /dir1/drop.bro gets overwritten by /usr/local/bro/share/bro/drop.bro when broctl install creates .site folder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 16 17:14:21 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 16 Aug 2013 19:14:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1058) Memory leak in sumstats (probably) In-Reply-To: References: Message-ID: Bernhard Amann created BIT-1058: ----------------------------------- Summary: Memory leak in sumstats (probably) Key: BIT-1058 URL: https://bro-tracker.atlassian.net/browse/BIT-1058 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.2 Reporter: Bernhard Amann Priority: High Fix For: 2.2 Attachments: out2.pdf At the moment, the core/leaks/basic-cluster.bro always fails; the gprof output is attached. Only the master node leaks memory, the two worker nodes are fine. >From the gprof output, it looks like an increment operation is somehow triggering a memory leak. Robin and me tried to dig through this for quite some time. From our current understanding it looks like the memory leak is (indirectly) caused by an increment operation in a function that is called by an event that is received through remoteserialization. The closest we were able to track the leak to is line 249 of scripts/base/frameworks/sumstats/cluster.bro: {noformat} event SumStats::cluster_send_result(uid: string, ss_name: string, key: Key, result: Result, cleanup: bool) { [...] ++done_with[uid]; } {noformat} Commenting out this line "fixes" the memory leak (and probably renders the sumstat framework inoperable); however we were not able to track it further to the exact cause; replacing the increment with an equivalent done_with[uid] = done_with[uid]+1; did not solve the problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 16 17:41:21 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 16 Aug 2013 19:41:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-995) Potential core reference counting bug through sumstats framework In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann updated BIT-995: ------------------------------- Resolution: Invalid Status: Closed (was: Open) This bug will probably be near-impossible to reproduce now due to all the changes in the sumstats framework. If I ever manage to - I will re-open this... > Potential core reference counting bug through sumstats framework > ---------------------------------------------------------------- > > Key: BIT-995 > URL: https://bro-tracker.atlassian.net/browse/BIT-995 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > Attachments: measure-comm.bro > > > Testing on our FreeBSD cluster has shown some strange behavior when > running the sumstats framework with scan.bro and using the attached > script to log exchanged cluster messages. > This is rather interesting, because the script only writes information to > the logging framework and interacts in no way with anything that should > be able to mess with the reference counting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From bernhard at ICSI.Berkeley.EDU Fri Aug 16 17:44:56 2013 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Fri, 16 Aug 2013 17:44:56 -0700 Subject: [Bro-Dev] Move 3rdparty into a separate submodule In-Reply-To: <0416A42A-5926-42BC-A158-C4D850A7398E@icir.org> References: <20130815183318.GR34835@icir.org> <0416A42A-5926-42BC-A158-C4D850A7398E@icir.org> Message-ID: If someone can create a new git-repo for it, I can move it there? or I can file a bug-report :) Bernhard On Aug 15, 2013, at 11:37 AM, Seth Hall wrote: > On Aug 15, 2013, at 2:33 PM, Robin Sommer wrote: > >> I think it's a good idea. > > > Me too. > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From robin at icir.org Fri Aug 16 18:14:09 2013 From: robin at icir.org (Robin Sommer) Date: Fri, 16 Aug 2013 18:14:09 -0700 Subject: [Bro-Dev] Move 3rdparty into a separate submodule In-Reply-To: References: <20130815183318.GR34835@icir.org> <0416A42A-5926-42BC-A158-C4D850A7398E@icir.org> Message-ID: <20130817011409.GP79130@icir.org> bro-3rdparty exists now. You have admin privs, I suggest you prepare that one directly in its master branch, and then do a topic branch in bro that pulls that in. Robin On Fri, Aug 16, 2013 at 17:44 -0700, you wrote: > If someone can create a new git-repo for it, I can move it there? > > or I can file a bug-report :) > > Bernhard > > On Aug 15, 2013, at 11:37 AM, Seth Hall wrote: > > > On Aug 15, 2013, at 2:33 PM, Robin Sommer wrote: > > > >> I think it's a good idea. > > > > > > Me too. > > > > -- > > Seth Hall > > International Computer Science Institute > > (Bro) because everyone has a network > > http://www.bro.org/ > > > > > > _______________________________________________ > > bro-dev mailing list > > bro-dev at bro.org > > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Fri Aug 16 18:45:21 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 16 Aug 2013 20:45:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1059) merge topic/bernhard/3rdparty In-Reply-To: References: Message-ID: Bernhard Amann created BIT-1059: ----------------------------------- Summary: merge topic/bernhard/3rdparty Key: BIT-1059 URL: https://bro-tracker.atlassian.net/browse/BIT-1059 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Bernhard Amann Fix For: 2.2 please merge topic/bernhard/3rdparty - sqlite moved there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 16 18:45:21 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 16 Aug 2013 20:45:21 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1059) merge topic/bernhard/3rdparty In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann updated BIT-1059: -------------------------------- Status: Merge Request (was: Open) > merge topic/bernhard/3rdparty > ----------------------------- > > Key: BIT-1059 > URL: https://bro-tracker.atlassian.net/browse/BIT-1059 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > > please merge topic/bernhard/3rdparty - sqlite moved there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Sat Aug 17 00:00:10 2013 From: noreply at bro.org (Merge Tracker) Date: Sat, 17 Aug 2013 00:00:10 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308170700.r7H70AdJ004062@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- -------------- ---------- ---------- ------------- ---------- ----------------------------- BIT-1059 [1] Bro Bernhard Amann - 2013-08-16 2.2 Normal merge topic/bernhard/3rdparty Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ---------------------------------------------- #1 [2] bro anthonykasza [3] 2013-08-13 levenshtein distance [4] #1 [5] broccoli dcode [6] 2013-08-12 Updated specfile. Works under mock for EL6 [7] [1] BIT-1059 https://bro-tracker.atlassian.net/browse/BIT-1059 [2] Pull Request #1 https://github.com/bro/bro/issues/1 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #1 with git pull https://github.com/anthonykasza/bro.git master [5] Pull Request #1 https://github.com/bro/broccoli/issues/1 [6] dcode https://github.com/dcode [7] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From noreply at bro.org Sun Aug 18 00:00:11 2013 From: noreply at bro.org (Merge Tracker) Date: Sun, 18 Aug 2013 00:00:11 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308180700.r7I70BwS007438@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- -------------- ---------- ---------- ------------- ---------- ----------------------------- BIT-1059 [1] Bro Bernhard Amann - 2013-08-16 2.2 Normal merge topic/bernhard/3rdparty Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ---------------------------------------------- #1 [2] bro anthonykasza [3] 2013-08-17 levenshtein distance [4] #1 [5] broccoli dcode [6] 2013-08-12 Updated specfile. Works under mock for EL6 [7] [1] BIT-1059 https://bro-tracker.atlassian.net/browse/BIT-1059 [2] Pull Request #1 https://github.com/bro/bro/issues/1 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #1 with git pull https://github.com/anthonykasza/bro.git master [5] Pull Request #1 https://github.com/bro/broccoli/issues/1 [6] dcode https://github.com/dcode [7] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From noreply at bro.org Mon Aug 19 00:00:11 2013 From: noreply at bro.org (Merge Tracker) Date: Mon, 19 Aug 2013 00:00:11 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308190700.r7J70BOI014580@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- -------------- ---------- ---------- ------------- ---------- ----------------------------- BIT-1059 [1] Bro Bernhard Amann - 2013-08-16 2.2 Normal merge topic/bernhard/3rdparty Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ---------------------------------------------- #1 [2] bro anthonykasza [3] 2013-08-17 levenshtein distance [4] #1 [5] broccoli dcode [6] 2013-08-12 Updated specfile. Works under mock for EL6 [7] [1] BIT-1059 https://bro-tracker.atlassian.net/browse/BIT-1059 [2] Pull Request #1 https://github.com/bro/bro/issues/1 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #1 with git pull https://github.com/anthonykasza/bro.git master [5] Pull Request #1 https://github.com/bro/broccoli/issues/1 [6] dcode https://github.com/dcode [7] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Mon Aug 19 03:08:39 2013 From: jira at bro-tracker.atlassian.net (Jakub Svoboda (JIRA)) Date: Mon, 19 Aug 2013 05:08:39 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1056) incorrect values in num_pkts and num_bytes_ip In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600#comment-13600 ] Jakub Svoboda commented on BIT-1056: ------------------------------------ The problem was indeed caused by the offloading. It works correctly with offloading disabled. Thank you for your help and your time! > incorrect values in num_pkts and num_bytes_ip > --------------------------------------------- > > Key: BIT-1056 > URL: https://bro-tracker.atlassian.net/browse/BIT-1056 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.1 > Environment: Two separate testing environments: > 1. Intel i3, OpenSuse x64, public IP, Bro is on the computer monitoring its own traffic > 2. Intel i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine has private IP, Bro is on virtual machine monitoring its own traffic > Reporter: Jakub Svoboda > Labels: issue > > Problem: Bro does not see traffic correctly. > * conn.log contains invalid information, with values differing from a network monitoring probe. (The probe runs on a different computer, so there's no interference.) > * Our own scripts logging connections using the connection_state_remove event report the same bad values as conn.log. > * Specifically, all following values are off in the connection_state_remove event handler: c$orig$num_pkts c$resp$num_pkts c$conn$orig_pkts c$conn$resp_pkts c$orig$num_bytes_ip c$resp$num_bytes_ip c$conn$orig_ip_bytes c$conn$resp_ip_bytes. (Yes, we know some of the values are copied one from another in conn/main.bro.) > * Problem is the same on both tested machines. > * Problem is the same both when capturing an interface and when analyzing a pcap. > * Problem is the same on both Bro 2.1 and the current git version (as of 14.08.2013) > The problem is exactly the same on two different computers. > 1. i3, OpenSuse, public IP > 2. i3, Ubuntu 12.04 x64 in VirtualBox, host has public IP, virtual machine has private IP, Bro is on virtual machine monitoring its own traffic > Both computers use 64bit OS and Bro 2.1 compiled from source. > Notes: The chance of the network monitoring probe reporting bad values is very low as this probe is used by our department of computer science and several people checked it works correctly. > Also, the values reported by Bro are obviously wrong even without no comparison with the probe---e.g. an outgoing flow with 0 outgoing packets, 0 outgoing bytes and nonzero incoming packets/bytes. > Test 1: > Bro log files were read after bro shutdown so that there should be nothing missing. > Bro was installed on a computer with IP 147.251.13.65, monitoring its own traffic. A web browser was used to generate connections. One of the contacted addresses was 77.75.77.170. > Data from conn.log: > id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes > 147.251.13.65 44690 77.75.77.170 993 tcp 0 151 0 0 3 307 > (this was the only line with 147.251.13.65 and 77.75.77.170) > Data from the network monitoring probe: > nfdump filter: > ip 147.251.13.65 and ip 77.75.77.170 > Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows > 2013-08-14 09:54:30.355 0.054 TCP 77.75.77.170:993 -> 147.251.13.65:44690 .AP... 0 3 307 55 45481 102 1 > 2013-08-14 09:54:30.347 0.062 TCP 147.251.13.65:44690 -> 77.75.77.170:993 .AP... 0 5 348 80 44903 69 1 > Summary: total flows: 2, total bytes: 655, total packets: 8, avg bps: 84516, avg pps: 129, avg bpp: 81 > Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00 > Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460 > Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5795248.1 > Test 2: > Bro log files were read after bro shutdown so that there should be nothing missing. > Bro was installed on a computer with IP 147.251.13.65, monitoring its own traffic. A web browser was used to generate connections. One of the contacted addresses was 195.113.232.73. > Debug output of our own Bro script watching connections using connection_state_remove event handler. > The script's code for the event handler is: > event connection_state_remove(c: connection) { > NOTICE([$note=profile_creation::Test3, $msg=cat("DBG: ", c$id$orig_h, "->", c$id$resp_h, " c$orig$num_pkts=", c$orig$num_pkts, " c$resp$num_pkts=", c$resp$num_pkts, " c$conn$orig_pkts=", c$conn$orig_pkts, " c$conn$resp_pkts=", c$conn$resp_pkts, " c$orig$num_bytes_ip=", c$orig$num_bytes_ip, " c$resp$num_bytes_ip=", c$resp$num_bytes_ip, " c$conn$orig_ip_bytes=", c$conn$orig_ip_bytes, " c$conn$resp_ip_bytes=", c$conn$resp_ip_bytes, " c$id$orig_p=", c$id$orig_p, " c$id$resp_p=", c$id$resp_p )]); > } > The cut out output: > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49631/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49721/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49723/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49724/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=0 c$conn$orig_pkts=0 c$conn$resp_pkts=0 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=0 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=0 c$id$orig_p=49725/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=17 c$conn$orig_pkts=0 c$conn$resp_pkts=17 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=21613 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=21613 c$id$orig_p=49725/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=5 c$conn$orig_pkts=0 c$conn$resp_pkts=5 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=5032 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=5032 c$id$orig_p=49721/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=5 c$conn$orig_pkts=0 c$conn$resp_pkts=5 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=5056 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=5056 c$id$orig_p=49723/tcp c$id$resp_p=80/tcp > DBG: 147.251.13.65->195.113.232.73 c$orig$num_pkts=0 c$resp$num_pkts=4 c$conn$orig_pkts=0 c$conn$resp_pkts=4 c$orig$num_bytes_ip=0 c$resp$num_bytes_ip=4110 c$conn$orig_ip_bytes=0 c$conn$resp_ip_bytes=4110 c$id$orig_p=49724/tcp c$id$resp_p=80/tcp > All relevant data from conn.log: > id.orig_h id.orig_p id.resp_h id.resp_p proto orig_bytes resp_bytes orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes > 147.251.13.65 49631 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49721 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49723 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49724 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49725 195.113.232.73 80 tcp - - 0 0 0 0 > 147.251.13.65 49725 195.113.232.73 80 tcp 0 20729 0 0 17 21613 > 147.251.13.65 49721 195.113.232.73 80 tcp 0 4772 0 0 5 5032 > 147.251.13.65 49723 195.113.232.73 80 tcp 0 4796 0 0 5 5056 > 147.251.13.65 49724 195.113.232.73 80 tcp 0 3902 0 0 4 4110 > Data from the network monitoring probe: > ** nfdump -M /data/nfsen/profiles-data/live/p3006:p3005:p3004:p3002:p3003:p3001:p3000 -T -r 2013/08/14/nfcapd.201308140955 -o extended -c 500 > nfdump filter: > ip 147.251.13.65 and ip 195.113.232.73 > Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows > 2013-08-14 09:54:33.929 28.188 TCP 195.113.232.73:80 -> 147.251.13.65:49725 .AP..F 0 15 19873 0 5640 1324 1 > 2013-08-14 09:54:17.976 44.141 TCP 195.113.232.73:80 -> 147.251.13.65:49631 .AP.SF 0 6 3709 0 672 618 1 > 2013-08-14 09:54:33.958 28.159 TCP 195.113.232.73:80 -> 147.251.13.65:49723 .AP..F 0 5 5056 0 1436 1011 1 > 2013-08-14 09:54:33.948 28.169 TCP 195.113.232.73:80 -> 147.251.13.65:49721 .AP..F 0 5 5032 0 1429 1006 1 > 2013-08-14 09:54:33.950 28.167 TCP 195.113.232.73:80 -> 147.251.13.65:49724 .AP..F 0 4 4110 0 1167 1027 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49725 -> 195.113.232.73:80 .AP..F 0 13 1732 0 488 133 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49724 -> 195.113.232.73:80 .AP..F 0 6 1356 0 382 226 1 > 2013-08-14 09:54:17.972 44.145 TCP 147.251.13.65:49631 -> 195.113.232.73:80 .AP.SF 0 8 746 0 135 93 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49723 -> 195.113.232.73:80 .AP..F 0 7 1408 0 397 201 1 > 2013-08-14 09:54:33.777 28.340 TCP 147.251.13.65:49721 -> 195.113.232.73:80 .AP..F 0 5 1304 0 368 260 1 > Summary: total flows: 10, total bytes: 44326, total packets: 74, avg bps: 8032, avg pps: 1, avg bpp: 599 > Time window: 2013-08-14 09:48:47 - 2013-08-14 10:00:00 > Total flows processed: 881266, Blocks skipped: 0, Bytes read: 77680460 > Sys: 0.153s flows/second: 5723435.6 Wall: 0.152s flows/second: 5782205.9 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From robin at icir.org Mon Aug 19 12:00:21 2013 From: robin at icir.org (Robin Sommer) Date: Mon, 19 Aug 2013 12:00:21 -0700 Subject: [Bro-Dev] git.bro.org update Message-ID: <20130819190021.GS1481@icir.org> Fyi, as we're now mirroring most Bro repositories on GitHub, we've disabled gitweb at http://git.bro.org. That now redirects to GitHub. But git.bro.org will keep providing the master repositories for cloning via git://git.bro.org/ Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Mon Aug 19 12:10:38 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 19 Aug 2013 14:10:38 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1059) merge topic/bernhard/3rdparty In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1059: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > merge topic/bernhard/3rdparty > ----------------------------- > > Key: BIT-1059 > URL: https://bro-tracker.atlassian.net/browse/BIT-1059 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > > please merge topic/bernhard/3rdparty - sqlite moved there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 19 13:04:38 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 19 Aug 2013 15:04:38 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1060) topic/jsiwek/misc In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1060: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/misc > ----------------- > > Key: BIT-1060 > URL: https://bro-tracker.atlassian.net/browse/BIT-1060 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.2 > > > This branch is in {{bro}} and {{btest}} repos w/ various fixes/workarounds, probably easiest to read commit log, but here's highlight that I remember: > - Improve btest's ability to kill processes that don't terminate > - Workaround a deadlock in gperftools > - Fix a deadlock in SQLite-using threads > - Workaround a problem w/ raw input reader's exec'd child not getting an EOF on its stdin pipe > - Unit test improvements -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 19 13:04:38 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 19 Aug 2013 15:04:38 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1060) topic/jsiwek/misc In-Reply-To: References: Message-ID: Jon Siwek created BIT-1060: ------------------------------ Summary: topic/jsiwek/misc Key: BIT-1060 URL: https://bro-tracker.atlassian.net/browse/BIT-1060 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Jon Siwek Assignee: Robin Sommer Fix For: 2.2 This branch is in {{bro}} and {{btest}} repos w/ various fixes/workarounds, probably easiest to read commit log, but here's highlight that I remember: - Improve btest's ability to kill processes that don't terminate - Workaround a deadlock in gperftools - Fix a deadlock in SQLite-using threads - Workaround a problem w/ raw input reader's exec'd child not getting an EOF on its stdin pipe - Unit test improvements -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 19 14:27:38 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 19 Aug 2013 16:27:38 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1058) Memory leak in sumstats (probably) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601#comment-13601 ] Jon Siwek commented on BIT-1058: -------------------------------- I was looking at this a bit last week, but haven't got back to it yet. My current guess is it's something to do with the Trigger/StateAccess/NotifierRegistry code. The sumstats scripts have a couple places where the condition of a {{when()}} block depends on {{done_with\[uid\]}} so places that do {{++done_with\[uid\]}} trigger creation of a new value that's reported as leaked possibly because that value is being held by the trigger-notification code or because Bro terminates before that trigger-notification code has a chance to run and release resources. > Memory leak in sumstats (probably) > ---------------------------------- > > Key: BIT-1058 > URL: https://bro-tracker.atlassian.net/browse/BIT-1058 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Bernhard Amann > Priority: High > Labels: leak, sumstats > Fix For: 2.2 > > Attachments: out2.pdf > > > At the moment, the core/leaks/basic-cluster.bro always fails; the gprof output is attached. Only the master node leaks memory, the two worker nodes are fine. > From the gprof output, it looks like an increment operation is somehow triggering a memory leak. > Robin and me tried to dig through this for quite some time. From our current understanding it looks like the memory leak is (indirectly) caused by an increment operation in a function that is called by an event that is received through remoteserialization. > The closest we were able to track the leak to is line 249 of scripts/base/frameworks/sumstats/cluster.bro: > {noformat} > event SumStats::cluster_send_result(uid: string, ss_name: string, key: Key, result: Result, cleanup: bool) > { > [...] > ++done_with[uid]; > } > {noformat} > Commenting out this line "fixes" the memory leak (and probably renders the sumstat framework inoperable); however we were not able to track it further to the exact cause; replacing the increment with an equivalent done_with[uid] = done_with[uid]+1; did not solve the problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 19 15:39:38 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Mon, 19 Aug 2013 17:39:38 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1061) topic/dnthayer/cleanup4 In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1061: ---------------------------------- Summary: topic/dnthayer/cleanup4 Key: BIT-1061 URL: https://bro-tracker.atlassian.net/browse/BIT-1061 Project: Bro Issue Tracker Issue Type: Patch Components: BroControl Reporter: Daniel Thayer Fix For: 2.2 This branch contains some fixes for broctl, its tests, and documentation. It also contains some code cleanup. Here are the one-line commit messages for all commits in this branch: Documentation fixes Minor fixes for broctl tests Fix bug with usage of cmd_restart_pre method Remove unused subdirectory "spool/scripts" Remove unused imports, variables, and semicolons -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 19 15:41:38 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 19 Aug 2013 17:41:38 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1060) topic/jsiwek/misc In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1060: --------------------------------- Assignee: Bernhard Amann (was: Robin Sommer) > topic/jsiwek/misc > ----------------- > > Key: BIT-1060 > URL: https://bro-tracker.atlassian.net/browse/BIT-1060 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Bernhard Amann > Fix For: 2.2 > > > This branch is in {{bro}} and {{btest}} repos w/ various fixes/workarounds, probably easiest to read commit log, but here's highlight that I remember: > - Improve btest's ability to kill processes that don't terminate > - Workaround a deadlock in gperftools > - Fix a deadlock in SQLite-using threads > - Workaround a problem w/ raw input reader's exec'd child not getting an EOF on its stdin pipe > - Unit test improvements -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 19 15:41:39 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 19 Aug 2013 17:41:39 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1060) topic/jsiwek/misc In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602#comment-13602 ] Robin Sommer commented on BIT-1060: ----------------------------------- Bernhard, can you review the input framework changes here? I'm not familiar enough with the internals here. > topic/jsiwek/misc > ----------------- > > Key: BIT-1060 > URL: https://bro-tracker.atlassian.net/browse/BIT-1060 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Bernhard Amann > Fix For: 2.2 > > > This branch is in {{bro}} and {{btest}} repos w/ various fixes/workarounds, probably easiest to read commit log, but here's highlight that I remember: > - Improve btest's ability to kill processes that don't terminate > - Workaround a deadlock in gperftools > - Fix a deadlock in SQLite-using threads > - Workaround a problem w/ raw input reader's exec'd child not getting an EOF on its stdin pipe > - Unit test improvements -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 19 15:41:39 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Mon, 19 Aug 2013 17:41:39 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1061) topic/dnthayer/cleanup4 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1061: ------------------------------- Status: Merge Request (was: Open) > topic/dnthayer/cleanup4 > ----------------------- > > Key: BIT-1061 > URL: https://bro-tracker.atlassian.net/browse/BIT-1061 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.2 > > > This branch contains some fixes for broctl, its tests, and documentation. > It also contains some code cleanup. > Here are the one-line commit messages for all commits in this branch: > Documentation fixes > Minor fixes for broctl tests > Fix bug with usage of cmd_restart_pre method > Remove unused subdirectory "spool/scripts" > Remove unused imports, variables, and semicolons -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 19 15:44:39 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 19 Aug 2013 17:44:39 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1060) topic/jsiwek/misc In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603#comment-13603 ] Robin Sommer commented on BIT-1060: ----------------------------------- Jon, one small thing I noticed during a quick lock: it seems there are two error cases where the new mutex isn't unlocked (right after the call to pthread_mutex_lock in Raw::Execute()). > topic/jsiwek/misc > ----------------- > > Key: BIT-1060 > URL: https://bro-tracker.atlassian.net/browse/BIT-1060 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Bernhard Amann > Fix For: 2.2 > > > This branch is in {{bro}} and {{btest}} repos w/ various fixes/workarounds, probably easiest to read commit log, but here's highlight that I remember: > - Improve btest's ability to kill processes that don't terminate > - Workaround a deadlock in gperftools > - Fix a deadlock in SQLite-using threads > - Workaround a problem w/ raw input reader's exec'd child not getting an EOF on its stdin pipe > - Unit test improvements -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 19 22:08:39 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 20 Aug 2013 00:08:39 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1061) topic/dnthayer/cleanup4 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1061: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) . > topic/dnthayer/cleanup4 > ----------------------- > > Key: BIT-1061 > URL: https://bro-tracker.atlassian.net/browse/BIT-1061 > Project: Bro Issue Tracker > Issue Type: Patch > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.2 > > > This branch contains some fixes for broctl, its tests, and documentation. > It also contains some code cleanup. > Here are the one-line commit messages for all commits in this branch: > Documentation fixes > Minor fixes for broctl tests > Fix bug with usage of cmd_restart_pre method > Remove unused subdirectory "spool/scripts" > Remove unused imports, variables, and semicolons -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Mon Aug 19 22:12:39 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 20 Aug 2013 00:12:39 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1057) topic/jsiwek/bloomfilter-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1057: ------------------------------ Resolution: Merged Status: Closed (was: Open) > topic/jsiwek/bloomfilter-fix > ---------------------------- > > Key: BIT-1057 > URL: https://bro-tracker.atlassian.net/browse/BIT-1057 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Matthias Vallentin > Fix For: 2.2 > > > The change (1) I did on this branch makes the bloomfilter-seed test start passing on 32-bit systems, though I'm not confident how correct it is. Matthias, can you review it and change this to a merge request if it looks sane? > (1) https://github.com/bro/bro/commit/774dadfe9aedc9fed98df69abcc83a3068bdf06b -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Tue Aug 20 00:00:12 2013 From: noreply at bro.org (Merge Tracker) Date: Tue, 20 Aug 2013 00:00:12 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308200700.r7K70CSg025260@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- -------------- ---------- ------------- ---------- --------------------- BIT-1060 [1] Bro Jon Siwek Bernhard Amann 2013-08-19 2.2 Normal topic/jsiwek/misc [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [3] broccoli dcode [4] 2013-08-12 Updated specfile. Works under mock for EL6 [5] [1] BIT-1060 https://bro-tracker.atlassian.net/browse/BIT-1060 [2] misc https://github.com/bro/bro/tree/topic/jsiwek/misc [3] Pull Request #1 https://github.com/bro/broccoli/issues/1 [4] dcode https://github.com/dcode [5] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Tue Aug 20 07:59:38 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 20 Aug 2013 09:59:38 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1060) topic/jsiwek/misc In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13605#comment-13605 ] Jon Siwek commented on BIT-1060: -------------------------------- {quote} Jon, one small thing I noticed during a quick lock: it seems there are two error cases where the new mutex isn't unlocked {quote} Thanks, added that to the branch. > topic/jsiwek/misc > ----------------- > > Key: BIT-1060 > URL: https://bro-tracker.atlassian.net/browse/BIT-1060 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Bernhard Amann > Fix For: 2.2 > > > This branch is in {{bro}} and {{btest}} repos w/ various fixes/workarounds, probably easiest to read commit log, but here's highlight that I remember: > - Improve btest's ability to kill processes that don't terminate > - Workaround a deadlock in gperftools > - Fix a deadlock in SQLite-using threads > - Workaround a problem w/ raw input reader's exec'd child not getting an EOF on its stdin pipe > - Unit test improvements -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From robin at icir.org Tue Aug 20 08:13:20 2013 From: robin at icir.org (Robin Sommer) Date: Tue, 20 Aug 2013 08:13:20 -0700 Subject: [Bro-Dev] Extending Jenkins tests Message-ID: <20130820151320.GA43380@icir.org> Jon, I'd like to integrate the new BroControl and BTests test suites into the Jenkins runs, what's the best way to do that? Is Jenkins triggering the tests via "make test" or in some other way? Also, I'm thinking we should switch Jenkins to use btest's brief output so that it skips all the "ok" tests in its mails for brevity. Maybe be could create separate target "jenkins" for running the tests there? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jsiwek at illinois.edu Tue Aug 20 14:23:22 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Tue, 20 Aug 2013 21:23:22 +0000 Subject: [Bro-Dev] Extending Jenkins tests In-Reply-To: <20130820151320.GA43380@icir.org> References: <20130820151320.GA43380@icir.org> Message-ID: > I'd like to integrate the new BroControl and BTests test suites into > the Jenkins runs, what's the best way to do that? Since the BTest tests don't depend on Bro at all, it's probably best to set up a new job that polls the btest master branch for changes directly and then runs the test suite. For BroControl tests, I think maybe it should be in a new job that's triggered from the UpdateRepos job (alongside the Compile* jobs). > Is Jenkins triggering the tests via "make test" or in some other way? Another way. > Also, I'm thinking we should switch Jenkins to use btest's brief output so that > it skips all the "ok" tests in its mails for brevity. Makes sense. Does that still show "skipped" tests? I think that may have been a reason for the more verbose output -- I wanted to know whether tests were passing versus just being skipped. > Maybe be could create separate target "jenkins" for running the tests there? Don't think we need something like that. Did you want to play around with making changes to the Jenkins config? Else I'll try to get BroControl/BTest stuff added sometime this week. - Jon From robin at icir.org Tue Aug 20 14:29:28 2013 From: robin at icir.org (Robin Sommer) Date: Tue, 20 Aug 2013 14:29:28 -0700 Subject: [Bro-Dev] Extending Jenkins tests In-Reply-To: References: <20130820151320.GA43380@icir.org> Message-ID: <20130820212928.GB96302@icir.org> On Tue, Aug 20, 2013 at 21:23 +0000, you wrote: > Since the BTest tests don't depend on Bro at all, it's probably best > to set up a new job that polls the btest master branch for changes > directly and then runs the test suite. > > For BroControl tests, I think maybe it should be in a new job that's > triggered from the UpdateRepos job (alongside the Compile* jobs). Makes sense. > Makes sense. Does that still show "skipped" tests? Good point, I don't think so, but we can change that. > Did you want to play around with making changes to the Jenkins config? No, I was thinking if it's just the Makefile I could do it, but otherwise I prefer to leave it in your hands, I'd just mess it up. :-) So just go ahead when it works for you, not pressing. Thanks, Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From noreply at bro.org Wed Aug 21 00:00:13 2013 From: noreply at bro.org (Merge Tracker) Date: Wed, 21 Aug 2013 00:00:13 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308210700.r7L70DoE021056@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- -------------- ---------- ------------- ---------- --------------------- BIT-1060 [1] Bro Jon Siwek Bernhard Amann 2013-08-20 2.2 Normal topic/jsiwek/misc [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [3] broccoli dcode [4] 2013-08-12 Updated specfile. Works under mock for EL6 [5] [1] BIT-1060 https://bro-tracker.atlassian.net/browse/BIT-1060 [2] misc https://github.com/bro/bro/tree/topic/jsiwek/misc [3] Pull Request #1 https://github.com/bro/broccoli/issues/1 [4] dcode https://github.com/dcode [5] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Wed Aug 21 10:22:31 2013 From: jira at bro-tracker.atlassian.net (john blaze (JIRA)) Date: Wed, 21 Aug 2013 12:22:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1062) Issues fragmented packets and BRO In-Reply-To: References: Message-ID: john blaze created BIT-1062: ------------------------------- Summary: Issues fragmented packets and BRO Key: BIT-1062 URL: https://bro-tracker.atlassian.net/browse/BIT-1062 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.1 Environment: Ubuntu/Debian Reporter: john blaze Attachments: fraggy_out_EVILSTUFF, more_frag.pcap I was doing some testing with fragmented attacks trying to bypass IDS sensors and noticed that BRO does not identify/populate the SRC & DST IP's in the weird log and other fields such as the URI in the http.log when doing stuff like: >>> f=fragment(IP(dst="80.69.77.211")/ICMP()/("X"*50), fragsize=10) >>> for frag in f: ... send(frag) 1377062338.222065 - - - - - excessively_small_fragment - F bro Also,. I fragmented a GET /EVILSTUFF HTTP request,. and noticed: 1377056289.770819 - - - - - excessively_small_fragment - F bro 1377056289.787032 - - - - - fragment_inconsistency - F bro 1377056290.141267 iL6Ki3ncjV1 192.168.1.5 17384 192.168.1.16 80 unmatched_HTTP_reply - F bro PCAPS are attached. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 21 11:18:31 2013 From: jira at bro-tracker.atlassian.net (Anthony Verez (JIRA)) Date: Wed, 21 Aug 2013 13:18:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1063) Patch for documentation In-Reply-To: References: Message-ID: Anthony Verez created BIT-1063: ---------------------------------- Summary: Patch for documentation Key: BIT-1063 URL: https://bro-tracker.atlassian.net/browse/BIT-1063 Project: Bro Issue Tracker Issue Type: Patch Components: Website Affects Versions: git/master Reporter: Anthony Verez I fixed examples, a link and a typing error in the docs for the git/master version. Great docs btw ;-) Patch: diff --git a/doc/notice.rst b/doc/notice.rst index 76d5bcd..b4b375c 100644 --- a/doc/notice.rst +++ b/doc/notice.rst @@ -98,9 +98,9 @@ type :bro:see:`SSH::Password_Guessing` if the server is 10.0.0.1: .. note:: - Keep in mind that the semantics of the SSH::Password_Guessing notice are - such that it is only raised when Bro heuristically detects a failed - login. + Keep in mind that the semantics of the :bro:see:`SSH::Password_Guessing` + notice are such that it is only raised when Bro heuristically detects + a failed login. Hooks can also have priorities applied to order their execution like events with a default priority of 0. Greater values are executed first. Setting @@ -339,7 +339,7 @@ included below. hook Notice::policy(n: Notice::Info) { if ( n?$conn && n$conn?$http && n$conn$http?$host ) - n$email_body_sections[|email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); + n$email_body_sections[|n$email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); } @@ -348,7 +348,7 @@ Cluster Considerations As a user/developer of Bro, the main cluster concern with the notice framework is understanding what runs where. When a notice is generated on a worker, the -worker checks to see if the notice shoudl be suppressed based on information +worker checks to see if the notice should be suppressed based on information locally maintained in the worker process. If it's not being suppressed, the worker forwards the notice directly to the manager and does no more local processing. The manager then runs the :bro:see:`Notice::policy` hook and diff --git a/doc/quickstart.rst b/doc/quickstart.rst index 9f64e36..b5ac4ee 100644 --- a/doc/quickstart.rst +++ b/doc/quickstart.rst @@ -270,14 +270,11 @@ that only takes the email action for SSH logins to a defined set of servers: 192.168.1.102, } &redef; - redef Notice::policy += { - [$action = Notice::ACTION_EMAIL, - $pred(n: Notice::Info) = - { - return n$note == SSH::Login && n$id$resp_h in watched_servers; - } - ] - }; + hook Notice::policy(n: Notice::Info) + { + if ( n$note == SSH::SUCCESSFUL_LOGIN && n$id$resp_h in watched_servers ) + add n$actions[Notice::ACTION_EMAIL]; + } You'll just have to trust the syntax for now, but what we've done is first declare our own variable to hold a set of watched addresses, -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 21 12:36:31 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 21 Aug 2013 14:36:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1058) Memory leak in sumstats (probably) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1058: --------------------------- Assignee: Robin Sommer > Memory leak in sumstats (probably) > ---------------------------------- > > Key: BIT-1058 > URL: https://bro-tracker.atlassian.net/browse/BIT-1058 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Bernhard Amann > Assignee: Robin Sommer > Priority: High > Labels: leak, sumstats > Fix For: 2.2 > > Attachments: out2.pdf > > > At the moment, the core/leaks/basic-cluster.bro always fails; the gprof output is attached. Only the master node leaks memory, the two worker nodes are fine. > From the gprof output, it looks like an increment operation is somehow triggering a memory leak. > Robin and me tried to dig through this for quite some time. From our current understanding it looks like the memory leak is (indirectly) caused by an increment operation in a function that is called by an event that is received through remoteserialization. > The closest we were able to track the leak to is line 249 of scripts/base/frameworks/sumstats/cluster.bro: > {noformat} > event SumStats::cluster_send_result(uid: string, ss_name: string, key: Key, result: Result, cleanup: bool) > { > [...] > ++done_with[uid]; > } > {noformat} > Commenting out this line "fixes" the memory leak (and probably renders the sumstat framework inoperable); however we were not able to track it further to the exact cause; replacing the increment with an equivalent done_with[uid] = done_with[uid]+1; did not solve the problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 21 12:36:31 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 21 Aug 2013 14:36:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1058) Memory leak in sumstats (probably) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13700#comment-13700 ] Jon Siwek commented on BIT-1058: -------------------------------- Fix in {{topic/jsiwek/when-leak}}. > Memory leak in sumstats (probably) > ---------------------------------- > > Key: BIT-1058 > URL: https://bro-tracker.atlassian.net/browse/BIT-1058 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Bernhard Amann > Priority: High > Labels: leak, sumstats > Fix For: 2.2 > > Attachments: out2.pdf > > > At the moment, the core/leaks/basic-cluster.bro always fails; the gprof output is attached. Only the master node leaks memory, the two worker nodes are fine. > From the gprof output, it looks like an increment operation is somehow triggering a memory leak. > Robin and me tried to dig through this for quite some time. From our current understanding it looks like the memory leak is (indirectly) caused by an increment operation in a function that is called by an event that is received through remoteserialization. > The closest we were able to track the leak to is line 249 of scripts/base/frameworks/sumstats/cluster.bro: > {noformat} > event SumStats::cluster_send_result(uid: string, ss_name: string, key: Key, result: Result, cleanup: bool) > { > [...] > ++done_with[uid]; > } > {noformat} > Commenting out this line "fixes" the memory leak (and probably renders the sumstat framework inoperable); however we were not able to track it further to the exact cause; replacing the increment with an equivalent done_with[uid] = done_with[uid]+1; did not solve the problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 21 12:36:31 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 21 Aug 2013 14:36:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1058) Memory leak in sumstats (probably) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1058: --------------------------- Status: Merge Request (was: Open) > Memory leak in sumstats (probably) > ---------------------------------- > > Key: BIT-1058 > URL: https://bro-tracker.atlassian.net/browse/BIT-1058 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Bernhard Amann > Assignee: Robin Sommer > Priority: High > Labels: leak, sumstats > Fix For: 2.2 > > Attachments: out2.pdf > > > At the moment, the core/leaks/basic-cluster.bro always fails; the gprof output is attached. Only the master node leaks memory, the two worker nodes are fine. > From the gprof output, it looks like an increment operation is somehow triggering a memory leak. > Robin and me tried to dig through this for quite some time. From our current understanding it looks like the memory leak is (indirectly) caused by an increment operation in a function that is called by an event that is received through remoteserialization. > The closest we were able to track the leak to is line 249 of scripts/base/frameworks/sumstats/cluster.bro: > {noformat} > event SumStats::cluster_send_result(uid: string, ss_name: string, key: Key, result: Result, cleanup: bool) > { > [...] > ++done_with[uid]; > } > {noformat} > Commenting out this line "fixes" the memory leak (and probably renders the sumstat framework inoperable); however we were not able to track it further to the exact cause; replacing the increment with an equivalent done_with[uid] = done_with[uid]+1; did not solve the problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 21 12:50:31 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 21 Aug 2013 14:50:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1063) Patch for documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1063: --------------------------- Status: Merge Request (was: Open) > Patch for documentation > ----------------------- > > Key: BIT-1063 > URL: https://bro-tracker.atlassian.net/browse/BIT-1063 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Website > Affects Versions: git/master > Reporter: Anthony Verez > > I fixed examples, a link and a typing error in the docs for the git/master version. Great docs btw ;-) > Patch: > diff --git a/doc/notice.rst b/doc/notice.rst > index 76d5bcd..b4b375c 100644 > --- a/doc/notice.rst > +++ b/doc/notice.rst > @@ -98,9 +98,9 @@ type :bro:see:`SSH::Password_Guessing` if the server is 10.0.0.1: > .. note:: > - Keep in mind that the semantics of the SSH::Password_Guessing notice are > - such that it is only raised when Bro heuristically detects a failed > - login. > + Keep in mind that the semantics of the :bro:see:`SSH::Password_Guessing` > + notice are such that it is only raised when Bro heuristically detects > + a failed login. > Hooks can also have priorities applied to order their execution like events > with a default priority of 0. Greater values are executed first. Setting > @@ -339,7 +339,7 @@ included below. > hook Notice::policy(n: Notice::Info) > { > if ( n?$conn && n$conn?$http && n$conn$http?$host ) > - n$email_body_sections[|email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > + n$email_body_sections[|n$email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > } > @@ -348,7 +348,7 @@ Cluster Considerations > As a user/developer of Bro, the main cluster concern with the notice framework > is understanding what runs where. When a notice is generated on a worker, the > -worker checks to see if the notice shoudl be suppressed based on information > +worker checks to see if the notice should be suppressed based on information > locally maintained in the worker process. If it's not being > suppressed, the worker forwards the notice directly to the manager and does no more > local processing. The manager then runs the :bro:see:`Notice::policy` hook and > diff --git a/doc/quickstart.rst b/doc/quickstart.rst > index 9f64e36..b5ac4ee 100644 > --- a/doc/quickstart.rst > +++ b/doc/quickstart.rst > @@ -270,14 +270,11 @@ that only takes the email action for SSH logins to a defined set of servers: > 192.168.1.102, > } &redef; > - redef Notice::policy += { > - [$action = Notice::ACTION_EMAIL, > - $pred(n: Notice::Info) = > - { > - return n$note == SSH::Login && n$id$resp_h in watched_servers; > - } > - ] > - }; > + hook Notice::policy(n: Notice::Info) > + { > + if ( n$note == SSH::SUCCESSFUL_LOGIN && n$id$resp_h in watched_servers ) > + add n$actions[Notice::ACTION_EMAIL]; > + } > You'll just have to trust the syntax for now, but what we've done is > first declare our own variable to hold a set of watched addresses, -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 21 13:18:31 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 21 Aug 2013 15:18:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1063) Patch for documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701#comment-13701 ] Robin Sommer commented on BIT-1063: ----------------------------------- Please attach the patch as a separate file. > Patch for documentation > ----------------------- > > Key: BIT-1063 > URL: https://bro-tracker.atlassian.net/browse/BIT-1063 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Website > Affects Versions: git/master > Reporter: Anthony Verez > > I fixed examples, a link and a typing error in the docs for the git/master version. Great docs btw ;-) > Patch: > diff --git a/doc/notice.rst b/doc/notice.rst > index 76d5bcd..b4b375c 100644 > --- a/doc/notice.rst > +++ b/doc/notice.rst > @@ -98,9 +98,9 @@ type :bro:see:`SSH::Password_Guessing` if the server is 10.0.0.1: > .. note:: > - Keep in mind that the semantics of the SSH::Password_Guessing notice are > - such that it is only raised when Bro heuristically detects a failed > - login. > + Keep in mind that the semantics of the :bro:see:`SSH::Password_Guessing` > + notice are such that it is only raised when Bro heuristically detects > + a failed login. > Hooks can also have priorities applied to order their execution like events > with a default priority of 0. Greater values are executed first. Setting > @@ -339,7 +339,7 @@ included below. > hook Notice::policy(n: Notice::Info) > { > if ( n?$conn && n$conn?$http && n$conn$http?$host ) > - n$email_body_sections[|email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > + n$email_body_sections[|n$email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > } > @@ -348,7 +348,7 @@ Cluster Considerations > As a user/developer of Bro, the main cluster concern with the notice framework > is understanding what runs where. When a notice is generated on a worker, the > -worker checks to see if the notice shoudl be suppressed based on information > +worker checks to see if the notice should be suppressed based on information > locally maintained in the worker process. If it's not being > suppressed, the worker forwards the notice directly to the manager and does no more > local processing. The manager then runs the :bro:see:`Notice::policy` hook and > diff --git a/doc/quickstart.rst b/doc/quickstart.rst > index 9f64e36..b5ac4ee 100644 > --- a/doc/quickstart.rst > +++ b/doc/quickstart.rst > @@ -270,14 +270,11 @@ that only takes the email action for SSH logins to a defined set of servers: > 192.168.1.102, > } &redef; > - redef Notice::policy += { > - [$action = Notice::ACTION_EMAIL, > - $pred(n: Notice::Info) = > - { > - return n$note == SSH::Login && n$id$resp_h in watched_servers; > - } > - ] > - }; > + hook Notice::policy(n: Notice::Info) > + { > + if ( n$note == SSH::SUCCESSFUL_LOGIN && n$id$resp_h in watched_servers ) > + add n$actions[Notice::ACTION_EMAIL]; > + } > You'll just have to trust the syntax for now, but what we've done is > first declare our own variable to hold a set of watched addresses, -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 21 14:48:31 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 21 Aug 2013 16:48:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1058) Memory leak in sumstats (probably) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1058: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Memory leak in sumstats (probably) > ---------------------------------- > > Key: BIT-1058 > URL: https://bro-tracker.atlassian.net/browse/BIT-1058 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Bernhard Amann > Assignee: Robin Sommer > Priority: High > Labels: leak, sumstats > Fix For: 2.2 > > Attachments: out2.pdf > > > At the moment, the core/leaks/basic-cluster.bro always fails; the gprof output is attached. Only the master node leaks memory, the two worker nodes are fine. > From the gprof output, it looks like an increment operation is somehow triggering a memory leak. > Robin and me tried to dig through this for quite some time. From our current understanding it looks like the memory leak is (indirectly) caused by an increment operation in a function that is called by an event that is received through remoteserialization. > The closest we were able to track the leak to is line 249 of scripts/base/frameworks/sumstats/cluster.bro: > {noformat} > event SumStats::cluster_send_result(uid: string, ss_name: string, key: Key, result: Result, cleanup: bool) > { > [...] > ++done_with[uid]; > } > {noformat} > Commenting out this line "fixes" the memory leak (and probably renders the sumstat framework inoperable); however we were not able to track it further to the exact cause; replacing the increment with an equivalent done_with[uid] = done_with[uid]+1; did not solve the problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From robin at icir.org Wed Aug 21 15:18:26 2013 From: robin at icir.org (Robin Sommer) Date: Wed, 21 Aug 2013 15:18:26 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/when-leak: Fix memory leak w/ when statements - BIT-1058 (8432f05) In-Reply-To: <201308211935.r7LJZdVM003587@bro-ids.icir.org> References: <201308211935.r7LJZdVM003587@bro-ids.icir.org> Message-ID: <20130821221826.GA62664@icir.org> On Wed, Aug 21, 2013 at 12:35 -0700, Jonathan Siwek wrote: > Fix memory leak w/ when statements - BIT-1058 Very cool, thanks a lot for tracking that down! Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Wed Aug 21 17:02:31 2013 From: jira at bro-tracker.atlassian.net (Anthony Verez (JIRA)) Date: Wed, 21 Aug 2013 19:02:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1063) Patch for documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702#comment-13702 ] Anthony Verez edited comment on BIT-1063 at 8/21/13 7:02 PM: ------------------------------------------------------------- Sure, here is the patch. was (Author: netantho): Patch > Patch for documentation > ----------------------- > > Key: BIT-1063 > URL: https://bro-tracker.atlassian.net/browse/BIT-1063 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Website > Affects Versions: git/master > Reporter: Anthony Verez > Attachments: 0001-Docs-fix.patch > > > I fixed examples, a link and a typing error in the docs for the git/master version. Great docs btw ;-) > Patch: > diff --git a/doc/notice.rst b/doc/notice.rst > index 76d5bcd..b4b375c 100644 > --- a/doc/notice.rst > +++ b/doc/notice.rst > @@ -98,9 +98,9 @@ type :bro:see:`SSH::Password_Guessing` if the server is 10.0.0.1: > .. note:: > - Keep in mind that the semantics of the SSH::Password_Guessing notice are > - such that it is only raised when Bro heuristically detects a failed > - login. > + Keep in mind that the semantics of the :bro:see:`SSH::Password_Guessing` > + notice are such that it is only raised when Bro heuristically detects > + a failed login. > Hooks can also have priorities applied to order their execution like events > with a default priority of 0. Greater values are executed first. Setting > @@ -339,7 +339,7 @@ included below. > hook Notice::policy(n: Notice::Info) > { > if ( n?$conn && n$conn?$http && n$conn$http?$host ) > - n$email_body_sections[|email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > + n$email_body_sections[|n$email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > } > @@ -348,7 +348,7 @@ Cluster Considerations > As a user/developer of Bro, the main cluster concern with the notice framework > is understanding what runs where. When a notice is generated on a worker, the > -worker checks to see if the notice shoudl be suppressed based on information > +worker checks to see if the notice should be suppressed based on information > locally maintained in the worker process. If it's not being > suppressed, the worker forwards the notice directly to the manager and does no more > local processing. The manager then runs the :bro:see:`Notice::policy` hook and > diff --git a/doc/quickstart.rst b/doc/quickstart.rst > index 9f64e36..b5ac4ee 100644 > --- a/doc/quickstart.rst > +++ b/doc/quickstart.rst > @@ -270,14 +270,11 @@ that only takes the email action for SSH logins to a defined set of servers: > 192.168.1.102, > } &redef; > - redef Notice::policy += { > - [$action = Notice::ACTION_EMAIL, > - $pred(n: Notice::Info) = > - { > - return n$note == SSH::Login && n$id$resp_h in watched_servers; > - } > - ] > - }; > + hook Notice::policy(n: Notice::Info) > + { > + if ( n$note == SSH::SUCCESSFUL_LOGIN && n$id$resp_h in watched_servers ) > + add n$actions[Notice::ACTION_EMAIL]; > + } > You'll just have to trust the syntax for now, but what we've done is > first declare our own variable to hold a set of watched addresses, -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Wed Aug 21 17:02:31 2013 From: jira at bro-tracker.atlassian.net (Anthony Verez (JIRA)) Date: Wed, 21 Aug 2013 19:02:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1063) Patch for documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Verez updated BIT-1063: ------------------------------- Attachment: 0001-Docs-fix.patch Patch > Patch for documentation > ----------------------- > > Key: BIT-1063 > URL: https://bro-tracker.atlassian.net/browse/BIT-1063 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Website > Affects Versions: git/master > Reporter: Anthony Verez > Attachments: 0001-Docs-fix.patch > > > I fixed examples, a link and a typing error in the docs for the git/master version. Great docs btw ;-) > Patch: > diff --git a/doc/notice.rst b/doc/notice.rst > index 76d5bcd..b4b375c 100644 > --- a/doc/notice.rst > +++ b/doc/notice.rst > @@ -98,9 +98,9 @@ type :bro:see:`SSH::Password_Guessing` if the server is 10.0.0.1: > .. note:: > - Keep in mind that the semantics of the SSH::Password_Guessing notice are > - such that it is only raised when Bro heuristically detects a failed > - login. > + Keep in mind that the semantics of the :bro:see:`SSH::Password_Guessing` > + notice are such that it is only raised when Bro heuristically detects > + a failed login. > Hooks can also have priorities applied to order their execution like events > with a default priority of 0. Greater values are executed first. Setting > @@ -339,7 +339,7 @@ included below. > hook Notice::policy(n: Notice::Info) > { > if ( n?$conn && n$conn?$http && n$conn$http?$host ) > - n$email_body_sections[|email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > + n$email_body_sections[|n$email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > } > @@ -348,7 +348,7 @@ Cluster Considerations > As a user/developer of Bro, the main cluster concern with the notice framework > is understanding what runs where. When a notice is generated on a worker, the > -worker checks to see if the notice shoudl be suppressed based on information > +worker checks to see if the notice should be suppressed based on information > locally maintained in the worker process. If it's not being > suppressed, the worker forwards the notice directly to the manager and does no more > local processing. The manager then runs the :bro:see:`Notice::policy` hook and > diff --git a/doc/quickstart.rst b/doc/quickstart.rst > index 9f64e36..b5ac4ee 100644 > --- a/doc/quickstart.rst > +++ b/doc/quickstart.rst > @@ -270,14 +270,11 @@ that only takes the email action for SSH logins to a defined set of servers: > 192.168.1.102, > } &redef; > - redef Notice::policy += { > - [$action = Notice::ACTION_EMAIL, > - $pred(n: Notice::Info) = > - { > - return n$note == SSH::Login && n$id$resp_h in watched_servers; > - } > - ] > - }; > + hook Notice::policy(n: Notice::Info) > + { > + if ( n$note == SSH::SUCCESSFUL_LOGIN && n$id$resp_h in watched_servers ) > + add n$actions[Notice::ACTION_EMAIL]; > + } > You'll just have to trust the syntax for now, but what we've done is > first declare our own variable to hold a set of watched addresses, -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Thu Aug 22 00:00:12 2013 From: noreply at bro.org (Merge Tracker) Date: Thu, 22 Aug 2013 00:00:12 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308220700.r7M70C1f003598@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- -------------- ---------- ------------- ---------- ----------------------- BIT-1063 [1] Website Anthony Verez - 2013-08-21 - Normal Patch for documentation BIT-1060 [2] Bro Jon Siwek Bernhard Amann 2013-08-20 2.2 Normal topic/jsiwek/misc [3] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [4] broccoli dcode [5] 2013-08-12 Updated specfile. Works under mock for EL6 [6] [1] BIT-1063 https://bro-tracker.atlassian.net/browse/BIT-1063 [2] BIT-1060 https://bro-tracker.atlassian.net/browse/BIT-1060 [3] misc https://github.com/bro/bro/tree/topic/jsiwek/misc [4] Pull Request #1 https://github.com/bro/broccoli/issues/1 [5] dcode https://github.com/dcode [6] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Thu Aug 22 08:08:31 2013 From: jira at bro-tracker.atlassian.net (Pietro Delsante (JIRA)) Date: Thu, 22 Aug 2013 10:08:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers In-Reply-To: References: Message-ID: Pietro Delsante created BIT-1064: ------------------------------------ Summary: DNS Analyzer does not correctly log NXDOMAIN answers Key: BIT-1064 URL: https://bro-tracker.atlassian.net/browse/BIT-1064 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.1 Environment: Bro 2.1 running on SecurityOnion 12.04-2 Reporter: Pietro Delsante Attachments: nxdomain_pcap.png Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest available packages. It looks like Bro's DNS analyzer is not assigning the correct rcode and rcode_name in the output log when the query is of type A and the server answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both fields, like this: {noformat} 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|- {noformat} that is, exploded: {noformat} ts: 1377179281.104465 uid: prGZzGRr1M4 id: 192.168.X.Y|45406|8.8.8.8|53 proto: udp trans_id: 64928 query: www.this-domain-does-not-exist.it qclass: 1 qclass_name: C_INTERNET qtype: 1 qtype_name: A rcode: - rcode_name: - AA: F TC: F RD: T RA: F Z: 0 answers: - TTLs: - {noformat} The only case in which I see those values set correctly (rcode: 3, rcode_name: NXDOMAIN) is when Bro is logging a PTR query: {noformat} 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|- {noformat} The attachment is a screenshot from a wireshark capture of the DNS query showing that the server is actually answering with NXDOMAIN. The only change I made to the default configuration was to enable the extraction of executable files from HTTP and SMTP fluxes, so this should have nothing to do with this issue. Should you need any more info about my setup, please let me know. Thanks, Pietro -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 08:42:31 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 22 Aug 2013 10:42:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13703#comment-13703 ] Seth Hall commented on BIT-1064: -------------------------------- It seems to be working correctly for me. Could you send a packet capture that exhibits the problem? > DNS Analyzer does not correctly log NXDOMAIN answers > ---------------------------------------------------- > > Key: BIT-1064 > URL: https://bro-tracker.atlassian.net/browse/BIT-1064 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.1 > Environment: Bro 2.1 running on SecurityOnion 12.04-2 > Reporter: Pietro Delsante > Labels: dns, nxdomain > Attachments: nxdomain_pcap.png > > > Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest available packages. > It looks like Bro's DNS analyzer is not assigning the correct rcode and rcode_name in the output log when the query is of type A and the server answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both fields, like this: > {noformat} > 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|- > {noformat} > that is, exploded: > {noformat} > ts: 1377179281.104465 > uid: prGZzGRr1M4 > id: 192.168.X.Y|45406|8.8.8.8|53 > proto: udp > trans_id: 64928 > query: www.this-domain-does-not-exist.it > qclass: 1 > qclass_name: C_INTERNET > qtype: 1 > qtype_name: A > rcode: - > rcode_name: - > AA: F > TC: F > RD: T > RA: F > Z: 0 > answers: - > TTLs: - > {noformat} > The only case in which I see those values set correctly (rcode: 3, rcode_name: NXDOMAIN) is when Bro is logging a PTR query: > {noformat} > 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|- > {noformat} > The attachment is a screenshot from a wireshark capture of the DNS query showing that the server is actually answering with NXDOMAIN. > The only change I made to the default configuration was to enable the extraction of executable files from HTTP and SMTP fluxes, so this should have nothing to do with this issue. > Should you need any more info about my setup, please let me know. > Thanks, > Pietro -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 08:46:31 2013 From: jira at bro-tracker.atlassian.net (Pietro Delsante (JIRA)) Date: Thu, 22 Aug 2013 10:46:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro Delsante updated BIT-1064: --------------------------------- Attachment: nxdomain.pcap PCAP file containing a request and response of a nonexistent domain, the server is answering with RCODE=3 (NXDOMAIN). This happens both with my internal DNS server and with Google's 8.8.8.8. > DNS Analyzer does not correctly log NXDOMAIN answers > ---------------------------------------------------- > > Key: BIT-1064 > URL: https://bro-tracker.atlassian.net/browse/BIT-1064 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.1 > Environment: Bro 2.1 running on SecurityOnion 12.04-2 > Reporter: Pietro Delsante > Labels: dns, nxdomain > Attachments: nxdomain.pcap, nxdomain_pcap.png > > > Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest available packages. > It looks like Bro's DNS analyzer is not assigning the correct rcode and rcode_name in the output log when the query is of type A and the server answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both fields, like this: > {noformat} > 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|- > {noformat} > that is, exploded: > {noformat} > ts: 1377179281.104465 > uid: prGZzGRr1M4 > id: 192.168.X.Y|45406|8.8.8.8|53 > proto: udp > trans_id: 64928 > query: www.this-domain-does-not-exist.it > qclass: 1 > qclass_name: C_INTERNET > qtype: 1 > qtype_name: A > rcode: - > rcode_name: - > AA: F > TC: F > RD: T > RA: F > Z: 0 > answers: - > TTLs: - > {noformat} > The only case in which I see those values set correctly (rcode: 3, rcode_name: NXDOMAIN) is when Bro is logging a PTR query: > {noformat} > 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|- > {noformat} > The attachment is a screenshot from a wireshark capture of the DNS query showing that the server is actually answering with NXDOMAIN. > The only change I made to the default configuration was to enable the extraction of executable files from HTTP and SMTP fluxes, so this should have nothing to do with this issue. > Should you need any more info about my setup, please let me know. > Thanks, > Pietro -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 08:48:31 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 22 Aug 2013 10:48:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1060) topic/jsiwek/misc In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13706#comment-13706 ] Robin Sommer commented on BIT-1060: ----------------------------------- I'm going ahead with the merge but Bernhard, please still take a look when you get a chance. > topic/jsiwek/misc > ----------------- > > Key: BIT-1060 > URL: https://bro-tracker.atlassian.net/browse/BIT-1060 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Bernhard Amann > Fix For: 2.2 > > > This branch is in {{bro}} and {{btest}} repos w/ various fixes/workarounds, probably easiest to read commit log, but here's highlight that I remember: > - Improve btest's ability to kill processes that don't terminate > - Workaround a deadlock in gperftools > - Fix a deadlock in SQLite-using threads > - Workaround a problem w/ raw input reader's exec'd child not getting an EOF on its stdin pipe > - Unit test improvements -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 08:48:31 2013 From: jira at bro-tracker.atlassian.net (Pietro Delsante (JIRA)) Date: Thu, 22 Aug 2013 10:48:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13705#comment-13705 ] Pietro Delsante commented on BIT-1064: -------------------------------------- Hi Seth, there you go! Hope this helps... should you need any other info, I'm here! > DNS Analyzer does not correctly log NXDOMAIN answers > ---------------------------------------------------- > > Key: BIT-1064 > URL: https://bro-tracker.atlassian.net/browse/BIT-1064 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.1 > Environment: Bro 2.1 running on SecurityOnion 12.04-2 > Reporter: Pietro Delsante > Labels: dns, nxdomain > Attachments: nxdomain.pcap, nxdomain_pcap.png > > > Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest available packages. > It looks like Bro's DNS analyzer is not assigning the correct rcode and rcode_name in the output log when the query is of type A and the server answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both fields, like this: > {noformat} > 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|- > {noformat} > that is, exploded: > {noformat} > ts: 1377179281.104465 > uid: prGZzGRr1M4 > id: 192.168.X.Y|45406|8.8.8.8|53 > proto: udp > trans_id: 64928 > query: www.this-domain-does-not-exist.it > qclass: 1 > qclass_name: C_INTERNET > qtype: 1 > qtype_name: A > rcode: - > rcode_name: - > AA: F > TC: F > RD: T > RA: F > Z: 0 > answers: - > TTLs: - > {noformat} > The only case in which I see those values set correctly (rcode: 3, rcode_name: NXDOMAIN) is when Bro is logging a PTR query: > {noformat} > 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|- > {noformat} > The attachment is a screenshot from a wireshark capture of the DNS query showing that the server is actually answering with NXDOMAIN. > The only change I made to the default configuration was to enable the extraction of executable files from HTTP and SMTP fluxes, so this should have nothing to do with this issue. > Should you need any more info about my setup, please let me know. > Thanks, > Pietro -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 08:48:31 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 22 Aug 2013 10:48:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1060) topic/jsiwek/misc In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1060: ------------------------------ Status: Open (was: Merge Request) > topic/jsiwek/misc > ----------------- > > Key: BIT-1060 > URL: https://bro-tracker.atlassian.net/browse/BIT-1060 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Bernhard Amann > Fix For: 2.2 > > > This branch is in {{bro}} and {{btest}} repos w/ various fixes/workarounds, probably easiest to read commit log, but here's highlight that I remember: > - Improve btest's ability to kill processes that don't terminate > - Workaround a deadlock in gperftools > - Fix a deadlock in SQLite-using threads > - Workaround a problem w/ raw input reader's exec'd child not getting an EOF on its stdin pipe > - Unit test improvements -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 09:00:31 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 22 Aug 2013 11:00:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1063) Patch for documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1063: ------------------------------ Status: Closed (was: Merge Request) > Patch for documentation > ----------------------- > > Key: BIT-1063 > URL: https://bro-tracker.atlassian.net/browse/BIT-1063 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Website > Affects Versions: git/master > Reporter: Anthony Verez > Attachments: 0001-Docs-fix.patch > > > I fixed examples, a link and a typing error in the docs for the git/master version. Great docs btw ;-) > Patch: > diff --git a/doc/notice.rst b/doc/notice.rst > index 76d5bcd..b4b375c 100644 > --- a/doc/notice.rst > +++ b/doc/notice.rst > @@ -98,9 +98,9 @@ type :bro:see:`SSH::Password_Guessing` if the server is 10.0.0.1: > .. note:: > - Keep in mind that the semantics of the SSH::Password_Guessing notice are > - such that it is only raised when Bro heuristically detects a failed > - login. > + Keep in mind that the semantics of the :bro:see:`SSH::Password_Guessing` > + notice are such that it is only raised when Bro heuristically detects > + a failed login. > Hooks can also have priorities applied to order their execution like events > with a default priority of 0. Greater values are executed first. Setting > @@ -339,7 +339,7 @@ included below. > hook Notice::policy(n: Notice::Info) > { > if ( n?$conn && n$conn?$http && n$conn$http?$host ) > - n$email_body_sections[|email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > + n$email_body_sections[|n$email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > } > @@ -348,7 +348,7 @@ Cluster Considerations > As a user/developer of Bro, the main cluster concern with the notice framework > is understanding what runs where. When a notice is generated on a worker, the > -worker checks to see if the notice shoudl be suppressed based on information > +worker checks to see if the notice should be suppressed based on information > locally maintained in the worker process. If it's not being > suppressed, the worker forwards the notice directly to the manager and does no more > local processing. The manager then runs the :bro:see:`Notice::policy` hook and > diff --git a/doc/quickstart.rst b/doc/quickstart.rst > index 9f64e36..b5ac4ee 100644 > --- a/doc/quickstart.rst > +++ b/doc/quickstart.rst > @@ -270,14 +270,11 @@ that only takes the email action for SSH logins to a defined set of servers: > 192.168.1.102, > } &redef; > - redef Notice::policy += { > - [$action = Notice::ACTION_EMAIL, > - $pred(n: Notice::Info) = > - { > - return n$note == SSH::Login && n$id$resp_h in watched_servers; > - } > - ] > - }; > + hook Notice::policy(n: Notice::Info) > + { > + if ( n$note == SSH::SUCCESSFUL_LOGIN && n$id$resp_h in watched_servers ) > + add n$actions[Notice::ACTION_EMAIL]; > + } > You'll just have to trust the syntax for now, but what we've done is > first declare our own variable to hold a set of watched addresses, -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 09:00:31 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 22 Aug 2013 11:00:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1063) Patch for documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13707#comment-13707 ] Robin Sommer commented on BIT-1063: ----------------------------------- Thanks. I've applied this to a separate documentation branch that's currently in progress, it will show in master once that's merged (should be soon). > Patch for documentation > ----------------------- > > Key: BIT-1063 > URL: https://bro-tracker.atlassian.net/browse/BIT-1063 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Website > Affects Versions: git/master > Reporter: Anthony Verez > Attachments: 0001-Docs-fix.patch > > > I fixed examples, a link and a typing error in the docs for the git/master version. Great docs btw ;-) > Patch: > diff --git a/doc/notice.rst b/doc/notice.rst > index 76d5bcd..b4b375c 100644 > --- a/doc/notice.rst > +++ b/doc/notice.rst > @@ -98,9 +98,9 @@ type :bro:see:`SSH::Password_Guessing` if the server is 10.0.0.1: > .. note:: > - Keep in mind that the semantics of the SSH::Password_Guessing notice are > - such that it is only raised when Bro heuristically detects a failed > - login. > + Keep in mind that the semantics of the :bro:see:`SSH::Password_Guessing` > + notice are such that it is only raised when Bro heuristically detects > + a failed login. > Hooks can also have priorities applied to order their execution like events > with a default priority of 0. Greater values are executed first. Setting > @@ -339,7 +339,7 @@ included below. > hook Notice::policy(n: Notice::Info) > { > if ( n?$conn && n$conn?$http && n$conn$http?$host ) > - n$email_body_sections[|email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > + n$email_body_sections[|n$email_body_sections|] = fmt("HTTP host header: %s", n$conn$http$host); > } > @@ -348,7 +348,7 @@ Cluster Considerations > As a user/developer of Bro, the main cluster concern with the notice framework > is understanding what runs where. When a notice is generated on a worker, the > -worker checks to see if the notice shoudl be suppressed based on information > +worker checks to see if the notice should be suppressed based on information > locally maintained in the worker process. If it's not being > suppressed, the worker forwards the notice directly to the manager and does no more > local processing. The manager then runs the :bro:see:`Notice::policy` hook and > diff --git a/doc/quickstart.rst b/doc/quickstart.rst > index 9f64e36..b5ac4ee 100644 > --- a/doc/quickstart.rst > +++ b/doc/quickstart.rst > @@ -270,14 +270,11 @@ that only takes the email action for SSH logins to a defined set of servers: > 192.168.1.102, > } &redef; > - redef Notice::policy += { > - [$action = Notice::ACTION_EMAIL, > - $pred(n: Notice::Info) = > - { > - return n$note == SSH::Login && n$id$resp_h in watched_servers; > - } > - ] > - }; > + hook Notice::policy(n: Notice::Info) > + { > + if ( n$note == SSH::SUCCESSFUL_LOGIN && n$id$resp_h in watched_servers ) > + add n$actions[Notice::ACTION_EMAIL]; > + } > You'll just have to trust the syntax for now, but what we've done is > first declare our own variable to hold a set of watched addresses, -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 09:06:31 2013 From: jira at bro-tracker.atlassian.net (Pietro Delsante (JIRA)) Date: Thu, 22 Aug 2013 11:06:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13708#comment-13708 ] Pietro Delsante edited comment on BIT-1064 at 8/22/13 11:05 AM: ---------------------------------------------------------------- By the way, the logs I pasted above are extracted from ELSA; however, things don't change if I read bro's logs directly: {noformat} root at myhost:/nsm/bro/logs# for f in $(find . -type f -name "dns.*"); do zcat $f | grep "www.this-domain-does-not-exist.it"; done 1377179281.104465 prGZzGRr1M4 192.168.X.Y 45406 8.8.8.8 53 udp 64928 www.this-domain-does-not-exist.it 1 C_INTERNET 1 A - - F F T F 0 - - 1377179281.146009 Xd0O3YLn3ch 192.168.X.Y 44310 8.8.8.8 53 udp 52665 www.this-domain-does-not-exist.it.XXX.dom 1 C_INTERNET 1 A - - F F T F 0 - - {noformat} was (Author: pdelsante): By the way, the logs I pasted above are extracted from ELSA; however, things don't change if I read bro's logs directly: {noformat} root at myhost:/nsm/bro/logs# for f in $(find . -type f -name "dns.*"); do zcat $f | grep "www.this-domain-does-not-exist.it"; done 1377179281.104465 prGZzGRr1M4 192.168.X.Y 45406 8.8.8.8 53 udp 64928 www.this-domain-does-not-exist.it 1 C_INTERNET 1 A - - F F T F 0- - 1377179281.146009 Xd0O3YLn3ch 192.168.X.Y 44310 8.8.8.8 53 udp 52665 www.this-domain-does-not-exist.it.XXX.dom 1 C_INTERNET 1 A - - F F T F0 - - {noformat} > DNS Analyzer does not correctly log NXDOMAIN answers > ---------------------------------------------------- > > Key: BIT-1064 > URL: https://bro-tracker.atlassian.net/browse/BIT-1064 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.1 > Environment: Bro 2.1 running on SecurityOnion 12.04-2 > Reporter: Pietro Delsante > Labels: dns, nxdomain > Attachments: nxdomain.pcap, nxdomain_pcap.png > > > Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest available packages. > It looks like Bro's DNS analyzer is not assigning the correct rcode and rcode_name in the output log when the query is of type A and the server answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both fields, like this: > {noformat} > 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|- > {noformat} > that is, exploded: > {noformat} > ts: 1377179281.104465 > uid: prGZzGRr1M4 > id: 192.168.X.Y|45406|8.8.8.8|53 > proto: udp > trans_id: 64928 > query: www.this-domain-does-not-exist.it > qclass: 1 > qclass_name: C_INTERNET > qtype: 1 > qtype_name: A > rcode: - > rcode_name: - > AA: F > TC: F > RD: T > RA: F > Z: 0 > answers: - > TTLs: - > {noformat} > The only case in which I see those values set correctly (rcode: 3, rcode_name: NXDOMAIN) is when Bro is logging a PTR query: > {noformat} > 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|- > {noformat} > The attachment is a screenshot from a wireshark capture of the DNS query showing that the server is actually answering with NXDOMAIN. > The only change I made to the default configuration was to enable the extraction of executable files from HTTP and SMTP fluxes, so this should have nothing to do with this issue. > Should you need any more info about my setup, please let me know. > Thanks, > Pietro -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 09:06:31 2013 From: jira at bro-tracker.atlassian.net (Pietro Delsante (JIRA)) Date: Thu, 22 Aug 2013 11:06:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13708#comment-13708 ] Pietro Delsante edited comment on BIT-1064 at 8/22/13 11:06 AM: ---------------------------------------------------------------- By the way, the logs I pasted above are extracted from ELSA; however, things don't change if I read bro's logs directly: {noformat} root at myhost:/nsm/bro/logs# for f in $(find . -type f -name "dns.*"); do zcat $f | grep "www.this-domain-does-not-exist.it"; done 1377179281.104465 prGZzGRr1M4 192.168.X.Y 45406 8.8.8.8 53 udp 64928 www.this-domain-does-not-exist.it 1 C_INTERNET 1 A - - F F T F 0 - - 1377179281.146009 Xd0O3YLn3ch 192.168.X.Y 44310 8.8.8.8 53 udp 52665 www.this-domain-does-not-exist.it.XXX.dom 1 C_INTERNET 1 A - - F F T F 0 - - {noformat} was (Author: pdelsante): By the way, the logs I pasted above are extracted from ELSA; however, things don't change if I read bro's logs directly: {noformat} root at myhost:/nsm/bro/logs# for f in $(find . -type f -name "dns.*"); do zcat $f | grep "www.this-domain-does-not-exist.it"; done 1377179281.104465 prGZzGRr1M4 192.168.X.Y 45406 8.8.8.8 53 udp 64928 www.this-domain-does-not-exist.it 1 C_INTERNET 1 A - - F F T F 0 - - 1377179281.146009 Xd0O3YLn3ch 192.168.X.Y 44310 8.8.8.8 53 udp 52665 www.this-domain-does-not-exist.it.XXX.dom 1 C_INTERNET 1 A - - F F T F 0 - - {noformat} > DNS Analyzer does not correctly log NXDOMAIN answers > ---------------------------------------------------- > > Key: BIT-1064 > URL: https://bro-tracker.atlassian.net/browse/BIT-1064 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.1 > Environment: Bro 2.1 running on SecurityOnion 12.04-2 > Reporter: Pietro Delsante > Labels: dns, nxdomain > Attachments: nxdomain.pcap, nxdomain_pcap.png > > > Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest available packages. > It looks like Bro's DNS analyzer is not assigning the correct rcode and rcode_name in the output log when the query is of type A and the server answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both fields, like this: > {noformat} > 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|- > {noformat} > that is, exploded: > {noformat} > ts: 1377179281.104465 > uid: prGZzGRr1M4 > id: 192.168.X.Y|45406|8.8.8.8|53 > proto: udp > trans_id: 64928 > query: www.this-domain-does-not-exist.it > qclass: 1 > qclass_name: C_INTERNET > qtype: 1 > qtype_name: A > rcode: - > rcode_name: - > AA: F > TC: F > RD: T > RA: F > Z: 0 > answers: - > TTLs: - > {noformat} > The only case in which I see those values set correctly (rcode: 3, rcode_name: NXDOMAIN) is when Bro is logging a PTR query: > {noformat} > 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|- > {noformat} > The attachment is a screenshot from a wireshark capture of the DNS query showing that the server is actually answering with NXDOMAIN. > The only change I made to the default configuration was to enable the extraction of executable files from HTTP and SMTP fluxes, so this should have nothing to do with this issue. > Should you need any more info about my setup, please let me know. > Thanks, > Pietro -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 09:06:31 2013 From: jira at bro-tracker.atlassian.net (Pietro Delsante (JIRA)) Date: Thu, 22 Aug 2013 11:06:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13708#comment-13708 ] Pietro Delsante commented on BIT-1064: -------------------------------------- By the way, the logs I pasted above are extracted from ELSA; however, things don't change if I read bro's logs directly: {noformat} root at myhost:/nsm/bro/logs# for f in $(find . -type f -name "dns.*"); do zcat $f | grep "www.this-domain-does-not-exist.it"; done 1377179281.104465 prGZzGRr1M4 192.168.X.Y 45406 8.8.8.8 53 udp 64928 www.this-domain-does-not-exist.it 1 C_INTERNET 1 A - - F F T F 0- - 1377179281.146009 Xd0O3YLn3ch 192.168.X.Y 44310 8.8.8.8 53 udp 52665 www.this-domain-does-not-exist.it.XXX.dom 1 C_INTERNET 1 A - - F F T F0 - - {noformat} > DNS Analyzer does not correctly log NXDOMAIN answers > ---------------------------------------------------- > > Key: BIT-1064 > URL: https://bro-tracker.atlassian.net/browse/BIT-1064 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.1 > Environment: Bro 2.1 running on SecurityOnion 12.04-2 > Reporter: Pietro Delsante > Labels: dns, nxdomain > Attachments: nxdomain.pcap, nxdomain_pcap.png > > > Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest available packages. > It looks like Bro's DNS analyzer is not assigning the correct rcode and rcode_name in the output log when the query is of type A and the server answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both fields, like this: > {noformat} > 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|- > {noformat} > that is, exploded: > {noformat} > ts: 1377179281.104465 > uid: prGZzGRr1M4 > id: 192.168.X.Y|45406|8.8.8.8|53 > proto: udp > trans_id: 64928 > query: www.this-domain-does-not-exist.it > qclass: 1 > qclass_name: C_INTERNET > qtype: 1 > qtype_name: A > rcode: - > rcode_name: - > AA: F > TC: F > RD: T > RA: F > Z: 0 > answers: - > TTLs: - > {noformat} > The only case in which I see those values set correctly (rcode: 3, rcode_name: NXDOMAIN) is when Bro is logging a PTR query: > {noformat} > 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|- > {noformat} > The attachment is a screenshot from a wireshark capture of the DNS query showing that the server is actually answering with NXDOMAIN. > The only change I made to the default configuration was to enable the extraction of executable files from HTTP and SMTP fluxes, so this should have nothing to do with this issue. > Should you need any more info about my setup, please let me know. > Thanks, > Pietro -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 09:40:31 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 22 Aug 2013 11:40:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1065) packet_filter.log is broken In-Reply-To: References: Message-ID: Seth Hall created BIT-1065: ------------------------------ Summary: packet_filter.log is broken Key: BIT-1065 URL: https://bro-tracker.atlassian.net/browse/BIT-1065 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.2 Reporter: Seth Hall Apparently I broke the packet_filter.log when I overhauled the packet filter framework. I need to make sure and fix that prior to the 2.2 release (I think it's an ok thing to be broken during the beta). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 09:44:31 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 22 Aug 2013 11:44:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1064: --------------------------- Resolution: Fixed Fix Version/s: 2.2 Status: Closed (was: Open) > DNS Analyzer does not correctly log NXDOMAIN answers > ---------------------------------------------------- > > Key: BIT-1064 > URL: https://bro-tracker.atlassian.net/browse/BIT-1064 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.1 > Environment: Bro 2.1 running on SecurityOnion 12.04-2 > Reporter: Pietro Delsante > Labels: dns, nxdomain > Fix For: 2.2 > > Attachments: nxdomain.pcap, nxdomain_pcap.png > > > Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest available packages. > It looks like Bro's DNS analyzer is not assigning the correct rcode and rcode_name in the output log when the query is of type A and the server answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both fields, like this: > {noformat} > 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|- > {noformat} > that is, exploded: > {noformat} > ts: 1377179281.104465 > uid: prGZzGRr1M4 > id: 192.168.X.Y|45406|8.8.8.8|53 > proto: udp > trans_id: 64928 > query: www.this-domain-does-not-exist.it > qclass: 1 > qclass_name: C_INTERNET > qtype: 1 > qtype_name: A > rcode: - > rcode_name: - > AA: F > TC: F > RD: T > RA: F > Z: 0 > answers: - > TTLs: - > {noformat} > The only case in which I see those values set correctly (rcode: 3, rcode_name: NXDOMAIN) is when Bro is logging a PTR query: > {noformat} > 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|- > {noformat} > The attachment is a screenshot from a wireshark capture of the DNS query showing that the server is actually answering with NXDOMAIN. > The only change I made to the default configuration was to enable the extraction of executable files from HTTP and SMTP fluxes, so this should have nothing to do with this issue. > Should you need any more info about my setup, please let me know. > Thanks, > Pietro -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 09:44:31 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 22 Aug 2013 11:44:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13709#comment-13709 ] Seth Hall commented on BIT-1064: -------------------------------- It's fixed in 2.2 (git master). I think this was related to some bugs I fixed a while ago in the DNS base scripts. I'm closing the ticket because we aren't going to back port the fix to prior releases. > DNS Analyzer does not correctly log NXDOMAIN answers > ---------------------------------------------------- > > Key: BIT-1064 > URL: https://bro-tracker.atlassian.net/browse/BIT-1064 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.1 > Environment: Bro 2.1 running on SecurityOnion 12.04-2 > Reporter: Pietro Delsante > Labels: dns, nxdomain > Fix For: 2.2 > > Attachments: nxdomain.pcap, nxdomain_pcap.png > > > Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest available packages. > It looks like Bro's DNS analyzer is not assigning the correct rcode and rcode_name in the output log when the query is of type A and the server answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both fields, like this: > {noformat} > 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|- > {noformat} > that is, exploded: > {noformat} > ts: 1377179281.104465 > uid: prGZzGRr1M4 > id: 192.168.X.Y|45406|8.8.8.8|53 > proto: udp > trans_id: 64928 > query: www.this-domain-does-not-exist.it > qclass: 1 > qclass_name: C_INTERNET > qtype: 1 > qtype_name: A > rcode: - > rcode_name: - > AA: F > TC: F > RD: T > RA: F > Z: 0 > answers: - > TTLs: - > {noformat} > The only case in which I see those values set correctly (rcode: 3, rcode_name: NXDOMAIN) is when Bro is logging a PTR query: > {noformat} > 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|- > {noformat} > The attachment is a screenshot from a wireshark capture of the DNS query showing that the server is actually answering with NXDOMAIN. > The only change I made to the default configuration was to enable the extraction of executable files from HTTP and SMTP fluxes, so this should have nothing to do with this issue. > Should you need any more info about my setup, please let me know. > Thanks, > Pietro -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From robin at icir.org Thu Aug 22 09:50:27 2013 From: robin at icir.org (Robin Sommer) Date: Thu, 22 Aug 2013 09:50:27 -0700 Subject: [Bro-Dev] [JIRA] (BIT-1065) packet_filter.log is broken In-Reply-To: References: Message-ID: <20130822165027.GW25209@icir.org> On Thu, Aug 22, 2013 at 11:40 -0500, you wrote: > Apparently I broke the packet_filter.log when I overhauled the packet > filter framework. I need to make sure and fix that prior to the 2.2 > release (I think it's an ok thing to be broken during the beta). Looks like we need a test case for that. :) Is it difficult to fix? I'd prefer to have that done before the beta, it's not great to have "known brokeness" in a version we ask people to test. Robin From jira at bro-tracker.atlassian.net Thu Aug 22 09:50:31 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 22 Aug 2013 11:50:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1065) packet_filter.log is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13710#comment-13710 ] Robin Sommer commented on BIT-1065: ----------------------------------- Looks like we need a test case for that. :) Is it difficult to fix? I'd prefer to have that done before the beta, it's not great to have "known brokeness" in a version we ask people to test. Robin > packet_filter.log is broken > --------------------------- > > Key: BIT-1065 > URL: https://bro-tracker.atlassian.net/browse/BIT-1065 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > > Apparently I broke the packet_filter.log when I overhauled the packet filter framework. I need to make sure and fix that prior to the 2.2 release (I think it's an ok thing to be broken during the beta). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From robin at icir.org Thu Aug 22 09:58:22 2013 From: robin at icir.org (Robin Sommer) Date: Thu, 22 Aug 2013 09:58:22 -0700 Subject: [Bro-Dev] Planing for a 2.2 beta In-Reply-To: <20130812160442.GC1711@icir.org> References: <20130812160442.GC1711@icir.org> Message-ID: <20130822165822.GA32483@icir.org> Let me update this: On Mon, Aug 12, 2013 at 09:04 -0700, I wrote: > - Fix sumstats framework (Seth; or is it done already now?) Done I believe. > - HyperLogLog (Bernhard) Waiting for Bernhard but I believe it's now ready for merging as the memory leak was likely related to the when problem. > - DHCP script cleanup (Seth/Vlad; see BIT-1050) Pending. > - DNP3 finalizing (Robin, Hui) Done, except that one unit tests fails on some platform. > - Windows executable analyzer (Seth; going to happen?) Pending. > - SIP analyzer (Vlad; going to happen?) Pending. > - Bloomfilter test failures (Matthias) Done. > - Input framework test failures (Bernhard) Done. > - X509 extensions (going to happen? can somebody remind we what this is about?) We'll skip these. Plus potentially the packet-filter.log fix. Anything else? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Thu Aug 22 10:04:31 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 22 Aug 2013 12:04:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1065) packet_filter.log is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13711#comment-13711 ] Seth Hall commented on BIT-1065: -------------------------------- I doubt it would be hard, but I wanted to add a feature too where each worker reports it's packet filter too (which mostly just involves adding a "node" field to the log). > packet_filter.log is broken > --------------------------- > > Key: BIT-1065 > URL: https://bro-tracker.atlassian.net/browse/BIT-1065 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > > Apparently I broke the packet_filter.log when I overhauled the packet filter framework. I need to make sure and fix that prior to the 2.2 release (I think it's an ok thing to be broken during the beta). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From robin at icir.org Thu Aug 22 10:11:39 2013 From: robin at icir.org (Robin Sommer) Date: Thu, 22 Aug 2013 10:11:39 -0700 Subject: [Bro-Dev] [JIRA] (BIT-1065) packet_filter.log is broken In-Reply-To: References: Message-ID: <20130822171139.GZ25209@icir.org> Another reason to do it before the beta. :) Robin On Thu, Aug 22, 2013 at 12:04 -0500, you wrote: > > [ https://bro-tracker.atlassian.net/browse/BIT-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13711#comment-13711 ] > > Seth Hall commented on BIT-1065: > -------------------------------- > > I doubt it would be hard, but I wanted to add a feature too where each worker reports it's packet filter too (which mostly just involves adding a "node" field to the log). > > > packet_filter.log is broken > > --------------------------- > > > > Key: BIT-1065 > > URL: https://bro-tracker.atlassian.net/browse/BIT-1065 > > Project: Bro Issue Tracker > > Issue Type: Problem > > Components: Bro > > Affects Versions: 2.2 > > Reporter: Seth Hall > > > > Apparently I broke the packet_filter.log when I overhauled the packet filter framework. I need to make sure and fix that prior to the 2.2 release (I think it's an ok thing to be broken during the beta). > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa > For more information on JIRA, see: http://www.atlassian.com/software/jira > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > From jira at bro-tracker.atlassian.net Thu Aug 22 10:12:31 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 22 Aug 2013 12:12:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1065) packet_filter.log is broken In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13712#comment-13712 ] Robin Sommer commented on BIT-1065: ----------------------------------- Another reason to do it before the beta. :) Robin > packet_filter.log is broken > --------------------------- > > Key: BIT-1065 > URL: https://bro-tracker.atlassian.net/browse/BIT-1065 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Seth Hall > > Apparently I broke the packet_filter.log when I overhauled the packet filter framework. I need to make sure and fix that prior to the 2.2 release (I think it's an ok thing to be broken during the beta). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From seth at icir.org Thu Aug 22 10:31:02 2013 From: seth at icir.org (Seth Hall) Date: Thu, 22 Aug 2013 13:31:02 -0400 Subject: [Bro-Dev] Planing for a 2.2 beta In-Reply-To: <20130822165822.GA32483@icir.org> References: <20130812160442.GC1711@icir.org> <20130822165822.GA32483@icir.org> Message-ID: <3DF5137C-86AB-4A40-A5CC-A3E79E01CF88@icir.org> On Aug 22, 2013, at 12:58 PM, Robin Sommer wrote: > Plus potentially the packet-filter.log fix. > > Anything else? I would like two relatively small changes that will result in some massive headache in test baseline updates. Sorry I hadn't been pushing on this harder until now. Could we do the ticket that moves uid values to 92 to 96 bits? I forget what we agreed on. Also, I'd like to see uid values prepended with either 'F' or 'C' for file uids and connection uids. It would help me keep them straight in mind, especially with the designator being part of the uid. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130822/8d9b9c08/attachment.bin From vladg at cmu.edu Thu Aug 22 10:50:25 2013 From: vladg at cmu.edu (Vlad Grigorescu) Date: Thu, 22 Aug 2013 17:50:25 +0000 Subject: [Bro-Dev] Planing for a 2.2 beta In-Reply-To: <7702_1377191014_r7MH3Xxf009724_20130822165822.GA32483@icir.org> References: <20130812160442.GC1711@icir.org> <7702_1377191014_r7MH3Xxf009724_20130822165822.GA32483@icir.org> Message-ID: <1202BE242E080642B0CD0AD0A03E8552DA9940@PGH-MSGMB-03.andrew.ad.cmu.edu> On Aug 22, 2013, at 12:58 PM, Robin Sommer wrote: >> - DHCP script cleanup (Seth/Vlad; see BIT-1050) > > Pending. > >> - SIP analyzer (Vlad; going to happen?) > > Pending. Yeah, sorry about that. Getting SumStats working again was a big priority for CMU, so I've been focusing on that. I hope to work on those this weekend. --Vlad From robin at icir.org Thu Aug 22 11:12:40 2013 From: robin at icir.org (Robin Sommer) Date: Thu, 22 Aug 2013 11:12:40 -0700 Subject: [Bro-Dev] Planing for a 2.2 beta In-Reply-To: <1202BE242E080642B0CD0AD0A03E8552DA9940@PGH-MSGMB-03.andrew.ad.cmu.edu> References: <20130812160442.GC1711@icir.org> <7702_1377191014_r7MH3Xxf009724_20130822165822.GA32483@icir.org> <1202BE242E080642B0CD0AD0A03E8552DA9940@PGH-MSGMB-03.andrew.ad.cmu.edu> Message-ID: <20130822181240.GC25209@icir.org> On Thu, Aug 22, 2013 at 17:50 +0000, you wrote: > Yeah, sorry about that. Getting SumStats working again was a big > priority for CMU, so I've been focusing on that. I hope to work on > those this weekend. No worries, sumstats clearly had priority, just wanted to update the list. :) Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From robin at icir.org Thu Aug 22 11:14:47 2013 From: robin at icir.org (Robin Sommer) Date: Thu, 22 Aug 2013 11:14:47 -0700 Subject: [Bro-Dev] Planing for a 2.2 beta In-Reply-To: <3DF5137C-86AB-4A40-A5CC-A3E79E01CF88@icir.org> References: <20130812160442.GC1711@icir.org> <20130822165822.GA32483@icir.org> <3DF5137C-86AB-4A40-A5CC-A3E79E01CF88@icir.org> Message-ID: <20130822181447.GE25209@icir.org> On Thu, Aug 22, 2013 at 13:31 -0400, you wrote: > Could we do the ticket that moves uid values to 92 to 96 bits? I > forget what we agreed on. Also, I'd like to see uid values prepended > with either 'F' or 'C' for file uids and connection uids. It would > help me keep them straight in mind, especially with the designator > being part of the uid. Yeah, both make sense. Any volunteers to take the lead on implemtentation and baseline updates? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Thu Aug 22 12:14:31 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 22 Aug 2013 14:14:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1066) topic/dnthayer/oldwiki-binpac In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1066: ---------------------------------- Summary: topic/dnthayer/oldwiki-binpac Key: BIT-1066 URL: https://bro-tracker.atlassian.net/browse/BIT-1066 Project: Bro Issue Tracker Issue Type: Improvement Components: BinPAC Reporter: Daniel Thayer Fix For: 2.2 This branch adds BinPAC documentation (just copied the "BinPAC Userguide" from the old Bro wiki, and reformatted to reST). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 13:44:31 2013 From: jira at bro-tracker.atlassian.net (Adam Slagell (JIRA)) Date: Thu, 22 Aug 2013 15:44:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13713#comment-13713 ] Adam Slagell commented on BIT-1016: ----------------------------------- I think we want 96 bits as it is divisible by 8 and half way between 64 and 128. > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 15:10:31 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 22 Aug 2013 17:10:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1067) topic/jsiwek/extract-limit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1067: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/extract-limit > -------------------------- > > Key: BIT-1067 > URL: https://bro-tracker.atlassian.net/browse/BIT-1067 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.2 > > > Two changes in this branch: > - Add ability to limit size of extracted files. > - Refactor file analyzer plugins to create classes via macros. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 15:10:31 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 22 Aug 2013 17:10:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1067) topic/jsiwek/extract-limit In-Reply-To: References: Message-ID: Jon Siwek created BIT-1067: ------------------------------ Summary: topic/jsiwek/extract-limit Key: BIT-1067 URL: https://bro-tracker.atlassian.net/browse/BIT-1067 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Jon Siwek Assignee: Robin Sommer Fix For: 2.2 Two changes in this branch: - Add ability to limit size of extracted files. - Refactor file analyzer plugins to create classes via macros. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 16:28:31 2013 From: jira at bro-tracker.atlassian.net (john blaze (JIRA)) Date: Thu, 22 Aug 2013 18:28:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1062) Issues fragmented packets and BRO In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] john blaze updated BIT-1062: ---------------------------- Status: Open (was: In Progress) > Issues fragmented packets and BRO > --------------------------------- > > Key: BIT-1062 > URL: https://bro-tracker.atlassian.net/browse/BIT-1062 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.1 > Environment: Ubuntu/Debian > Reporter: john blaze > Attachments: fraggy_out_EVILSTUFF, more_frag.pcap > > > I was doing some testing with fragmented attacks trying to bypass IDS sensors and noticed that BRO does not identify/populate the SRC & DST IP's in the weird log and other fields such as the URI in the http.log when doing stuff like: > >>> f=fragment(IP(dst="80.69.77.211")/ICMP()/("X"*50), fragsize=10) > >>> for frag in f: > ... send(frag) > 1377062338.222065 - - - - - excessively_small_fragment - F bro > Also,. I fragmented a GET /EVILSTUFF HTTP request,. and noticed: > 1377056289.770819 - - - - - excessively_small_fragment - F bro > 1377056289.787032 - - - - - fragment_inconsistency - F bro > 1377056290.141267 iL6Ki3ncjV1 192.168.1.5 17384 192.168.1.16 80 unmatched_HTTP_reply - F bro > PCAPS are attached. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Thu Aug 22 16:28:31 2013 From: jira at bro-tracker.atlassian.net (john blaze (JIRA)) Date: Thu, 22 Aug 2013 18:28:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1062) Issues fragmented packets and BRO In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] john blaze updated BIT-1062: ---------------------------- Status: In Progress (was: Open) > Issues fragmented packets and BRO > --------------------------------- > > Key: BIT-1062 > URL: https://bro-tracker.atlassian.net/browse/BIT-1062 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.1 > Environment: Ubuntu/Debian > Reporter: john blaze > Attachments: fraggy_out_EVILSTUFF, more_frag.pcap > > > I was doing some testing with fragmented attacks trying to bypass IDS sensors and noticed that BRO does not identify/populate the SRC & DST IP's in the weird log and other fields such as the URI in the http.log when doing stuff like: > >>> f=fragment(IP(dst="80.69.77.211")/ICMP()/("X"*50), fragsize=10) > >>> for frag in f: > ... send(frag) > 1377062338.222065 - - - - - excessively_small_fragment - F bro > Also,. I fragmented a GET /EVILSTUFF HTTP request,. and noticed: > 1377056289.770819 - - - - - excessively_small_fragment - F bro > 1377056289.787032 - - - - - fragment_inconsistency - F bro > 1377056290.141267 iL6Ki3ncjV1 192.168.1.5 17384 192.168.1.16 80 unmatched_HTTP_reply - F bro > PCAPS are attached. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From robin at icir.org Thu Aug 22 17:03:58 2013 From: robin at icir.org (Robin Sommer) Date: Thu, 22 Aug 2013 17:03:58 -0700 Subject: [Bro-Dev] extract_files Message-ID: <20130823000358.GO25209@icir.org> That directory gets created in bro_init(), can we change that so that it happens only when something's actually extracted into it? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From noreply at bro.org Fri Aug 23 00:00:14 2013 From: noreply at bro.org (Merge Tracker) Date: Fri, 23 Aug 2013 00:00:14 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308230700.r7N70E9B024047@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------ BIT-1067 [1] Bro Jon Siwek Robin Sommer 2013-08-22 2.2 Normal topic/jsiwek/extract-limit [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [3] broccoli dcode [4] 2013-08-12 Updated specfile. Works under mock for EL6 [5] [1] BIT-1067 https://bro-tracker.atlassian.net/browse/BIT-1067 [2] extract-limit https://github.com/bro/bro/tree/topic/jsiwek/extract-limit [3] Pull Request #1 https://github.com/bro/broccoli/issues/1 [4] dcode https://github.com/dcode [5] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Fri Aug 23 06:30:31 2013 From: jira at bro-tracker.atlassian.net (Pietro Delsante (JIRA)) Date: Fri, 23 Aug 2013 08:30:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1064) DNS Analyzer does not correctly log NXDOMAIN answers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13714#comment-13714 ] Pietro Delsante commented on BIT-1064: -------------------------------------- Thanks Seth, I'll wait for Doug to include Bro 2.2 in Security Onion :-) > DNS Analyzer does not correctly log NXDOMAIN answers > ---------------------------------------------------- > > Key: BIT-1064 > URL: https://bro-tracker.atlassian.net/browse/BIT-1064 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.1 > Environment: Bro 2.1 running on SecurityOnion 12.04-2 > Reporter: Pietro Delsante > Labels: dns, nxdomain > Fix For: 2.2 > > Attachments: nxdomain.pcap, nxdomain_pcap.png > > > Hi, I am running Bro 2.1 on Security Onion 12.04-2 updated to the latest available packages. > It looks like Bro's DNS analyzer is not assigning the correct rcode and rcode_name in the output log when the query is of type A and the server answers with a rcode=3 (NXDOMAIN): instead, it puts a dash "-" in both fields, like this: > {noformat} > 1377179281.104465|prGZzGRr1M4|192.168.X.Y|45406|8.8.8.8|53|udp|64928|www.this-domain-does-not-exist.it|1|C_INTERNET|1|A|-|-|F|F|T|F|0|-|- > {noformat} > that is, exploded: > {noformat} > ts: 1377179281.104465 > uid: prGZzGRr1M4 > id: 192.168.X.Y|45406|8.8.8.8|53 > proto: udp > trans_id: 64928 > query: www.this-domain-does-not-exist.it > qclass: 1 > qclass_name: C_INTERNET > qtype: 1 > qtype_name: A > rcode: - > rcode_name: - > AA: F > TC: F > RD: T > RA: F > Z: 0 > answers: - > TTLs: - > {noformat} > The only case in which I see those values set correctly (rcode: 3, rcode_name: NXDOMAIN) is when Bro is logging a PTR query: > {noformat} > 1377079094.159646|XRRCSUItHlj|192.168.X.Y|39362|8.8.8.8|53|udp|54306|1.0.168.192.in-addr.arpa|1|C_INTERNET|12|PTR|3|NXDOMAIN|F|F|T|F|0|-|- > {noformat} > The attachment is a screenshot from a wireshark capture of the DNS query showing that the server is actually answering with NXDOMAIN. > The only change I made to the default configuration was to enable the extraction of executable files from HTTP and SMTP fluxes, so this should have nothing to do with this issue. > Should you need any more info about my setup, please let me know. > Thanks, > Pietro -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From seth at icir.org Fri Aug 23 07:11:23 2013 From: seth at icir.org (Seth Hall) Date: Fri, 23 Aug 2013 10:11:23 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/extract-limit: Add options to limit extracted file sizes w/ 100MB default. (89ae4ff) In-Reply-To: <201308222145.r7MLjwXF029337@bro-ids.icir.org> References: <201308222145.r7MLjwXF029337@bro-ids.icir.org> Message-ID: On Aug 22, 2013, at 5:45 PM, Jonathan Siwek wrote: > Add options to limit extracted file sizes w/ 100MB default. Instead of having a 100M default, what do you think about having it unlimited by default but then making a tuning script which changes that to 100MB which broctl would then load by default? I think I might get surprised by that limit if I'm running on trace files and might want it to be unlimited. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130823/efc8cdda/attachment.bin From jira at bro-tracker.atlassian.net Fri Aug 23 07:38:31 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 23 Aug 2013 09:38:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1067) topic/jsiwek/extract-limit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1067: ------------------------------ Status: Closed (was: Merge Request) > topic/jsiwek/extract-limit > -------------------------- > > Key: BIT-1067 > URL: https://bro-tracker.atlassian.net/browse/BIT-1067 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.2 > > > Two changes in this branch: > - Add ability to limit size of extracted files. > - Refactor file analyzer plugins to create classes via macros. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 23 12:02:31 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Fri, 23 Aug 2013 14:02:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1068) cpu_pin problem In-Reply-To: References: Message-ID: Seth Hall created BIT-1068: ------------------------------ Summary: cpu_pin problem Key: BIT-1068 URL: https://bro-tracker.atlassian.net/browse/BIT-1068 Project: Bro Issue Tracker Issue Type: Problem Components: BroControl Affects Versions: 2.2 Reporter: Seth Hall Assignee: Daniel Thayer I seem to be having a problem with the cpu_pin feature of broctl. I'm getting the following output... [rootsh at xxxxxx worker-1-6]# cat stderr.log sched_setaffinity: Invalid argument failed to set pid 0's affinity. Daniel, any clue what I should be looking into or information I can provide? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 23 12:08:31 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Fri, 23 Aug 2013 14:08:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1068) cpu_pin problem In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715#comment-13715 ] Daniel Thayer commented on BIT-1068: ------------------------------------ Which OS are you using? What does your node.cfg look like? > cpu_pin problem > --------------- > > Key: BIT-1068 > URL: https://bro-tracker.atlassian.net/browse/BIT-1068 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > > I seem to be having a problem with the cpu_pin feature of broctl. I'm getting the following output... > [rootsh at xxxxxx worker-1-6]# cat stderr.log > sched_setaffinity: Invalid argument > failed to set pid 0's affinity. > Daniel, any clue what I should be looking into or information I can provide? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From jira at bro-tracker.atlassian.net Fri Aug 23 15:30:31 2013 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Fri, 23 Aug 2013 17:30:31 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1068) cpu_pin problem In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13716#comment-13716 ] Seth Hall commented on BIT-1068: -------------------------------- I feel so ashamed. :) I entered '8' as a cpu because it was an 8 core cpu. Unfortunately I was off by one since the last cpu core is addressed at '7'. Sigh. Do you think that's something you could catch and detect? A better error message would be super helpful. :) > cpu_pin problem > --------------- > > Key: BIT-1068 > URL: https://bro-tracker.atlassian.net/browse/BIT-1068 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > > I seem to be having a problem with the cpu_pin feature of broctl. I'm getting the following output... > [rootsh at xxxxxx worker-1-6]# cat stderr.log > sched_setaffinity: Invalid argument > failed to set pid 0's affinity. > Daniel, any clue what I should be looking into or information I can provide? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://bro-tracker.atlassian.net/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira From noreply at bro.org Sat Aug 24 00:00:13 2013 From: noreply at bro.org (Merge Tracker) Date: Sat, 24 Aug 2013 00:00:13 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308240700.r7O70DRo032277@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------- ---------- ------------------------------------------- 6dbbce8 [1] bro Jon Siwek 2013-08-23 Remove code relict pointed out by Bernhard. 288ef20 [2] bro Jon Siwek 2013-08-23 Fix wrong documentation for mkdir BIF. 17d0ecd [3] bro Jon Siwek 2013-08-23 File extraction tweaks. Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [4] broccoli dcode [5] 2013-08-12 Updated specfile. Works under mock for EL6 [6] [1] 6dbbce8 https://github.com/bro/bro/commit/6dbbce8e0582db89c6a69148c73bf0e517f07cbc [2] 288ef20 https://github.com/bro/bro/commit/288ef20a4e8ae939238728ed4b1cb8c07b36eb5a [3] 17d0ecd https://github.com/bro/bro/commit/17d0ecd388581d702c27a4eea259cc5bcc0ca465 [4] Pull Request #1 https://github.com/bro/broccoli/issues/1 [5] dcode https://github.com/dcode [6] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From noreply at bro.org Sun Aug 25 00:00:11 2013 From: noreply at bro.org (Merge Tracker) Date: Sun, 25 Aug 2013 00:00:11 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308250700.r7P70Bx9013490@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------- ---------- ------------------------------------------- 6dbbce8 [1] bro Jon Siwek 2013-08-23 Remove code relict pointed out by Bernhard. 288ef20 [2] bro Jon Siwek 2013-08-23 Fix wrong documentation for mkdir BIF. 17d0ecd [3] bro Jon Siwek 2013-08-23 File extraction tweaks. Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [4] broccoli dcode [5] 2013-08-12 Updated specfile. Works under mock for EL6 [6] [1] 6dbbce8 https://github.com/bro/bro/commit/6dbbce8e0582db89c6a69148c73bf0e517f07cbc [2] 288ef20 https://github.com/bro/bro/commit/288ef20a4e8ae939238728ed4b1cb8c07b36eb5a [3] 17d0ecd https://github.com/bro/bro/commit/17d0ecd388581d702c27a4eea259cc5bcc0ca465 [4] Pull Request #1 https://github.com/bro/broccoli/issues/1 [5] dcode https://github.com/dcode [6] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From noreply at bro.org Mon Aug 26 07:35:23 2013 From: noreply at bro.org (Merge Tracker) Date: Mon, 26 Aug 2013 07:35:23 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308261435.r7QEZN7q017080@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [1] broccoli dcode [2] 2013-08-12 Updated specfile. Works under mock for EL6 [3] [1] Pull Request #1 https://github.com/bro/broccoli/issues/1 [2] dcode https://github.com/dcode [3] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Mon Aug 26 13:35:00 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Mon, 26 Aug 2013 15:35:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1068) pin_cpus error message In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1068: ------------------------------- Summary: pin_cpus error message (was: cpu_pin problem) > pin_cpus error message > ---------------------- > > Key: BIT-1068 > URL: https://bro-tracker.atlassian.net/browse/BIT-1068 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > > I seem to be having a problem with the cpu_pin feature of broctl. I'm getting the following output... > [rootsh at xxxxxx worker-1-6]# cat stderr.log > sched_setaffinity: Invalid argument > failed to set pid 0's affinity. > Daniel, any clue what I should be looking into or information I can provide? -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Mon Aug 26 14:21:01 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 26 Aug 2013 16:21:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1016: --------------------------- Status: Merge Request (was: Open) > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Mon Aug 26 14:25:00 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 26 Aug 2013 16:25:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800#comment-13800 ] Jon Siwek commented on BIT-1016: -------------------------------- A fix for this is in topic/jsiwek/uid in bro, bro-testing, and bro-testing-private repos. Note that the default is 96 bits per UID, though the bits_per_uid script variable can be redef'd to satisfy the original request for 128 bits (or any other desired bit-length). > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Mon Aug 26 14:45:00 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Mon, 26 Aug 2013 16:45:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1060) topic/jsiwek/misc In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann reassigned BIT-1060: ----------------------------------- Assignee: Jon Siwek (was: Bernhard Amann) I found a very small thing (errno check). John will remove it and test if everything still works :) > topic/jsiwek/misc > ----------------- > > Key: BIT-1060 > URL: https://bro-tracker.atlassian.net/browse/BIT-1060 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.2 > > > This branch is in {{bro}} and {{btest}} repos w/ various fixes/workarounds, probably easiest to read commit log, but here's highlight that I remember: > - Improve btest's ability to kill processes that don't terminate > - Workaround a deadlock in gperftools > - Fix a deadlock in SQLite-using threads > - Workaround a problem w/ raw input reader's exec'd child not getting an EOF on its stdin pipe > - Unit test improvements -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Mon Aug 26 14:46:59 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Mon, 26 Aug 2013 16:46:59 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1060) topic/jsiwek/misc In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann updated BIT-1060: -------------------------------- Resolution: Fixed Status: Closed (was: Open) Ok, I was too slow, it already is in fastpath. > topic/jsiwek/misc > ----------------- > > Key: BIT-1060 > URL: https://bro-tracker.atlassian.net/browse/BIT-1060 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.2 > > > This branch is in {{bro}} and {{btest}} repos w/ various fixes/workarounds, probably easiest to read commit log, but here's highlight that I remember: > - Improve btest's ability to kill processes that don't terminate > - Workaround a deadlock in gperftools > - Fix a deadlock in SQLite-using threads > - Workaround a problem w/ raw input reader's exec'd child not getting an EOF on its stdin pipe > - Unit test improvements -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Mon Aug 26 20:35:00 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Mon, 26 Aug 2013 22:35:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1068) pin_cpus error message In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1068: ------------------------------- Status: Merge Request (was: Open) > pin_cpus error message > ---------------------- > > Key: BIT-1068 > URL: https://bro-tracker.atlassian.net/browse/BIT-1068 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > > I seem to be having a problem with the cpu_pin feature of broctl. I'm getting the following output... > [rootsh at xxxxxx worker-1-6]# cat stderr.log > sched_setaffinity: Invalid argument > failed to set pid 0's affinity. > Daniel, any clue what I should be looking into or information I can provide? -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Mon Aug 26 20:35:00 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Mon, 26 Aug 2013 22:35:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1068) pin_cpus error message In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13803#comment-13803 ] Daniel Thayer commented on BIT-1068: ------------------------------------ In branch topic/dnthayer/ticket1068, I've added a check of the CPU pin command which generates a more user-friendly error message if it fails. > pin_cpus error message > ---------------------- > > Key: BIT-1068 > URL: https://bro-tracker.atlassian.net/browse/BIT-1068 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > > I seem to be having a problem with the cpu_pin feature of broctl. I'm getting the following output... > [rootsh at xxxxxx worker-1-6]# cat stderr.log > sched_setaffinity: Invalid argument > failed to set pid 0's affinity. > Daniel, any clue what I should be looking into or information I can provide? -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Mon Aug 26 20:37:00 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Mon, 26 Aug 2013 22:37:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1068) pin_cpus error message In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1068: ------------------------------- Fix Version/s: 2.2 > pin_cpus error message > ---------------------- > > Key: BIT-1068 > URL: https://bro-tracker.atlassian.net/browse/BIT-1068 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > Fix For: 2.2 > > > I seem to be having a problem with the cpu_pin feature of broctl. I'm getting the following output... > [rootsh at xxxxxx worker-1-6]# cat stderr.log > sched_setaffinity: Invalid argument > failed to set pid 0's affinity. > Daniel, any clue what I should be looking into or information I can provide? -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From noreply at bro.org Tue Aug 27 00:00:11 2013 From: noreply at bro.org (Merge Tracker) Date: Tue, 27 Aug 2013 00:00:11 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308270700.r7R70B4p030288@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------- ---------- ------------- ---------- -------------------------------- BIT-1068 [1] BroControl Seth Hall Daniel Thayer 2013-08-26 2.2 Normal pin_cpus error message BIT-1016 [2] Bro rhave - 2013-08-26 2.2 Low Option to extend uids to 128 bit Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [3] broccoli dcode [4] 2013-08-12 Updated specfile. Works under mock for EL6 [5] [1] BIT-1068 https://bro-tracker.atlassian.net/browse/BIT-1068 [2] BIT-1016 https://bro-tracker.atlassian.net/browse/BIT-1016 [3] Pull Request #1 https://github.com/bro/broccoli/issues/1 [4] dcode https://github.com/dcode [5] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Tue Aug 27 12:43:59 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Tue, 27 Aug 2013 14:43:59 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1069) merge topic/bernhard/hexstr In-Reply-To: References: Message-ID: Bernhard Amann created BIT-1069: ----------------------------------- Summary: merge topic/bernhard/hexstr Key: BIT-1069 URL: https://bro-tracker.atlassian.net/browse/BIT-1069 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: git/master Reporter: Bernhard Amann Priority: Low topic/bernhard/hexstr contains a bif function "hexstr_to_bytestring" which does exactly the opposite of the existing "bytestring_to_hexstr" -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 12:43:59 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Tue, 27 Aug 2013 14:43:59 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1069) merge topic/bernhard/hexstr In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann updated BIT-1069: -------------------------------- Status: Merge Request (was: Open) > merge topic/bernhard/hexstr > --------------------------- > > Key: BIT-1069 > URL: https://bro-tracker.atlassian.net/browse/BIT-1069 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Priority: Low > > topic/bernhard/hexstr contains a bif function "hexstr_to_bytestring" which does exactly the opposite of the existing "bytestring_to_hexstr" -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 12:48:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 27 Aug 2013 14:48:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1016: --------------------------------- Assignee: Jon Siwek > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 12:48:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 27 Aug 2013 14:48:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1016: ------------------------------ Status: Open (was: Merge Request) > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 12:50:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 27 Aug 2013 14:50:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13804#comment-13804 ] Robin Sommer commented on BIT-1016: ----------------------------------- Merged. Three comments: - while I generally like the bits_per_uid option, I'm wondering about the performance impact. If it were a static number, UID could just use a correspondingly sized array, vs. now using a relative expensive std::vector. Is the flexibility worth that here? I think a good alternative would be to just #define the bit length in UID, then we could at least easily increase it later. - If we want to keep the bits_per_uid option, could we at least switch to malloced array instead of std::vector with several pushs per UID? - If keeping bits_per_uid, please add a test case that tries a few different values for it. - We could use "C-" instead of "C" to make it more obvious that the first character is special. But not sure if it's worth it. > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 12:54:00 2013 From: jira at bro-tracker.atlassian.net (Adam Slagell (JIRA)) Date: Tue, 27 Aug 2013 14:54:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13805#comment-13805 ] Adam Slagell commented on BIT-1016: ----------------------------------- C- probably doesn't matter. They could treat it as part of the hex and it would still be unique. > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 13:23:59 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 27 Aug 2013 15:23:59 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1068) pin_cpus error message In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13806#comment-13806 ] Robin Sommer commented on BIT-1068: ----------------------------------- Is the "true" here intentional? {code} [?] # message just in case there's some other reason for the failure). true if [ $? -eq 0 ]; then [?] {code} > pin_cpus error message > ---------------------- > > Key: BIT-1068 > URL: https://bro-tracker.atlassian.net/browse/BIT-1068 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > Fix For: 2.2 > > > I seem to be having a problem with the cpu_pin feature of broctl. I'm getting the following output... > [rootsh at xxxxxx worker-1-6]# cat stderr.log > sched_setaffinity: Invalid argument > failed to set pid 0's affinity. > Daniel, any clue what I should be looking into or information I can provide? -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 13:29:59 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 27 Aug 2013 15:29:59 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1068) pin_cpus error message In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13807#comment-13807 ] Daniel Thayer commented on BIT-1068: ------------------------------------ Yes, it was intentional. Without that check, then if "true" is not found (very unlikely), then bro will always fail to start (when CPU pinning is in use). > pin_cpus error message > ---------------------- > > Key: BIT-1068 > URL: https://bro-tracker.atlassian.net/browse/BIT-1068 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > Fix For: 2.2 > > > I seem to be having a problem with the cpu_pin feature of broctl. I'm getting the following output... > [rootsh at xxxxxx worker-1-6]# cat stderr.log > sched_setaffinity: Invalid argument > failed to set pid 0's affinity. > Daniel, any clue what I should be looking into or information I can provide? -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From vladg at cmu.edu Tue Aug 27 13:35:21 2013 From: vladg at cmu.edu (Vlad Grigorescu) Date: Tue, 27 Aug 2013 20:35:21 +0000 Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: <28407_1377634793_r7RKJqC3028637_JIRA.11022.1370459545645.52.1377633000879@bro-tracker.atlassian.net> References: <28407_1377634793_r7RKJqC3028637_JIRA.11022.1370459545645.52.1377633000879@bro-tracker.atlassian.net> Message-ID: <1202BE242E080642B0CD0AD0A03E8552DD63A3@PGH-MSGMB-03.andrew.ad.cmu.edu> On Aug 27, 2013, at 3:50 PM, Robin Sommer (JIRA) wrote: > - We could use "C-" instead of "C" to make it more obvious that the first character is special. But not sure if it's worth it. FWIW, I prefer C for the simple reason that if I double-click it, it selects the whole uid (including the C), and I can then copy the whole thing. I also feel like when you actually look at a log, and one column always starts with a C and one starts with an F, it will be obvious enough. --Vlad From robin at icir.org Tue Aug 27 13:37:23 2013 From: robin at icir.org (Robin Sommer) Date: Tue, 27 Aug 2013 13:37:23 -0700 Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: <1202BE242E080642B0CD0AD0A03E8552DD63A3@PGH-MSGMB-03.andrew.ad.cmu.edu> References: <28407_1377634793_r7RKJqC3028637_JIRA.11022.1370459545645.52.1377633000879@bro-tracker.atlassian.net> <1202BE242E080642B0CD0AD0A03E8552DD63A3@PGH-MSGMB-03.andrew.ad.cmu.edu> Message-ID: <20130827203723.GH1797@icir.org> On Tue, Aug 27, 2013 at 20:35 +0000, you wrote: > FWIW, I prefer C for the simple reason that if I double-click it, > it selects the whole uid (including the C), and I can then copy the > whole thing. Good point. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Tue Aug 27 13:38:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 27 Aug 2013 15:38:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13808#comment-13808 ] Robin Sommer commented on BIT-1016: ----------------------------------- Good point. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 14:02:00 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 27 Aug 2013 16:02:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13809#comment-13809 ] Jon Siwek commented on BIT-1016: -------------------------------- {quote}while I generally like the bits_per_uid option, I'm wondering about the performance impact. If it were a static number, UID could just use a correspondingly sized array, vs. now using a relative expensive std::vector. Is the flexibility worth that here? I think a good alternative would be to just #define the bit length in UID, then we could at least easily increase it later.{quote} We know that there's people (person) that may want something other than our default and we know that not everyone runs Bro by compiling it from source, so having the option to change it at parse-time I thought was nice. {quote}If we want to keep the bits_per_uid option, could we at least switch to malloced array instead of std::vector with several pushs per UID?{quote} I preferred vector to start because it's safer/simpler to write correct code, but yeah I can look in to that optimization especially since there's not a lot of code involved. {quote}If keeping bits_per_uid, please add a test case that tries a few different values for it.{quote} Ok. > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 14:04:00 2013 From: jira at bro-tracker.atlassian.net (Matthias Vallentin (JIRA)) Date: Tue, 27 Aug 2013 16:04:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13810#comment-13810 ] Matthias Vallentin commented on BIT-1016: ----------------------------------------- Regarding performance: another option would be to use 128-bit UUIDs internally and just chop of 32 bytes if a 96-bit UUID is desired, assuming the bits in the UUID are distributed uniformly. Then we could use a fixed-size array and just change how the data is interpreted at script land. > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From robin at icir.org Tue Aug 27 14:10:15 2013 From: robin at icir.org (Robin Sommer) Date: Tue, 27 Aug 2013 14:10:15 -0700 Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: <20130827211015.GI1797@icir.org> I like that idea. Storage-wise it's 2x64 bit anyways. Robin On Tue, Aug 27, 2013 at 16:04 -0500, you wrote: > Regarding performance: another option would be to use 128-bit UUIDs > internally and just chop of 32 bytes if a 96-bit UUID is desired, > assuming the bits in the UUID are distributed uniformly. Then we could > use a fixed-size array and just change how the data is interpreted at > script land. > > > Option to extend uids to 128 bit > > -------------------------------- > > > > Key: BIT-1016 > > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > > Project: Bro Issue Tracker > > Issue Type: New Feature > > Components: Bro > > Affects Versions: git/master > > Reporter: rhave > > Assignee: Jon Siwek > > Priority: Low > > Fix For: 2.2 > > > > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc > > > > -- > This message was sent by Atlassian JIRA > (v6.1-OD-06#6139) > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > From jira at bro-tracker.atlassian.net Tue Aug 27 14:12:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 27 Aug 2013 16:12:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811#comment-13811 ] Robin Sommer commented on BIT-1016: ----------------------------------- I like that idea. Storage-wise it's 2x64 bit anyways. Robin > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 14:14:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 27 Aug 2013 16:14:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1069) merge topic/bernhard/hexstr In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1069: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > merge topic/bernhard/hexstr > --------------------------- > > Key: BIT-1069 > URL: https://bro-tracker.atlassian.net/browse/BIT-1069 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Priority: Low > > topic/bernhard/hexstr contains a bif function "hexstr_to_bytestring" which does exactly the opposite of the existing "bytestring_to_hexstr" -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 14:14:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 27 Aug 2013 16:14:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1068) pin_cpus error message In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1068: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > pin_cpus error message > ---------------------- > > Key: BIT-1068 > URL: https://bro-tracker.atlassian.net/browse/BIT-1068 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > Fix For: 2.2 > > > I seem to be having a problem with the cpu_pin feature of broctl. I'm getting the following output... > [rootsh at xxxxxx worker-1-6]# cat stderr.log > sched_setaffinity: Invalid argument > failed to set pid 0's affinity. > Daniel, any clue what I should be looking into or information I can provide? -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 14:14:00 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 27 Aug 2013 16:14:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13812#comment-13812 ] Jon Siwek commented on BIT-1016: -------------------------------- {quote} Regarding performance: another option would be to use 128-bit UUIDs internally and just chop of 32 bytes if a 96-bit UUID is desired, assuming the bits in the UUID are distributed uniformly. Then we could use a fixed-size array and just change how the data is interpreted at script land. {quote} This is kind of what currently happens: the UID gets calculated in 64-bit chunks, then truncated if necessary. Except you can ask for more than 128-bits if you want. > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 14:22:00 2013 From: jira at bro-tracker.atlassian.net (Matthias Vallentin (JIRA)) Date: Tue, 27 Aug 2013 16:22:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13813#comment-13813 ] Matthias Vallentin commented on BIT-1016: ----------------------------------------- {quote} This is kind of what currently happens: the UID gets calculated in 64-bit chunks, then truncated if necessary. Except you can ask for more than 128-bits if you want. {quote} I wonder if we'd ever need more than 128-bit. I'm not aware of any UUID implementation that uses more than that. Using a {{uint64_t[2]}} wrapped in a small class with value semantics should suffice in my eyes. > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 14:34:00 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 27 Aug 2013 16:34:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13814#comment-13814 ] Jon Siwek commented on BIT-1016: -------------------------------- {quote}I wonder if we'd ever need more than 128-bit. I'm not aware of any UUID implementation that uses more than that. Using a uint64_t[2] wrapped in a small class with value semantics should suffice in my eyes.{quote} Maybe. Could be the generality of it won't ever be realized. But also, I don't think we have any measurements yet that show how concerned/picky we need to be about how it's implemented? > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Tue Aug 27 14:40:00 2013 From: jira at bro-tracker.atlassian.net (Matthias Vallentin (JIRA)) Date: Tue, 27 Aug 2013 16:40:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815#comment-13815 ] Matthias Vallentin commented on BIT-1016: ----------------------------------------- {quote} But also, I don't think we have any measurements yet that show how concerned/picky we need to be about how it's implemented? {quote} True that. A {{std::vector}} is not that much slower in terms of speed, it's just that it has some constant space overhead, roughly ~24 bytes. I cannot gauge how many UUID objects reside in memory at the same time, but this may be the ultimate measure. > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From noreply at bro.org Wed Aug 28 00:00:11 2013 From: noreply at bro.org (Merge Tracker) Date: Wed, 28 Aug 2013 00:00:11 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308280700.r7S70BpE012415@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [1] broccoli dcode [2] 2013-08-12 Updated specfile. Works under mock for EL6 [3] [1] Pull Request #1 https://github.com/bro/broccoli/issues/1 [2] dcode https://github.com/dcode [3] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Wed Aug 28 14:42:00 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 28 Aug 2013 16:42:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13816#comment-13816 ] Jon Siwek commented on BIT-1016: -------------------------------- I changed to a {{uint64[2]}} implementation that's increased with a {{#define}}. This means the max {{bits_per_uid}} is 128 right now, so at least can satisfy the original request and increasing beyond that is a trivial source code change. > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Wed Aug 28 14:44:00 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 28 Aug 2013 16:44:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1016: --------------------------- Status: Merge Request (was: Open) > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Wed Aug 28 15:40:03 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 28 Aug 2013 17:40:03 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1070) topic/dnthayer/bug-fixes In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1070: ---------------------------------- Summary: topic/dnthayer/bug-fixes Key: BIT-1070 URL: https://bro-tracker.atlassian.net/browse/BIT-1070 Project: Bro Issue Tracker Issue Type: Problem Components: BTest Reporter: Daniel Thayer Fix For: 2.2 This branch contains some fixes to btest and the README. -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Wed Aug 28 15:40:03 2013 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 28 Aug 2013 17:40:03 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1070) topic/dnthayer/bug-fixes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1070: ------------------------------- Status: Merge Request (was: Open) > topic/dnthayer/bug-fixes > ------------------------ > > Key: BIT-1070 > URL: https://bro-tracker.atlassian.net/browse/BIT-1070 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BTest > Reporter: Daniel Thayer > Fix For: 2.2 > > > This branch contains some fixes to btest and the README. -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Wed Aug 28 21:09:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 28 Aug 2013 23:09:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1016) Option to extend uids to 128 bit In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1016: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Option to extend uids to 128 bit > -------------------------------- > > Key: BIT-1016 > URL: https://bro-tracker.atlassian.net/browse/BIT-1016 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: rhave > Assignee: Jon Siwek > Priority: Low > Fix For: 2.2 > > > Bro's uids are currently 64 bits, which makes them collide with a 50% chance after 5.1 x 10^9^ different uids (see http://en.wikipedia.org/wiki/Birthday_problem#Probability_table). > I'm currently generating uuids of 128 bit to replace the native uids in bro, as I'm using them as keys in a database, but this requires rewriting of the bro-logs. I suspect that more people could benefit from an option to extend the uids to 128 bit. > I've made a quick and dirty patch to change most of the uids to 128 bit (file_analysis uids are missing). The patch is ugly, and is only to show some of the functionality I would like: http://pastebin.com/GkaGejNc -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Wed Aug 28 21:09:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 28 Aug 2013 23:09:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1070) topic/dnthayer/bug-fixes In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1070: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/dnthayer/bug-fixes > ------------------------ > > Key: BIT-1070 > URL: https://bro-tracker.atlassian.net/browse/BIT-1070 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BTest > Reporter: Daniel Thayer > Fix For: 2.2 > > > This branch contains some fixes to btest and the README. -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From noreply at bro.org Thu Aug 29 00:00:12 2013 From: noreply at bro.org (Merge Tracker) Date: Thu, 29 Aug 2013 00:00:12 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308290700.r7T70CSU027315@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [1] broccoli dcode [2] 2013-08-12 Updated specfile. Works under mock for EL6 [3] [1] Pull Request #1 https://github.com/bro/broccoli/issues/1 [2] dcode https://github.com/dcode [3] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Thu Aug 29 13:38:59 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Thu, 29 Aug 2013 15:38:59 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1071) Initializing an opaqueval as a global segfaults Bro In-Reply-To: References: Message-ID: Bernhard Amann created BIT-1071: ----------------------------------- Summary: Initializing an opaqueval as a global segfaults Bro Key: BIT-1071 URL: https://bro-tracker.atlassian.net/browse/BIT-1071 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Reporter: Bernhard Amann Fix For: 2.2 Running Bro with a file containing {quote} global test = md5_hash_init(); {quote} crashes Bro and results in the following backtrace: {quote} #0 0x08259e95 in Ref (o=0x0) at /home/bernhard/bro/src/Obj.h:205 #1 0x0825a037 in BroType::Ref (this=0x0) at /home/bernhard/bro/src/Type.h:225 #2 0x0825a35b in Val::Val (this=0x87c7580, t=0x0) at /home/bernhard/bro/src/Val.h:387 #3 0x08377d46 in OpaqueVal::OpaqueVal (this=0x87c7580, t=0x0) at /home/bernhard/bro/src/Val.cc:3133 #4 0x0830b6a8 in HashVal::HashVal (this=0x87c7580, t=0x0) at /home/bernhard/bro/src/OpaqueVal.cc:62 #5 0x0830b8bd in MD5Val::MD5Val (this=0x87c7580) at /home/bernhard/bro/src/OpaqueVal.cc:81 #6 0x082d68aa in BifFunc::bro_md5_hash_init (frame=0x0, BiF_ARGS=0x93db520) at bro.bif:631 #7 0x082d4df1 in BuiltinFunc::Call (this=0x8b57e40, args=0x93db520, parent=0x0) at /home/bernhard/bro/src/Func.cc:507 #8 0x082c76a6 in CallExpr::Eval (this=0x93e4300, f=0x0) at /home/bernhard/bro/src/Expr.cc:4827 #9 0x082b438b in Expr::InitVal (this=0x93e4300, t=0x897c740, aggr=0x0) at /home/bernhard/bro/src/Expr.cc:130 #10 0x0837a4de in init_val (init=0x93e4300, t=0x897c740, aggr=0x0) at /home/bernhard/bro/src/Var.cc:17 #11 0x0837abc5 in make_var (id=0x93d0c30, t=0x897c740, c=INIT_FULL, init=0x93e4300, attr=0x0, dt=VAR_REGULAR, do_init=1) at /home/bernhard/bro/src/Var.cc:188 #12 0x0837adc5 in add_global (id=0x93d0c30, t=0x0, c=INIT_FULL, init=0x93e4300, attr=0x0, dt=VAR_REGULAR) at /home/bernhard/bro/src/Var.cc:226 #13 0x08255c0c in yyparse () at parse.y:1068 #14 0x08268da1 in main (argc=2, argv=0xbffff624) at /home/bernhard/bro/src/main.cc:859 {quote} Seems to work for any opaque val. -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Thu Aug 29 14:10:59 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Thu, 29 Aug 2013 16:10:59 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog In-Reply-To: References: Message-ID: Bernhard Amann created BIT-1072: ----------------------------------- Summary: merge topic/bernhard/hyperloglog Key: BIT-1072 URL: https://bro-tracker.atlassian.net/browse/BIT-1072 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: git/master Reporter: Bernhard Amann Fix For: 2.2 Attachments: out.pdf The branch adds support for the hyperloglog data structure. In the branch, core/leaks/basic-cluster.bro currently faisl. However, this seems to be unrelated to hll and just to be triggered by the addition of it to the sumstats tests. It looks like some kind of scriptland issue. pprof output is attached. (master, workers don't leak memory) -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Thu Aug 29 14:10:59 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Thu, 29 Aug 2013 16:10:59 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann updated BIT-1072: -------------------------------- Status: Merge Request (was: Open) > merge topic/bernhard/hyperloglog > -------------------------------- > > Key: BIT-1072 > URL: https://bro-tracker.atlassian.net/browse/BIT-1072 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > Attachments: out.pdf > > > The branch adds support for the hyperloglog data structure. > In the branch, core/leaks/basic-cluster.bro currently faisl. However, this seems to be unrelated to hll and just to be triggered by the addition of it to the sumstats tests. It looks like some kind of scriptland issue. pprof output is attached. (master, workers don't leak memory) -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From noreply at bro.org Fri Aug 30 00:00:14 2013 From: noreply at bro.org (Merge Tracker) Date: Fri, 30 Aug 2013 00:00:14 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308300700.r7U70EFn014962@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- -------------- ---------- ---------- ------------- ---------- -------------------------------- BIT-1072 [1] Bro Bernhard Amann - 2013-08-29 2.2 Normal merge topic/bernhard/hyperloglog Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------- ---------- -------------------------------------------------- dc2e3d6 [2] bro Jon Siwek 2013-08-29 Fix global opaque val segfault, addresses BIT-1071 742a047 [3] bro Jon Siwek 2013-08-29 Fix malloc/delete mismatch. c4e8908 [4] bro Jon Siwek 2013-08-29 Fix invalid pointer dereference in AsciiFormatter. Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [5] broccoli dcode [6] 2013-08-12 Updated specfile. Works under mock for EL6 [7] [1] BIT-1072 https://bro-tracker.atlassian.net/browse/BIT-1072 [2] dc2e3d6 https://github.com/bro/bro/commit/dc2e3d6e04ad7504989b0b377f3c2f6cb9e1a2ef [3] 742a047 https://github.com/bro/bro/commit/742a047a40e6eeb32aea9ededf6bd294d5263ff8 [4] c4e8908 https://github.com/bro/bro/commit/c4e8908c8e267c1e93fd7fa0c9df22f474ec47f3 [5] Pull Request #1 https://github.com/bro/broccoli/issues/1 [6] dcode https://github.com/dcode [7] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Fri Aug 30 08:32:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 30 Aug 2013 10:32:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1072: ------------------------------ Status: Open (was: Merge Request) > merge topic/bernhard/hyperloglog > -------------------------------- > > Key: BIT-1072 > URL: https://bro-tracker.atlassian.net/browse/BIT-1072 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > Attachments: out.pdf > > > The branch adds support for the hyperloglog data structure. > In the branch, core/leaks/basic-cluster.bro currently faisl. However, this seems to be unrelated to hll and just to be triggered by the addition of it to the sumstats tests. It looks like some kind of scriptland issue. pprof output is attached. (master, workers don't leak memory) -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Fri Aug 30 08:32:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 30 Aug 2013 10:32:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817#comment-13817 ] Robin Sommer commented on BIT-1072: ----------------------------------- I'm getting a number of conflicts when merging into master. Please merge the branch with master first. > merge topic/bernhard/hyperloglog > -------------------------------- > > Key: BIT-1072 > URL: https://bro-tracker.atlassian.net/browse/BIT-1072 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > Attachments: out.pdf > > > The branch adds support for the hyperloglog data structure. > In the branch, core/leaks/basic-cluster.bro currently faisl. However, this seems to be unrelated to hll and just to be triggered by the addition of it to the sumstats tests. It looks like some kind of scriptland issue. pprof output is attached. (master, workers don't leak memory) -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Fri Aug 30 08:48:00 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 30 Aug 2013 10:48:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann updated BIT-1072: -------------------------------- Status: Merge Request (was: Open) > merge topic/bernhard/hyperloglog > -------------------------------- > > Key: BIT-1072 > URL: https://bro-tracker.atlassian.net/browse/BIT-1072 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > Attachments: out.pdf > > > The branch adds support for the hyperloglog data structure. > In the branch, core/leaks/basic-cluster.bro currently faisl. However, this seems to be unrelated to hll and just to be triggered by the addition of it to the sumstats tests. It looks like some kind of scriptland issue. pprof output is attached. (master, workers don't leak memory) -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Fri Aug 30 08:48:00 2013 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 30 Aug 2013 10:48:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13818#comment-13818 ] Bernhard Amann commented on BIT-1072: ------------------------------------- Sorry - re-merged master > merge topic/bernhard/hyperloglog > -------------------------------- > > Key: BIT-1072 > URL: https://bro-tracker.atlassian.net/browse/BIT-1072 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > Attachments: out.pdf > > > The branch adds support for the hyperloglog data structure. > In the branch, core/leaks/basic-cluster.bro currently faisl. However, this seems to be unrelated to hll and just to be triggered by the addition of it to the sumstats tests. It looks like some kind of scriptland issue. pprof output is attached. (master, workers don't leak memory) -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Fri Aug 30 11:30:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 30 Aug 2013 13:30:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1072: ------------------------------ Status: Open (was: Merge Request) > merge topic/bernhard/hyperloglog > -------------------------------- > > Key: BIT-1072 > URL: https://bro-tracker.atlassian.net/browse/BIT-1072 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > Attachments: out.pdf > > > The branch adds support for the hyperloglog data structure. > In the branch, core/leaks/basic-cluster.bro currently faisl. However, this seems to be unrelated to hll and just to be triggered by the addition of it to the sumstats tests. It looks like some kind of scriptland issue. pprof output is attached. (master, workers don't leak memory) -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Fri Aug 30 11:30:00 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 30 Aug 2013 13:30:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13819#comment-13819 ] Robin Sommer commented on BIT-1072: ----------------------------------- I ended up refactoring and reformatting this quite a bit, it's in topic/robin/hyperlolog-merge. However, I broke something, the tests aren't working. Need to debug that later. In the meantime, some requests/questions: - Please look over my changes and see if they make sense. (You don't need to track down the bug; I take the blame for that :). - Can you please rework the Doxygen comments in HyperLogLog.h so that the descriptions for the public methods are understandable on their own. Right now I can't really follow them as often they talk about internal parameters/functionality. What you could do is provide a short overview of the data structure parameters in the class' doc string, and then refer to that in the methods. Also, please use the @param and @return syntax. (Start from my branch with this: I already reformatted and reordered things there quite a bit.) - I don't understand what can be parameterized by the user and what not (and why not). One can give an error margin to the actor, but the confidence is a compile time constant. Also, where are the magic alpha_m values in *.cc coming from? Are these indeed always static values that don't depend on any parameters? > merge topic/bernhard/hyperloglog > -------------------------------- > > Key: BIT-1072 > URL: https://bro-tracker.atlassian.net/browse/BIT-1072 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > Attachments: out.pdf > > > The branch adds support for the hyperloglog data structure. > In the branch, core/leaks/basic-cluster.bro currently faisl. However, this seems to be unrelated to hll and just to be triggered by the addition of it to the sumstats tests. It looks like some kind of scriptland issue. pprof output is attached. (master, workers don't leak memory) -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From jira at bro-tracker.atlassian.net Fri Aug 30 12:26:00 2013 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 30 Aug 2013 14:26:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1071) Initializing an opaqueval as a global segfaults Bro In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1071: --------------------------- Resolution: Fixed Status: Closed (was: Open) > Initializing an opaqueval as a global segfaults Bro > --------------------------------------------------- > > Key: BIT-1071 > URL: https://bro-tracker.atlassian.net/browse/BIT-1071 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > > Running Bro with a file containing > {quote} > global test = md5_hash_init(); > {quote} > crashes Bro and results in the following backtrace: > {quote} > #0 0x08259e95 in Ref (o=0x0) at /home/bernhard/bro/src/Obj.h:205 > #1 0x0825a037 in BroType::Ref (this=0x0) at /home/bernhard/bro/src/Type.h:225 > #2 0x0825a35b in Val::Val (this=0x87c7580, t=0x0) at /home/bernhard/bro/src/Val.h:387 > #3 0x08377d46 in OpaqueVal::OpaqueVal (this=0x87c7580, t=0x0) at /home/bernhard/bro/src/Val.cc:3133 > #4 0x0830b6a8 in HashVal::HashVal (this=0x87c7580, t=0x0) at /home/bernhard/bro/src/OpaqueVal.cc:62 > #5 0x0830b8bd in MD5Val::MD5Val (this=0x87c7580) at /home/bernhard/bro/src/OpaqueVal.cc:81 > #6 0x082d68aa in BifFunc::bro_md5_hash_init (frame=0x0, BiF_ARGS=0x93db520) at bro.bif:631 > #7 0x082d4df1 in BuiltinFunc::Call (this=0x8b57e40, args=0x93db520, parent=0x0) at /home/bernhard/bro/src/Func.cc:507 > #8 0x082c76a6 in CallExpr::Eval (this=0x93e4300, f=0x0) at /home/bernhard/bro/src/Expr.cc:4827 > #9 0x082b438b in Expr::InitVal (this=0x93e4300, t=0x897c740, aggr=0x0) at /home/bernhard/bro/src/Expr.cc:130 > #10 0x0837a4de in init_val (init=0x93e4300, t=0x897c740, aggr=0x0) at /home/bernhard/bro/src/Var.cc:17 > #11 0x0837abc5 in make_var (id=0x93d0c30, t=0x897c740, c=INIT_FULL, init=0x93e4300, attr=0x0, dt=VAR_REGULAR, do_init=1) at /home/bernhard/bro/src/Var.cc:188 > #12 0x0837adc5 in add_global (id=0x93d0c30, t=0x0, c=INIT_FULL, init=0x93e4300, attr=0x0, dt=VAR_REGULAR) at /home/bernhard/bro/src/Var.cc:226 > #13 0x08255c0c in yyparse () at parse.y:1068 > #14 0x08268da1 in main (argc=2, argv=0xbffff624) at /home/bernhard/bro/src/main.cc:859 > {quote} > Seems to work for any opaque val. -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139) From noreply at bro.org Sat Aug 31 00:00:15 2013 From: noreply at bro.org (Merge Tracker) Date: Sat, 31 Aug 2013 00:00:15 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201308310700.r7V70FJ0024050@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------- #1 [1] broccoli dcode [2] 2013-08-12 Updated specfile. Works under mock for EL6 [3] [1] Pull Request #1 https://github.com/bro/broccoli/issues/1 [2] dcode https://github.com/dcode [3] Merge Pull Request #1 with git pull https://github.com/dcode/broccoli.git master From jira at bro-tracker.atlassian.net Sat Aug 31 11:20:59 2013 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Sat, 31 Aug 2013 13:20:59 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1072) merge topic/bernhard/hyperloglog In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820#comment-13820 ] Robin Sommer commented on BIT-1072: ----------------------------------- I fixed the bug introduced, did some more polishing, and also made the confidence a parameter to hll init. Merged into master now, but please still work on the Doxygen comments. > merge topic/bernhard/hyperloglog > -------------------------------- > > Key: BIT-1072 > URL: https://bro-tracker.atlassian.net/browse/BIT-1072 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Bernhard Amann > Fix For: 2.2 > > Attachments: out.pdf > > > The branch adds support for the hyperloglog data structure. > In the branch, core/leaks/basic-cluster.bro currently faisl. However, this seems to be unrelated to hll and just to be triggered by the addition of it to the sumstats tests. It looks like some kind of scriptland issue. pprof output is attached. (master, workers don't leak memory) -- This message was sent by Atlassian JIRA (v6.1-OD-06#6139)