From noreply at bro-ids.org Sun Jul 1 00:00:06 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 1 Jul 2012 00:00:06 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207010700.q61706OG012785@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 833 [1] | aashish | | High | ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 34ead91 | Jon Siwek | 2012-06-29 | Fix inconsistencies in random number generation. [2] bro | 0e48fda | Jon Siwek | 2012-06-29 | Updating input framework unit tests. [3] bro | 41f1544 | Jon Siwek | 2012-06-28 | Add front-end name to InitMessage from WriterFrontend to Backend. [4] bro | 1bbd639 | Jon Siwek | 2012-06-28 | Small tweak to make test complete quicker. [5] bro | 21a0e74 | Jon Siwek | 2012-06-28 | Drain events before terminating log/thread managers. [6] bro | a651185 | Jon Siwek | 2012-06-27 | Fix strict-aliasing warning in RemoteSerializer.cc (fixes #834). [7] bro | 94f0bf2 | Daniel Thayer | 2012-06-26 | Fix typos in event documentation [8] bro | 5ab2545 | Daniel Thayer | 2012-06-26 | Fix typos in NEWS for Bro 2.1 beta [9] pysubnettree | 00cc7fa | Daniel Thayer | 2012-06-28 | Fix indentation of an "else" statement [10] pysubnettree | d9c2160 | Jon Siwek | 2012-06-28 | Fix compile warnings and dependencies of swig-generated files. [11] [1] #833: http://tracker.bro-ids.org/bro/ticket/833 [2] fastpath: http://tracker.bro-ids.org/bro/changeset/34ead91f992cbc40dcb81053343e2ef60a3aff61/bro [3] fastpath: http://tracker.bro-ids.org/bro/changeset/0e48fda6ffa0be4cec2d763305a1394e19b32778/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/41f1544332cddfa9a636c05f41371698a891de63/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/1bbd63970a9fe5529cc9c6898c510d47ea5472af/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/21a0e74d682f0584288c6e631496bb4083e5d33f/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/a651185ff9f93fedb3a82575e5107dd7460475de/bro [8] fastpath: http://tracker.bro-ids.org/bro/changeset/94f0bf215783b7b529a7960da6bb463e4fe8c0cf/bro [9] fastpath: http://tracker.bro-ids.org/bro/changeset/5ab2545ff3da7b210e368b81fff87c12614d6ab8/bro [1] fastpath: http://tracker.bro-ids.org/bro/changeset/00cc7fa9a53410cff1369501eb3d4ae28ca4bc9d/pysubnettree [1] fastpath: http://tracker.bro-ids.org/bro/changeset/d9c2160980c319db5df0c3c6d958270f71743622/pysubnettree From noreply at bro-ids.org Mon Jul 2 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 2 Jul 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207020700.q62703pr027169@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 833 [1] | aashish | | High | ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 34ead91 | Jon Siwek | 2012-06-29 | Fix inconsistencies in random number generation. [2] bro | 0e48fda | Jon Siwek | 2012-06-29 | Updating input framework unit tests. [3] bro | 41f1544 | Jon Siwek | 2012-06-28 | Add front-end name to InitMessage from WriterFrontend to Backend. [4] bro | 1bbd639 | Jon Siwek | 2012-06-28 | Small tweak to make test complete quicker. [5] bro | 21a0e74 | Jon Siwek | 2012-06-28 | Drain events before terminating log/thread managers. [6] bro | a651185 | Jon Siwek | 2012-06-27 | Fix strict-aliasing warning in RemoteSerializer.cc (fixes #834). [7] bro | 94f0bf2 | Daniel Thayer | 2012-06-26 | Fix typos in event documentation [8] bro | 5ab2545 | Daniel Thayer | 2012-06-26 | Fix typos in NEWS for Bro 2.1 beta [9] pysubnettree | 00cc7fa | Daniel Thayer | 2012-06-28 | Fix indentation of an "else" statement [10] pysubnettree | d9c2160 | Jon Siwek | 2012-06-28 | Fix compile warnings and dependencies of swig-generated files. [11] [1] #833: http://tracker.bro-ids.org/bro/ticket/833 [2] fastpath: http://tracker.bro-ids.org/bro/changeset/34ead91f992cbc40dcb81053343e2ef60a3aff61/bro [3] fastpath: http://tracker.bro-ids.org/bro/changeset/0e48fda6ffa0be4cec2d763305a1394e19b32778/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/41f1544332cddfa9a636c05f41371698a891de63/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/1bbd63970a9fe5529cc9c6898c510d47ea5472af/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/21a0e74d682f0584288c6e631496bb4083e5d33f/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/a651185ff9f93fedb3a82575e5107dd7460475de/bro [8] fastpath: http://tracker.bro-ids.org/bro/changeset/94f0bf215783b7b529a7960da6bb463e4fe8c0cf/bro [9] fastpath: http://tracker.bro-ids.org/bro/changeset/5ab2545ff3da7b210e368b81fff87c12614d6ab8/bro [1] fastpath: http://tracker.bro-ids.org/bro/changeset/00cc7fa9a53410cff1369501eb3d4ae28ca4bc9d/pysubnettree [1] fastpath: http://tracker.bro-ids.org/bro/changeset/d9c2160980c319db5df0c3c6d958270f71743622/pysubnettree From bro at tracker.bro-ids.org Mon Jul 2 07:50:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 14:50:19 -0000 Subject: [Bro-Dev] #659: NMI builds check for compiler warnings In-Reply-To: <048.98a7f5e7f513339f0f639eb6aee714a6@tracker.bro-ids.org> References: <048.98a7f5e7f513339f0f639eb6aee714a6@tracker.bro-ids.org> Message-ID: <063.bca17e5e52a26439552520ad2ec11416@tracker.bro-ids.org> #659: NMI builds check for compiler warnings ------------------------------+------------------------ Reporter: jsiwek | Owner: dnthayer Type: Feature Request | Status: closed Priority: Low | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Rejected | Keywords: nmi ------------------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Rejected Comment: There's a Jenkins plugin to parse/monitor warnings, so this will be done as part of that setup now. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 09:24:30 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 16:24:30 -0000 Subject: [Bro-Dev] #838: topic/dnthayer/broctl-doc-fixes Message-ID: <050.07f6114e2ec257c19822a06d155f2e88@tracker.bro-ids.org> #838: topic/dnthayer/broctl-doc-fixes ---------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Keywords: | ---------------------------+------------------------ This branch fixes the broctl README to better explain which scripts relevant to site-specific customization get loaded (and in what order) when using broctl. Also removed description of several features that do not seem to work. The actual behavior is: "worker-1.local.bro" is not automatically loaded, there is no example policy in local-manager.bro, local-manager.bro and local-worker.bro do not automatically load local.bro, and proxies do not automatically load local-worker.bro. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 09:25:30 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 16:25:30 -0000 Subject: [Bro-Dev] #837: broctl load order incorrect In-Reply-To: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> References: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> Message-ID: <061.bcb94b4e8c9a90c2801780cb5652079b@tracker.bro-ids.org> #837: broctl load order incorrect -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by dnthayer): I've filed ticket 838 to address the documentation aspect of this issue. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 13:22:08 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 20:22:08 -0000 Subject: [Bro-Dev] #839: topic/dnthayer/load-balancing Message-ID: <050.21555e4bd2ef85158ed0085fa255a574@tracker.bro-ids.org> #839: topic/dnthayer/load-balancing ---------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Keywords: | ---------------------------+------------------------ This branch adds improvements to load-balancing support in broctl. Instead of adding a separate worker entry in node.cfg for each Bro worker process on each worker host, with this branch it is possible to just specify the number of worker processes on each host. This branch adds three new keywords to the node.cfg file (to be used with worker entries): lb_procs (specifies number of workers on a host), lb_method (specifies what type of load balancing to use: pf_ring, myricom, or interfaces), and lb_interfaces (used only with "lb_method=interfaces" to specify which interfaces to load-balance on). Two new broctl plugins (these operate automatically and the user doesn't need to be aware of them) are added to set the appropriate environment variables when either PF_RING or myricom load-balancing is being used. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 13:37:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 20:37:02 -0000 Subject: [Bro-Dev] #840: Merge topic/bernhard/input-crash-search Message-ID: <048.491f92b6c7214116c74620e8c554b356@tracker.bro-ids.org> #840: Merge topic/bernhard/input-crash-search ---------------------------+------------------------ Reporter: amannb | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ Branch sets the available parts of the frontend names before a thread is started, making debugging a little bit easier. It also adds errno output for a few pthread functions. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 13:38:32 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 20:38:32 -0000 Subject: [Bro-Dev] #841: Merge topic/bernhard/reader-info Message-ID: <048.8636b6a10bf644f44d49580659807837@tracker.bro-ids.org> #841: Merge topic/bernhard/reader-info ---------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ Branch adds the same configuration interface to the input framework that is exposed by the logging framework. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 13:38:58 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 20:38:58 -0000 Subject: [Bro-Dev] #841: Merge topic/bernhard/reader-info In-Reply-To: <048.8636b6a10bf644f44d49580659807837@tracker.bro-ids.org> References: <048.8636b6a10bf644f44d49580659807837@tracker.bro-ids.org> Message-ID: <063.13fb2d746617ba9209a33f0d2394e048@tracker.bro-ids.org> #841: Merge topic/bernhard/reader-info ----------------------------+------------------------ Reporter: amannb | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by amannb): * type: Problem => Merge Request -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 14:24:16 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 21:24:16 -0000 Subject: [Bro-Dev] #837: broctl load order incorrect In-Reply-To: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> References: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> Message-ID: <061.690b5d1e34559b430f8447338ee5e688@tracker.bro-ids.org> #837: broctl load order incorrect -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by robin): On Thu, Jun 28, 2012 at 22:45 -0000, you wrote: > Alternatively, how about we just remove those broctl options > that clash with bro script variables? What other options would be affected by this? Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 14:25:01 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 21:25:01 -0000 Subject: [Bro-Dev] #836: Make reporter.log errors go to stderr when run from command-line In-Reply-To: <048.f55ab9255b4a29b56b8218bba951e661@tracker.bro-ids.org> References: <048.f55ab9255b4a29b56b8218bba951e661@tracker.bro-ids.org> Message-ID: <063.ca9beea350b8741672b18a2ded78e3ce@tracker.bro-ids.org> #836: Make reporter.log errors go to stderr when run from command-line ------------------------------+------------------------ Reporter: amannb | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Comment (by robin): Makes sense. Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 14:39:30 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 21:39:30 -0000 Subject: [Bro-Dev] #835: Porting Drop and Catch-n-release to 2.0 In-Reply-To: <049.54449caa3016b34f217c36247f756cfa@tracker.bro-ids.org> References: <049.54449caa3016b34f217c36247f756cfa@tracker.bro-ids.org> Message-ID: <064.129689043abb782eed7265ee6db5dc5b@tracker.bro-ids.org> #835: Porting Drop and Catch-n-release to 2.0 ------------------------------+------------------------ Reporter: aashish | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Comment (by robin): An adapted version of scan.bro is already in the contributed scripts repo (http://git.bro-ids.org/bro-scripts.git/tree). However, the plan is for that and also drop.bro to be superseded by new upcoming frameworks so I don't think we should merge this in at this time. But we can add the scripts to the contrib repo as well. Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From jsiwek at illinois.edu Mon Jul 2 15:11:56 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Mon, 2 Jul 2012 22:11:56 +0000 Subject: [Bro-Dev] UDP payload signatures Message-ID: Since there are now UDP payload signatures by default for Teredo/AYIYA DPD, we had talked about checking out the potential/necessity for optimizing those signatures to only check for matches on first packets of a connection. I don't think it's worth doing now because (1) the default settings only do matching on a connection for the first 1K payload and (2) the internals don't seem to support such an option that well because, internally, multiple patterns get compiled together into a DFA to check matching and the interface to it is geared towards checking if any pattern was matched, not checking if a given pattern didn't match. So does it sound reasonable to leave out this feature? Unrelated to that, I was checking how UDP payload patterns were actually matched and found unexpected behavior. The docs say: "Regular expressions are implicitly anchored, i.e., they work as if prefixed with the ^ operator. For reassembled TCP connections, they are anchored at the first byte of the payload stream. For all other connections, they are anchored at the first payload byte of each packet. To match at arbitrary positions, you can prefix the regular expression with .*, as done in the examples above." But for a UDP connection made up of 2 packets with payloads "XXXX'" and then "YYYY", I still need the ".*" prefix to match on the 2nd: signature yyyy { ip-proto = udp payload /.*YYYY/ event "Found YYYY" } Changing the pattern to /YYYY/ or /^YYYY/ results in no match (but does match if I flip order of packets). Is the bug in the docs or the code? Jon From bro at tracker.bro-ids.org Mon Jul 2 15:23:27 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 22:23:27 -0000 Subject: [Bro-Dev] #841: Merge topic/bernhard/reader-info In-Reply-To: <048.8636b6a10bf644f44d49580659807837@tracker.bro-ids.org> References: <048.8636b6a10bf644f44d49580659807837@tracker.bro-ids.org> Message-ID: <063.67838c812203f23d67e9b5dcca412c24@tracker.bro-ids.org> #841: Merge topic/bernhard/reader-info ----------------------------+------------------------ Reporter: amannb | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Merging with after 0e48fda6ffa0be4cec2d763305a1394e19b32778 gives me conflicts in the test base lines. Please merge fastpath into this branch. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 15:35:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 22:35:20 -0000 Subject: [Bro-Dev] #841: Merge topic/bernhard/reader-info In-Reply-To: <048.8636b6a10bf644f44d49580659807837@tracker.bro-ids.org> References: <048.8636b6a10bf644f44d49580659807837@tracker.bro-ids.org> Message-ID: <063.fcc9c8becf1cbf8e85437711d059f350@tracker.bro-ids.org> #841: Merge topic/bernhard/reader-info ----------------------------+------------------------ Reporter: amannb | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by amannb): Done -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 16:01:39 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 23:01:39 -0000 Subject: [Bro-Dev] #839: topic/dnthayer/load-balancing In-Reply-To: <050.21555e4bd2ef85158ed0085fa255a574@tracker.bro-ids.org> References: <050.21555e4bd2ef85158ed0085fa255a574@tracker.bro-ids.org> Message-ID: <065.3d943e9d7e92a94a450784bb41e497f0@tracker.bro-ids.org> #839: topic/dnthayer/load-balancing ----------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): I like using plugins for this. It would be nice eventually to move all of the code specific to a balancing type to the plugins (i.e., in config.py the checks for "pf_ring", etc; and the special casing of "interfaces"). But that needs further extension of the plugin API and can wait. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 16:02:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 02 Jul 2012 23:02:09 -0000 Subject: [Bro-Dev] #839: topic/dnthayer/load-balancing In-Reply-To: <050.21555e4bd2ef85158ed0085fa255a574@tracker.bro-ids.org> References: <050.21555e4bd2ef85158ed0085fa255a574@tracker.bro-ids.org> Message-ID: <065.eb62b14abf6036a63d647bc8919899bb@tracker.bro-ids.org> #839: topic/dnthayer/load-balancing ----------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Please add documentation for the load-balancing, including an example. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 17:07:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 00:07:56 -0000 Subject: [Bro-Dev] #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events In-Reply-To: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> References: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> Message-ID: <064.8614de68baaa1ff5eba3152d267d5e0e@tracker.bro-ids.org> #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events ----------------------------+------------------------ Reporter: aashish | Owner: robin Type: Merge Request | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [de6e5c951a9b1408eb1ef6eef33115449e8a12f8/bro]: {{{ #!CommitTicketReference repository="bro" revision="de6e5c951a9b1408eb1ef6eef33115449e8a12f8" Merge remote-tracking branch 'origin/topic/jsiwek/icmp6-ndp-options' * origin/topic/jsiwek/icmp6-ndp-options: Extract ICMPv6 NDP options and include in ICMP events (addresses #833). Closes #833. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 17:07:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 00:07:56 -0000 Subject: [Bro-Dev] #840: Merge topic/bernhard/input-crash-search In-Reply-To: <048.491f92b6c7214116c74620e8c554b356@tracker.bro-ids.org> References: <048.491f92b6c7214116c74620e8c554b356@tracker.bro-ids.org> Message-ID: <063.8004ac9434a3ab6002ea9be20e1d9743@tracker.bro-ids.org> #840: Merge topic/bernhard/input-crash-search ----------------------------+------------------------ Reporter: amannb | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [918330948208e43ccf9173811dbf310eb75bf5a9/bro]: {{{ #!CommitTicketReference repository="bro" revision="918330948208e43ccf9173811dbf310eb75bf5a9" Merge remote-tracking branch 'origin/topic/bernhard/input-crash-search' * origin/topic/bernhard/input-crash-search: for bug-searching: Closes #840 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 17:07:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 00:07:56 -0000 Subject: [Bro-Dev] #841: Merge topic/bernhard/reader-info In-Reply-To: <048.8636b6a10bf644f44d49580659807837@tracker.bro-ids.org> References: <048.8636b6a10bf644f44d49580659807837@tracker.bro-ids.org> Message-ID: <063.7093f6923dd3bdf021e11bf94447e686@tracker.bro-ids.org> #841: Merge topic/bernhard/reader-info ----------------------------+------------------------ Reporter: amannb | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [06d2fd52bd2ee1c28462c64908d13a60dae0af86/bro]: {{{ #!CommitTicketReference repository="bro" revision="06d2fd52bd2ee1c28462c64908d13a60dae0af86" Merge remote-tracking branch 'origin/topic/bernhard/reader-info' * origin/topic/bernhard/reader-info: fix small bug - now configuration actually is passed. add mode to readerinfo - no need to have it separately everywhere anymore. introduce reader-info struct analogous to writer-info. Introduce support for a table of key/value pairs with further configuration options, with the same userinterface as in the logging interface. make writer-info work when debugging is enabled Conflicts: testing/btest/Baseline/scripts.base.frameworks.input.event/out testing/btest/Baseline/scripts.base.frameworks.input.executeraw/out testing/btest/Baseline/scripts.base.frameworks.input.raw/out testing/btest/Baseline/scripts.base.frameworks.input.rereadraw/out testing/btest/Baseline/scripts.base.frameworks.input.tableevent/out Closes #841. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 17:07:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 00:07:56 -0000 Subject: [Bro-Dev] #834: Compile warning on RHEL 6.2 In-Reply-To: <046.fd396e56746c3541b2691ce86ee8cf2b@tracker.bro-ids.org> References: <046.fd396e56746c3541b2691ce86ee8cf2b@tracker.bro-ids.org> Message-ID: <061.2fde9a120e3bd1c8a5fd0de7b40a350f@tracker.bro-ids.org> #834: Compile warning on RHEL 6.2 ----------------------+------------------------ Reporter: seth | Owner: jsiwek Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------+------------------------ Comment (by robin): In [9f6cf1ad901bd5eef3801c87c419b43340ba4567/bro]: {{{ #!CommitTicketReference repository="bro" revision="9f6cf1ad901bd5eef3801c87c419b43340ba4567" Merge remote-tracking branch 'origin/fastpath' * origin/fastpath: Fix inconsistencies in random number generation. Updating input framework unit tests. Add front-end name to InitMessage from WriterFrontend to Backend. Small tweak to make test complete quicker. Drain events before terminating log/thread managers. Fix strict-aliasing warning in RemoteSerializer.cc (fixes #834). Fix typos in event documentation Fix typos in NEWS for Bro 2.1 beta }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 17:08:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 00:08:34 -0000 Subject: [Bro-Dev] #839: topic/dnthayer/load-balancing In-Reply-To: <050.21555e4bd2ef85158ed0085fa255a574@tracker.bro-ids.org> References: <050.21555e4bd2ef85158ed0085fa255a574@tracker.bro-ids.org> Message-ID: <065.1c14fdcc43cc53247baa7f1e41ab79e5@tracker.bro-ids.org> #839: topic/dnthayer/load-balancing ----------------------------+------------------------ Reporter: dnthayer | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [27c6e97619ddfd4edc987de7c081f92dbfc58148/broctl]: {{{ #!CommitTicketReference repository="broctl" revision="27c6e97619ddfd4edc987de7c081f92dbfc58148" Merge remote-tracking branch 'origin/topic/dnthayer/load-balancing' * origin/topic/dnthayer/load-balancing: Add error checks for load-balancing config Remove plugins that do nothing Set the load balancing env. vars at init time Add a numerical suffix to all worker node names Fix the setting of myricom env. variable Add support for "interfaces" load balancing Add setting environment variables for myricom Add ability to specify multiple workers on a node Fix runtime error and warning messages due to new plugins Checkpoint. Closes #839. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 2 17:08:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 00:08:34 -0000 Subject: [Bro-Dev] #838: topic/dnthayer/broctl-doc-fixes In-Reply-To: <050.07f6114e2ec257c19822a06d155f2e88@tracker.bro-ids.org> References: <050.07f6114e2ec257c19822a06d155f2e88@tracker.bro-ids.org> Message-ID: <065.c7cbfb339dace90dce319b70d635cbc5@tracker.bro-ids.org> #838: topic/dnthayer/broctl-doc-fixes ----------------------------+------------------------ Reporter: dnthayer | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [0f9497bed6ba25f19cd6143ee3693a6d1194ce22/broctl]: {{{ #!CommitTicketReference repository="broctl" revision="0f9497bed6ba25f19cd6143ee3693a6d1194ce22" Merge remote-tracking branch 'origin/topic/dnthayer/broctl-doc-fixes' * origin/topic/dnthayer/broctl-doc-fixes: Improve broctl README Closes #838. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Mon Jul 2 17:13:11 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 2 Jul 2012 17:13:11 -0700 Subject: [Bro-Dev] Feature freeze for 2.1 Beta Message-ID: <20120703001311.GB43789@icir.org> I've merged in the remaining items and think we're pretty much ready for a public beta of Bro 2.1. I'm planing to push that out later this week, probably on Thursday if no showstoppers come up. Until then, let's please all do more testing on different platforms, both with the test suite and live. Right now, all tests pass for me on my development machine. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Tue Jul 3 01:54:04 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 08:54:04 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro Message-ID: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro ---------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ The following short script brings Bro to a standstill on my system when encountering the first http request {{{ event bro_init() { local filter: Log::Filter = [$name="host-only", $include=set("host")]; Log::add_filter(HTTP::LOG, filter); } }}} When a path is added to the filter definition, everything works fine. However, the path is marked as optional in the framework. When running with an attached gdb, the following error is shown on the first http access: {{{ Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: 13 at address: 0x0000000000000000 [Switching to process 6370 thread 0x2503] 0x00000001003dec18 in threading::Value::~Value (this=0x30000000102ed731) at SerialTypes.cc:52 52 if ( (type == TYPE_ENUM || type == TYPE_STRING || type == TYPE_FILE || type == TYPE_FUNC) (gdb) bt #0 0x00000001003dec18 in threading::Value::~Value (this=0x30000000102ed731) at SerialTypes.cc:52 #1 0x00000001003f205c in logging::WriterBackend::DeleteVals (this=0x10224c800, num_writes=66, vals=0x102252400) at WriterBackend.cc:139 #2 0x00000001003f227c in logging::WriterBackend::Write (this=0x10224c800, arg_num_fields=26, num_writes=66, vals=0x102252400) at WriterBackend.cc:209 #3 0x00000001003f789e in logging::WriteMessage::Process (this=0x104a13d30) at WriterFrontend.cc:60 #4 0x00000001003d477a in threading::MsgThread::Run (this=0x10224c800) at MsgThread.cc:302 #5 0x00000001003cfa57 in threading::BasicThread::launcher (arg=0x10224c800) at BasicThread.cc:170 #6 0x00007fff87f738bf in _pthread_start () #7 0x00007fff87f76b75 in thread_start () }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 3 07:00:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 14:00:06 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro In-Reply-To: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> References: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> Message-ID: <063.a3df642bbd9dd61b422f629ded81ed2a@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by robin): * priority: Normal => High * milestone: => Bro2.1 -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Tue Jul 3 07:48:54 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 3 Jul 2012 07:48:54 -0700 Subject: [Bro-Dev] UDP payload signatures In-Reply-To: References: Message-ID: <20120703144854.GH75784@icir.org> On Mon, Jul 02, 2012 at 22:11 +0000, you wrote: > payload and (2) the internals don't seem to support such an option > that well because, internally, multiple patterns get compiled together > into a DFA to check matching and the interface to it is geared towards > checking if any pattern was matched, not checking if a given pattern > didn't match. That's indeed something hard to get around, and we wouldn't change that. The performance savings would only kick in later (there's potentially more logic that triggers upon a regexp match). However, it's hard to say if that would change much, in particular with the 1K buffer as you say. So yes, assuming nobody is seeing signficant performance impact with the recent changes (which I haven't in my tests on traces), we can leave things as they are right now. As a test, we could create something like a "worst-case trace" that only has traffic of the kind relevent here and measure if the signature matching makes a noticable difference. > "Regular expressions are implicitly anchored, i.e., they work as if > prefixed with the ^ operator. For reassembled TCP connections, they > are anchored at the first byte of the payload stream. For all other > connections, they are anchored at the first payload byte of each > packet. To match at arbitrary positions, you can prefix the regular > expression with .*, as done in the examples above." This is indeed the intended behaviour. > Changing the pattern to /YYYY/ or /^YYYY/ results in no match (but > does match if I flip order of packets). Is the bug in the docs or the > code? That looks like a bug in the code. Also reminds me that we should really have unit tests for the signature engine ... Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Tue Jul 3 08:18:59 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 15:18:59 -0000 Subject: [Bro-Dev] #843: Swig Error Message Message-ID: <052.bbbed534a2d74c8c0fe5c7743b6acb86@tracker.bro-ids.org> #843: Swig Error Message ------------------------+-------------------------- Reporter: grigorescu | Type: Problem Status: new | Priority: Low Milestone: | Component: pysubnettree Version: git/master | Keywords: ------------------------+-------------------------- Minor error message issue: The following error message could be a bit more clear: {{{ -- Could NOT find SWIG (missing: SWIG_EXECUTABLE SWIG_DIR) -- Found PythonInterp: /usr/bin/python (found version "2.4.2") -- Found PythonLibs: /usr/lib/libpython2.4.so (found version "2.4.2") -- Found PythonDev: /usr/include/python2.4 Found swig version: Found python version: 2.4.2 CMake Error at aux/broctl/aux/pysubnettree/CMakeLists.txt:18 (message): Swig versions less than 1.3.30 are incompatible with Python versions greater than or equal to 2.5, upgrading your swig installation is recommended. }}} The information about Swig < 1.3.30 and Python >= 2.5 really doesn't apply here. The issue is that Swig isn't installed. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 3 08:39:24 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 15:39:24 -0000 Subject: [Bro-Dev] #843: Swig Error Message In-Reply-To: <052.bbbed534a2d74c8c0fe5c7743b6acb86@tracker.bro-ids.org> References: <052.bbbed534a2d74c8c0fe5c7743b6acb86@tracker.bro-ids.org> Message-ID: <067.cfef8e910e16d54e115eebdfdc206bed@tracker.bro-ids.org> #843: Swig Error Message ---------------------------+------------------------ Reporter: grigorescu | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.1 Component: pysubnettree | Version: git/master Resolution: | Keywords: ---------------------------+------------------------ Changes (by robin): * milestone: => Bro2.1 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 3 09:59:22 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 16:59:22 -0000 Subject: [Bro-Dev] #843: Swig Error Message In-Reply-To: <052.bbbed534a2d74c8c0fe5c7743b6acb86@tracker.bro-ids.org> References: <052.bbbed534a2d74c8c0fe5c7743b6acb86@tracker.bro-ids.org> Message-ID: <067.e329806392d5686c7d0d85a08eddb348@tracker.bro-ids.org> #843: Swig Error Message ---------------------------+------------------------ Reporter: grigorescu | Owner: jsiwek Type: Problem | Status: closed Priority: Low | Milestone: Bro2.1 Component: pysubnettree | Version: git/master Resolution: fixed | Keywords: ---------------------------+------------------------ Changes (by jsiwek): * owner: => jsiwek * status: new => closed * resolution: => fixed Comment: In [7e016f0b94c80e15392ab931cf4b9a625060ffa3/pysubnettree]: {{{ #!CommitTicketReference repository="pysubnettree" revision="7e016f0b94c80e15392ab931cf4b9a625060ffa3" Improve check for swig/python version incompatibility (fixes #843). The error message for swig/python version incompatibilities should not appear for the cases where swig or python was not installed (or where their versions could not be found/set by CMake). }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 3 11:15:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 18:15:52 -0000 Subject: [Bro-Dev] #837: broctl load order incorrect In-Reply-To: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> References: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> Message-ID: <061.a8bc4685c7e02ba8be1d12d3f4c80e79@tracker.bro-ids.org> #837: broctl load order incorrect -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by dnthayer): Replying to [comment:5 robin]: > On Thu, Jun 28, 2012 at 22:45 -0000, you wrote: > > > Alternatively, how about we just remove those broctl options > > that clash with bro script variables? > > What other options would be affected by this? > > Robin These broctl options (associated bro script variable also shown) would be affected: zoneid (Cluster::nodes) ipv6comm (Communication::listen_ipv6) timemachinehost (Cluster::nodes) timemachineport (Cluster::nodes) mailto (Notice::mail_dest) sendmail (Notice::sendmail) mailsubjectprefix (Notice::mail_subject_prefix) logrotationinterval (Log::default_rotation_interval) logdir (changes Cluster::log_dir, but this is commented-out with a TODO comment) -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 3 14:00:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 21:00:14 -0000 Subject: [Bro-Dev] #837: broctl load order incorrect In-Reply-To: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> References: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> Message-ID: <061.03bbf0a0f7dd1065590b4f6885955d0e@tracker.bro-ids.org> #837: broctl load order incorrect -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by robin): On Tue, Jul 03, 2012 at 18:15 -0000, you wrote: > zoneid (Cluster::nodes) > ipv6comm (Communication::listen_ipv6) > timemachinehost (Cluster::nodes) > timemachineport (Cluster::nodes) > mailto (Notice::mail_dest) > sendmail (Notice::sendmail) > mailsubjectprefix (Notice::mail_subject_prefix) > logrotationinterval (Log::default_rotation_interval) > > logdir (changes Cluster::log_dir, but this is commented-out with a TODO > comment) Hmm ... There are some I'm reluctant to remove from BroControl, like mailto and mailsubjectprefix (and logdir, though I'm not sure why Bro needs to know about that). How about instead (1) allowing people to override them manually, but (2) giving a warning, like in "check", if they configure seomthing else than BroControl would set (with an option to suppress such warnings). Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 3 15:07:22 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 03 Jul 2012 22:07:22 -0000 Subject: [Bro-Dev] #844: UDP payload signature patterns don't match packet-wise Message-ID: <048.d46f01c364d671e22ed72b7671c2688d@tracker.bro-ids.org> #844: UDP payload signature patterns don't match packet-wise ---------------------+------------------------ Reporter: jsiwek | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ The docs say: {{{ Regular expressions are implicitly anchored, i.e., they work as if prefixed with the ^ operator. For reassembled TCP connections, they are anchored at the first byte of the payload stream. For all other connections, they are anchored at the first payload byte of each packet. To match at arbitrary positions, you can prefix the regular expression with .*, as done in the examples above. }}} But for a UDP connection made up of 2 packets with payloads "XXXX'" and then "YYYY", I still need the ".*" prefix to match on the 2nd: {{{ signature yyyy { ip-proto = udp payload /.*YYYY/ event "Found YYYY" } }}} Changing the pattern to `/YYYY/` or `/^YYYY/` results in no match (but does match if I flip order of packets). -- Ticket URL: Bro Tracker Bro Issue Tracker From jsiwek at illinois.edu Tue Jul 3 15:10:52 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Tue, 3 Jul 2012 22:10:52 +0000 Subject: [Bro-Dev] UDP payload signatures In-Reply-To: <20120703144854.GH75784@icir.org> References: <20120703144854.GH75784@icir.org> Message-ID: > As a test, we could create > something like a "worst-case trace" that only has traffic of the kind > relevent here and measure if the signature matching makes a noticable > difference. I did some tests with 2,5702,400 total 1-byte (\x58) payload UDP packets over 25,100 connections comprised of 1,024 packets each and the worst performance impact I saw was a +0.2% difference when adding the new UDP signatures. > That looks like a bug in the code. Also reminds me that we should > really have unit tests for the signature engine ... Just made a ticket for now: http://tracker.bro-ids.org/bro/ticket/844 Jon From robin at icir.org Tue Jul 3 17:01:12 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 3 Jul 2012 17:01:12 -0700 Subject: [Bro-Dev] UDP payload signatures In-Reply-To: References: <20120703144854.GH75784@icir.org> Message-ID: <20120704000112.GA94758@icir.org> On Tue, Jul 03, 2012 at 22:10 +0000, you wrote: > I did some tests with 2,5702,400 total 1-byte (\x58) payload UDP > packets over 25,100 connections comprised of 1,024 packets each and > the worst performance impact I saw was a +0.2% difference when adding > the new UDP signatures. Cool, that sounds good. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Wed Jul 4 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 4 Jul 2012 00:00:04 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207040700.q6470441010697@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 8dc1e41 | Daniel Thayer | 2012-07-03 | Fix minor typos in dataseries documentation [1] broctl | f332b9f | Daniel Thayer | 2012-07-03 | Update broctl docs [2] pysubnettree | 7e016f0 | Jon Siwek | 2012-07-03 | Improve check for swig/python version incompatibility (fixes #843). [3] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/8dc1e418761ce95964cfb66d6dc6128ce6ce2d90/bro [2] fastpath: http://tracker.bro-ids.org/bro/changeset/f332b9f69592e64d1f105cef7befb8e8d7d5ae9e/broctl [3] fastpath: http://tracker.bro-ids.org/bro/changeset/7e016f0b94c80e15392ab931cf4b9a625060ffa3/pysubnettree From noreply at bro-ids.org Thu Jul 5 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 5 Jul 2012 00:00:04 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207050700.q65704Oq023921@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 8dc1e41 | Daniel Thayer | 2012-07-03 | Fix minor typos in dataseries documentation [1] broctl | f332b9f | Daniel Thayer | 2012-07-03 | Update broctl docs [2] pysubnettree | 7e016f0 | Jon Siwek | 2012-07-03 | Improve check for swig/python version incompatibility (fixes #843). [3] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/8dc1e418761ce95964cfb66d6dc6128ce6ce2d90/bro [2] fastpath: http://tracker.bro-ids.org/bro/changeset/f332b9f69592e64d1f105cef7befb8e8d7d5ae9e/broctl [3] fastpath: http://tracker.bro-ids.org/bro/changeset/7e016f0b94c80e15392ab931cf4b9a625060ffa3/pysubnettree From bro at tracker.bro-ids.org Thu Jul 5 12:36:57 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 05 Jul 2012 19:36:57 -0000 Subject: [Bro-Dev] #843: Swig Error Message In-Reply-To: <052.bbbed534a2d74c8c0fe5c7743b6acb86@tracker.bro-ids.org> References: <052.bbbed534a2d74c8c0fe5c7743b6acb86@tracker.bro-ids.org> Message-ID: <067.5a634cc544b10ef7b4e9dec785af2937@tracker.bro-ids.org> #843: Swig Error Message ---------------------------+------------------------ Reporter: grigorescu | Owner: jsiwek Type: Problem | Status: closed Priority: Low | Milestone: Bro2.1 Component: pysubnettree | Version: git/master Resolution: fixed | Keywords: ---------------------------+------------------------ Comment (by robin): In [a44c4b5a40a3d29e8d4cdeb787c000099559e54d/pysubnettree]: {{{ #!CommitTicketReference repository="pysubnettree" revision="a44c4b5a40a3d29e8d4cdeb787c000099559e54d" Merge remote-tracking branch 'origin/fastpath' * origin/fastpath: Improve check for swig/python version incompatibility (fixes #843). }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Jul 6 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 6 Jul 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207060700.q66702JH020895@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 1b8673f | Daniel Thayer | 2012-07-05 | Remove a non-portable test case [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/1b8673f4b2622a2aec73ca148aab88bde834eb3b/bro From bro at tracker.bro-ids.org Fri Jul 6 09:22:50 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 06 Jul 2012 16:22:50 -0000 Subject: [Bro-Dev] #845: PF_RING+DNA Message-ID: <050.33eaa70484a10db210df057a0be688c7@tracker.bro-ids.org> #845: PF_RING+DNA -----------------------------+------------------------ Reporter: dnthayer | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Component: BroControl | Version: git/master Keywords: | -----------------------------+------------------------ This is a feature that didn't make it into 2.1-beta. The idea is to have a broctl plugin that has a pre-start hook to automatically run this on each worker host: pfdnacluster_master -i dna0 -c 21 -n A worker entry in node.cfg would look something like this: [worker-1] type=worker host=host1 interface=dna0 lb_procs=4 lb_method=pf_ring_dna -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 9 09:04:42 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 09 Jul 2012 16:04:42 -0000 Subject: [Bro-Dev] #846: Tests Failures Message-ID: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> #846: Tests Failures ---------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ The is collected from mails: {{{ core.checksums ? failed [Gilbert, Matthias; non-determinisitc] bifs.system [Gilbert; should should be fixed by now, please double-check] istate.bro-ipv6-socket ? failed [Fails if IPv6 connectivity not available (fw in this case); can we test for that somehow? Otherwise, fine to leave as it is for now.) istate.broccoli-ipv6-socket ? failed [Same] scripts.base.protocols.smtp.basic [Matthias; with clang] scripts.base.frameworks.logging.rotate-custom [Matthias; with clang] core.dns-init [Adam; when using dnscrypt from OpenDNS] }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 9 09:39:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 09 Jul 2012 16:39:10 -0000 Subject: [Bro-Dev] #846: Tests Failures In-Reply-To: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> References: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> Message-ID: <062.9868a116a00a0faeee37c5f896f4c6af@tracker.bro-ids.org> #846: Tests Failures ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): One more: rerunning the prelim beta tar ball on my laptop, all tests passed except one: {{{ core.disable-mobile-ipv6 ... failed % 'btest-diff weird.log' failed unexpectedly (exit code 1) % cat .diag == File =============================== == Diff =============================== --- /tmp/test-diff.75669.weird.log.baseline.tmp 2012-07-09 16:04:46.000000000 +0000 +++ /tmp/test-diff.75669.weird.log.tmp 2012-07-09 16:04:46.000000000 +0000 @@ -1,8 +0,0 @@ -#separator \x09 -#set_separator , -#empty_field (empty) -#unset_field - -#path weird -#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer -#types time string addr port addr port string string bool string -XXXXXXXXXX.XXXXXX - - - - - unknown_protocol_135 - F bro ======================================= % cat .stderr }}} I've never seen that one fail before? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 9 13:54:08 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 09 Jul 2012 20:54:08 -0000 Subject: [Bro-Dev] #846: Tests Failures In-Reply-To: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> References: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> Message-ID: <062.8a4275ca19fcd30925b67fc3900430d7@tracker.bro-ids.org> #846: Tests Failures ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by amannb): bifs.system is fixed, at least for the test system where I found the problem. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 10 09:17:04 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 10 Jul 2012 16:17:04 -0000 Subject: [Bro-Dev] #846: Tests Failures In-Reply-To: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> References: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> Message-ID: <062.0b707c2a6adf7a6a1f1f25b5dd980ff0@tracker.bro-ids.org> #846: Tests Failures ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [c0bbd78ee1c856ea62687fe3f10d867e8dd760c4/bro]: {{{ #!CommitTicketReference repository="bro" revision="c0bbd78ee1c856ea62687fe3f10d867e8dd760c4" Fix segfault when there's an error/timeout resolving DNS requests. Addresses #846. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 10 09:45:31 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 10 Jul 2012 16:45:31 -0000 Subject: [Bro-Dev] #846: Tests Failures In-Reply-To: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> References: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> Message-ID: <062.9cbb6811809465bef90cf8ed486a5050@tracker.bro-ids.org> #846: Tests Failures ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): > {{{ > istate.bro-ipv6-socket ? failed [Fails if IPv6 connectivity not available (fw in this case); can we test for that somehow? Otherwise, fine to leave as it is for now.) > istate.broccoli-ipv6-socket ? failed [Same] > }}} Yeah, I don't think that's a big deal, either. They could probably extend their `@TEST-REQUIRES` to check if netcat is available and can do a connection over `::1`, but maybe it's actually more informative for the test to fail than to just skip. > {{{ > core.dns-init [Adam; when using dnscrypt from OpenDNS] > }}} The segfault is at least fixed by the commit in comment:3, but the test may still fail in this case, at least with the current OS X DNSCrypt client, because the local proxy is giving DNS responses with extra data at the end which causes a call to `ns_initparse()` in `nb_dns.c` to return -1 and set `errno` to `EMSGSIZE`, giving us warnings like: {{{ NB-DNS error in DNS_Mgr::WaitForReplies (ns_initparse(): Message too long) }}} I thought about trying to handle this special case by also checking for whether `ns_initparse()` was able to parse at least one answer section before failing and then just go forward with that, but it didn't seem like a great idea (I'm not sure it's "right" to trust things in the returned `ns_msg` buffer when there's an error, I couldn't find examples in the wild of any code bases doing that). But in the latest https://github.com/opendns/dnscrypt-proxy, the proxy is at least not returning these DNS responses with extra data at the end, so it works fine with Bro. > {{{ > core.disable-mobile-ipv6 ... failed > }}} > I've never seen that one fail before? Is that persistent or transient for you? I think it probably is due to the same issue causing the following group of failures: > {{{ > core.checksums ? failed [Gilbert, Matthias; non-determinisitc] > scripts.base.protocols.smtp.basic [Matthias; with clang] > scripts.base.frameworks.logging.rotate-custom [Matthias; with clang] > }}} I don't think Clang is a culprit in any of these, I was able to get an occasional failure using GCC on OS X. For any given test, it seemed like I could run it hundreds of times serially without getting a failure, but when increasing the load on the system (e.g. running more tests in parallel), the chances of getting a failure also increased. And the failures all looked like they were due to not writing out a log file, so my hunch is it's a general issue with the new threaded logging and not anything particular to these specific tests. Any idea what to start looking at? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 10 13:14:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 10 Jul 2012 20:14:20 -0000 Subject: [Bro-Dev] #847: binpac::SOCKS::SOCKS5_Address::Parse Assertion Error Message-ID: <052.50a568914ac210fa9d0aaf11555b1657@tracker.bro-ids.org> #847: binpac::SOCKS::SOCKS5_Address::Parse Assertion Error ------------------------+--------------------- Reporter: grigorescu | Type: Problem Status: new | Priority: High Milestone: Bro2.1 | Component: BinPAC Version: git/master | Keywords: socks ------------------------+--------------------- Running git master, had a worker crash with the following error: {{{ bro: bro/build/src/socks_pac.cc:848: int binpac::SOCKS::SOCKS5_Address::Parse(const binpac::uint8*, const binpac::uint8*): Assertion `t_dataptr_after_addr <= t_end_of_data' failed. /usr/local/bro/share/broctl/scripts/run-bro: line 60: 7094 Aborted (core dumped) nohup $mybro $@ }}} The version of code that I have is up to: commit acb6c0a0a5cf45079252fc6dfb5fef93df897fe2 Merge: 3fcece4 d26a96b Author: Robin Sommer Date: Mon Jul 2 16:59:56 2012 -0700 Unfortunately, I don't have a backtrace. Please let me know if I can provide any other information. --Vlad -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 10 13:18:43 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 10 Jul 2012 20:18:43 -0000 Subject: [Bro-Dev] #847: binpac::SOCKS::SOCKS5_Address::Parse Assertion Error In-Reply-To: <052.50a568914ac210fa9d0aaf11555b1657@tracker.bro-ids.org> References: <052.50a568914ac210fa9d0aaf11555b1657@tracker.bro-ids.org> Message-ID: <067.7a65140357206c25194c39eff79630ed@tracker.bro-ids.org> #847: binpac::SOCKS::SOCKS5_Address::Parse Assertion Error -------------------------+------------------------ Reporter: grigorescu | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: BinPAC | Version: git/master Resolution: | Keywords: socks -------------------------+------------------------ Comment (by seth): > bro: bro/build/src/socks_pac.cc:848: int > binpac::SOCKS::SOCKS5_Address::Parse(const binpac::uint8*, const > binpac::uint8*): Assertion `t_dataptr_after_addr <= t_end_of_data' failed. > /usr/local/bro/share/broctl/scripts/run-bro: line 60: 7094 Aborted > (core dumped) nohup $mybro $@ I know what this problem is and I'm planning on working on it tonight. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 10 13:21:54 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 10 Jul 2012 20:21:54 -0000 Subject: [Bro-Dev] #847: binpac::SOCKS::SOCKS5_Address::Parse Assertion Error In-Reply-To: <052.50a568914ac210fa9d0aaf11555b1657@tracker.bro-ids.org> References: <052.50a568914ac210fa9d0aaf11555b1657@tracker.bro-ids.org> Message-ID: <067.617a181ce9984a5cc7c5ecaead720a9a@tracker.bro-ids.org> #847: binpac::SOCKS::SOCKS5_Address::Parse Assertion Error -------------------------+------------------------ Reporter: grigorescu | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: BinPAC | Version: git/master Resolution: | Keywords: socks -------------------------+------------------------ Comment (by grigorescu): I lied. Backtrace attached, for the sake of completeness. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 10 14:02:57 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 10 Jul 2012 21:02:57 -0000 Subject: [Bro-Dev] #846: Tests Failures In-Reply-To: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> References: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> Message-ID: <062.aa9b889375f51204dd7b195bbd0b7e17@tracker.bro-ids.org> #846: Tests Failures ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): On Tue, Jul 10, 2012 at 16:45 -0000, you wrote: > Yeah, I don't think that's a big deal, either. Ack. > The segfault is at least fixed by the commit in comment:3, > but the test may still fail in this case, Ok. I'm starting to think though that we need a list of reasons which tests are expected to fail under what circumentances (and how to identify that that is indeed the case). As we're getting more of them, it's becoming hard to track (another example: no libgeop triggers additional reporter messages). > parallel), the chances of getting a failure also increased. And the > failures all looked like they were due to not writing out a log file, so > my hunch is it's a general issue with the new threaded logging and not > anything particular to these specific tests. Any idea what to start > looking at? That would indeed make a good explanation. That must be a race condition in the threat termination code. When that happens, do you see any reporter messages about threads that didn't finish in time? There's code that aborts a thread forcefully if it doesn't terminate sufficiently quickly by itself (with "quickly" being a hardcoded value in MsgThread::OnStop(); not great admittedly ...) Generally, threading::Manager::Terminate() is the entry point for the thread shutdown. The output of -Bthreading would be interesting to see in for a case where output doesn't appear. Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 10 14:42:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 10 Jul 2012 21:42:03 -0000 Subject: [Bro-Dev] #846: Tests Failures In-Reply-To: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> References: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> Message-ID: <062.8ed072024e07ec96c811511eb2cafdee@tracker.bro-ids.org> #846: Tests Failures ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [c4b6499d858c5799845f9f312931bc845e506e05/bro]: {{{ #!CommitTicketReference repository="bro" revision="c4b6499d858c5799845f9f312931bc845e506e05" Add sorting canonifier to rotate-custom unit test. (addresses #846) The output on stderr for this test is the results of many backgrounded "echo" commands, one for each rotation, so the order in which they occur may be subject to OS process scheduling and can't be relied upon }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 10 16:51:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 10 Jul 2012 23:51:48 -0000 Subject: [Bro-Dev] #846: Tests Failures In-Reply-To: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> References: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> Message-ID: <062.e186123debb8eab0ebe0acfb4f9958a4@tracker.bro-ids.org> #846: Tests Failures ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): In [e3f6a467a4c093aedfd15501d46c31730927ace4/bro]: {{{ #!CommitTicketReference repository="bro" revision="e3f6a467a4c093aedfd15501d46c31730927ace4" Merge remote-tracking branch 'origin/fastpath' * origin/fastpath: Add sorting canonifier to rotate-custom unit test. (addresses #846) Fix compiler warnings Fix segfault when there's an error/timeout resolving DNS requests. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From guanhua.tu at gmail.com Wed Jul 11 07:26:29 2012 From: guanhua.tu at gmail.com (Scott Guan-Hua Tu) Date: Wed, 11 Jul 2012 10:26:29 -0400 Subject: [Bro-Dev] One question about connection between Broccoli-Python and Bro Message-ID: Hi, I encountered a problem to connect Broccoli-Python to Bro. Specifically speaking, I can not connect Broccoli-Python to Bro when Bro is processing tcpdump file as follows. "Bro -r test.pcap" However, I am able to connect Broccoli-Python to Bro when Bro is monitoring network interface as follows. "Bro -i eth0" My Broccoli-Python is able to send/receive to/from Bro in the case above. Is there anyone know the restriction to use Broccoli-Python? How can I get notification from Bro when it is processing tcpdump file? Thanks a lot. Scott -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120711/9be65499/attachment.html From seth at icir.org Wed Jul 11 08:02:27 2012 From: seth at icir.org (Seth Hall) Date: Wed, 11 Jul 2012 11:02:27 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Fixing memory leak. (b31ef8c) In-Reply-To: <201207111459.q6BEx2jP018776@bro-ids.icir.org> References: <201207111459.q6BEx2jP018776@bro-ids.icir.org> Message-ID: <0514F940-E32C-4E0B-B62C-2CC04F07EED1@icir.org> On Jul 11, 2012, at 10:59 AM, Seth Hall wrote: > Fixing memory leak. This was a leak in my code that was exposed when Daniel fixed a compiler warning in the ssl analyzer. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Wed Jul 11 08:19:46 2012 From: seth at icir.org (Seth Hall) Date: Wed, 11 Jul 2012 11:19:46 -0400 Subject: [Bro-Dev] One question about connection between Broccoli-Python and Bro In-Reply-To: References: Message-ID: On Jul 11, 2012, at 10:26 AM, Scott Guan-Hua Tu wrote: > Is there anyone know the restriction to use Broccoli-Python? > How can I get notification from Bro when it is processing tcpdump file? Bro's event interface is primarily for realtime analysis which you get from sniffing traffic on an interface. If you are reading a tracefile, "real time" typically proceeds much faster than the wall clock and since Bro's communication protocol was originally intended for multiple Bro instances to communicate with each other things could become pretty badly confused if different Bro processes think the time is different. Now, I agree with you that it seems like a very reasonable request for broccoli to be allowed to connect even when reading trace files (i've probably requested that feature myself at some point), but I'll leave it up to Robin or someone else to see if that's something that we could reasonably do (allow communication with broccoli even if reading trace files). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Wed Jul 11 08:39:37 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 11 Jul 2012 08:39:37 -0700 Subject: [Bro-Dev] One question about connection between Broccoli-Python and Bro In-Reply-To: References: Message-ID: <20120711153937.GT67997@icir.org> On Wed, Jul 11, 2012 at 11:19 -0400, you wrote: > Bro's event interface is primarily for realtime analysis which you get > from sniffing traffic on an interface. If you are reading a > tracefile, "real time" typically proceeds much faster than the wall > clock There's actually a way to make it work: if you start Bro with the option '--pseudo-realtime' it will enable the communication system even when reading a trace. There's a catch though: it will now "simulate" real-time by delaying processing of the trace according to the timestamps in there, i.e., if you have a trace covering an interval T, it will take Bro the same time T to process the trace offline. As that's however often inconvinient, there's one more knob: you can give the option an integer factor (e.g., --pseudo-realtime=10), and it will then scale up the time accordingly, i.e., process the trace 10 times as fast as real-time (i.e., M/10). By using a suitable large factor, you may get the effect you're looking for. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Wed Jul 11 13:32:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 11 Jul 2012 20:32:21 -0000 Subject: [Bro-Dev] #848: Crashes in sub-threads do not propagate to main Bro Message-ID: <048.f7796931648451485812b4da6626efe2@tracker.bro-ids.org> #848: Crashes in sub-threads do not propagate to main Bro ---------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ At the moment it is possible that a (logging and/or input) thread crashes without the user being notified of this in any way. For example, if one adds something like {{{ int *a = 0; *a = 5; }}} to the beginning of Ascii::DoWrite and starts up Bro, all logfiles are generated, no data is written into them and Bro just sits there without doing anything. Attaching a gdb shows the null-pointer exception. The main Bro thread probably should probably just crash in case there is a null-pointer violation or a similar exception in one of the child threads. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 14:18:11 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 11 Jul 2012 21:18:11 -0000 Subject: [Bro-Dev] #847: binpac::SOCKS::SOCKS5_Address::Parse Assertion Error In-Reply-To: <052.50a568914ac210fa9d0aaf11555b1657@tracker.bro-ids.org> References: <052.50a568914ac210fa9d0aaf11555b1657@tracker.bro-ids.org> Message-ID: <067.2f6f01c8c9414be179c4966bfacb4f96@tracker.bro-ids.org> #847: binpac::SOCKS::SOCKS5_Address::Parse Assertion Error ----------------------------+------------------------ Reporter: grigorescu | Owner: robin Type: Merge Request | Status: assigned Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: socks ----------------------------+------------------------ Changes (by seth): * owner: => robin * status: new => assigned * component: BinPAC => Bro * type: Problem => Merge Request Comment: I think this should be fixed by my changes in topic/seth/socks-fixes. It's hard to tell without an example tracefile, but the changes are something I should have done in the first place anyway. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 15:54:50 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 11 Jul 2012 22:54:50 -0000 Subject: [Bro-Dev] #762: Add eof line to logfiles In-Reply-To: <048.b69152fe8b8e0e80e5715b13977d82cb@tracker.bro-ids.org> References: <048.b69152fe8b8e0e80e5715b13977d82cb@tracker.bro-ids.org> Message-ID: <063.8cab66d8ef3ebda87a292455556609d3@tracker.bro-ids.org> #762: Add eof line to logfiles ----------------------------+------------------------ Reporter: amannb | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by amannb): * type: Feature Request => Merge Request Comment: Patch for this is in branch topic/bernhard/logging-ascii-eof (Does not update the baseline of external tests because I was not entirely sure how to do this) -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 16:03:58 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 11 Jul 2012 23:03:58 -0000 Subject: [Bro-Dev] #762: Add eof line to logfiles In-Reply-To: <048.b69152fe8b8e0e80e5715b13977d82cb@tracker.bro-ids.org> References: <048.b69152fe8b8e0e80e5715b13977d82cb@tracker.bro-ids.org> Message-ID: <063.ff352d0564d75bcbec55dbde9ba5a91e@tracker.bro-ids.org> #762: Add eof line to logfiles ------------------------------+------------------------ Reporter: amannb | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Changes (by amannb): * type: Merge Request => Feature Request Comment: Ah, skip that, forgot to add the additional configuration optionw we talked about in the diskussion. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 16:22:53 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 11 Jul 2012 23:22:53 -0000 Subject: [Bro-Dev] #762: Add eof line to logfiles In-Reply-To: <048.b69152fe8b8e0e80e5715b13977d82cb@tracker.bro-ids.org> References: <048.b69152fe8b8e0e80e5715b13977d82cb@tracker.bro-ids.org> Message-ID: <063.fd75a1561062213f0ca02af38552279d@tracker.bro-ids.org> #762: Add eof line to logfiles ----------------------------+------------------------ Reporter: amannb | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by amannb): * type: Feature Request => Merge Request Comment: Second try. Now the option include_headers is renamed to include_format (as mentioned in the discussion back then) and the footer line is not printed in this case. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 19:59:40 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 02:59:40 -0000 Subject: [Bro-Dev] #848: Crashes in sub-threads do not propagate to main Bro In-Reply-To: <048.f7796931648451485812b4da6626efe2@tracker.bro-ids.org> References: <048.f7796931648451485812b4da6626efe2@tracker.bro-ids.org> Message-ID: <063.e1fdb277226a7588b2f369f117fe8eab@tracker.bro-ids.org> #848: Crashes in sub-threads do not propagate to main Bro ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): On Wed, Jul 11, 2012 at 20:32 -0000, you wrote: > The main Bro thread probably should probably just crash in case there is a > null-pointer violation or a similar exception in one of the child threads. I would have expected it to do that actually. Guess I need to read up on how signals are handled in the threaded setting ... Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:27:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:27:14 -0000 Subject: [Bro-Dev] #647: Extend HTTP analyzer to support multiply encoded content. (was: SDCH support) In-Reply-To: <046.95060f0c54e55e518c6e6817c50c1af5@tracker.bro-ids.org> References: <046.95060f0c54e55e518c6e6817c50c1af5@tracker.bro-ids.org> Message-ID: <061.66f8673f3cc9f2175910d45a0b7b25a9@tracker.bro-ids.org> #647: Extend HTTP analyzer to support multiply encoded content. ----------------------+---------------------- Reporter: seth | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ----------------------+---------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:27:44 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:27:44 -0000 Subject: [Bro-Dev] #816: Reworked PacketFilter framework In-Reply-To: <046.1cbe2b89832a98cbd82f94acb45002cf@tracker.bro-ids.org> References: <046.1cbe2b89832a98cbd82f94acb45002cf@tracker.bro-ids.org> Message-ID: <061.0f748980901dcbb9abfef67850787a95@tracker.bro-ids.org> #816: Reworked PacketFilter framework ---------------------+------------------------ Reporter: seth | Owner: seth Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: This is now coming in 2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:28:22 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:28:22 -0000 Subject: [Bro-Dev] #579: Syslog logging writer In-Reply-To: <046.4e6efad585e65a2ccb68427348524651@tracker.bro-ids.org> References: <046.4e6efad585e65a2ccb68427348524651@tracker.bro-ids.org> Message-ID: <061.8a25e1c0e0f462e59179437da7aa2997@tracker.bro-ids.org> #579: Syslog logging writer ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Not happening yet. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:28:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:28:56 -0000 Subject: [Bro-Dev] #741: Remove HTTP verbs from HTTP analyzer In-Reply-To: <046.33191738c5e4da46b75e4c33d1e6e42a@tracker.bro-ids.org> References: <046.33191738c5e4da46b75e4c33d1e6e42a@tracker.bro-ids.org> Message-ID: <061.492cdc7dae0ed6f1f91c158fe06a892a@tracker.bro-ids.org> #741: Remove HTTP verbs from HTTP analyzer ----------------------+-------------------- Reporter: seth | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ----------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:29:24 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:29:24 -0000 Subject: [Bro-Dev] #253: Can't bind to port 47760, Address already in use In-Reply-To: <056.21a044ffe4916939ea16cb28b2316505@tracker.bro-ids.org> References: <056.21a044ffe4916939ea16cb28b2316505@tracker.bro-ids.org> Message-ID: <071.3622dd7ee860e7b0b10a7f3b7de0ccc8@tracker.bro-ids.org> #253: Can't bind to port 47760, Address already in use ------------------------------+-------------------- Reporter: Tyler.Schoenke | Owner: robin Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: BroControl | Version: 1.5.2 Resolution: | Keywords: ------------------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:33:05 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:33:05 -0000 Subject: [Bro-Dev] #237: SSH connection states not correct In-Reply-To: <047.aa22f7b6ea44599cd97222bae31793f3@tracker.bro-ids.org> References: <047.aa22f7b6ea44599cd97222bae31793f3@tracker.bro-ids.org> Message-ID: <062.49c0903a0ef009a882989c31738fe34a@tracker.bro-ids.org> #237: SSH connection states not correct ---------------------------+-------------------- Reporter: robin | Owner: robin Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: 1.5.1 Resolution: Works for Me | Keywords: ---------------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Works for Me Comment: Just tested and can no longer reproduce this problem (with SSH::skip_processing_after_detection) in current 2.1 pre-beta. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:34:31 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:34:31 -0000 Subject: [Bro-Dev] #316: Bro integer type cleanup meta ticket In-Reply-To: <048.7cf1ce0a62f4bc35b77008e881ac7d60@tracker.bro-ids.org> References: <048.7cf1ce0a62f4bc35b77008e881ac7d60@tracker.bro-ids.org> Message-ID: <063.7a0802c98f1bf262fb8cb08138a20f92@tracker.bro-ids.org> #316: Bro integer type cleanup meta ticket ---------------------+---------------------- Reporter: gregor | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: inttypes ---------------------+---------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: At some point soon we're going to have to decide if we leave this as a formal task or just fix these problems as they arise. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:35:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:35:18 -0000 Subject: [Bro-Dev] #318: Use inttypes.h instead of home-made ifdefs In-Reply-To: <048.72e2a02b9b1688932f3025cc3b21bfbb@tracker.bro-ids.org> References: <048.72e2a02b9b1688932f3025cc3b21bfbb@tracker.bro-ids.org> Message-ID: <063.dd9c43ace5178152d4bbed5f4d090978@tracker.bro-ids.org> #318: Use inttypes.h instead of home-made ifdefs ---------------------+------------------------ Reporter: gregor | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: inttypes ---------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:35:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:35:48 -0000 Subject: [Bro-Dev] #320: Check for counters, length fields, etc. that can overflow and change to 64 bit In-Reply-To: <048.85c02dc81710813bab89e027ab272d66@tracker.bro-ids.org> References: <048.85c02dc81710813bab89e027ab272d66@tracker.bro-ids.org> Message-ID: <063.f246896003d41230519f3a68ee3a848f@tracker.bro-ids.org> #320: Check for counters, length fields, etc. that can overflow and change to 64 bit ---------------------+------------------------ Reporter: gregor | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: inttypes ---------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:37:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:37:02 -0000 Subject: [Bro-Dev] #353: Restore and improve IDMEF support In-Reply-To: <046.3a20146b674cb9679a0be4ea55b0c147@tracker.bro-ids.org> References: <046.3a20146b674cb9679a0be4ea55b0c147@tracker.bro-ids.org> Message-ID: <061.5621b2ee032228b6b60f5984429ef30e@tracker.bro-ids.org> #353: Restore and improve IDMEF support -----------------------+-------------------- Reporter: seth | Owner: Type: Task | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: Rejected | Keywords: -----------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Rejected Comment: I'm going to close this ticket for now. It doesn't serve much purpose and once IDMEF becomes more important for us the subject will come back up. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:37:26 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:37:26 -0000 Subject: [Bro-Dev] #389: Extend script level DNS to do different query classes In-Reply-To: <046.73b6016b4f21f90de0f60ffaf492631e@tracker.bro-ids.org> References: <046.73b6016b4f21f90de0f60ffaf492631e@tracker.bro-ids.org> Message-ID: <061.e500744f4d408c17cc9b0653e53272ad@tracker.bro-ids.org> #389: Extend script level DNS to do different query classes ------------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ------------------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Not happening yet. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:40:49 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:40:49 -0000 Subject: [Bro-Dev] #395: Cannot add global to module if it already exists in global namespace In-Reply-To: <048.ed7c07f7b808821a854fad6bf4ed9504@tracker.bro-ids.org> References: <048.ed7c07f7b808821a854fad6bf4ed9504@tracker.bro-ids.org> Message-ID: <063.eb28aacf33a797f32677272d326a6e36@tracker.bro-ids.org> #395: Cannot add global to module if it already exists in global namespace -------------------------------+------------------------ Reporter: gregor | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Feedback Missing | Keywords: -------------------------------+------------------------ Changes (by seth): * status: new => closed * resolution: => Feedback Missing Comment: I'm going to close this ticket for now, it gets into really hairy and undefined behavior and is unlikely to be encountered often. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:41:24 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:41:24 -0000 Subject: [Bro-Dev] #342: Add payload to ICMP analyzer In-Reply-To: <046.dc4cf3cae0b93ae2cca66efcf0ce60b4@tracker.bro-ids.org> References: <046.dc4cf3cae0b93ae2cca66efcf0ce60b4@tracker.bro-ids.org> Message-ID: <061.c9f84e4ed67ac230566a72cef64c2fd4@tracker.bro-ids.org> #342: Add payload to ICMP analyzer ---------------------+-------------------- Reporter: seth | Owner: Type: Patch | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: 1.5.2 Resolution: | Keywords: ---------------------+-------------------- Comment (by seth): Did this happen? I didn't track the ICMP changes closely. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:42:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:42:18 -0000 Subject: [Bro-Dev] #423: Additional dynamic init time pattern construction In-Reply-To: <046.c0d291f90db51e06469d074bd4b43b08@tracker.bro-ids.org> References: <046.c0d291f90db51e06469d074bd4b43b08@tracker.bro-ids.org> Message-ID: <061.018ac4d9642ab40c2b479faee7aa49ef@tracker.bro-ids.org> #423: Additional dynamic init time pattern construction ----------------------+---------------------- Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: language ----------------------+---------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Not happening yet. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:49:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:49:09 -0000 Subject: [Bro-Dev] #434: Fix secondary path In-Reply-To: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> References: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> Message-ID: <062.3fe295c453145402fd531e8428ebed56@tracker.bro-ids.org> #434: Fix secondary path ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Ah, I just reread large-conns.bro and I finally fully understand how it works. Can I ask for a concrete example of what you use it for? I don't know exactly the use cases where it would come in handy. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:50:13 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:50:13 -0000 Subject: [Bro-Dev] #437: Unify tables/set/vectors/records In-Reply-To: <047.c10b9b126ae075f31f818caebb8639e1@tracker.bro-ids.org> References: <047.c10b9b126ae075f31f818caebb8639e1@tracker.bro-ids.org> Message-ID: <062.87eae8c8dbf97fad23f97455f09e57ec@tracker.bro-ids.org> #437: Unify tables/set/vectors/records ---------------------+---------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: language ---------------------+---------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Unfortunately not happening yet. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:51:35 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:51:35 -0000 Subject: [Bro-Dev] #451: Remove DNS options for skipping auth/addl events In-Reply-To: <047.8ae817995be2255322e68a394e45df3e@tracker.bro-ids.org> References: <047.8ae817995be2255322e68a394e45df3e@tracker.bro-ids.org> Message-ID: <062.8f5059d267b106a485803fc42939158c@tracker.bro-ids.org> #451: Remove DNS options for skipping auth/addl events -----------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: Rejected | Keywords: -----------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Rejected Comment: Closing this ticket since it seems like we likely won't be removing those options any time soon. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:52:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:52:00 -0000 Subject: [Bro-Dev] #533: Support STARTTLS in various other protocols In-Reply-To: <046.eb31182326c73895e8a74c14690577fd@tracker.bro-ids.org> References: <046.eb31182326c73895e8a74c14690577fd@tracker.bro-ids.org> Message-ID: <061.b6c9fdaca42a31f53480f0e1a9d324a7@tracker.bro-ids.org> #533: Support STARTTLS in various other protocols ------------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ------------------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Yet another "not happening yet". -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:54:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:54:14 -0000 Subject: [Bro-Dev] #560: Child analyzer Init() problem In-Reply-To: <048.34d603b1c85589c490200b63ec47eb7f@tracker.bro-ids.org> References: <048.34d603b1c85589c490200b63ec47eb7f@tracker.bro-ids.org> Message-ID: <063.bf1ccc551b59156c00d415abae61ee7e@tracker.bro-ids.org> #560: Child analyzer Init() problem ----------------------+------------------------ Reporter: gregor | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:55:12 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:55:12 -0000 Subject: [Bro-Dev] #389: Extend script level DNS to do different query classes In-Reply-To: <046.73b6016b4f21f90de0f60ffaf492631e@tracker.bro-ids.org> References: <046.73b6016b4f21f90de0f60ffaf492631e@tracker.bro-ids.org> Message-ID: <061.68c5257a7fc970b7dca4126a80cb55ad@tracker.bro-ids.org> #389: Extend script level DNS to do different query classes ------------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ------------------------------+-------------------- Comment (by seth): Closing this ticket in light of ticket #584 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:55:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:55:20 -0000 Subject: [Bro-Dev] #389: Extend script level DNS to do different query classes In-Reply-To: <046.73b6016b4f21f90de0f60ffaf492631e@tracker.bro-ids.org> References: <046.73b6016b4f21f90de0f60ffaf492631e@tracker.bro-ids.org> Message-ID: <061.7083dc8a7f71ffcacf2560d29bf83d66@tracker.bro-ids.org> #389: Extend script level DNS to do different query classes ------------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: Duplicate | Keywords: ------------------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Duplicate -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:55:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:55:51 -0000 Subject: [Bro-Dev] #478: Move BinPAC docs over to new server In-Reply-To: <047.826126acda25dd38cf616630391698f4@tracker.bro-ids.org> References: <047.826126acda25dd38cf616630391698f4@tracker.bro-ids.org> Message-ID: <062.80ee9a6a54395a71f3a3101ac14d3ce0@tracker.bro-ids.org> #478: Move BinPAC docs over to new server -----------------------------+-------------------- Reporter: robin | Owner: seth Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Website / Wiki | Version: Resolution: | Keywords: -----------------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Bumping again. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:57:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:57:10 -0000 Subject: [Bro-Dev] #578: Add ICMPv6 support to Bro In-Reply-To: <048.1e6e618d9d8a166299c6c8582e9c8511@tracker.bro-ids.org> References: <048.1e6e618d9d8a166299c6c8582e9c8511@tracker.bro-ids.org> Message-ID: <063.c72cfc5a82aa3c0cbf0a14a120c3e97e@tracker.bro-ids.org> #578: Add ICMPv6 support to Bro ----------------------+------------------------ Reporter: gregor | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: IPv6 ----------------------+------------------------ Comment (by seth): Does the lack of script-level login prevent closure of this ticket? I would say it doesn't since we don't currently have any script level implementation for ICMP (non-v6) either. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 21:59:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 04:59:56 -0000 Subject: [Bro-Dev] #634: CouchDB writer In-Reply-To: <053.06c318cee769c34ae468c38f0621a66a@tracker.bro-ids.org> References: <053.06c318cee769c34ae468c38f0621a66a@tracker.bro-ids.org> Message-ID: <068.4786806f19b6a1d893b4e0e4a7ca16ce@tracker.bro-ids.org> #634: CouchDB writer --------------------------+-------------------- Reporter: jeff.baumes | Owner: Type: patch | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: --------------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Jeff, do you have any interest in updating this patch? It probably needs to be reworked quite a bit, but is not unreasonable considering how similar it should be to the elasticsearch plugin we're going to be including. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:00:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:00:46 -0000 Subject: [Bro-Dev] #672: Bring POP3 back into the distribution In-Reply-To: <050.6f055a004d2f3a5791fd7de74ec82190@tracker.bro-ids.org> References: <050.6f055a004d2f3a5791fd7de74ec82190@tracker.bro-ids.org> Message-ID: <065.84d8bb58656bf1b8e49598127306391a@tracker.bro-ids.org> #672: Bring POP3 back into the distribution -----------------------+------------------------ Reporter: matthias | Owner: seth Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Didn't happen. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:01:49 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:01:49 -0000 Subject: [Bro-Dev] #735: Clean up and merge the TCPStats analyzer In-Reply-To: <046.98d32efb2f183e28fd41325e04632c26@tracker.bro-ids.org> References: <046.98d32efb2f183e28fd41325e04632c26@tracker.bro-ids.org> Message-ID: <061.d53ce5155b5417e462bc3af6b95c5077@tracker.bro-ids.org> #735: Clean up and merge the TCPStats analyzer ---------------------+-------------------- Reporter: seth | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Bumping back, I haven't heard if this code is ready or not. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:03:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:03:52 -0000 Subject: [Bro-Dev] #827: MIME analyzer doesn't decode "encoded-word" encoding. In-Reply-To: <046.c53d86328568edbcd557ac89698567bb@tracker.bro-ids.org> References: <046.c53d86328568edbcd557ac89698567bb@tracker.bro-ids.org> Message-ID: <061.a4c79df569d2a44ed69ebfc6c87fc113@tracker.bro-ids.org> #827: MIME analyzer doesn't decode "encoded-word" encoding. ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: analyzer ----------------------+------------------------ Changes (by seth): * keywords: => analyzer * milestone: Bro2.1 => Comment: Removing milestone and adding the new "analyzer" keyword for work that needs to be done to analyzers. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:16:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:16:09 -0000 Subject: [Bro-Dev] #683: Some BiFs should return a vector instead of a set/table In-Reply-To: <050.1a279086ae98a29669599697952475ed@tracker.bro-ids.org> References: <050.1a279086ae98a29669599697952475ed@tracker.bro-ids.org> Message-ID: <065.d3f45578e80692d2d8852c04de0c9414@tracker.bro-ids.org> #683: Some BiFs should return a vector instead of a set/table -----------------------+------------------------ Reporter: matthias | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: language -----------------------+------------------------ Changes (by seth): * keywords: => language * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:16:50 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:16:50 -0000 Subject: [Bro-Dev] #747: Scheduled event misfire In-Reply-To: <046.a42f21c91392be1b926e37206f116085@tracker.bro-ids.org> References: <046.a42f21c91392be1b926e37206f116085@tracker.bro-ids.org> Message-ID: <061.ffbf1534eda2dd47593df86262a397fb@tracker.bro-ids.org> #747: Scheduled event misfire ----------------------+----------------- Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: Resolution: | Keywords: ----------------------+----------------- Changes (by seth): * milestone: Bro2.1 => Comment: I'm going to remove this from the milestone because it's still worthwhile and should be fixed, but it's really low priority. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:17:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:17:19 -0000 Subject: [Bro-Dev] #759: Increase test coverage In-Reply-To: <047.80b3470f0ca6911e07abe4991010ff19@tracker.bro-ids.org> References: <047.80b3470f0ca6911e07abe4991010ff19@tracker.bro-ids.org> Message-ID: <062.bbaf61a1737d273e64d9fabfe6168a33@tracker.bro-ids.org> #759: Increase test coverage -----------------------------+------------------------ Reporter: robin | Owner: Type: Task | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by seth): * status: new => closed * resolution: => Solved/Applied Comment: Too vague. Closing. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:18:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:18:06 -0000 Subject: [Bro-Dev] #811: Redefing Notice::policy in local.bro not removing default notice action In-Reply-To: <046.697ad84ed40fde86da0e018602a995df@tracker.bro-ids.org> References: <046.697ad84ed40fde86da0e018602a995df@tracker.bro-ids.org> Message-ID: <061.6ba6870ffda8fe8d1a5fb61a807a4ac4@tracker.bro-ids.org> #811: Redefing Notice::policy in local.bro not removing default notice action -------------------------+------------------------------------------------- Reporter: will | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: 2.0 Resolution: Works for | Keywords: Notice, action, redef, Me | PacketFilter::Dropped_Packets -------------------------+------------------------------------------------- Changes (by seth): * status: new => closed * resolution: => Works for Me Comment: Closing. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:19:11 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:19:11 -0000 Subject: [Bro-Dev] #836: Make reporter.log errors go to stderr when run from command-line In-Reply-To: <048.f55ab9255b4a29b56b8218bba951e661@tracker.bro-ids.org> References: <048.f55ab9255b4a29b56b8218bba951e661@tracker.bro-ids.org> Message-ID: <063.19b09d21f44182c1f5a93b9b2426d15c@tracker.bro-ids.org> #836: Make reporter.log errors go to stderr when run from command-line ------------------------------+------------------------ Reporter: amannb | Owner: seth Type: Feature Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Changes (by seth): * owner: => seth * status: new => assigned * milestone: => Bro2.1 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:19:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:19:34 -0000 Subject: [Bro-Dev] #465: Fix up the MIME analyzer In-Reply-To: <046.cab68644f8f2eb3e5bb44bf373b6902d@tracker.bro-ids.org> References: <046.cab68644f8f2eb3e5bb44bf373b6902d@tracker.bro-ids.org> Message-ID: <061.7e85b31301a07f63efed9115e372863b@tracker.bro-ids.org> #465: Fix up the MIME analyzer ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: analyzer ----------------------+------------------------ Changes (by seth): * keywords: => analyzer * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:20:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:20:20 -0000 Subject: [Bro-Dev] #631: Special message for broctl locking when done by cron In-Reply-To: <046.60d83c62aa090d2979230708fe26b94e@tracker.bro-ids.org> References: <046.60d83c62aa090d2979230708fe26b94e@tracker.bro-ids.org> Message-ID: <061.05c75462c32a0749ed805118ae75f19f@tracker.bro-ids.org> #631: Special message for broctl locking when done by cron ------------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: BroControl | Version: Resolution: | Keywords: ------------------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Low priority, but would be nice still. We need to discuss to see if this fits into some larger strategy for broctl. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:22:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:22:09 -0000 Subject: [Bro-Dev] #646: Cleanup interpreter error handling. In-Reply-To: <047.f66ed674d262f7ebac52f1c4e6df3a3e@tracker.bro-ids.org> References: <047.f66ed674d262f7ebac52f1c4e6df3a3e@tracker.bro-ids.org> Message-ID: <062.2534d22c1a7291f69e406df53573ef31@tracker.bro-ids.org> #646: Cleanup interpreter error handling. ---------------------+---------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: language ---------------------+---------------------- Comment (by seth): How much of this is done now? Most of the problems that I used to see with Bro shutting itself down because of an error are gone now. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:22:49 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:22:49 -0000 Subject: [Bro-Dev] #700: PacketSorter In-Reply-To: <048.3b6beb7959a1599da5788f14c206fd37@tracker.bro-ids.org> References: <048.3b6beb7959a1599da5788f14c206fd37@tracker.bro-ids.org> Message-ID: <063.7cd5e242712880e1e18a3890532c266a@tracker.bro-ids.org> #700: PacketSorter ----------------------+------------------------- Reporter: gregor | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: BroV6, IPv6 ----------------------+------------------------- Comment (by seth): I have no clue what to do with this ticket. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:23:11 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:23:11 -0000 Subject: [Bro-Dev] #698: HTTP vs MIME events In-Reply-To: <047.7f6f47529b886fee475b591da0309f6c@tracker.bro-ids.org> References: <047.7f6f47529b886fee475b591da0309f6c@tracker.bro-ids.org> Message-ID: <062.93c2367994ff572cfde74f9ef283d35f@tracker.bro-ids.org> #698: HTTP vs MIME events ----------------------+------------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: cleanup analyzer ----------------------+------------------------------ Changes (by seth): * keywords: cleanup => cleanup analyzer * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:23:27 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:23:27 -0000 Subject: [Bro-Dev] #742: Maintain constant order for hostname notice email extension In-Reply-To: <046.95889261c6d0b1c7495e259f92d824b3@tracker.bro-ids.org> References: <046.95889261c6d0b1c7495e259f92d824b3@tracker.bro-ids.org> Message-ID: <061.7b596c74bcbc18a439a4aea748395c72@tracker.bro-ids.org> #742: Maintain constant order for hostname notice email extension ----------------------+-------------------- Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ----------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:23:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:23:48 -0000 Subject: [Bro-Dev] #760: Lift Server Alternative Name (SAN) field to scripting layer In-Reply-To: <050.4ffe419969135e0b86fba4a2e5f44f36@tracker.bro-ids.org> References: <050.4ffe419969135e0b86fba4a2e5f44f36@tracker.bro-ids.org> Message-ID: <065.ca7186c7d1747271285b15c7d6284547@tracker.bro-ids.org> #760: Lift Server Alternative Name (SAN) field to scripting layer ------------------------------+------------------------ Reporter: matthias | Owner: seth Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: analyzer ------------------------------+------------------------ Changes (by seth): * keywords: => analyzer * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:27:05 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:27:05 -0000 Subject: [Bro-Dev] #763: Escape # when first character in log file line In-Reply-To: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> References: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> Message-ID: <063.6b817324ac90f873f0ec9964f86d6267@tracker.bro-ids.org> #763: Escape # when first character in log file line ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by seth): * priority: Normal => High * milestone: Bro2.1 => Bro2.2 Comment: Bumping priority on this for the next release because it's probably pretty important to fix since user'd could cause problems for themselves with a logging filter by making the first field in a log an analyzer supplied string. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:27:47 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:27:47 -0000 Subject: [Bro-Dev] #749: Extend the file type to be able to represent sockets In-Reply-To: <046.2f100e2100d0e990ec4fe371768923af@tracker.bro-ids.org> References: <046.2f100e2100d0e990ec4fe371768923af@tracker.bro-ids.org> Message-ID: <061.048c8ed100cd6b9a536ed22b23a69dbd@tracker.bro-ids.org> #749: Extend the file type to be able to represent sockets ------------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: Rejected | Keywords: ------------------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Rejected Comment: I'm just going to close this. We aren't quite ready for it I think. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:35:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:35:14 -0000 Subject: [Bro-Dev] #837: broctl load order incorrect In-Reply-To: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> References: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> Message-ID: <061.24306f006e5a1fff8ce21c276ed729a9@tracker.bro-ids.org> #837: broctl load order incorrect -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Replying to [comment:7 robin]: > How about instead (1) allowing people to override them manually, but > (2) giving a warning, like in "check", if they configure seomthing > else than BroControl would set (with an option to suppress such > warnings). I suppose that would be ok. Would we need to extend Bro's parser somehow to detect multiple overriding redefs? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:36:58 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:36:58 -0000 Subject: [Bro-Dev] #348: Reassembler integer overflow issues. Data not delivered after 2GB In-Reply-To: <048.e61375ac2d702203a810377b29931bd9@tracker.bro-ids.org> References: <048.e61375ac2d702203a810377b29931bd9@tracker.bro-ids.org> Message-ID: <063.87349decad042b3bc0d1f1a4ec4ca82a@tracker.bro-ids.org> #348: Reassembler integer overflow issues. Data not delivered after 2GB ----------------------+------------------------ Reporter: gregor | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: inttypes ----------------------+------------------------ Changes (by seth): * priority: Normal => High * milestone: Bro2.1 => Bro2.2 Comment: I'm going to mark this as high priority for 2.2. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:38:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:38:56 -0000 Subject: [Bro-Dev] #768: Inline monitoring of modified scripts. In-Reply-To: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> References: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> Message-ID: <061.bf74b2e78bd68ec0aa16654caa04ad0f@tracker.bro-ids.org> #768: Inline monitoring of modified scripts. -------------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by seth): Replying to [comment:2 justin]: > Additionally, it would be really great if broctl could somehow tell if a full restart is needed, or just an update. Unfortunately that would be right up there near impossible as far as I can tell. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:40:07 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:40:07 -0000 Subject: [Bro-Dev] #768: Inline monitoring of modified scripts. In-Reply-To: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> References: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> Message-ID: <061.bec995d730d5eebff0d48ee6ac57d1ec@tracker.bro-ids.org> #768: Inline monitoring of modified scripts. -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Changes (by seth): * owner: => dnthayer * status: new => assigned Comment: Daniel, do you think you could pick up this ticket for 2.2? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:40:15 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:40:15 -0000 Subject: [Bro-Dev] #768: Inline monitoring of modified scripts. In-Reply-To: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> References: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> Message-ID: <061.402e817c8332fe384de2c887ac03b010@tracker.bro-ids.org> #768: Inline monitoring of modified scripts. -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:41:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:41:02 -0000 Subject: [Bro-Dev] #725: Incorrect weird (unmatched_HTTP_reply) in the HTTP analyzer. In-Reply-To: <046.f0b2b0cf581674f47a5bca69fa93e12b@tracker.bro-ids.org> References: <046.f0b2b0cf581674f47a5bca69fa93e12b@tracker.bro-ids.org> Message-ID: <061.68d9937bda8f973e17276be101b75ce1@tracker.bro-ids.org> #725: Incorrect weird (unmatched_HTTP_reply) in the HTTP analyzer. ----------------------+---------------------- Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: analyzer ----------------------+---------------------- Changes (by seth): * keywords: => analyzer * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:41:54 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:41:54 -0000 Subject: [Bro-Dev] #755: Bogus DNS_truncated_ans_too_short notice in weird.log for NetBIOS DNS responses In-Reply-To: <050.d53f19c650c18d43fd5c8bf59f396338@tracker.bro-ids.org> References: <050.d53f19c650c18d43fd5c8bf59f396338@tracker.bro-ids.org> Message-ID: <065.06405441a65d2d98bae3a5dc06a3470b@tracker.bro-ids.org> #755: Bogus DNS_truncated_ans_too_short notice in weird.log for NetBIOS DNS responses -----------------------+------------------------ Reporter: matthias | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Kills me not to fix this for 2.1, I see it all the time but I don't think we'll get to it. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:43:13 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:43:13 -0000 Subject: [Bro-Dev] #719: SMTP policy blocklist: Added originator only logging In-Reply-To: <047.9e4c3286edf66f09984c98d8e2df69ab@tracker.bro-ids.org> References: <047.9e4c3286edf66f09984c98d8e2df69ab@tracker.bro-ids.org> Message-ID: <062.a5a9a1aa93d65e3cca31d04258621056@tracker.bro-ids.org> #719: SMTP policy blocklist: Added originator only logging -----------------------+---------------------- Reporter: eddyg | Owner: Type: Patch | Status: closed Priority: Low | Milestone: Bro2.1 Component: Bro | Version: 2.0 Beta Resolution: Rejected | Keywords: -----------------------+---------------------- Changes (by seth): * status: new => closed * resolution: => Rejected Comment: I'm going to close this ticket. The default notice policy change may happen at some point in the future still though. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:44:55 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:44:55 -0000 Subject: [Bro-Dev] #610: topic/seth/syslog-analyzer-updates - Updates for syslog analyzer In-Reply-To: <046.cb3dee1a160d6374e0f15924076c0111@tracker.bro-ids.org> References: <046.cb3dee1a160d6374e0f15924076c0111@tracker.bro-ids.org> Message-ID: <061.d6c17344abe6782d03c70f3bd0821f16@tracker.bro-ids.org> #610: topic/seth/syslog-analyzer-updates - Updates for syslog analyzer ---------------------+---------------------- Reporter: seth | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: analyzer ---------------------+---------------------- Changes (by seth): * keywords: beta => analyzer * milestone: Bro2.1 => Bro2.2 Comment: Bumping this back. We still need TCP syslog examples for both TCP syslog methods (octet stuff and line based). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:45:49 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:45:49 -0000 Subject: [Bro-Dev] #566: Binpac analyzers and content gaps In-Reply-To: <048.f660012aacff96470eebea84cb8c5530@tracker.bro-ids.org> References: <048.f660012aacff96470eebea84cb8c5530@tracker.bro-ids.org> Message-ID: <063.e2e6d48ca74322e737a4fa7d07bacf47@tracker.bro-ids.org> #566: Binpac analyzers and content gaps -----------------------+------------------------ Reporter: gregor | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: BinPAC | Version: git/master Resolution: Rejected | Keywords: -----------------------+------------------------ Changes (by seth): * status: new => closed * resolution: => Rejected Comment: Closing this ticket. It almost certainly won't be added to Binpac. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:46:33 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:46:33 -0000 Subject: [Bro-Dev] #519: policy/protocols/http/headers.bro only logs client headers In-Reply-To: <046.5777f3c379efa3c38e8a08a6979a91db@tracker.bro-ids.org> References: <046.5777f3c379efa3c38e8a08a6979a91db@tracker.bro-ids.org> Message-ID: <061.c4a7d0dfad6afb78bfdd4407e2ea799d@tracker.bro-ids.org> #519: policy/protocols/http/headers.bro only logs client headers ----------------------+---------------------- Reporter: vern | Owner: seth Type: Problem | Status: reopened Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ----------------------+---------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:47:45 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:47:45 -0000 Subject: [Bro-Dev] #30: Drop logic doesn't pass reason to external script In-Reply-To: <057.0ad45d6f5bf6892f3fcea8ee22660335@tracker.bro-ids.org> References: <057.0ad45d6f5bf6892f3fcea8ee22660335@tracker.bro-ids.org> Message-ID: <072.c9430a9892156adef17ce5e4e1bb2d56@tracker.bro-ids.org> #30: Drop logic doesn't pass reason to external script -----------------------+------------------------------- Reporter: rreitz@? | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: branch-robin-work Resolution: Invalid | Keywords: -----------------------+------------------------------- Changes (by seth): * status: seen => closed * resolution: => Invalid Comment: I'm going to close this ticket. It's unlikely to be referenced when writing the reaction framework. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:48:28 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:48:28 -0000 Subject: [Bro-Dev] #410: Extension to init time pattern construction In-Reply-To: <046.38b30bae927e7dd85cebc62450515fe7@tracker.bro-ids.org> References: <046.38b30bae927e7dd85cebc62450515fe7@tracker.bro-ids.org> Message-ID: <061.6da42565895bafcd9577bae68d0dd09a@tracker.bro-ids.org> #410: Extension to init time pattern construction ------------------------------+---------------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: language ------------------------------+---------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Would still be nice to have fixed, but not crucial. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:49:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:49:52 -0000 Subject: [Bro-Dev] #576: Conn.log does not use well known ports for service field anymore In-Reply-To: <048.db8d9c5b5dbcaeb9188fac26e3c86ea5@tracker.bro-ids.org> References: <048.db8d9c5b5dbcaeb9188fac26e3c86ea5@tracker.bro-ids.org> Message-ID: <063.232c8a033072c09b5b1746c338aeb13a@tracker.bro-ids.org> #576: Conn.log does not use well known ports for service field anymore ------------------------------+-------------------- Reporter: gregor | Owner: Type: Feature Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: Rejected | Keywords: BETA ------------------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Rejected Comment: I'm closing this. The topic will come back if people really want it. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:56:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:56:14 -0000 Subject: [Bro-Dev] #697: Equivalent of capture-events.bro in 2.x In-Reply-To: <050.4a84d4a1f0ff3e14c2a28b6cd50f737e@tracker.bro-ids.org> References: <050.4a84d4a1f0ff3e14c2a28b6cd50f737e@tracker.bro-ids.org> Message-ID: <065.5caa24c527e6b36eac520d1c00dad13a@tracker.bro-ids.org> #697: Equivalent of capture-events.bro in 2.x -----------------------+------------------------ Reporter: matthias | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: -----------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Comment: I'm going to remove this from a milestone because it has very limited utility for most people. Anyone will still be free to pick up the ticket and add the feature though. :) -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:59:01 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:59:01 -0000 Subject: [Bro-Dev] #779: missing values cause bro to crash when used inside of a 'when' statement. In-Reply-To: <048.d7eccddacf988dcfe011e1dac3c48b28@tracker.bro-ids.org> References: <048.d7eccddacf988dcfe011e1dac3c48b28@tracker.bro-ids.org> Message-ID: <063.6c98415b3e3404c697e14b98db7b87ab@tracker.bro-ids.org> #779: missing values cause bro to crash when used inside of a 'when' statement. ----------------------+--------------------------------------- Reporter: justin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: when InterpreterException ----------------------+--------------------------------------- Changes (by seth): * priority: Normal => High * milestone: Bro2.1 => Bro2.2 Comment: This is another one of those potentially surprising shutdown bugs and really should be fixed. Fortunately it's not a huge priority because it's in a little-used feature (the when statement). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 22:59:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 05:59:46 -0000 Subject: [Bro-Dev] #751: Broxygen Wishlist In-Reply-To: <047.4b4140bee19348deee804641b473e786@tracker.bro-ids.org> References: <047.4b4140bee19348deee804641b473e786@tracker.bro-ids.org> Message-ID: <062.59ac62c8a696e77d6c4b903802a7cd2a@tracker.bro-ids.org> #751: Broxygen Wishlist ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: 2.0 Resolution: | Keywords: ---------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:00:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:00:09 -0000 Subject: [Bro-Dev] #584: DNS TXT record lookup bif In-Reply-To: <046.2077446f247be33289c55cd0258699d3@tracker.bro-ids.org> References: <046.2077446f247be33289c55cd0258699d3@tracker.bro-ids.org> Message-ID: <061.5bdcebcb304c4d327069f6a1064ab65e@tracker.bro-ids.org> #584: DNS TXT record lookup bif ------------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ------------------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:01:28 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:01:28 -0000 Subject: [Bro-Dev] #427: Assertion failed: (!v), function Eval, file Trigger.cc In-Reply-To: <047.734f3fa34279853408b9e9abca37a2b4@tracker.bro-ids.org> References: <047.734f3fa34279853408b9e9abca37a2b4@tracker.bro-ids.org> Message-ID: <062.445c9543c5d74eb903c73257da5ff25e@tracker.bro-ids.org> #427: Assertion failed: (!v), function Eval, file Trigger.cc -----------------------------+-------------------- Reporter: leres | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: 1.5.2 Resolution: Solved/Applied | Keywords: ipv6 -----------------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Solved/Applied Comment: We haven't heard anything about this in a while and the IPv6 code was all rewritten so I'm going to go ahead and close this. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:02:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:02:19 -0000 Subject: [Bro-Dev] #701: Autodoc final version of redef'ed records (for logging) In-Reply-To: <048.f55cbfb9e2a966664745393c7e203879@tracker.bro-ids.org> References: <048.f55cbfb9e2a966664745393c7e203879@tracker.bro-ids.org> Message-ID: <063.dfc7acfab6147339d5ceb2224ec55467@tracker.bro-ids.org> #701: Autodoc final version of redef'ed records (for logging) ----------------------+-------------------- Reporter: gregor | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: docs ----------------------+-------------------- Changes (by seth): * keywords: => docs * milestone: Bro2.1 => Bro2.2 Comment: This would be really helpful, but I think it may require more discussion. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:03:23 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:03:23 -0000 Subject: [Bro-Dev] #419: Add a "real" list type to the scripting language. In-Reply-To: <047.a101407502c977f85e7e61f3b4b1b71d@tracker.bro-ids.org> References: <047.a101407502c977f85e7e61f3b4b1b71d@tracker.bro-ids.org> Message-ID: <062.deef444518e9d206bd9a3da0c3b3ba73@tracker.bro-ids.org> #419: Add a "real" list type to the scripting language. ----------------------+---------------------- Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: Resolution: | Keywords: language ----------------------+---------------------- Changes (by seth): * milestone: Bro2.1 => Comment: Is there a point to leaving this ticket open? I'm pretty sure we aren't forgetting about it. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:06:27 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:06:27 -0000 Subject: [Bro-Dev] #703: NUL_in_line and line_terminated_with_single_CR complaints In-Reply-To: <046.a178d88fb4159c54e316119fd1f5ded7@tracker.bro-ids.org> References: <046.a178d88fb4159c54e316119fd1f5ded7@tracker.bro-ids.org> Message-ID: <061.6ec931f7307003a6ec7c461a24000e46@tracker.bro-ids.org> #703: NUL_in_line and line_terminated_with_single_CR complaints ------------------------+-------------------- Reporter: vern | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: Duplicate | Keywords: ------------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Duplicate Comment: Replying to [comment:4 gregor]: > But this string ("Ready to start TLS") is not standardized, right? Only the 220 is. Good point. Once the TLS upgrade is supported it should be noticeable by the presence of SMTP (or other) and SSL in the service field in conn.log. I'm going to close this ticket because it probably won't be needed once the TLS upgrade support is in place. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:07:58 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:07:58 -0000 Subject: [Bro-Dev] #306: Write a new user manual In-Reply-To: <046.b5f6eda94a2759af201753eaf30711c5@tracker.bro-ids.org> References: <046.b5f6eda94a2759af201753eaf30711c5@tracker.bro-ids.org> Message-ID: <061.a589bcc2db03343107df1da9b3f229ed@tracker.bro-ids.org> #306: Write a new user manual -------------------+-------------------- Reporter: seth | Owner: Type: Task | Status: new Priority: High | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: -------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Robin has started to put some infrastructure in place, but nothing has been written yet. Maybe for 2.2? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:15:31 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:15:31 -0000 Subject: [Bro-Dev] #772: Problem with $path_func in Log filters In-Reply-To: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> References: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> Message-ID: <061.6fb1a31627c1408864e87bfbbb08afa6@tracker.bro-ids.org> #772: Problem with $path_func in Log filters ----------------------+------------------------ Reporter: seth | Owner: seth Type: Problem | Status: assigned Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Not ready to do this, but putting at high priority for 2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:16:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:16:03 -0000 Subject: [Bro-Dev] #474: &raw_output turns null values into \0 In-Reply-To: <046.21443edb25709a7dff08ac2dadf6c6e0@tracker.bro-ids.org> References: <046.21443edb25709a7dff08ac2dadf6c6e0@tracker.bro-ids.org> Message-ID: <061.2e3f4101880bdc7df25af8e91878b1da@tracker.bro-ids.org> #474: &raw_output turns null values into \0 ----------------------+------------------------ Reporter: seth | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: preview ----------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: I think this is low priority right now so I'm going to bump it back. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:17:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:17:06 -0000 Subject: [Bro-Dev] #671: Test Bro core and script layer simultaneously In-Reply-To: <050.bdf7bb369f1c9115ad64afb0c5f4fe7a@tracker.bro-ids.org> References: <050.bdf7bb369f1c9115ad64afb0c5f4fe7a@tracker.bro-ids.org> Message-ID: <065.ae363a5e181be3c0226beae8a127b684@tracker.bro-ids.org> #671: Test Bro core and script layer simultaneously -----------------------+------------------------ Reporter: matthias | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: BTest | Version: git/master Resolution: | Keywords: -----------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: If this strategy works and isn't too much effort I'd really like to see it make it into the test suite for 2.2. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:17:37 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:17:37 -0000 Subject: [Bro-Dev] #80: Checking for the existance keys in multi-dimensional tables gives error message In-Reply-To: <050.c01d8a094407695f72fd3d3d599c4f48@tracker.bro-ids.org> References: <050.c01d8a094407695f72fd3d3d599c4f48@tracker.bro-ids.org> Message-ID: <065.f04d25d905714dc65f2146d87471f4c5@tracker.bro-ids.org> #80: Checking for the existance keys in multi-dimensional tables gives error message -----------------------+------------------------ Reporter: bernhard | Owner: Type: Problem | Status: seen Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------+------------------------ Changes (by seth): * priority: Normal => Low * version: 1.4 => git/master * milestone: Bro2.1 => Bro2.2 Comment: I hate this problem. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:18:53 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:18:53 -0000 Subject: [Bro-Dev] #774: IPv6 in signatures In-Reply-To: <046.e3fc459c715e558f55a8d0d7807eaeac@tracker.bro-ids.org> References: <046.e3fc459c715e558f55a8d0d7807eaeac@tracker.bro-ids.org> Message-ID: <061.4584a35191b7735e6f79b66ab8ca90ab@tracker.bro-ids.org> #774: IPv6 in signatures ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------+------------------------ Changes (by seth): * priority: Normal => Low * milestone: Bro2.1 => Bro2.2 Comment: Still low priority, but needs to be fixed at some point. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:19:42 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:19:42 -0000 Subject: [Bro-Dev] #824: Default the connection and alarm summaries to once per day In-Reply-To: <056.dadb50a21009b083c39e1a280f60c9e9@tracker.bro-ids.org> References: <056.dadb50a21009b083c39e1a280f60c9e9@tracker.bro-ids.org> Message-ID: <071.afd6733d43ce369a1e54c95ecd2ebff1@tracker.bro-ids.org> #824: Default the connection and alarm summaries to once per day ------------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:20:23 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:20:23 -0000 Subject: [Bro-Dev] #603: Checking correctness of logs In-Reply-To: <047.5d54b4bcd94fce279ea5de274dc02946@tracker.bro-ids.org> References: <047.5d54b4bcd94fce279ea5de274dc02946@tracker.bro-ids.org> Message-ID: <062.f5102fda69a8322df05fcd96c783bb9d@tracker.bro-ids.org> #603: Checking correctness of logs ---------------------+------------------------ Reporter: robin | Owner: seth Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:20:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:20:51 -0000 Subject: [Bro-Dev] #640: BiFs to enable or disable events. In-Reply-To: <046.1f1e3ac9563bd29d452598383f9647bd@tracker.bro-ids.org> References: <046.1f1e3ac9563bd29d452598383f9647bd@tracker.bro-ids.org> Message-ID: <061.d9af77ba83ccf592d7ab9002a41f37cf@tracker.bro-ids.org> #640: BiFs to enable or disable events. ------------------------------+---------------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: High | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: language ------------------------------+---------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:22:25 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:22:25 -0000 Subject: [Bro-Dev] #730: Find and fix tcp sequence counting bugs In-Reply-To: <046.dcc46d96df7639fb8de3a135dbe7e0cc@tracker.bro-ids.org> References: <046.dcc46d96df7639fb8de3a135dbe7e0cc@tracker.bro-ids.org> Message-ID: <061.7e92add20a2383e72dba93c8998fbc5f@tracker.bro-ids.org> #730: Find and fix tcp sequence counting bugs ----------------------+-------------------- Reporter: seth | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ----------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: I don't want to make changes as large as this would involve before the 2.1 release. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:28:42 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:28:42 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro In-Reply-To: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> References: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> Message-ID: <063.c0708c3c6860d369f3936953ad59a783@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: beta ----------------------+------------------------ Changes (by seth): * keywords: => beta Comment: Let's get this fixed during the beta if it's not too much work. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:30:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:30:51 -0000 Subject: [Bro-Dev] #781: Case sensitive (non-normalized) HTTP header names In-Reply-To: <048.1cb22f83281a8d29460fe152fe9d53ce@tracker.bro-ids.org> References: <048.1cb22f83281a8d29460fe152fe9d53ce@tracker.bro-ids.org> Message-ID: <063.a6186ee59b1bc1e28e3d93ee688091e7@tracker.bro-ids.org> #781: Case sensitive (non-normalized) HTTP header names ------------------------------+------------------------ Reporter: sconzo | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: analyzer ------------------------------+------------------------ Changes (by seth): * keywords: => analyzer * milestone: Bro2.1 => Bro2.2 Comment: We aren't making many analyzer improvements for 2.1, we'll try to get to this and the other HTTP analyzer tickets for 2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 11 23:31:39 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 06:31:39 -0000 Subject: [Bro-Dev] #340: Cleanup: unify where global consts are defined (access from policy layer and event engine) In-Reply-To: <048.454303efc01b5748989d5c65cf7c8dd3@tracker.bro-ids.org> References: <048.454303efc01b5748989d5c65cf7c8dd3@tracker.bro-ids.org> Message-ID: <063.1989e60aa0da2722ca98b50c8c289900@tracker.bro-ids.org> #340: Cleanup: unify where global consts are defined (access from policy layer and event engine) ------------------------------+--------------------- Reporter: gregor | Owner: Type: Feature Request | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: cleanup ------------------------------+--------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: This isn't going to happen for 2.1 at this point. -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Wed Jul 11 23:35:05 2012 From: seth at icir.org (Seth Hall) Date: Thu, 12 Jul 2012 02:35:05 -0400 Subject: [Bro-Dev] Whew. Message-ID: <834ABE3B-3B19-4EC2-B8B3-C7221E2158FA@icir.org> What a night? http://tracker.bro-ids.org/bro/roadmap Not too many tickets left now. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From noreply at bro-ids.org Thu Jul 12 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 12 Jul 2012 00:00:04 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207120700.q6C704qQ013100@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 762 [1] | amannb | | Normal | Add eof line to logfiles Bro | 847 [2] | grigorescu | robin | High | binpac::SOCKS::SOCKS5_Address::Parse Assertion Error > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 8ff8c66 | Bernhard Amann | 2012-07-11 | make pthread_mutex_unlock include the reason for why the unlock fails. [3] bro | b31ef8c | Seth Hall | 2012-07-11 | Fixing memory leak. [4] [1] #762: http://tracker.bro-ids.org/bro/ticket/762 [2] #847: http://tracker.bro-ids.org/bro/ticket/847 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/8ff8c66655fdfc2dfec703b332dfad02226be775/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/b31ef8cde5c6b40a736a88ea1354f3073f99c9b1/bro From robin at icir.org Thu Jul 12 07:16:50 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 12 Jul 2012 07:16:50 -0700 Subject: [Bro-Dev] Whew. In-Reply-To: <834ABE3B-3B19-4EC2-B8B3-C7221E2158FA@icir.org> References: <834ABE3B-3B19-4EC2-B8B3-C7221E2158FA@icir.org> Message-ID: <20120712141650.GA28069@icir.org> On Thu, Jul 12, 2012 at 02:35 -0400, you wrote: > What a night? http://tracker.bro-ids.org/bro/roadmap Impressive, thanks for going through! Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Thu Jul 12 07:27:55 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 14:27:55 -0000 Subject: [Bro-Dev] #578: Add ICMPv6 support to Bro In-Reply-To: <048.1e6e618d9d8a166299c6c8582e9c8511@tracker.bro-ids.org> References: <048.1e6e618d9d8a166299c6c8582e9c8511@tracker.bro-ids.org> Message-ID: <063.d8e26dc0f437a5ecac7aac7df47a57e5@tracker.bro-ids.org> #578: Add ICMPv6 support to Bro ----------------------+------------------------ Reporter: gregor | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: IPv6 ----------------------+------------------------ Comment (by slagell): I would say no and close it. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 07:41:39 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 14:41:39 -0000 Subject: [Bro-Dev] #342: Add payload to ICMP analyzer In-Reply-To: <046.dc4cf3cae0b93ae2cca66efcf0ce60b4@tracker.bro-ids.org> References: <046.dc4cf3cae0b93ae2cca66efcf0ce60b4@tracker.bro-ids.org> Message-ID: <061.5ede99e0d8875c42a3c4d0a5d4c289ec@tracker.bro-ids.org> #342: Add payload to ICMP analyzer ---------------------+-------------------- Reporter: seth | Owner: Type: Patch | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: 1.5.2 Resolution: | Keywords: ---------------------+-------------------- Comment (by jsiwek): Replying to [comment:11 seth]: > Did this happen? I didn't track the ICMP changes closely. No `icmp_sent` events still don't include a payload parameter. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 07:56:25 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 14:56:25 -0000 Subject: [Bro-Dev] #646: Cleanup interpreter error handling. In-Reply-To: <047.f66ed674d262f7ebac52f1c4e6df3a3e@tracker.bro-ids.org> References: <047.f66ed674d262f7ebac52f1c4e6df3a3e@tracker.bro-ids.org> Message-ID: <062.fcfdc3f804eb45ec7e742abec99dff4a@tracker.bro-ids.org> #646: Cleanup interpreter error handling. ---------------------+---------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: language ---------------------+---------------------- Comment (by jsiwek): Replying to [comment:2 seth]: > How much of this is done now? Most of the problems that I used to see with Bro shutting itself down because of an error are gone now. I'd say full completion of this task also depends on fixing the memory management issues discussed in #831, which is not trivial. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 07:57:22 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 14:57:22 -0000 Subject: [Bro-Dev] #831: Interpreter exceptions cause memory leaks (was "Memory leak in print") (was: Memory leak in print) In-Reply-To: <048.ea67e86bfb2a6633bbc5ada34ca295e5@tracker.bro-ids.org> References: <048.ea67e86bfb2a6633bbc5ada34ca295e5@tracker.bro-ids.org> Message-ID: <063.ec21891cd5bb7744068e99f0c4858251@tracker.bro-ids.org> #831: Interpreter exceptions cause memory leaks (was "Memory leak in print") ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by jsiwek): * milestone: => Bro2.2 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 08:53:13 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 15:53:13 -0000 Subject: [Bro-Dev] #849: SMTP analyzer and reporter warnings Message-ID: <046.6fc0d12f96c161a2045d175866dd8f48@tracker.bro-ids.org> #849: SMTP analyzer and reporter warnings ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: analyzer | ----------------------+------------------------ There are some warnings in the SMTP analyzer (ultimately from using the MIME analyzer) that go to reporter but they are wildly unhelpful in reporter.log. Here's an example line from reporter.log: {{{ 1342043855.564338 Reporter::WARNING nested mail transaction (empty) - }}} Doing protocol violations on the smtp analyzer wouldn't quite be the right thing either because the dpd framework might remove the smtp analyzer from the connection. Part of the problem may stem from the fact that MIME analyzer isn't a true analyzer (doesn't descend from Analyzer). There is some obvious analyzer restructuring that needs to happen here but that can wait for the larger analyzer work that is coming up. Does anyone have thoughts about what we could do with this message now to make it more useful? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 08:54:32 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 15:54:32 -0000 Subject: [Bro-Dev] #342: Add payload to ICMP analyzer In-Reply-To: <046.dc4cf3cae0b93ae2cca66efcf0ce60b4@tracker.bro-ids.org> References: <046.dc4cf3cae0b93ae2cca66efcf0ce60b4@tracker.bro-ids.org> Message-ID: <061.4c4bb1550cf0b29be662e62a61bfbac8@tracker.bro-ids.org> #342: Add payload to ICMP analyzer ---------------------+-------------------- Reporter: seth | Owner: Type: Patch | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: 1.5.2 Resolution: | Keywords: ---------------------+-------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Bumping this then. It's too big of a change to include now. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 08:55:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 15:55:03 -0000 Subject: [Bro-Dev] #646: Cleanup interpreter error handling. In-Reply-To: <047.f66ed674d262f7ebac52f1c4e6df3a3e@tracker.bro-ids.org> References: <047.f66ed674d262f7ebac52f1c4e6df3a3e@tracker.bro-ids.org> Message-ID: <062.21d56fc6796c5b7655bfa4497e48edae@tracker.bro-ids.org> #646: Cleanup interpreter error handling. ---------------------+---------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: language ---------------------+---------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: Bumping back. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 09:07:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 16:07:20 -0000 Subject: [Bro-Dev] #847: binpac::SOCKS::SOCKS5_Address::Parse Assertion Error In-Reply-To: <052.50a568914ac210fa9d0aaf11555b1657@tracker.bro-ids.org> References: <052.50a568914ac210fa9d0aaf11555b1657@tracker.bro-ids.org> Message-ID: <067.7ed74343e05028d782b2b84cdb2a6488@tracker.bro-ids.org> #847: binpac::SOCKS::SOCKS5_Address::Parse Assertion Error ----------------------------+------------------------ Reporter: grigorescu | Owner: robin Type: Merge Request | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: socks ----------------------------+------------------------ Changes (by robin): * status: assigned => closed * resolution: => fixed Comment: In [5d13e4f94996a435e1b179060451720060d96088/bro]: {{{ #!CommitTicketReference repository="bro" revision="5d13e4f94996a435e1b179060451720060d96088" Merge remote-tracking branch 'origin/topic/seth/socks-fixes' * origin/topic/seth/socks-fixes: Some small fixes to further reduce SOCKS false positive logs. Closes #847. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 11:49:16 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 18:49:16 -0000 Subject: [Bro-Dev] #850: topic/seth/elasticsearch Message-ID: <046.c8ab851cf8d2d187966f09f844526fca@tracker.bro-ids.org> #850: topic/seth/elasticsearch ---------------------------+------------------------ Reporter: seth | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This should be ready for merging as long as it's labelled as "in testing" -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 14:28:11 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 21:28:11 -0000 Subject: [Bro-Dev] #783: Update IPv6-related website documentation In-Reply-To: <048.b6917f135ad80ce9768ff7e76b549342@tracker.bro-ids.org> References: <048.b6917f135ad80ce9768ff7e76b549342@tracker.bro-ids.org> Message-ID: <063.016873f21b548b3447e5f0a19d3eebbb@tracker.bro-ids.org> #783: Update IPv6-related website documentation -----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Task | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: ipv6 -----------------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied Comment: I've removed the old docs. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 16:46:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 23:46:20 -0000 Subject: [Bro-Dev] #700: PacketSorter In-Reply-To: <048.3b6beb7959a1599da5788f14c206fd37@tracker.bro-ids.org> References: <048.3b6beb7959a1599da5788f14c206fd37@tracker.bro-ids.org> Message-ID: <063.ff42f784751290e29bd8a2c48eaefae1@tracker.bro-ids.org> #700: PacketSorter ----------------------+------------------------- Reporter: gregor | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: BroV6, IPv6 ----------------------+------------------------- Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: We definitely aren't doing anything with this for 2.1. I'll bump it back and try to bring it up again for 2.2. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 16:46:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 23:46:52 -0000 Subject: [Bro-Dev] #578: Add ICMPv6 support to Bro In-Reply-To: <048.1e6e618d9d8a166299c6c8582e9c8511@tracker.bro-ids.org> References: <048.1e6e618d9d8a166299c6c8582e9c8511@tracker.bro-ids.org> Message-ID: <063.b49148985611b02a4fb0e5b56b1c0ece@tracker.bro-ids.org> #578: Add ICMPv6 support to Bro -----------------------------+------------------------ Reporter: gregor | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: IPv6 -----------------------------+------------------------ Changes (by seth): * status: new => closed * resolution: => Solved/Applied Comment: Seems reasonable. Closing. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 16:55:33 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 23:55:33 -0000 Subject: [Bro-Dev] #763: Escape # when first character in log file line In-Reply-To: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> References: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> Message-ID: <063.2c124498e0d78751af81abbe1fd35a26@tracker.bro-ids.org> #763: Escape # when first character in log file line ----------------------+------------------------ Reporter: amannb | Owner: robin Type: Problem | Status: closed Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [c3c6967c5e262a166d3beb73908e882966156b56/bro]: {{{ #!CommitTicketReference repository="bro" revision="c3c6967c5e262a166d3beb73908e882966156b56" Reworking thread termination logic. Turns out the finish methods weren't called correctly, caused by a mess up with method names which all sounded too similar and the wrong one ended up being called. I've reworked this by changing the thread/writer/reader interfaces, which actually also simplifies them by getting rid of the requirement for writer backends to call their parent methods (i.e., less opportunity for errors). This commit also includes the following (because I noticed the problem above when working on some of these): - The ASCII log writer now includes "#start " and "#end lines in the each file. The latter supersedes Bernhard's "EOF" patch. This required a number of tests updates. The standard canonifier removes the timestamps, but some tests compare files directly, which doesn't work if they aren't printing out the same timestamps (like the comm tests). - The above required yet another change to the writer API to network_time to methods. - Renamed ASCII logger "header" options to "meta". - Fixes #763 "Escape # when first character in log file line". All btests pass for me on Linux FC15. Will try MacOS next. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 16:57:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 12 Jul 2012 23:57:38 -0000 Subject: [Bro-Dev] #850: topic/seth/elasticsearch In-Reply-To: <046.c8ab851cf8d2d187966f09f844526fca@tracker.bro-ids.org> References: <046.c8ab851cf8d2d187966f09f844526fca@tracker.bro-ids.org> Message-ID: <061.cf4e0c067ab9d8c91117cb377fb19574@tracker.bro-ids.org> #850: topic/seth/elasticsearch ----------------------------+------------------------ Reporter: seth | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): This will need some further changes after the thread API changes are merged in ? -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Thu Jul 12 17:10:09 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 12 Jul 2012 17:10:09 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/threading-fix: Reworking thread termination logic. (c3c6967) In-Reply-To: <201207122355.q6CNtaIl004130@bro-ids.icir.org> References: <201207122355.q6CNtaIl004130@bro-ids.icir.org> Message-ID: <20120713001009.GQ30253@icir.org> On Thu, Jul 12, 2012 at 16:55 -0700, I wrote: > All btests pass for me on Linux FC15. Will try MacOS next. Turns out I still get occasional missing output on Mac (but only in optimized mode so far). :-( Don't know why, but I have to stop now, and not sure if I'll get back to it tomorrow. If anyone wants to take a look, please go ahead. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Thu Jul 12 17:34:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 13 Jul 2012 00:34:02 -0000 Subject: [Bro-Dev] #762: Add eof line to logfiles In-Reply-To: <048.b69152fe8b8e0e80e5715b13977d82cb@tracker.bro-ids.org> References: <048.b69152fe8b8e0e80e5715b13977d82cb@tracker.bro-ids.org> Message-ID: <063.6ddd6220c59b78e08725ba9cfed37e09@tracker.bro-ids.org> #762: Add eof line to logfiles -----------------------------+------------------------ Reporter: amannb | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied Comment: Will be supersede by including start/end times. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 12 18:03:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 13 Jul 2012 01:03:00 -0000 Subject: [Bro-Dev] #763: Escape # when first character in log file line In-Reply-To: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> References: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> Message-ID: <063.2c595635aa16be1d0d514613ef02067c@tracker.bro-ids.org> #763: Escape # when first character in log file line ----------------------+------------------------ Reporter: amannb | Owner: robin Type: Problem | Status: reopened Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by robin): * status: closed => reopened * resolution: fixed => Comment: Not merged yet. -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Thu Jul 12 18:41:44 2012 From: seth at icir.org (Seth Hall) Date: Thu, 12 Jul 2012 21:41:44 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/threading-fix: Reworking thread termination logic. (c3c6967) In-Reply-To: <201207122355.q6CNtaIl004130@bro-ids.icir.org> References: <201207122355.q6CNtaIl004130@bro-ids.icir.org> Message-ID: <9C123C8F-4BFA-456D-B326-58965819522A@icir.org> On Jul 12, 2012, at 7:55 PM, Robin Sommer wrote: > - The ASCII log writer now includes "#start " and > "#end lines in the each file. The latter supersedes > Bernhard's "EOF" patch. Are these timestamps from the wall clock? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Thu Jul 12 18:43:49 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 12 Jul 2012 18:43:49 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/threading-fix: Reworking thread termination logic. (c3c6967) In-Reply-To: <9C123C8F-4BFA-456D-B326-58965819522A@icir.org> References: <201207122355.q6CNtaIl004130@bro-ids.icir.org> <9C123C8F-4BFA-456D-B326-58965819522A@icir.org> Message-ID: <20120713014349.GA56263@icir.org> On Thu, Jul 12, 2012 at 21:41 -0400, you wrote: > Are these timestamps from the wall clock? network time. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bernhard at ICSI.Berkeley.EDU Thu Jul 12 18:44:13 2012 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Thu, 12 Jul 2012 18:44:13 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/threading-fix: Reworking thread termination logic. (c3c6967) In-Reply-To: <9C123C8F-4BFA-456D-B326-58965819522A@icir.org> References: <201207122355.q6CNtaIl004130@bro-ids.icir.org> <9C123C8F-4BFA-456D-B326-58965819522A@icir.org> Message-ID: On Jul 12, 2012, at 6:41 PM, Seth Hall wrote: > > On Jul 12, 2012, at 7:55 PM, Robin Sommer wrote: > >> - The ASCII log writer now includes "#start " and >> "#end lines in the each file. The latter supersedes >> Bernhard's "EOF" patch. > > > Are these timestamps from the wall clock? Robin mentioned earlier to me that he wants to use network_time for them. So I guess not. Bernhard From seth at icir.org Thu Jul 12 18:48:06 2012 From: seth at icir.org (Seth Hall) Date: Thu, 12 Jul 2012 21:48:06 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/threading-fix: Reworking thread termination logic. (c3c6967) In-Reply-To: <20120713014349.GA56263@icir.org> References: <201207122355.q6CNtaIl004130@bro-ids.icir.org> <9C123C8F-4BFA-456D-B326-58965819522A@icir.org> <20120713014349.GA56263@icir.org> Message-ID: <3490CDCE-DF26-4A45-A64F-F575C5BE809B@icir.org> On Jul 12, 2012, at 9:43 PM, Robin Sommer wrote: > On Thu, Jul 12, 2012 at 21:41 -0400, you wrote: > >> Are these timestamps from the wall clock? > > network time. Where are they from on a cluster manager since it doesn't have network time? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From noreply at bro-ids.org Fri Jul 13 00:00:06 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 13 Jul 2012 00:00:06 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207130700.q6D706X6019657@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch From robin at icir.org Fri Jul 13 07:33:12 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 13 Jul 2012 07:33:12 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/threading-fix: Reworking thread termination logic. (c3c6967) In-Reply-To: <3490CDCE-DF26-4A45-A64F-F575C5BE809B@icir.org> References: <201207122355.q6CNtaIl004130@bro-ids.icir.org> <9C123C8F-4BFA-456D-B326-58965819522A@icir.org> <20120713014349.GA56263@icir.org> <3490CDCE-DF26-4A45-A64F-F575C5BE809B@icir.org> Message-ID: <20120713143312.GG71135@icir.org> On Thu, Jul 12, 2012 at 21:48 -0400, you wrote: > Where are they from on a cluster manager since it doesn't have network time? Without packet input, network time indeed follows wall clock (that's generally the case, i.e. the timestamps you see in the manager's own logs (i.e., not the ones received from workers) are also coming from network_time internally, but the I/O loop sets that to current_time() in that case). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Fri Jul 13 07:45:19 2012 From: seth at icir.org (Seth Hall) Date: Fri, 13 Jul 2012 10:45:19 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/threading-fix: Reworking thread termination logic. (c3c6967) In-Reply-To: <20120713143312.GG71135@icir.org> References: <201207122355.q6CNtaIl004130@bro-ids.icir.org> <9C123C8F-4BFA-456D-B326-58965819522A@icir.org> <20120713014349.GA56263@icir.org> <3490CDCE-DF26-4A45-A64F-F575C5BE809B@icir.org> <20120713143312.GG71135@icir.org> Message-ID: <8C5E185F-F33A-4719-9BEB-EB4A03463A51@icir.org> On Jul 13, 2012, at 10:33 AM, Robin Sommer wrote: > Without packet input, network time indeed follows wall clock (that's > generally the case, i.e. the timestamps you see in the manager's own > logs (i.e., not the ones received from workers) are also coming from > network_time internally, but the I/O loop sets that to current_time() > in that case). Ah, of course. It's amazing how many scenarios that approach covers perfectly. The only situation I can see trouble in the future is when I start bugging people about doing cluster based tracefile processing. ;) That will need a packet distributor that is keeping a steady heartbeat for the whole cluster anyway so it would still not cause a problem because the manager's clock would inherently need to be driven by that anyway. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Fri Jul 13 11:16:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 13 Jul 2012 18:16:19 -0000 Subject: [Bro-Dev] #851: Info Record Doc Fixes Message-ID: <052.b7c8291976c888bdb40e7b28ec10ebd9@tracker.bro-ids.org> #851: Info Record Doc Fixes ------------------------+--------------------------- Reporter: grigorescu | Type: Merge Request Status: new | Priority: Low Milestone: Bro2.1 | Component: Bro Version: git/master | Keywords: docs ------------------------+--------------------------- After noticing that the [http://www.bro- ids.org/documentation/scripts/base/protocols/smtp/main.html#id2 SMTP Info record documentation] is missing many comments, I've done a pass of the comments for the Info records, doing some general clean-up and trying to make things a bit clearer and more consistent. Commit is available [https://github.com/grigorescu/bro/commit/f43576cff346bcecde12bd477f444d532e4b0632 here]. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 13 13:41:41 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 13 Jul 2012 20:41:41 -0000 Subject: [Bro-Dev] #852: Update cert bundle from mozilla before release Message-ID: <046.03be5949fa44efa71d66c15d8d09104f@tracker.bro-ids.org> #852: Update cert bundle from mozilla before release ---------------------+------------------------ Reporter: seth | Owner: seth Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ like the summary says. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 13 14:54:15 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 13 Jul 2012 21:54:15 -0000 Subject: [Bro-Dev] #434: Fix secondary path In-Reply-To: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> References: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> Message-ID: <062.bdd55c3227cbe6a58b7fd4776879e4e4@tracker.bro-ids.org> #434: Fix secondary path ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by vern): Concrete example comes from wanting to estimate flow sizes in TCP traffic without having to capture all TCP packets. You can do this with SYN+FIN packets, except for very large flows that wrap the sequence space, you can't figure out the correct connection size. large-conns.bro allows figuring this out at only the cost of a pretty inexpensive additional packet filter. -- Ticket URL: Bro Tracker Bro Issue Tracker From bernhard at ICSI.Berkeley.EDU Fri Jul 13 15:00:22 2012 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Fri, 13 Jul 2012 15:00:22 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/gilbert/dtrace-probes: OSX support for dtrace built into bro. Probes supported are: (20fe478) In-Reply-To: <201207130345.q6D3jpK0023399@bro-ids.icir.org> References: <201207130345.q6D3jpK0023399@bro-ids.icir.org> Message-ID: <94B5DC40-6B59-47CC-B981-A50EEE459B5B@icsi.berkeley.edu> This is cool :). I always wanted a reason to play around with dtrace and never really found a good reason for it. Might this also work on FreeBSD? If I am not very much mistaken, dtrace support has been added to it quite a while ago. Bernhard On Jul 12, 2012, at 8:45 PM, Gilbert Clark wrote: > Repository : ssh://git at bro-ids.icir.org/bro > > On branch : topic/gilbert/dtrace-probes > Link : http://tracker.bro-ids.org/bro/changeset/20fe4788fa96a8855d0dc1ce4c12576d01dea3d8/bro > >> --------------------------------------------------------------- > > commit 20fe4788fa96a8855d0dc1ce4c12576d01dea3d8 > Author: Gilbert Clark > Date: Thu Jul 12 19:39:51 2012 -0400 > > OSX support for dtrace built into bro. Probes supported are: > > bro_script -- builtin_entry, builtin_return, function_entry, function_return >> Offers support for bro script-land tracing (via --enable-dtrace configure option). > > bro_checkpoint -- fire, clear >> Meant to support incremental statistics (e.g. time elapsed between two checkpoints). From gc355804 at ohio.edu Fri Jul 13 15:53:11 2012 From: gc355804 at ohio.edu (Clark, Gilbert) Date: Fri, 13 Jul 2012 18:53:11 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/gilbert/dtrace-probes: OSX support for dtrace built into bro. Probes supported are: (20fe478) In-Reply-To: <94B5DC40-6B59-47CC-B981-A50EEE459B5B@icsi.berkeley.edu> Message-ID: The way the probes are compiled, it *should* work with FreeBSD. I'm in the process of rebuilding my FreeBSD VM, so no way to easily test that yet, though. This will *not* work on Solaris. In case anyone would like to play with this, I've attached a simple bro-trace.d that will trace calls to bro-builtins. On OS X, you'll *need* to actually sudo -i to run the script; if you don't, END {} won't fire (not sure why). You'll also probably need 'dtrace -Z -s bro-trace.d' if you run dtrace before bro is started. By the way, to give some idea of overhead, I ran an analysis of the 700 MB pcap here: http://2009.hack.lu/index.php/InfoVisContest via: bro -r ~/Downloads/jubrowska-capture_1.cap On my laptop (quad-core i7, 5400 RPM HD, 8 GB DDR3), without probes compiled into bro, I got: real 3m33.206s user 4m15.808s sys 0m25.539s With probes compiled but nothing using them: real 3m41.651s user 4m18.873s sys 0m25.759s With bro-trace.d running while bro processed the above capture file: real 4m29.553s user 4m26.601s sys 1m10.063s --Gilbert On 7/13/12 6:00 PM, "Bernhard Amann" wrote: >This is cool :). I always wanted a reason to play around with dtrace and >never really found a good >reason for it. > >Might this also work on FreeBSD? If I am not very much mistaken, dtrace >support has been added >to it quite a while ago. > >Bernhard > >On Jul 12, 2012, at 8:45 PM, Gilbert Clark wrote: > >> Repository : ssh://git at bro-ids.icir.org/bro >> >> On branch : topic/gilbert/dtrace-probes >> Link : >>http://tracker.bro-ids.org/bro/changeset/20fe4788fa96a8855d0dc1ce4c12576d >>01dea3d8/bro >> >>> --------------------------------------------------------------- >> >> commit 20fe4788fa96a8855d0dc1ce4c12576d01dea3d8 >> Author: Gilbert Clark >> Date: Thu Jul 12 19:39:51 2012 -0400 >> >> OSX support for dtrace built into bro. Probes supported are: >> >> bro_script -- builtin_entry, builtin_return, function_entry, >>function_return >>> Offers support for bro script-land tracing (via --enable-dtrace >>>configure option). >> >> bro_checkpoint -- fire, clear >>> Meant to support incremental statistics (e.g. time elapsed between two >>>checkpoints). > -------------- next part -------------- A non-text attachment was scrubbed... Name: bro-trace.d Type: application/octet-stream Size: 901 bytes Desc: bro-trace.d Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120713/738b57c4/attachment.obj From bro at tracker.bro-ids.org Fri Jul 13 16:46:54 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 13 Jul 2012 23:46:54 -0000 Subject: [Bro-Dev] #851: Info Record Doc Fixes In-Reply-To: <052.b7c8291976c888bdb40e7b28ec10ebd9@tracker.bro-ids.org> References: <052.b7c8291976c888bdb40e7b28ec10ebd9@tracker.bro-ids.org> Message-ID: <067.2d8f94b36fcd544fc9b0d23f40f2827d@tracker.bro-ids.org> #851: Info Record Doc Fixes ----------------------------+------------------------ Reporter: grigorescu | Owner: robin Type: Merge Request | Status: closed Priority: Low | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: docs ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [86f6f36f70f8d740dca081b976d35157a6ca4f18/bro]: {{{ #!CommitTicketReference repository="bro" revision="86f6f36f70f8d740dca081b976d35157a6ca4f18" Merge remote-tracking branch 'vlad/info_record_fixes' * vlad/info_record_fixes: Fix some Info:Record field documentation. Closes #851. Many thanks! }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 13 19:14:29 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 14 Jul 2012 02:14:29 -0000 Subject: [Bro-Dev] #434: Fix secondary path In-Reply-To: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> References: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> Message-ID: <062.c942bba70154063aa5c9e2d54e28558e@tracker.bro-ids.org> #434: Fix secondary path ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by seth): > Concrete example comes from wanting to estimate flow sizes in TCP traffic > without having to capture all TCP packets. You can do this with SYN+FIN > packets, except for very large flows that wrap the sequence space, you > can't figure out the correct connection size. large-conns.bro allows > figuring this out at only the cost of a pretty inexpensive additional > packet filter. Instead of doing a secondary filter, does it make sense to have a filter which leaves ack packets being allowed through? You will increase the number of packets making it through (the default window for large-conns is 16KB so you may even get ~16x as many packets). The difference I see though is that for each captured packet with the secondary path mechanism it ends up creating a Bro data structure and inserting an event in the event queue which is going to have a fair amount of load by itself. You have the obvious benefits of complete ack tracking such as monitoring for gaps and reusing the existing connection size counting code in the core. There is a lot of infrastructure type work I have ready for the 2.2 release too which will make a lot of this sort of large and (and possibly dynamic) filter creation much easier. If you write filters dynamically that become too large they can start to take a significant amount of time to compile but we should be able to work around that by threading the bpf compilation step (another use for the 'when' statement?). Compiled BPF filter insertion time doesn't seem to increase with an increase in the size of the filter either. There is another implicit problem with this example that I don't like in that it relies on TCP offsets which don't work in BPF for IPv6 traffic and I don't expect that to start working anytime soon. I'm not necessarily fighting for or against the secondary path, but I'm having a hard time finding a concrete example that can't be done without the secondary path. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Jul 14 00:00:10 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 14 Jul 2012 00:00:10 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207140700.q6E70ASL009884@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | ce05600 | Seth Hall | 2012-07-13 | Mozilla's current certificate bundle. [3] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch [3] fastpath: http://tracker.bro-ids.org/bro/changeset/ce05600a717e31f36170d6c47dabd91bd914cd2d/bro From bro at tracker.bro-ids.org Sat Jul 14 01:28:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 14 Jul 2012 08:28:06 -0000 Subject: [Bro-Dev] #848: Crashes in sub-threads do not propagate to main Bro In-Reply-To: <048.f7796931648451485812b4da6626efe2@tracker.bro-ids.org> References: <048.f7796931648451485812b4da6626efe2@tracker.bro-ids.org> Message-ID: <063.1e04eac6e3ab7aaa685b94e25d57d978@tracker.bro-ids.org> #848: Crashes in sub-threads do not propagate to main Bro -----------------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by amannb): * status: new => closed * resolution: => Solved/Applied Comment: Should be solved by e1bd9609264a4d067e3c58016806877f0f859c8d -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sun Jul 15 00:00:37 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 15 Jul 2012 00:00:37 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207150700.q6F70bUU009758@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | ce05600 | Seth Hall | 2012-07-13 | Mozilla's current certificate bundle. [3] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch [3] fastpath: http://tracker.bro-ids.org/bro/changeset/ce05600a717e31f36170d6c47dabd91bd914cd2d/bro From noreply at bro-ids.org Mon Jul 16 00:02:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 16 Jul 2012 00:02:04 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207160702.q6G7244e029349@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | ce05600 | Seth Hall | 2012-07-13 | Mozilla's current certificate bundle. [3] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch [3] fastpath: http://tracker.bro-ids.org/bro/changeset/ce05600a717e31f36170d6c47dabd91bd914cd2d/bro From jsiwek at illinois.edu Mon Jul 16 13:22:00 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Mon, 16 Jul 2012 20:22:00 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Fixed a couple of init-time mem leaks. (750e1dd) In-Reply-To: <201207161952.q6GJq2Pk008570@bro-ids.icir.org> References: <201207161952.q6GJq2Pk008570@bro-ids.icir.org> Message-ID: > diff --git a/src/scan.l b/src/scan.l > index 645ce65..d90501d 100644 > --- a/src/scan.l > +++ b/src/scan.l > @@ -776,7 +776,7 @@ void add_input_file(const char* file) > if ( ! filename ) > (void) load_files(file); > else > - input_files.append(copy_string(file)); > + input_files.append((char*) file); > } Not sure that's going to work right: one call in scan.l gives as input the c_str() from an automatic std::string that goes out of scope soon afterwards, so it's relying on that copy to happen. Maybe a better thing would be to change input_files to use standard container types (e.g. vector of string or something). Jon From robin at icir.org Mon Jul 16 13:31:29 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 16 Jul 2012 13:31:29 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Fixed a couple of init-time mem leaks. (750e1dd) In-Reply-To: References: <201207161952.q6GJq2Pk008570@bro-ids.icir.org> Message-ID: <20120716203129.GB1624@icir.org> Also, we actually have a lot of init-time leaks, and I'm not sure it's worth fixing them individually. The chance of breaking something subtlety is quite large. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Mon Jul 16 13:36:39 2012 From: seth at icir.org (Seth Hall) Date: Mon, 16 Jul 2012 16:36:39 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Fixed a couple of init-time mem leaks. (750e1dd) In-Reply-To: <20120716203129.GB1624@icir.org> References: <201207161952.q6GJq2Pk008570@bro-ids.icir.org> <20120716203129.GB1624@icir.org> Message-ID: <332C172B-84D1-4F0A-A227-D7FC47DE58E2@icir.org> On Jul 16, 2012, at 4:31 PM, Robin Sommer wrote: > Also, we actually have a lot of init-time leaks, and I'm not sure it's > worth fixing them individually. The chance of breaking something > subtlety is quite large. True, feel free to get rid of that commit instead of merging. I was just trying to remove a couple of known memory leaks to try and find if we are in fact leaking memory somewhere. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Mon Jul 16 18:48:55 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 17 Jul 2012 01:48:55 -0000 Subject: [Bro-Dev] #763: Escape # when first character in log file line In-Reply-To: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> References: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> Message-ID: <063.6836bcb5790d3f5baa165d75f2da21c0@tracker.bro-ids.org> #763: Escape # when first character in log file line ----------------------+------------------------ Reporter: amannb | Owner: robin Type: Problem | Status: closed Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------+------------------------ Changes (by robin): * status: reopened => closed * resolution: => fixed Comment: In [8453f5958deed16be8a6569e7985c2a0777ce95d/bro]: {{{ #!CommitTicketReference repository="bro" revision="8453f5958deed16be8a6569e7985c2a0777ce95d" Reworking thread termination logic. Turns out the finish methods weren't called correctly, caused by a mess up with method names which all sounded too similar and the wrong one ended up being called. I've reworked this by changing the thread/writer/reader interfaces, which actually also simplifies them by getting rid of the requirement for writer backends to call their parent methods (i.e., less opportunity for errors). This commit also includes the following (because I noticed the problem above when working on some of these): - The ASCII log writer now includes "#start " and "#end lines in the each file. The latter supersedes Bernhard's "EOF" patch. This required a number of tests updates. The standard canonifier removes the timestamps, but some tests compare files directly, which doesn't work if they aren't printing out the same timestamps (like the comm tests). - The above required yet another change to the writer API to network_time to methods. - Renamed ASCII logger "header" options to "meta". - Fixes #763 "Escape # when first character in log file line". All btests pass for me on Linux FC15. Will try MacOS next. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 16 19:11:41 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 17 Jul 2012 02:11:41 -0000 Subject: [Bro-Dev] #434: Fix secondary path In-Reply-To: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> References: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> Message-ID: <062.bb46c88cfbdb38204287a6ec6119bcea@tracker.bro-ids.org> #434: Fix secondary path ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by vern): I believe your math is off. The default large-conns filter captures packets falling into 4*16KB = 64KB of the sequence space. If data packets average 1KB and ACK packets average 2KB (due to ack-every-other), then that will be a typical total of 64+32 = 96 packets for every 4GB transferred. Capturing all ACK packets, OTOH, will result in matching 4GB/2KB = 2M packets. So it's a very large difference. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jul 16 20:20:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 17 Jul 2012 03:20:20 -0000 Subject: [Bro-Dev] #434: Fix secondary path In-Reply-To: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> References: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> Message-ID: <062.833cccf14b348432990da9d950a1e2aa@tracker.bro-ids.org> #434: Fix secondary path ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by seth): > I believe your math is off. Damn, I'll think about this more. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Jul 17 00:01:32 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 17 Jul 2012 00:01:32 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207170701.q6H71W93018720@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 750e1dd | Seth Hall | 2012-07-16 | Fixed a couple of init-time mem leaks. [3] bro | ce05600 | Seth Hall | 2012-07-13 | Mozilla's current certificate bundle. [4] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch [3] fastpath: http://tracker.bro-ids.org/bro/changeset/750e1ddf69d9f3375801615e872ec42b8a8d5a6d/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/ce05600a717e31f36170d6c47dabd91bd914cd2d/bro From bernhard at ICSI.Berkeley.EDU Tue Jul 17 14:22:47 2012 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Tue, 17 Jul 2012 14:22:47 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Fixed a couple of init-time mem leaks. (750e1dd) In-Reply-To: <332C172B-84D1-4F0A-A227-D7FC47DE58E2@icir.org> References: <201207161952.q6GJq2Pk008570@bro-ids.icir.org> <20120716203129.GB1624@icir.org> <332C172B-84D1-4F0A-A227-D7FC47DE58E2@icir.org> Message-ID: I am not a hundred percent sure - but quite sure that that commit breaks core/load-prefixes.bro Bernhard On Jul 16, 2012, at 1:36 PM, Seth Hall wrote: > > On Jul 16, 2012, at 4:31 PM, Robin Sommer wrote: > >> Also, we actually have a lot of init-time leaks, and I'm not sure it's >> worth fixing them individually. The chance of breaking something >> subtlety is quite large. > > > True, feel free to get rid of that commit instead of merging. I was just trying to remove a couple of known memory leaks to try and find if we are in fact leaking memory somewhere. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From noreply at bro-ids.org Wed Jul 18 00:01:15 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 18 Jul 2012 00:01:15 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207180701.q6I71FPX020410@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 81edec8 | Bernhard Amann | 2012-07-17 | Fix bug, where in dns.log rcode always was set to 0/NOERROR when no reply package was seen. [3] bro | 750e1dd | Seth Hall | 2012-07-16 | Fixed a couple of init-time mem leaks. [4] bro | ce05600 | Seth Hall | 2012-07-13 | Mozilla's current certificate bundle. [5] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch [3] fastpath: http://tracker.bro-ids.org/bro/changeset/81edec8b2eeef682c4bb2639a0b191e12bc2f561/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/750e1ddf69d9f3375801615e872ec42b8a8d5a6d/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/ce05600a717e31f36170d6c47dabd91bd914cd2d/bro From bro at tracker.bro-ids.org Wed Jul 18 08:04:05 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 18 Jul 2012 15:04:05 -0000 Subject: [Bro-Dev] #853: problem with VLAN/MPLS packet dumping Message-ID: <048.ef8e24afbef1a1e09893e7e63ae551fc@tracker.bro-ids.org> #853: problem with VLAN/MPLS packet dumping ---------------------+------------------------ Reporter: jsiwek | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ report from Carsten Langer: {{{ By the way: you have in my opinion a problem with packet dumping. If the trace contains VLAN or MPLS, you strip off VLAN/MPLS and if then you dump the packet, then the dumped trace is missing the Ethernet header for these packets, while the Ethernet header is still there for packets which did not have VLAN/MPLS. My previous GTP-detunneling did the same mistake, now I have introduced a fake Ethernet header so that if the packet is dumped, is still has its Ethernet header. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 18 08:06:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 18 Jul 2012 15:06:18 -0000 Subject: [Bro-Dev] #854: problem with VLAN/MPLS packet dumping Message-ID: <048.59e87830ffcda9e10e957fd40878e16c@tracker.bro-ids.org> #854: problem with VLAN/MPLS packet dumping ---------------------+------------------------ Reporter: jsiwek | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ report from Carsten Langer: {{{ By the way: you have in my opinion a problem with packet dumping. If the trace contains VLAN or MPLS, you strip off VLAN/MPLS and if then you dump the packet, then the dumped trace is missing the Ethernet header for these packets, while the Ethernet header is still there for packets which did not have VLAN/MPLS. My previous GTP-detunneling did the same mistake, now I have introduced a fake Ethernet header so that if the packet is dumped, is still has its Ethernet header. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 18 08:10:22 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 18 Jul 2012 15:10:22 -0000 Subject: [Bro-Dev] #853: problem with VLAN/MPLS packet dumping In-Reply-To: <048.ef8e24afbef1a1e09893e7e63ae551fc@tracker.bro-ids.org> References: <048.ef8e24afbef1a1e09893e7e63ae551fc@tracker.bro-ids.org> Message-ID: <063.82b63477af0cdd9a451aa6d5198f6883@tracker.bro-ids.org> #853: problem with VLAN/MPLS packet dumping ------------------------+------------------------ Reporter: jsiwek | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: Duplicate | Keywords: ------------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Duplicate Comment: dup of #854 -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Jul 18 10:48:25 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 18 Jul 2012 10:48:25 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Fixing calc_next_rotate to use UTC based time functions. (6335dbb) In-Reply-To: <201207181743.q6IHhoiJ004990@bro-ids.icir.org> References: <201207181743.q6IHhoiJ004990@bro-ids.icir.org> Message-ID: <20120718174825.GI7392@icir.org> On Wed, Jul 18, 2012 at 10:43 -0700, Seth Hall wrote: > Fixing calc_next_rotate to use UTC based time functions. Does that mean it's now rotating on UTC time? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Wed Jul 18 11:09:48 2012 From: seth at icir.org (Seth Hall) Date: Wed, 18 Jul 2012 14:09:48 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Fixing calc_next_rotate to use UTC based time functions. (6335dbb) In-Reply-To: <20120718174825.GI7392@icir.org> References: <201207181743.q6IHhoiJ004990@bro-ids.icir.org> <20120718174825.GI7392@icir.org> Message-ID: <72A6F9F1-6738-44E8-BFE3-B5B5F1884FBE@icir.org> On Jul 18, 2012, at 1:48 PM, Robin Sommer wrote: > On Wed, Jul 18, 2012 at 10:43 -0700, Seth Hall wrote: > >> Fixing calc_next_rotate to use UTC based time functions. > > Does that mean it's now rotating on UTC time? Oh, yes. I wasn't considering that you'd want the rotation to happen on local time. I was bitten by the file rotation confusion again. This is apparently another change you can ignore. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Wed Jul 18 14:54:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 18 Jul 2012 21:54:51 -0000 Subject: [Bro-Dev] #855: unhelpful bro-cut error message for bad field Message-ID: <046.55ddac95c0b7d9e5de47f5b5061741f5@tracker.bro-ids.org> #855: unhelpful bro-cut error message for bad field ---------------------+------------------------ Reporter: vern | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ If I run "{{{bro-cut foo}}}" on a log file that doesn't have a field named {{{foo}}}, the message it generates is: {{{ bro-cut error: unknown field f[i] }}} Looks like a level of de-referencing got left off. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 18 15:06:12 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 18 Jul 2012 22:06:12 -0000 Subject: [Bro-Dev] #856: more documentation for utilities would be cool Message-ID: <046.83c9a4438eabd7f9583c66c29c3e71c7@tracker.bro-ids.org> #856: more documentation for utilities would be cool -----------------------------+------------------------ Reporter: vern | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Component: bro-aux | Version: git/master Keywords: | -----------------------------+------------------------ Utilities like bro-cut only supply --help documentation, as far as I can tell. Man pages would be handy. (In particular, I was looking for some sort of statement of exactly to what degree bro-cut can munch on the concatenation of multiple log files that have different column layouts.) -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 18 15:07:08 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 18 Jul 2012 22:07:08 -0000 Subject: [Bro-Dev] #855: unhelpful bro-cut error message for bad field In-Reply-To: <046.55ddac95c0b7d9e5de47f5b5061741f5@tracker.bro-ids.org> References: <046.55ddac95c0b7d9e5de47f5b5061741f5@tracker.bro-ids.org> Message-ID: <061.874d7264a550c8b864ca8e1e237fb412@tracker.bro-ids.org> #855: unhelpful bro-cut error message for bad field ----------------------+------------------------ Reporter: vern | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: bro-aux | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by vern): * component: Bro => bro-aux -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Jul 19 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 19 Jul 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207190700.q6J702aC031097@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 6335dbb | Seth Hall | 2012-07-18 | Fixing calc_next_rotate to use UTC based time functions. [3] bro | 81edec8 | Bernhard Amann | 2012-07-17 | Fix bug, where in dns.log rcode always was set to 0/NOERROR when no reply package was seen. [4] bro | 750e1dd | Seth Hall | 2012-07-16 | Fixed a couple of init-time mem leaks. [5] bro | ce05600 | Seth Hall | 2012-07-13 | Mozilla's current certificate bundle. [6] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch [3] fastpath: http://tracker.bro-ids.org/bro/changeset/6335dbb5e1cf694afea3c306012a258614d13880/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/81edec8b2eeef682c4bb2639a0b191e12bc2f561/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/750e1ddf69d9f3375801615e872ec42b8a8d5a6d/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/ce05600a717e31f36170d6c47dabd91bd914cd2d/bro From bro at tracker.bro-ids.org Thu Jul 19 10:59:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 19 Jul 2012 17:59:48 -0000 Subject: [Bro-Dev] #857: Change capture port in HTTP analyzer from 3138/tcp instead of 3128/tcp Message-ID: <049.2545d71b81bdfb740467af0719e58c1f@tracker.bro-ids.org> #857: Change capture port in HTTP analyzer from 3138/tcp instead of 3128/tcp ------------------------+--------------------- Reporter: aashish | Type: Problem Status: new | Priority: High Milestone: | Component: Bro Version: git/master | Keywords: ------------------------+--------------------- Port definitions in main.bro in ../share/bro/base/protocols/http/main.bro has 3138/tcp defined in structures "ports", "likely_server_ports" and "capture_filters" This should be 3128/tcp to capture traffic for squid proxy. Config below: # DPD configuration. const ports = { 80/tcp, 81/tcp, 631/tcp, 1080/tcp, 3138/tcp, 8000/tcp, 8080/tcp, 8888/tcp, }; redef dpd_config += { [[ANALYZER_HTTP, ANALYZER_HTTP_BINPAC]] = [$ports = ports], }; redef capture_filters += { ["http"] = "tcp and port (80 or 81 or 631 or 1080 or 3138 or 8000 or 8080 or 8888)" }; redef likely_server_ports += { 80/tcp, 81/tcp, 631/tcp, 1080/tcp, 3138/tcp, 8000/tcp, 8080/tcp, 8888/tcp, }; -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 19 11:02:44 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 19 Jul 2012 18:02:44 -0000 Subject: [Bro-Dev] #857: Change capture port in HTTP analyzer from 3138/tcp instead of 3128/tcp In-Reply-To: <049.2545d71b81bdfb740467af0719e58c1f@tracker.bro-ids.org> References: <049.2545d71b81bdfb740467af0719e58c1f@tracker.bro-ids.org> Message-ID: <064.e443f80914bc210505082870a1f1b114@tracker.bro-ids.org> #857: Change capture port in HTTP analyzer from 3138/tcp instead of 3128/tcp ----------------------+------------------------ Reporter: aashish | Owner: Type: Problem | Status: new Priority: High | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by aashish): Doh! Incorrect title: Rephrasing, we need to change port from 3138/tcp to 3128/tcp. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Jul 20 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 20 Jul 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207200700.q6K702qe020426@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 6335dbb | Seth Hall | 2012-07-18 | Fixing calc_next_rotate to use UTC based time functions. [3] bro | 81edec8 | Bernhard Amann | 2012-07-17 | Fix bug, where in dns.log rcode always was set to 0/NOERROR when no reply package was seen. [4] bro | 750e1dd | Seth Hall | 2012-07-16 | Fixed a couple of init-time mem leaks. [5] bro | ce05600 | Seth Hall | 2012-07-13 | Mozilla's current certificate bundle. [6] broctl | 57142bb | Jon Siwek | 2012-07-19 | Fix broctl startup when using custom config file dirs. [7] broctl | 7782a6b | Jon Siwek | 2012-07-19 | Change crash report info to include stack traces from all threads. [8] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch [3] fastpath: http://tracker.bro-ids.org/bro/changeset/6335dbb5e1cf694afea3c306012a258614d13880/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/81edec8b2eeef682c4bb2639a0b191e12bc2f561/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/750e1ddf69d9f3375801615e872ec42b8a8d5a6d/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/ce05600a717e31f36170d6c47dabd91bd914cd2d/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/57142bb5d6accca07d7b1a7e6633d6c3e0052c8b/broctl [8] fastpath: http://tracker.bro-ids.org/bro/changeset/7782a6b85cbc16c16ccf2f74434e1b94315c679c/broctl From vladg at cmu.edu Fri Jul 20 08:16:42 2012 From: vladg at cmu.edu (Vlad Grigorescu) Date: Fri, 20 Jul 2012 15:16:42 +0000 Subject: [Bro-Dev] SMTP Entities MD5 Hash Defaults Message-ID: Hi all, Currently, SMTP entities will calculate MD5 hashes for the following filetypes by default: application/x-dosexec, application/x-executable. I was a little surprised that common e-mail attack vectors like zip and PDF files don't have this hash calculated by default. I propose extending the default to also include application/zip and application/pdf. I think this is good default functionality, that won't cause a noticeable performance hit. Thoughts? Any other filetypes that would be useful to add there, while we're at it? --Vlad From bro at tracker.bro-ids.org Fri Jul 20 09:28:30 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 20 Jul 2012 16:28:30 -0000 Subject: [Bro-Dev] #763: Escape # when first character in log file line In-Reply-To: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> References: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> Message-ID: <063.f8724c2013194f83483a4329bec8279e@tracker.bro-ids.org> #763: Escape # when first character in log file line ----------------------+------------------------ Reporter: amannb | Owner: robin Type: Problem | Status: closed Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------+------------------------ Comment (by robin): In [f7a6407ab1213d95f074e47c39061f541f630944/bro]: {{{ #!CommitTicketReference repository="bro" revision="f7a6407ab1213d95f074e47c39061f541f630944" Reworking thread termination logic. Turns out the finish methods weren't called correctly, caused by a mess up with method names which all sounded too similar and the wrong one ended up being called. I've reworked this by changing the thread/writer/reader interfaces, which actually also simplifies them by getting rid of the requirement for writer backends to call their parent methods (i.e., less opportunity for errors). This commit also includes the following (because I noticed the problem above when working on some of these): - The ASCII log writer now includes "#start " and "#end lines in the each file. The latter supersedes Bernhard's "EOF" patch. This required a number of tests updates. The standard canonifier removes the timestamps, but some tests compare files directly, which doesn't work if they aren't printing out the same timestamps (like the comm tests). - The above required yet another change to the writer API to network_time to methods. - Renamed ASCII logger "header" options to "meta". - Fixes #763 "Escape # when first character in log file line". All btests pass for me on Linux FC15. Will try MacOS next. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 20 09:28:30 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 20 Jul 2012 16:28:30 -0000 Subject: [Bro-Dev] #763: Escape # when first character in log file line In-Reply-To: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> References: <048.3bfb7266147705e69563ae3f5f7bab89@tracker.bro-ids.org> Message-ID: <063.7fbd994dd2a60939e916874afaabd9b0@tracker.bro-ids.org> #763: Escape # when first character in log file line ----------------------+------------------------ Reporter: amannb | Owner: robin Type: Problem | Status: closed Priority: High | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------+------------------------ Comment (by robin): In [f73eb3b086c1ae88c122434613501af950a9dba0/bro]: {{{ #!CommitTicketReference repository="bro" revision="f73eb3b086c1ae88c122434613501af950a9dba0" Reworking thread termination logic. Turns out the finish methods weren't called correctly, caused by a mess up with method names which all sounded too similar and the wrong one ended up being called. I've reworked this by changing the thread/writer/reader interfaces, which actually also simplifies them by getting rid of the requirement for writer backends to call their parent methods (i.e., less opportunity for errors). This commit also includes the following (because I noticed the problem above when working on some of these): - The ASCII log writer now includes "#start " and "#end lines in the each file. The latter supersedes Bernhard's "EOF" patch. This required a number of tests updates. The standard canonifier removes the timestamps, but some tests compare files directly, which doesn't work if they aren't printing out the same timestamps (like the comm tests). - The above required yet another change to the writer API to network_time to methods. - Renamed ASCII logger "header" options to "meta". - Fixes #763 "Escape # when first character in log file line". All btests pass for me on Linux FC15. Will try MacOS next. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Fri Jul 20 09:36:31 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 20 Jul 2012 09:36:31 -0700 Subject: [Bro-Dev] topic/robin/master-test status In-Reply-To: <201207201628.q6KGSVQ2027250@bro-ids.icir.org> References: <201207201628.q6KGSVQ2027250@bro-ids.icir.org> Message-ID: <20120720163631.GK54398@icir.org> This branch has all my recent threading changes, as well as all the pending merge requests / fastpath commits. My plan is for this to become master asap, but it's not quite there yet: - I need to do more testing. Tried only Fedora Linux so far (and don't know yet if the earlier Mac problem is still there). - scripts.base.frameworks.logging.rotate fails sporadically by leaving out one rotation it seems. - When I add tuning/logs-to-elasticsearch.bro to test-all-policy, some tests hang. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Fri Jul 20 10:08:04 2012 From: seth at icir.org (Seth Hall) Date: Fri, 20 Jul 2012 13:08:04 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/master-test: Temporarily removing tuning/logs-to-elasticsearch.bro from the test-all-policy. (c5d1aeb) In-Reply-To: <201207201629.q6KGT4Rj027400@bro-ids.icir.org> References: <201207201629.q6KGT4Rj027400@bro-ids.icir.org> Message-ID: On Jul 20, 2012, at 12:29 PM, Robin Sommer wrote: > Temporarily removing tuning/logs-to-elasticsearch.bro from the > test-all-policy. > > Loading it in there can lead to some tests not terminating. We need to > fix that, it let's the coverage test fail. I'll play with it. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Fri Jul 20 10:10:14 2012 From: seth at icir.org (Seth Hall) Date: Fri, 20 Jul 2012 13:10:14 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/master-test: Merge remote-tracking branch 'remotes/origin/topic/seth/elasticsearch' into topic/robin/master-test (eef8b7d) In-Reply-To: <201207201629.q6KGT0Jo027382@bro-ids.icir.org> References: <201207201629.q6KGT0Jo027382@bro-ids.icir.org> Message-ID: <0764863C-24CC-49CF-80B8-895432EA1D20@icir.org> On Jul 20, 2012, at 12:29 PM, Robin Sommer wrote: > I've only tested that it compiles, not whether it still works. The > fact that we don't have any tests for this makes me uneasy ... I'll see if I can add a couple of tests. I suppose I can use nc to open a port to make sure that I get the correct data for a trace files being sent to elastic search. Definitely not the best approach, but it would be a bit more difficult to mock up with the full elastic search server. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Fri Jul 20 10:14:07 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 20 Jul 2012 10:14:07 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/master-test: Merge remote-tracking branch 'remotes/origin/topic/seth/elasticsearch' into topic/robin/master-test (eef8b7d) In-Reply-To: <0764863C-24CC-49CF-80B8-895432EA1D20@icir.org> References: <201207201629.q6KGT0Jo027382@bro-ids.icir.org> <0764863C-24CC-49CF-80B8-895432EA1D20@icir.org> Message-ID: <20120720171407.GA75482@icir.org> On Fri, Jul 20, 2012 at 13:10 -0400, you wrote: > I'll see if I can add a couple of tests. I suppose I can use nc to > open a port to make sure that I get the correct data for a trace files > being sent to elastic search. Yeah, that may be good compromise. Or maybe even a simple Python-based HTTP server that just logs what gets, that shouldn't be more than a few lines of Python code. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Fri Jul 20 10:18:56 2012 From: seth at icir.org (Seth Hall) Date: Fri, 20 Jul 2012 13:18:56 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/master-test: Merge remote-tracking branch 'remotes/origin/topic/seth/elasticsearch' into topic/robin/master-test (eef8b7d) In-Reply-To: <20120720171407.GA75482@icir.org> References: <201207201629.q6KGT0Jo027382@bro-ids.icir.org> <0764863C-24CC-49CF-80B8-895432EA1D20@icir.org> <20120720171407.GA75482@icir.org> Message-ID: On Jul 20, 2012, at 1:14 PM, Robin Sommer wrote: > On Fri, Jul 20, 2012 at 13:10 -0400, you wrote: > >> I'll see if I can add a couple of tests. I suppose I can use nc to >> open a port to make sure that I get the correct data for a trace files >> being sent to elastic search. > > Yeah, that may be good compromise. Or maybe even a simple Python-based > HTTP server that just logs what gets, that shouldn't be more than a > few lines of Python code. Yeah, that's what I started to think right after I sent the email. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Fri Jul 20 10:25:17 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 20 Jul 2012 17:25:17 -0000 Subject: [Bro-Dev] #857: Change capture port in HTTP analyzer from 3138/tcp instead of 3128/tcp In-Reply-To: <049.2545d71b81bdfb740467af0719e58c1f@tracker.bro-ids.org> References: <049.2545d71b81bdfb740467af0719e58c1f@tracker.bro-ids.org> Message-ID: <064.7b917e020fa89f566f47cf3dd40cc689@tracker.bro-ids.org> #857: Change capture port in HTTP analyzer from 3138/tcp instead of 3128/tcp ----------------------+------------------------ Reporter: aashish | Owner: Type: Problem | Status: new Priority: High | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): In [ce4b8dd4aca99c4e1013b5c843df30bfedc54cfd/bro]: {{{ #!CommitTicketReference repository="bro" revision="ce4b8dd4aca99c4e1013b5c843df30bfedc54cfd" Changing HTTP DPD port 3138 to 3128. Addresses #857. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Fri Jul 20 12:22:51 2012 From: seth at icir.org (Seth Hall) Date: Fri, 20 Jul 2012 15:22:51 -0400 Subject: [Bro-Dev] SMTP Entities MD5 Hash Defaults In-Reply-To: References: Message-ID: <46FD78FE-07A4-4EBB-9197-50DF4A24ABAE@icir.org> On Jul 20, 2012, at 11:16 AM, Vlad Grigorescu wrote: > Currently, SMTP entities will calculate MD5 hashes for the following > filetypes by default: application/x-dosexec, application/x-executable  Would you be up for just writing a script that does it for now? Maybe also a script that checks SMTP hashes with the malware hash registry like we're doing for HTTP? I'm not crazy about doing much work on the pre-2.2 because once the file analysis framework is integrated everything will be different and much better anyway. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From noreply at bro-ids.org Sat Jul 21 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 21 Jul 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207210700.q6L702DD008254@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 58e2b70 | Bernhard Amann | 2012-07-20 | make version_ok return true for TLSv12 [3] bro | 0a68136 | Robin Sommer | 2012-07-20 | Revert "Fixing calc_next_rotate to use UTC based time functions." [4] bro | 6335dbb | Seth Hall | 2012-07-18 | Fixing calc_next_rotate to use UTC based time functions. [5] bro | 50f5f81 | Robin Sommer | 2012-07-18 | Revert "Fixed a couple of init-time mem leaks." [6] bro | 81edec8 | Bernhard Amann | 2012-07-17 | Fix bug, where in dns.log rcode always was set to 0/NOERROR when no reply package was seen. [7] bro | 750e1dd | Seth Hall | 2012-07-16 | Fixed a couple of init-time mem leaks. [8] bro | ce05600 | Seth Hall | 2012-07-13 | Mozilla's current certificate bundle. [9] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch [3] fastpath: http://tracker.bro-ids.org/bro/changeset/58e2b70fc806621a833d13a88fbee4562f6753ba/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/0a681367b70e03fbb938146ec497546aa01d4ec8/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/6335dbb5e1cf694afea3c306012a258614d13880/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/50f5f8131df7691643209ccf2d058ab98a4ba6ad/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/81edec8b2eeef682c4bb2639a0b191e12bc2f561/bro [8] fastpath: http://tracker.bro-ids.org/bro/changeset/750e1ddf69d9f3375801615e872ec42b8a8d5a6d/bro [9] fastpath: http://tracker.bro-ids.org/bro/changeset/ce05600a717e31f36170d6c47dabd91bd914cd2d/bro From noreply at bro-ids.org Sun Jul 22 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 22 Jul 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207220700.q6M7022N030567@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 58e2b70 | Bernhard Amann | 2012-07-20 | make version_ok return true for TLSv12 [3] bro | 0a68136 | Robin Sommer | 2012-07-20 | Revert "Fixing calc_next_rotate to use UTC based time functions." [4] bro | 6335dbb | Seth Hall | 2012-07-18 | Fixing calc_next_rotate to use UTC based time functions. [5] bro | 50f5f81 | Robin Sommer | 2012-07-18 | Revert "Fixed a couple of init-time mem leaks." [6] bro | 81edec8 | Bernhard Amann | 2012-07-17 | Fix bug, where in dns.log rcode always was set to 0/NOERROR when no reply package was seen. [7] bro | 750e1dd | Seth Hall | 2012-07-16 | Fixed a couple of init-time mem leaks. [8] bro | ce05600 | Seth Hall | 2012-07-13 | Mozilla's current certificate bundle. [9] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch [3] fastpath: http://tracker.bro-ids.org/bro/changeset/58e2b70fc806621a833d13a88fbee4562f6753ba/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/0a681367b70e03fbb938146ec497546aa01d4ec8/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/6335dbb5e1cf694afea3c306012a258614d13880/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/50f5f8131df7691643209ccf2d058ab98a4ba6ad/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/81edec8b2eeef682c4bb2639a0b191e12bc2f561/bro [8] fastpath: http://tracker.bro-ids.org/bro/changeset/750e1ddf69d9f3375801615e872ec42b8a8d5a6d/bro [9] fastpath: http://tracker.bro-ids.org/bro/changeset/ce05600a717e31f36170d6c47dabd91bd914cd2d/bro From noreply at bro-ids.org Mon Jul 23 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 23 Jul 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207230700.q6N702Id011538@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 58e2b70 | Bernhard Amann | 2012-07-20 | make version_ok return true for TLSv12 [3] bro | 0a68136 | Robin Sommer | 2012-07-20 | Revert "Fixing calc_next_rotate to use UTC based time functions." [4] bro | 6335dbb | Seth Hall | 2012-07-18 | Fixing calc_next_rotate to use UTC based time functions. [5] bro | 50f5f81 | Robin Sommer | 2012-07-18 | Revert "Fixed a couple of init-time mem leaks." [6] bro | 81edec8 | Bernhard Amann | 2012-07-17 | Fix bug, where in dns.log rcode always was set to 0/NOERROR when no reply package was seen. [7] bro | 750e1dd | Seth Hall | 2012-07-16 | Fixed a couple of init-time mem leaks. [8] bro | ce05600 | Seth Hall | 2012-07-13 | Mozilla's current certificate bundle. [9] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch [3] fastpath: http://tracker.bro-ids.org/bro/changeset/58e2b70fc806621a833d13a88fbee4562f6753ba/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/0a681367b70e03fbb938146ec497546aa01d4ec8/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/6335dbb5e1cf694afea3c306012a258614d13880/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/50f5f8131df7691643209ccf2d058ab98a4ba6ad/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/81edec8b2eeef682c4bb2639a0b191e12bc2f561/bro [8] fastpath: http://tracker.bro-ids.org/bro/changeset/750e1ddf69d9f3375801615e872ec42b8a8d5a6d/bro [9] fastpath: http://tracker.bro-ids.org/bro/changeset/ce05600a717e31f36170d6c47dabd91bd914cd2d/bro From hlin33 at illinois.edu Mon Jul 23 14:46:29 2012 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Mon, 23 Jul 2012 16:46:29 -0500 Subject: [Bro-Dev] Hui Lin_std::length_error Message-ID: HI, So far my DNP3 analzyer works OK on well-formatted DNP3 dump, even the packets are not in the right order. However, when I test it again some fuzzied DNP3 packets, this error sometimes happens. The weird thing is that, I run the same dump several times, sometimes, it can finish the work with weird.log, and sometimes Bro throws out this error. terminate called after throwing an instance of 'std::length_error' what(): vector::reserve Aborted Any idea? -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120723/f3a789cc/attachment.html From robin at icir.org Mon Jul 23 17:08:52 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 23 Jul 2012 17:08:52 -0700 Subject: [Bro-Dev] Master-test merge (Re: [Bro-Commits] [git/bro] master: Merge branch 'topic/robin/master-test' (24aea29)) In-Reply-To: <201207240001.q6O01F5H013994@bro-ids.icir.org> Message-ID: <20120724000852.GJ77331@icir.org> On Mon, Jul 23, 2012 at 17:01 -0700, I wrote: > Merge branch 'topic/robin/master-test' This has the recent threading changes, plus all other pending merge requests. Please test, I hope it doesn't break anything .. Problems remaining: - Occasional tests failures reporting a bad file descriptor on Mac OS. Reason still unclear. - #start timestamps can be 1969-... when network time is not yet set. - tuning/logs-to-elasticsearch.bro not loaded by test-all-policy because it triggers lock-ups. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Mon Jul 23 22:16:04 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 24 Jul 2012 05:16:04 -0000 Subject: [Bro-Dev] #858: Logging framework stops completely when DoInit returns false Message-ID: <048.904aec95fb55939499acbd12b70008ad@tracker.bro-ids.org> #858: Logging framework stops completely when DoInit returns false ---------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ This has to be introduced quite recently - I tried to find the reason for the bug, but I am just too blind at the Moment. When one of the log writers returns from DoInit with false the whole logging framework just seems to stop - no output is done anymore to any file. To reproduce - change the return code of DoInit in None.cc to false and execute the attached Bro script. Logs will be written until the first http request is encountered. Starting from that moment, all activity stops and no more log messages are output to any file (including reporter/debug.log). -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Jul 24 00:00:01 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 24 Jul 2012 00:00:01 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207240700.q6O701ju011489@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 850 [1] | seth | | Normal | topic/seth/elasticsearch [2] [1] #850: http://tracker.bro-ids.org/bro/ticket/850 [2] elasticsearch: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/elasticsearch From bro at tracker.bro-ids.org Tue Jul 24 06:51:43 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 24 Jul 2012 13:51:43 -0000 Subject: [Bro-Dev] #850: topic/seth/elasticsearch In-Reply-To: <046.c8ab851cf8d2d187966f09f844526fca@tracker.bro-ids.org> References: <046.c8ab851cf8d2d187966f09f844526fca@tracker.bro-ids.org> Message-ID: <061.2aa71ec69a9f33537fea093c5aa2fe22@tracker.bro-ids.org> #850: topic/seth/elasticsearch -----------------------------+------------------------ Reporter: seth | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 24 07:02:15 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 24 Jul 2012 14:02:15 -0000 Subject: [Bro-Dev] #634: CouchDB writer In-Reply-To: <053.06c318cee769c34ae468c38f0621a66a@tracker.bro-ids.org> References: <053.06c318cee769c34ae468c38f0621a66a@tracker.bro-ids.org> Message-ID: <068.85a46a649d278e4ce5a74132d0aed734@tracker.bro-ids.org> #634: CouchDB writer --------------------------+-------------------- Reporter: jeff.baumes | Owner: Type: patch | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: Resolution: | Keywords: --------------------------+-------------------- Comment (by jeff.baumes): Unfortunately I do not have the time to do this, at least in the near term. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Tue Jul 24 07:41:23 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 24 Jul 2012 07:41:23 -0700 Subject: [Bro-Dev] Hui Lin_std::length_error In-Reply-To: References: Message-ID: <20120724144123.GO82027@icir.org> On Mon, Jul 23, 2012 at 16:46 -0500, you wrote: > terminate called after throwing an instance of 'std::length_error' > what(): vector::reserve A guess without looking at the code: could it be taking a value from the input to then determine the size of a vector? If so, a fuzzed value may turn into a value larger than the vector can grow to. If you attach a debugger, it should show you where the exception is raised (in the generated binpac code, though, so you may need to do a bit of digging what the corresponding .pac part is). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Tue Jul 24 12:04:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 24 Jul 2012 19:04:03 -0000 Subject: [Bro-Dev] #859: to_double bif Message-ID: <051.a2e917e9762f5bbb7a4a43f5902cd95a@tracker.bro-ids.org> #859: to_double bif ------------------------+----------------------------- Reporter: scampbell | Type: Feature Request Status: new | Priority: Normal Milestone: Bro2.1 | Component: Bro Version: git/master | Keywords: bif ------------------------+----------------------------- It would be quite useful to include a to_double bif as the new input framework makes this sort of thing much more common when digesting logs. sample code: ## Converts a :bro:type:`string` to a :bro:type:`double`. ## ## str: The :bro:type:`string` to convert. ## ## Returns: The :bro:type:`string` *str* as double, or 0 if *str* has ## an invalid format. ## function to_double%(str: string%): count %{ const char* s = str->CheckString(); char* end_s; double d = (double) strtod(s, &end_s); if ( s[0] == '\0' || end_s[0] != '\0' ) { builtin_error("bad conversion to count", @ARG@[0]); d = 0; } return new Val(d, TYPE_DOUBLE); %} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 24 15:46:26 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 24 Jul 2012 22:46:26 -0000 Subject: [Bro-Dev] #855: unhelpful bro-cut error message for bad field In-Reply-To: <046.55ddac95c0b7d9e5de47f5b5061741f5@tracker.bro-ids.org> References: <046.55ddac95c0b7d9e5de47f5b5061741f5@tracker.bro-ids.org> Message-ID: <061.aa9b9141e25356dc94916d1802ffddf1@tracker.bro-ids.org> #855: unhelpful bro-cut error message for bad field ----------------------+------------------------ Reporter: vern | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: bro-aux | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by dnthayer): Replying to [comment:1 vern]: I fixed this bug in January. Let me know if you still see this bug with the latest version (it works for me). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 24 15:55:12 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 24 Jul 2012 22:55:12 -0000 Subject: [Bro-Dev] #855: unhelpful bro-cut error message for bad field In-Reply-To: <046.55ddac95c0b7d9e5de47f5b5061741f5@tracker.bro-ids.org> References: <046.55ddac95c0b7d9e5de47f5b5061741f5@tracker.bro-ids.org> Message-ID: <061.1e69a23e059c51c6ffb1cd5e8a06d7ec@tracker.bro-ids.org> #855: unhelpful bro-cut error message for bad field ----------------------+------------------------ Reporter: vern | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: bro-aux | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by vern): Ah, ok. I was running 2.0 release (and didn't see anything about the bug in the tracker). I'll close this out. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jul 24 15:55:25 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 24 Jul 2012 22:55:25 -0000 Subject: [Bro-Dev] #855: unhelpful bro-cut error message for bad field In-Reply-To: <046.55ddac95c0b7d9e5de47f5b5061741f5@tracker.bro-ids.org> References: <046.55ddac95c0b7d9e5de47f5b5061741f5@tracker.bro-ids.org> Message-ID: <061.16036a768944dabe6e3fcbb2497be99a@tracker.bro-ids.org> #855: unhelpful bro-cut error message for bad field -----------------------------+------------------------ Reporter: vern | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Component: bro-aux | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by vern): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Tue Jul 24 16:12:29 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 24 Jul 2012 16:12:29 -0700 Subject: [Bro-Dev] Master-test merge (Re: [Bro-Commits] [git/bro] master: Merge branch 'topic/robin/master-test' (24aea29)) In-Reply-To: <20120724000852.GJ77331@icir.org> References: <201207240001.q6O01F5H013994@bro-ids.icir.org> <20120724000852.GJ77331@icir.org> Message-ID: <20120724231229.GA11582@icir.org> On Mon, Jul 23, 2012 at 17:08 -0700, I wrote: > Problems remaining: One more: I this occasionally: scripts.base.frameworks.logging.rotate-custom ... failed [...[ == Diff =============================== --- /tmp/test-diff.7552..stderr.baseline.tmp 2012-07-24 23:03:20.939242086 +0000 +++ /tmp/test-diff.7552..stderr.tmp 2012-07-24 23:03:20.959242847 +0000 @@ -3,7 +3,6 @@ 1st test.2011-03-07-04-00-05.log test 11-03-07_04.00.05 11-03-07_05.00.05 0 ascii 1st test.2011-03-07-05-00-05.log test 11-03-07_05.00.05 11-03-07_06.00.05 0 ascii 1st test.2011-03-07-06-00-05.log test 11-03-07_06.00.05 11-03-07_07.00.05 0 ascii -1st test.2011-03-07-07-00-05.log test 11-03-07_07.00.05 11-03-07_08.00.05 0 ascii 1st test.2011-03-07-08-00-05.log test 11-03-07_08.00.05 11-03-07_09.00.05 0 ascii 1st test.2011-03-07-09-00-05.log test 11-03-07_09.00.05 11-03-07_10.00.05 0 ascii 1st test.2011-03-07-10-00-05.log test 11-03-07_10.00.05 11-03-07_11.00.05 0 ascii ======================================= Anybody else? What makes me nervous is that it's missing an entry in the middle ... Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Tue Jul 24 16:33:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 24 Jul 2012 23:33:34 -0000 Subject: [Bro-Dev] #859: to_double bif In-Reply-To: <051.a2e917e9762f5bbb7a4a43f5902cd95a@tracker.bro-ids.org> References: <051.a2e917e9762f5bbb7a4a43f5902cd95a@tracker.bro-ids.org> Message-ID: <066.755038d0cf69b31852f3f5e61ef23ca1@tracker.bro-ids.org> #859: to_double bif ------------------------------+------------------------ Reporter: scampbell | Owner: robin Type: Feature Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: bif ------------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [c36a449c76cc442f64b97d1a7c11febf454304d9/bro]: {{{ #!CommitTicketReference repository="bro" revision="c36a449c76cc442f64b97d1a7c11febf454304d9" New built-in function to_double(s: string). Closes #859. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 25 10:40:44 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 25 Jul 2012 17:40:44 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro In-Reply-To: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> References: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> Message-ID: <063.4a118dacf73220ba9c3ef194be6c7715@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: beta ----------------------+------------------------ Comment (by jsiwek): In [2fafadd9300b2abdf9195f7270071d9549850084/bro]: {{{ #!CommitTicketReference repository="bro" revision="2fafadd9300b2abdf9195f7270071d9549850084" Fix differing log filters of streams from writing to same writer/path. Since WriterFrontend objects are looked up internally by writer type and path, and they also expect to write consistent field arguments, it could be the case that more than one filter of a given stream attempts to write to the same path (derived either from $path or $path_func fields of the filter) with the same writer type. This won't work, so now WriterFrontend objects are bound to the filter that instantiated them so that we can warn about other filters attempting to write to the conflicting writer/path and the write can be skipped. Remote logs don't appear to suffer the same issue due to pre-filtering. Addresses #842. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From slagell at illinois.edu Wed Jul 25 12:04:43 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Wed, 25 Jul 2012 19:04:43 +0000 Subject: [Bro-Dev] Testing errors with new XCode Message-ID: <558D23D33781EF45A69229CDAC6BF1511098A96A@CITESMBX6.ad.uillinois.edu> Gave master a try today. Hardly saw any clang warnings. Did get the following on tests (after turning off DNSCrypt). [ 25%] bifs.to_double_from_string ... failed 1 of 304 tests failed, 21 skipped make[2]: *** [btest-verbose] Error 1 Coverage for 'btest' dir: 1051/1711 (61.4%) Bro script statements covered. Coverage for 'external' dir: Complete test suite code coverage: 1051/1711 (61.4%) Bro script statements covered. Output of diag.log: bifs.to_double_from_string ... failed % 'btest-diff error' failed unexpectedly (exit code 1) % cat .diag == File =============================== error in /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 15: bad conversion to count (to_double(d) and NotADouble) error in /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 16: bad conversion to count (to_double(d) and ) == Diff =============================== --- /tmp/test-diff.33926.error.baseline.tmp 2012-07-25 18:52:04.000000000 +0000 +++ /tmp/test-diff.33926.error.tmp 2012-07-25 18:52:04.000000000 +0000 @@ -1,2 +1,2 @@ -error in /da/home/robin/bro/master/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /da/home/robin/bro/master/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 15: bad conversion to count (to_double(d) and NotADouble) -error in /da/home/robin/bro/master/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /da/home/robin/bro/master/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 16: bad conversion to count (to_double(d) and ) +error in /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 15: bad conversion to count (to_double(d) and NotADouble) +error in /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 16: bad conversion to count (to_double(d) and ) ======================================= % cat .stderr core.mobile-ipv6-home-addr ... not available, skipped core.mobile-ipv6-routing ... not available, skipped core.mobility-checksums ... not available, skipped core.mobility_msg ... not available, skipped core.leaks.ayiya ... not available, skipped core.leaks.basic-cluster ... not available, skipped core.leaks.dataseries-rotate ... not available, skipped core.leaks.dataseries ... not available, skipped core.leaks.dns ... not available, skipped core.leaks.incr-vec-expr ... not available, skipped core.leaks.ip-in-ip ... not available, skipped core.leaks.ipv6_ext_headers ... not available, skipped core.leaks.teredo ... not available, skipped core.leaks.remote ... not available, skipped core.leaks.vector-val-bifs ... not available, skipped core.leaks.test-all ... not available, skipped scripts.base.frameworks.logging.dataseries.options ... not available, skipped scripts.base.frameworks.logging.dataseries.rotate ... not available, skipped scripts.base.frameworks.logging.dataseries.test-logging ... not available, skipped scripts.base.frameworks.logging.dataseries.time-as-int ... not available, skipped scripts.base.frameworks.logging.dataseries.wikipedia ... not available, skipped slagell at prometheus: bro $ ------ Adam J. Slagell, CISO, CISSP Chief Information Security Officer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info 217.244.8965 "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From slagell at illinois.edu Wed Jul 25 12:05:27 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Wed, 25 Jul 2012 19:05:27 +0000 Subject: [Bro-Dev] Testing errors with new XCode In-Reply-To: <558D23D33781EF45A69229CDAC6BF1511098A96A@CITESMBX6.ad.uillinois.edu> References: <558D23D33781EF45A69229CDAC6BF1511098A96A@CITESMBX6.ad.uillinois.edu> Message-ID: <558D23D33781EF45A69229CDAC6BF1511098A9B0@CITESMBX6.ad.uillinois.edu> I should not this is still on Lion, not Mountain Lion. On Jul 25, 2012, at 2:04 PM, Slagell, Adam J wrote: > Gave master a try today. Hardly saw any clang warnings. Did get the following on tests (after turning off DNSCrypt). > > [ 25%] bifs.to_double_from_string ... failed > 1 of 304 tests failed, 21 skipped > make[2]: *** [btest-verbose] Error 1 > Coverage for 'btest' dir: > 1051/1711 (61.4%) Bro script statements covered. > Coverage for 'external' dir: > Complete test suite code coverage: > 1051/1711 (61.4%) Bro script statements covered. > > Output of diag.log: > > bifs.to_double_from_string ... failed > % 'btest-diff error' failed unexpectedly (exit code 1) > % cat .diag > == File =============================== > error in /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 15: bad conversion to count (to_double(d) and NotADouble) > error in /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 16: bad conversion to count (to_double(d) and ) > == Diff =============================== > --- /tmp/test-diff.33926.error.baseline.tmp 2012-07-25 18:52:04.000000000 +0000 > +++ /tmp/test-diff.33926.error.tmp 2012-07-25 18:52:04.000000000 +0000 > @@ -1,2 +1,2 @@ > -error in /da/home/robin/bro/master/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /da/home/robin/bro/master/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 15: bad conversion to count (to_double(d) and NotADouble) > -error in /da/home/robin/bro/master/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /da/home/robin/bro/master/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 16: bad conversion to count (to_double(d) and ) > +error in /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 15: bad conversion to count (to_double(d) and NotADouble) > +error in /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 7 and /Users/slagell/Downloads/bro/testing/btest/.tmp/bifs.to_double_from_string/to_double_from_string.bro, line 16: bad conversion to count (to_double(d) and ) > ======================================= > > % cat .stderr > > core.mobile-ipv6-home-addr ... not available, skipped > core.mobile-ipv6-routing ... not available, skipped > core.mobility-checksums ... not available, skipped > core.mobility_msg ... not available, skipped > core.leaks.ayiya ... not available, skipped > core.leaks.basic-cluster ... not available, skipped > core.leaks.dataseries-rotate ... not available, skipped > core.leaks.dataseries ... not available, skipped > core.leaks.dns ... not available, skipped > core.leaks.incr-vec-expr ... not available, skipped > core.leaks.ip-in-ip ... not available, skipped > core.leaks.ipv6_ext_headers ... not available, skipped > core.leaks.teredo ... not available, skipped > core.leaks.remote ... not available, skipped > core.leaks.vector-val-bifs ... not available, skipped > core.leaks.test-all ... not available, skipped > scripts.base.frameworks.logging.dataseries.options ... not available, skipped > scripts.base.frameworks.logging.dataseries.rotate ... not available, skipped > scripts.base.frameworks.logging.dataseries.test-logging ... not available, skipped > scripts.base.frameworks.logging.dataseries.time-as-int ... not available, skipped > scripts.base.frameworks.logging.dataseries.wikipedia ... not available, skipped > slagell at prometheus: bro $ > > > ------ > > Adam J. Slagell, CISO, CISSP > Chief Information Security Officer > National Center for Supercomputing Applications > University of Illinois at Urbana-Champaign > www.slagell.info > 217.244.8965 > > "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." > > > > > > > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > ------ Adam J. Slagell, CISO, CISSP Chief Information Security Officer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info 217.244.8965 "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From bro at tracker.bro-ids.org Wed Jul 25 13:13:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 25 Jul 2012 20:13:38 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro In-Reply-To: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> References: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> Message-ID: <063.7f008bf6bfff6acdb10817d8780c4d17@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: beta ----------------------+------------------------ Comment (by robin): Anohter good find! Merging, but two thoughts/questions: - can't this potentially generate a lot of reporter messages when there are a large number of writes going to the second one? - what would you think about "auto-renaming": if we detect a writer/path combination to be already taken, we rename the path to something unique ("http-2") and then proceed as normal? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 25 13:28:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 25 Jul 2012 20:28:52 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro In-Reply-To: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> References: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> Message-ID: <063.6dad27b7cbef61197fa5afb4b75fa4a8@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: beta ----------------------+------------------------ Comment (by jsiwek): > - can't this potentially generate a lot of reporter messages when there are a large number of writes going to the second one? Yes. > - what would you think about "auto-renaming": if we detect a writer/path combination to be already taken, we rename the path to something unique ("http-2") and then proceed as normal? Sounds like it could work well and fixes your above concern (just warn once when auto-renaming), are you going to try that or want me? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jul 25 13:30:35 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 25 Jul 2012 20:30:35 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro In-Reply-To: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> References: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> Message-ID: <063.385072eacb5b519137716baaf4210132@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: beta ----------------------+------------------------ Comment (by robin): On Wed, Jul 25, 2012 at 20:28 -0000, you wrote: > Sounds like it could work well and fixes your above concern (just warn > once when auto-renaming), are you going to try that or want me? If you don't mind, please go ahead. I'll still push the merge in a few minutes. Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Thu Jul 26 07:59:17 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 26 Jul 2012 07:59:17 -0700 Subject: [Bro-Dev] Testing errors with new XCode In-Reply-To: <558D23D33781EF45A69229CDAC6BF1511098A9B0@CITESMBX6.ad.uillinois.edu> References: <558D23D33781EF45A69229CDAC6BF1511098A96A@CITESMBX6.ad.uillinois.edu> <558D23D33781EF45A69229CDAC6BF1511098A9B0@CITESMBX6.ad.uillinois.edu> Message-ID: <20120726145917.GJ45020@icir.org> This should be fixed now, right? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From slagell at illinois.edu Thu Jul 26 09:10:25 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Thu, 26 Jul 2012 16:10:25 +0000 Subject: [Bro-Dev] Testing errors with new XCode In-Reply-To: <20120726145917.GJ45020@icir.org> References: <558D23D33781EF45A69229CDAC6BF1511098A96A@CITESMBX6.ad.uillinois.edu> <558D23D33781EF45A69229CDAC6BF1511098A9B0@CITESMBX6.ad.uillinois.edu> <20120726145917.GJ45020@icir.org> Message-ID: <558D23D33781EF45A69229CDAC6BF1511098C71B@CITESMBX6.ad.uillinois.edu> Tried again today. The bif test passes now, DNS tests still have a problem if DNScrypt is on, 21 tests are still skipped (only with dnscrypt on), and the coverage is the same. So I think we are all good. I just consider turning off the firewall and dnscrypt as standard procedure for running the tests. DNScrypt isn't segfaulting Bro anymore. ----- slagell at prometheus: bro $ make test [ 49%] core.dns-init ... failed 1 of 305 tests failed, 21 skipped make[2]: *** [btest-verbose] Error 1 Coverage for 'btest' dir: 1051/1711 (61.4%) Bro script statements covered. Coverage for 'external' dir: Complete test suite code coverage: 1051/1711 (61.4%) Bro script statements covered. slagell at prometheus: bro $ make test all 305 tests successful 1051/1711 (61.4%) Bro script statements covered. Coverage for 'btest' dir: 1051/1711 (61.4%) Bro script statements covered. Coverage for 'external' dir: Complete test suite code coverage: 1051/1711 (61.4%) Bro script statements covered. On Jul 26, 2012, at 9:59 AM, Robin Sommer wrote: > This should be fixed now, right? > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > ------ Adam J. Slagell, CISO, CISSP Chief Information Security Officer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info 217.244.8965 "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From bro at tracker.bro-ids.org Thu Jul 26 09:22:27 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 26 Jul 2012 16:22:27 -0000 Subject: [Bro-Dev] #858: Logging framework stops completely when DoInit returns false In-Reply-To: <048.904aec95fb55939499acbd12b70008ad@tracker.bro-ids.org> References: <048.904aec95fb55939499acbd12b70008ad@tracker.bro-ids.org> Message-ID: <063.219becb01e4f77e1d99b8fa39e63dd14@tracker.bro-ids.org> #858: Logging framework stops completely when DoInit returns false ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by amannb): ...this bug also means that when one tries to read a file with the input framework that does not exist, has the wrong fields, etc. one will not get any kind of error about it, because the logging stops :). That just took me a few minutes to figure out I looked into the source a bit more - but am still stumped. I think it has something to do with the thread being ended and threading stopping to work as soon as the first thread finishes. -- Ticket URL: Bro Tracker Bro Issue Tracker From jsiwek at illinois.edu Thu Jul 26 09:28:21 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Thu, 26 Jul 2012 16:28:21 +0000 Subject: [Bro-Dev] Testing errors with new XCode In-Reply-To: <558D23D33781EF45A69229CDAC6BF1511098C71B@CITESMBX6.ad.uillinois.edu> References: <558D23D33781EF45A69229CDAC6BF1511098A96A@CITESMBX6.ad.uillinois.edu> <558D23D33781EF45A69229CDAC6BF1511098A9B0@CITESMBX6.ad.uillinois.edu> <20120726145917.GJ45020@icir.org> <558D23D33781EF45A69229CDAC6BF1511098C71B@CITESMBX6.ad.uillinois.edu> Message-ID: On Jul 26, 2012, at 11:10 AM, Slagell, Adam J wrote: > Tried again today. The bif test passes now, DNS tests still have a problem if DNScrypt is on, 21 tests are still skipped (only with dnscrypt on), and the coverage is the same. I think the test failure in this case is OK as it is indicating a rejection of poorly formed DNS responses from the local DNSCrypt proxy. When I had used a more recent version of DNSCrypt from their GitHub (https://github.com/opendns/dnscrypt-proxy), the responses were well formed and worked compatibly with Bro. There may be be more we could do to still accept responses if at least one answer section was parseable from the response, but that seems like it could get sketchy. Jon From robin at icir.org Thu Jul 26 09:47:46 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 26 Jul 2012 09:47:46 -0700 Subject: [Bro-Dev] Testing errors with new XCode In-Reply-To: References: <558D23D33781EF45A69229CDAC6BF1511098A96A@CITESMBX6.ad.uillinois.edu> <558D23D33781EF45A69229CDAC6BF1511098A9B0@CITESMBX6.ad.uillinois.edu> <20120726145917.GJ45020@icir.org> <558D23D33781EF45A69229CDAC6BF1511098C71B@CITESMBX6.ad.uillinois.edu> Message-ID: <20120726164746.GB64324@icir.org> On Thu, Jul 26, 2012 at 16:28 +0000, you wrote: > I think the test failure in this case is OK as it is indicating a Yeah, I also think this is fine to leave as it is (same goes for my IPv6 problems; I still need to tweak my firewall settings to let them succeed). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Thu Jul 26 15:00:26 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 26 Jul 2012 22:00:26 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro In-Reply-To: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> References: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> Message-ID: <063.7cba7d651f14406c46853dd5f5e92417@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: beta ----------------------+------------------------ Comment (by jsiwek): In [63e8bf72edad62d4118e22be1e61e32404d03f30/bro]: {{{ #!CommitTicketReference repository="bro" revision="63e8bf72edad62d4118e22be1e61e32404d03f30" Change path conflicts between log filters to be auto-corrected. This change makes it so when differing logging filters on the same stream attempt to write to the same writer/path combination, the path of the filter doing the later write will be automatically adjusted so that it does not conflict with the other. The path is adjusted by appending "-N", where N is the smallest integer greater or equal to 2 required to resolve the path name conflict. Addresses #842. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Thu Jul 26 15:33:02 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 26 Jul 2012 15:33:02 -0700 Subject: [Bro-Dev] Master-test merge (Re: [Bro-Commits] [git/bro] master: Merge branch 'topic/robin/master-test' (24aea29)) In-Reply-To: <20120724231229.GA11582@icir.org> References: <201207240001.q6O01F5H013994@bro-ids.icir.org> <20120724000852.GJ77331@icir.org> <20120724231229.GA11582@icir.org> Message-ID: <20120726223302.GA73405@icir.org> On Tue, Jul 24, 2012 at 16:12 -0700, I wrote: > scripts.base.frameworks.logging.rotate-custom ... failed I believe I've figured this one out. Commit will come soon. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Thu Jul 26 15:43:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 26 Jul 2012 22:43:06 -0000 Subject: [Bro-Dev] #858: Logging framework stops completely when DoInit returns false In-Reply-To: <048.904aec95fb55939499acbd12b70008ad@tracker.bro-ids.org> References: <048.904aec95fb55939499acbd12b70008ad@tracker.bro-ids.org> Message-ID: <063.0964d6b7173a5d2f83dc7070781488cb@tracker.bro-ids.org> #858: Logging framework stops completely when DoInit returns false ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by robin): * milestone: => Bro2.1 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jul 26 17:28:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 00:28:19 -0000 Subject: [Bro-Dev] #858: Logging framework stops completely when DoInit returns false In-Reply-To: <048.904aec95fb55939499acbd12b70008ad@tracker.bro-ids.org> References: <048.904aec95fb55939499acbd12b70008ad@tracker.bro-ids.org> Message-ID: <063.ac7572a0fffff94de36cfc994657fb40@tracker.bro-ids.org> #858: Logging framework stops completely when DoInit returns false ----------------------+------------------------ Reporter: amannb | Owner: robin Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [743fc1680dc9d4c04f38ca80c7ef4e5b88e8f4cb/bro]: {{{ #!CommitTicketReference repository="bro" revision="743fc1680dc9d4c04f38ca80c7ef4e5b88e8f4cb" Improving error handling for threads. If a thread command fails (like the input framework not finding a file), that now (1) no longer hangs Bro, and (2) even allows for propagating error messages back before the thread is stops. (Actually, the thread doesn't really "stop"; the thread manager keeps threads around independent of their success; but it no longer polls them for input.) Closes #858. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Jul 27 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 27 Jul 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207270700.q6R7023l018188@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | f02ed65 | Bernhard Amann | 2012-07-26 | Fix crash when encountering an InterpreterException in a predicate in logging or input Framework. [1] bro | 76ea182 | Bernhard Amann | 2012-07-26 | make want_record=T the default for events [2] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/f02ed65878b81dfde81c2483887223bab99ad2e8/bro [2] fastpath: http://tracker.bro-ids.org/bro/changeset/76ea1823877677612e159c54edf1958898e7ceb2/bro From vladg at cmu.edu Fri Jul 27 05:17:23 2012 From: vladg at cmu.edu (Vlad Grigorescu) Date: Fri, 27 Jul 2012 12:17:23 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: make want_record=T the default for events (76ea182) In-Reply-To: <4947_1343362461_q6R4EJUx011316_201207270414.q6R4E7TJ016030@bro-ids.icir.org> Message-ID: So, the documentation right above it should probably be changed also. Also, while we're at it, "seperate," "signle," and "rised" might as well be fixed. --Vlad On 7/27/12 12:14 AM, "Bernhard Amann" wrote: diff --git a/scripts/base/frameworks/input/main.bro b/scripts/base/frameworks/input/main.bro index c31f92d..7f01540 100644 --- a/scripts/base/frameworks/input/main.bro +++ b/scripts/base/frameworks/input/main.bro @@ -84,7 +84,7 @@ export { ## If want_record if false (default), the event receives each value in fields as a seperate argument. ## If it is set to true, the event receives all fields in a signle record value. - want_record: bool &default=F; + want_record: bool &default=T; ## The event that is rised each time a new line is received from the reader. ## The event will receive an Input::Event enum as the first element, and the fields as the following arguments. From bernhard at ICSI.Berkeley.EDU Fri Jul 27 06:39:45 2012 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Fri, 27 Jul 2012 06:39:45 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: make want_record=T the default for events (76ea182) In-Reply-To: References: Message-ID: <3897A93E-F1D2-4C4B-9E4E-C6588EE530BD@icsi.berkeley.edu> Yes, you certainly are right about that? I completely forgot that. Thank you, will fix it in a few minutes. Bernhard On Jul 27, 2012, at 5:17 AM, Vlad Grigorescu wrote: > So, the documentation right above it should probably be changed also. > Also, while we're at it, "seperate," "signle," and "rised" might as well > be fixed. > > --Vlad > > On 7/27/12 12:14 AM, "Bernhard Amann" wrote: > > diff --git a/scripts/base/frameworks/input/main.bro > b/scripts/base/frameworks/input/main.bro > index c31f92d..7f01540 100644 > --- a/scripts/base/frameworks/input/main.bro > +++ b/scripts/base/frameworks/input/main.bro > @@ -84,7 +84,7 @@ export { > ## If want_record if false (default), the event receives each value in > fields as a seperate argument. > ## If it is set to true, the event receives all fields in a signle > record value. > - want_record: bool &default=F; > + want_record: bool &default=T; > ## The event that is rised each time a new line is received from the > reader. > ## The event will receive an Input::Event enum as the first element, and > the fields as the following arguments. > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From jsiwek at illinois.edu Fri Jul 27 08:03:42 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 27 Jul 2012 15:03:42 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/fastpath' (d262a70) In-Reply-To: <201207270028.q6R0SOl8004901@bro-ids.icir.org> References: <201207270028.q6R0SOl8004901@bro-ids.icir.org> Message-ID: Yes, thanks, that tweak makes sense to me: the loop should also terminate if an autocorrected path is found for which the same filter was previously responsible for writing (probably due to use of a path_func), and still do the warning about the original path conflict. Jon On Jul 26, 2012, at 7:28 PM, Robin Sommer wrote: > Repository : ssh://git at bro-ids.icir.org/bro > > On branch : master > Link : http://tracker.bro-ids.org/bro/changeset/d262a70509f4f7088c261509d6f6f4b5930f5896/bro > >> --------------------------------------------------------------- > > commit d262a70509f4f7088c261509d6f6f4b5930f5896 > Merge: 412bebb 63e8bf7 > Author: Robin Sommer > Date: Thu Jul 26 15:30:35 2012 -0700 > > Merge remote-tracking branch 'origin/fastpath' > > Small tweak: I added the "same writer" constraint to the loop > condition as well. Makes sense? > > * origin/fastpath: > Change path conflicts between log filters to be auto-corrected. > > > >> --------------------------------------------------------------- > > d262a70509f4f7088c261509d6f6f4b5930f5896 > scripts/base/frameworks/logging/main.bro | 11 +++++- > src/logging/Manager.cc | 42 +++++++++++++++----- > .../http-2-2.log} | 21 ++++++++-- > .../http-2.log | 23 +++++++++++ > .../http-3.log | 23 +++++++++++ > .../reporter.log | 17 +------- > .../frameworks/logging/writer-path-conflict.bro | 12 +++++- > 7 files changed, 119 insertions(+), 30 deletions(-) > > diff --cc src/logging/Manager.cc > index 3499d55,b1b289a..568a777 > --- a/src/logging/Manager.cc > +++ b/src/logging/Manager.cc > @@@ -758,9 -758,38 +758,39 @@@ bool Manager::Write(EnumVal* id, Record > #endif > } > > + Stream::WriterPathPair wpp(filter->writer->AsEnum(), path); > + > // See if we already have a writer for this path. > - Stream::WriterMap::iterator w = > - stream->writers.find(Stream::WriterPathPair(filter->writer->AsEnum(), path)); > + Stream::WriterMap::iterator w = stream->writers.find(wpp); > + > + if ( w != stream->writers.end() && > + w->second->instantiating_filter != filter->name ) > + { > + // Auto-correct path due to conflict with another filter over the > + // same writer/path pair > + string instantiator = w->second->instantiating_filter; > + string new_path; > + unsigned int i = 2; > + > + do { > + char num[32]; > + snprintf(num, sizeof(num), "-%u", i++); > + new_path = path + num; > + wpp.second = new_path; > + w = stream->writers.find(wpp); > - } while ( w != stream->writers.end()); > ++ } while ( w != stream->writers.end() && > ++ w->second->instantiating_filter != filter->name ); > + > + Unref(filter->path_val); > + filter->path_val = new StringVal(new_path.c_str()); > + > + reporter->Warning("Write using filter '%s' on path '%s' changed to" > + " use new path '%s' to avoid conflict with filter '%s'", > + filter->name.c_str(), path.c_str(), new_path.c_str(), > + instantiator.c_str()); > + > + path = filter->path = filter->path_val->AsString()->CheckString(); > + } > > WriterFrontend* writer = 0; > > > _______________________________________________ > bro-commits mailing list > bro-commits at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits > From bro at tracker.bro-ids.org Fri Jul 27 08:30:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 15:30:00 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro In-Reply-To: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> References: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> Message-ID: <063.150bc99f11360ae2f0eddd2736798ade@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro -----------------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: beta -----------------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied Comment: All fixes for this should be in master as of [d262a70509f4f7088c261509d6f6f4b5930f5896/bro], the "hang" in the original description was fixed by #848/[e1bd9609264a4d067e3c58016806877f0f859c8d/bro]. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 27 08:34:40 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 15:34:40 -0000 Subject: [Bro-Dev] #846: Tests Failures In-Reply-To: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> References: <047.5933ba48737e6ae2d4223b7a8cce9724@tracker.bro-ids.org> Message-ID: <062.7e8eb33c8f6f1af9b898ec4dd3a8792e@tracker.bro-ids.org> #846: Tests Failures -----------------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied Comment: I think all the intermittent test failures are addressed, there was [c48a16664b521bbcaa0fa60e37ae65b49202b168/bro] which fixed many of them and [412bebb7031d7954a1ce20deef3d6a2f2face192/bro] for the rotate-custom test in particular. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 27 08:48:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 15:48:48 -0000 Subject: [Bro-Dev] #836: Make reporter.log errors go to stderr when run from command-line In-Reply-To: <048.f55ab9255b4a29b56b8218bba951e661@tracker.bro-ids.org> References: <048.f55ab9255b4a29b56b8218bba951e661@tracker.bro-ids.org> Message-ID: <063.a7653a81c9888a7f670e9ce1d7c6b203@tracker.bro-ids.org> #836: Make reporter.log errors go to stderr when run from command-line ------------------------------+------------------------ Reporter: amannb | Owner: seth Type: Feature Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Comment (by jsiwek): Seth, what's the status of ` topic/seth/reporter-to-stderr` ? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 27 09:11:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 16:11:09 -0000 Subject: [Bro-Dev] #823: Remaining input framework todos In-Reply-To: <047.5e447178368a3a40b99fd343d1c743c6@tracker.bro-ids.org> References: <047.5e447178368a3a40b99fd343d1c743c6@tracker.bro-ids.org> Message-ID: <062.a28776d6a903b21f6db0328179ff37bc@tracker.bro-ids.org> #823: Remaining input framework todos ----------------------+------------------------ Reporter: robin | Owner: amannb Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): Bernhard, the leaks are fixed, right? (and/or not due to the input framework). If so, please close the ticket. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 27 09:12:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 16:12:03 -0000 Subject: [Bro-Dev] #814: Fix MailAlarmsTo In-Reply-To: <056.ad48e96857834b2e13a1d5fcdd547a93@tracker.bro-ids.org> References: <056.ad48e96857834b2e13a1d5fcdd547a93@tracker.bro-ids.org> Message-ID: <071.155d43f095d715eee7380c4581f5cc28@tracker.bro-ids.org> #814: Fix MailAlarmsTo -----------------------------+-------------------------- Reporter: Tyler.Schoenke | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: MailAlarmsTo -----------------------------+-------------------------- Comment (by robin): Is this still planned for 2.1? If so, before the beta? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 27 09:12:29 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 16:12:29 -0000 Subject: [Bro-Dev] #823: Remaining input framework todos In-Reply-To: <047.5e447178368a3a40b99fd343d1c743c6@tracker.bro-ids.org> References: <047.5e447178368a3a40b99fd343d1c743c6@tracker.bro-ids.org> Message-ID: <062.a78153e54e06ef57c55f9506fedbaef3@tracker.bro-ids.org> #823: Remaining input framework todos -----------------------------+------------------------ Reporter: robin | Owner: amannb Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by amannb): * status: assigned => closed * resolution: => Solved/Applied Comment: All leaks should be fixed, all other todos also have been applied. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 27 09:14:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 16:14:20 -0000 Subject: [Bro-Dev] #860: Rotation trouble Message-ID: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> #860: Rotation trouble ---------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ (From Seth). There seems to be a problem with rotation. Specifically, the rotations_pending variable in the logging manager. It looks like there is some problem at shutdown where a rotation is scheduled but the thread is shutdown before it receives the message so it never gets a chance to call the FinishedRotation method so the logging manager sits there waiting for all rotations to finish but they won't ever finish. Here's a snippet from my debug log showing the last scheduled rotation but the rotation never actually happening: $ grep dns/Log debug.log | grep -i rotat 1232039460.367675/1343270203.053525 [logging] Scheduled rotation timer for dns/Log::WRITER_ELASTICSEARCH to 1232039520.000000 1232039520.000107/1343270239.105484 [logging] Rotating dns/Log::WRITER_ELASTICSEARCH at 1232039520.000107 1232039520.000107/1343270239.105502 [threading] Sending 'Rotate' to dns/Log::WRITER_ELASTICSEARCH ... 1232039520.000107/1343270239.105523 [logging] Scheduled rotation timer for dns/Log::WRITER_ELASTICSEARCH to 1232039580.000000 You can replicate the problem by running Bro with a tracefile like this? bro -r ~/somepackets.trace Log::default_rotation_interval=1min You just need to make sure that your tracefile causes a rotation (the packets timestamps need to cross from one minute to another). When Bernhard and I were talking last night we realized that this problem has been hidden by the bg task executor in the tests because it's killing the processes even though they are probably in this eternal loop that we're seeing here. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 27 10:15:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 17:15:48 -0000 Subject: [Bro-Dev] #836: Make reporter.log errors go to stderr when run from command-line In-Reply-To: <048.f55ab9255b4a29b56b8218bba951e661@tracker.bro-ids.org> References: <048.f55ab9255b4a29b56b8218bba951e661@tracker.bro-ids.org> Message-ID: <063.94d5509df68caa0f129aa47626ad8cc3@tracker.bro-ids.org> #836: Make reporter.log errors go to stderr when run from command-line ------------------------------+------------------------ Reporter: amannb | Owner: seth Type: Feature Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Comment (by seth): > Seth, what's the status of ` topic/seth/reporter-to-stderr` ? I'll try and get it finished today. I'm a little unsure how to handle it because the code in the core prints to stderr already. Should I just make the core printing cease if the reporter events are being handled? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 27 11:40:55 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 18:40:55 -0000 Subject: [Bro-Dev] #860: Rotation trouble In-Reply-To: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> References: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> Message-ID: <062.1f05bc6267cfd348b5adcca0ea20a720@tracker.bro-ids.org> #860: Rotation trouble ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [503c1082355251808835acf03842cf4c7463cc1c/bro]: {{{ #!CommitTicketReference repository="bro" revision="503c1082355251808835acf03842cf4c7463cc1c" Fix log manager hanging on waiting for pending file rotations. This changes writer implementations to always respond to rotation messages in their DoRotate() method, even for failure/no-op cases with a new RotationFailedMessage. This informs the manager to decrement its count of pending rotations. Addresses #860. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 27 11:45:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 18:45:00 -0000 Subject: [Bro-Dev] #860: Rotation trouble In-Reply-To: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> References: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> Message-ID: <062.0663a945014fcb392c5494515aab8a9f@tracker.bro-ids.org> #860: Rotation trouble ----------------------------+------------------------ Reporter: robin | Owner: Type: Merge Request | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * type: Problem => Merge Request Comment: Try out `topic/jsiwek/rotation-fix` for this and see if it work ok, looks like it fixed it for me. The particular thing that was causing the hang usually was the no-op case for the ascii writer when there wasn't a log file for a given rotation interval (e.g. notice_policy.log and packet_filter.log typically are written to once at startup only), and it wasn't sending a FinishedRotation message back to the manager which is what decrements the pending rotations count. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 27 11:55:01 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 18:55:01 -0000 Subject: [Bro-Dev] #836: Make reporter.log errors go to stderr when run from command-line In-Reply-To: <048.f55ab9255b4a29b56b8218bba951e661@tracker.bro-ids.org> References: <048.f55ab9255b4a29b56b8218bba951e661@tracker.bro-ids.org> Message-ID: <063.80312846ae2abdbf955a526e7dd0ef83@tracker.bro-ids.org> #836: Make reporter.log errors go to stderr when run from command-line ------------------------------+------------------------ Reporter: amannb | Owner: seth Type: Feature Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Comment (by jsiwek): > I'll try and get it finished today. I'm a little unsure how to handle it because the code in the core prints to stderr already. Should I just make the core printing cease if the reporter events are being handled? That might be weird in the case where a user doesn't want to use the provided reporter framework, but still wants to use their own reporter event handlers for other purposes, they'd have to re-implement the script level stderr output on their own if they want it. Probably not a common case, but maybe it's better to just make the core do the stderr output, but still keep the script level options that can toggle whether to do it or not? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jul 27 12:05:25 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 27 Jul 2012 19:05:25 -0000 Subject: [Bro-Dev] #860: Rotation trouble In-Reply-To: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> References: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> Message-ID: <062.4e368713568c50cb2fa0b5bdac0df0a6@tracker.bro-ids.org> #860: Rotation trouble ----------------------------+------------------------ Reporter: robin | Owner: Type: Merge Request | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by seth): > Try out `topic/jsiwek/rotation-fix` for this and see if it work ok, looks > like it fixed it for me. Fixed it for me too. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Fri Jul 27 12:57:21 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 27 Jul 2012 12:57:21 -0700 Subject: [Bro-Dev] ElasticSearch problem (Re: [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/fastpath' (c66c6d7)) In-Reply-To: <20120727192745.GG96041@icir.org> References: <201207271926.q6RJQPRr018889@bro-ids.icir.org> <20120727192745.GG96041@icir.org> Message-ID: <20120727195721.GA95551@icir.org> On Fri, Jul 27, 2012 at 12:27 -0700, I wrote: > Oops, I pushed to early, I think not all baselines are updated yet. > Working on it. Pushed updates now, but the external tests produce new reporter messages (and thus fail right now): 1258831173.687091 Reporter::ERROR conn/Log::WRITER_ELASTICSEARCH: ElasticSearch server may not be accessible. (empty) 1258831173.687091 Reporter::WARNING conn/Log::WRITER_ELASTICSEARCH: HTTP operation with elasticsearch server timed out at 2000 msecs. (empty) 1258831173.687091 Reporter::ERROR conn/Log::WRITER_ELASTICSEARCH: Received a non-successful status code back from ElasticSearch server, check the elasticsearch server 1258850622.799571 Reporter::ERROR stats/Log::WRITER_ELASTICSEARCH: ElasticSearch server may not be accessible. (empty) 1258850622.799571 Reporter::WARNING stats/Log::WRITER_ELASTICSEARCH: HTTP operation with elasticsearch server timed out at 2000 msecs. (empty) 1258850622.799571 Reporter::ERROR stats/Log::WRITER_ELASTICSEARCH: Received a non-successful status code back from ElasticSearch server, check the elasticsearch serve 1258853364.046653 Reporter::ERROR dns/Log::WRITER_ELASTICSEARCH: ElasticSearch server may not be accessible. (empty) 1258853364.046653 Reporter::WARNING dns/Log::WRITER_ELASTICSEARCH: HTTP operation with elasticsearch server timed out at 2000 msecs. (empty) 1258853364.046653 Reporter::ERROR dns/Log::WRITER_ELASTICSEARCH: Received a non-suc I don't think these should be there in the standard configuration, as it's not supposed to use elasticsearch. Seth? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Fri Jul 27 13:06:42 2012 From: seth at icir.org (Seth Hall) Date: Fri, 27 Jul 2012 16:06:42 -0400 Subject: [Bro-Dev] ElasticSearch problem (Re: [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/fastpath' (c66c6d7)) In-Reply-To: <20120727195721.GA95551@icir.org> References: <201207271926.q6RJQPRr018889@bro-ids.icir.org> <20120727192745.GG96041@icir.org> <20120727195721.GA95551@icir.org> Message-ID: <1C186720-AF54-462B-B2C3-420A979E2EAB@icir.org> On Jul 27, 2012, at 3:57 PM, Robin Sommer wrote: > I don't think these should be there in the standard configuration, as > it's not supposed to use elasticsearch. Seth? The tuning/logs-to-elasticsearch script isn't being loaded is it? I'm surprised to see that warning. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Fri Jul 27 13:12:45 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 27 Jul 2012 13:12:45 -0700 Subject: [Bro-Dev] ElasticSearch problem (Re: [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/fastpath' (c66c6d7)) In-Reply-To: <1C186720-AF54-462B-B2C3-420A979E2EAB@icir.org> References: <201207271926.q6RJQPRr018889@bro-ids.icir.org> <20120727192745.GG96041@icir.org> <20120727195721.GA95551@icir.org> <1C186720-AF54-462B-B2C3-420A979E2EAB@icir.org> Message-ID: <20120727201245.GA1557@icir.org> On Fri, Jul 27, 2012 at 16:06 -0400, you wrote: > The tuning/logs-to-elasticsearch script isn't being loaded is it? I'm surprised to see that warning. Ah, turns out it is: the tests load test-all-policy.bro, to which I just added that script back (because, well, it's supposed to load all scripts we have). But shouldn't logs-to-elasticsearch depend on some other kind of configuration before it tries to do anything? Like configuring an ES server to talk to? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Fri Jul 27 13:16:53 2012 From: seth at icir.org (Seth Hall) Date: Fri, 27 Jul 2012 16:16:53 -0400 Subject: [Bro-Dev] ElasticSearch problem (Re: [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/fastpath' (c66c6d7)) In-Reply-To: <20120727201245.GA1557@icir.org> References: <201207271926.q6RJQPRr018889@bro-ids.icir.org> <20120727192745.GG96041@icir.org> <20120727195721.GA95551@icir.org> <1C186720-AF54-462B-B2C3-420A979E2EAB@icir.org> <20120727201245.GA1557@icir.org> Message-ID: <337E16F9-DC7E-4AC1-8BAE-B80D612908DD@icir.org> On Jul 27, 2012, at 4:12 PM, Robin Sommer wrote: > But shouldn't logs-to-elasticsearch depend on some other kind of > configuration before it tries to do anything? Like configuring an ES > server to talk to? The default is pre-configured (localhost:9200). I suspect we shouldn't load anything from the tuning/ directory in default tests. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Fri Jul 27 14:31:49 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 27 Jul 2012 14:31:49 -0700 Subject: [Bro-Dev] ElasticSearch problem (Re: [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/fastpath' (c66c6d7)) In-Reply-To: <337E16F9-DC7E-4AC1-8BAE-B80D612908DD@icir.org> References: <201207271926.q6RJQPRr018889@bro-ids.icir.org> <20120727192745.GG96041@icir.org> <20120727195721.GA95551@icir.org> <1C186720-AF54-462B-B2C3-420A979E2EAB@icir.org> <20120727201245.GA1557@icir.org> <337E16F9-DC7E-4AC1-8BAE-B80D612908DD@icir.org> Message-ID: <20120727213149.GB1557@icir.org> On Fri, Jul 27, 2012 at 16:16 -0400, you wrote: > The default is pre-configured (localhost:9200). I suspect we shouldn't load anything from the tuning/ directory in default tests. That would require restructuring some of the tests. Also, I do prefer having everything loaded. I see the problem here though. We could add the reporter.log to the baseline for now until we've figured out something better. But are they stable, or may the specifics look different everytime? Here's another idea: how about adding a way to disable the stuff in logs-to-elasticsearch even if loaded? Like by redefing the ES server to an empty string? That's something we could then add to the tests that load everything. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From jsiwek at illinois.edu Fri Jul 27 14:54:02 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 27 Jul 2012 21:54:02 +0000 Subject: [Bro-Dev] ElasticSearch problem (Re: [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/fastpath' (c66c6d7)) In-Reply-To: <20120727213149.GB1557@icir.org> References: <201207271926.q6RJQPRr018889@bro-ids.icir.org> <20120727192745.GG96041@icir.org> <20120727195721.GA95551@icir.org> <1C186720-AF54-462B-B2C3-420A979E2EAB@icir.org> <20120727201245.GA1557@icir.org> <337E16F9-DC7E-4AC1-8BAE-B80D612908DD@icir.org> <20120727213149.GB1557@icir.org> Message-ID: >> The default is pre-configured (localhost:9200). I suspect we shouldn't load anything from the tuning/ directory in default tests. > > That would require restructuring some of the tests. Also, I do prefer > having everything loaded. I see the problem here though. > > We could add the reporter.log to the baseline for now until we've > figured out something better. But are they stable, or may the > specifics look different everytime? > > Here's another idea: how about adding a way to disable the stuff in > logs-to-elasticsearch even if loaded? Like by redefing the ES server > to an empty string? That's something we could then add to the tests > that load everything. Another idea: testing/external/scripts/diff-all has a quick hack (that I'm not really sure works right still) for getting around the case where GeoIP support isn't enabled and shows up as a reporter message. Maybe that can be updated to also ignore lines regarding ElasticSearch. Jon From robin at icir.org Fri Jul 27 15:09:47 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 27 Jul 2012 15:09:47 -0700 Subject: [Bro-Dev] ElasticSearch problem (Re: [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/fastpath' (c66c6d7)) In-Reply-To: References: <201207271926.q6RJQPRr018889@bro-ids.icir.org> <20120727192745.GG96041@icir.org> <20120727195721.GA95551@icir.org> <1C186720-AF54-462B-B2C3-420A979E2EAB@icir.org> <20120727201245.GA1557@icir.org> <337E16F9-DC7E-4AC1-8BAE-B80D612908DD@icir.org> <20120727213149.GB1557@icir.org> Message-ID: <20120727220947.GD1557@icir.org> On Fri, Jul 27, 2012 at 21:54 +0000, you wrote: > Another idea: testing/external/scripts/diff-all has a quick hack (that > I'm not really sure works right still) for getting around the case > where GeoIP support isn't enabled and shows up as a reporter message. > Maybe that can be updated to also ignore lines regarding > ElasticSearch. I guess it could, but that's not really much nicer ... I can see eventuallu having more of these optional scripts that do something that might cause artefacts depending on config/capabilities. How about as a general policy we say that any script that may cause trouble when simply loaded on top of a standard configuration must have a way to be disabled. We then add a script disable-problematic-analyses.bro (or so :) to the external tests, and this script sets all those relevant options. (Hmm ... We might be able to achieve the same effect with some @unloads, but not sure I want to rely on that odd feature ...) I can think of one more alternative: split test-all-policy.bro into two parts, all the scripts that are fine to load with the external tests and those which aren't. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Sat Jul 28 00:00:01 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 28 Jul 2012 00:00:01 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207280700.q6S701GM005329@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 860 [1] | robin | | High | Rotation trouble [1] #860: http://tracker.bro-ids.org/bro/ticket/860 From bro at tracker.bro-ids.org Sat Jul 28 11:31:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 28 Jul 2012 18:31:56 -0000 Subject: [Bro-Dev] #860: Rotation trouble In-Reply-To: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> References: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> Message-ID: <062.e0e49ef0e5e4e586b32244218a557665@tracker.bro-ids.org> #860: Rotation trouble ----------------------------+------------------------ Reporter: robin | Owner: Type: Merge Request | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): The fix makes sense, but the interface gets a bit confusing with the two rotation-done methods. Also, do the writers actuallt need to call the FailedRotation() method when FinishedRotation() returns false? While FinishedRotation() currently always returns true anyways, even if it didn't, it could just call FailedRotation() itself. Here's a suggestion: what if we instead of adding FailedRotation(), we instead added a "success" parameter to FinishedRotation(). If false, it would just decrease the rotation count on the manager side, but don't do anything else. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Sat Jul 28 16:36:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 28 Jul 2012 23:36:21 -0000 Subject: [Bro-Dev] #860: Rotation trouble In-Reply-To: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> References: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> Message-ID: <062.5e0df53548da282aeebb77c5761ab3fc@tracker.bro-ids.org> #860: Rotation trouble ----------------------------+------------------------ Reporter: robin | Owner: Type: Merge Request | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): I went ahead and did a mix of the two: On the backend side, there are two methods but both are called FinishRotation (because even the 2nd version isn't necessarily "failed"); plus there's code to check that at least one indeed gets called. On the fronted/mgr side there's only one with a {{{success}}} parameter. See what you think. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Sat Jul 28 16:36:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 28 Jul 2012 23:36:46 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro In-Reply-To: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> References: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> Message-ID: <063.e17cd4fa3ac6339c057306db917bfe9e@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: reopened Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: beta ----------------------+------------------------ Changes (by robin): * status: closed => reopened * resolution: Solved/Applied => Comment: I just noticed that {{{scripts.base.frameworks.cluster.start-it-up}}} now produces two {{{reporter.log}}} for the manager: {{{ > ls manager-1/reporter* manager-1/reporter.2012-07-28-23-18-44.log manager-1/reporter-2.2012-07-28-23-18-55.log }}} Not quite sure where that is coming from. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Sat Jul 28 16:38:43 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 28 Jul 2012 23:38:43 -0000 Subject: [Bro-Dev] #860: Rotation trouble In-Reply-To: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> References: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> Message-ID: <062.312b81c390158abdf93b4f41d50c3eed@tracker.bro-ids.org> #860: Rotation trouble ----------------------------+------------------------ Reporter: robin | Owner: Type: Merge Request | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): In [4359bf6b42fb6438bf5d2285f07275625d9b542b/bro]: {{{ #!CommitTicketReference repository="bro" revision="4359bf6b42fb6438bf5d2285f07275625d9b542b" Fix log manager hanging on waiting for pending file rotations. This changes writer implementations to always respond to rotation messages in their DoRotate() method, even for failure/no-op cases with a new RotationFailedMessage. This informs the manager to decrement its count of pending rotations. Addresses #860. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Sat Jul 28 16:42:13 2012 From: robin at icir.org (Robin Sommer) Date: Sat, 28 Jul 2012 16:42:13 -0700 Subject: [Bro-Dev] ElasticSearch problem (Re: [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/fastpath' (c66c6d7)) In-Reply-To: <20120727220947.GD1557@icir.org> References: <201207271926.q6RJQPRr018889@bro-ids.icir.org> <20120727192745.GG96041@icir.org> <20120727195721.GA95551@icir.org> <1C186720-AF54-462B-B2C3-420A979E2EAB@icir.org> <20120727201245.GA1557@icir.org> <337E16F9-DC7E-4AC1-8BAE-B80D612908DD@icir.org> <20120727213149.GB1557@icir.org> <20120727220947.GD1557@icir.org> Message-ID: <20120728234213.GA36729@icir.org> On Fri, Jul 27, 2012 at 15:09 -0700, I wrote: > How about as a general policy we say that any script that may cause > trouble when simply loaded on top of a standard configuration must > have a way to be disabled. We then add a script > disable-problematic-analyses.bro (or so :) to the external tests, and > this script sets all those relevant options. I noticed that we already have script like that (testing-setup.bro), it just wasn't loaded late enough to help here. So I changed that and added a check to the ES script that if the server is not set, it doesn't do anything. That does the trick. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Sun Jul 29 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 29 Jul 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207290700.q6T702iE019533@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 860 [1] | robin | | High | Rotation trouble [1] #860: http://tracker.bro-ids.org/bro/ticket/860 From noreply at bro-ids.org Mon Jul 30 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 30 Jul 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201207300700.q6U702vG026789@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 860 [1] | robin | | High | Rotation trouble [1] #860: http://tracker.bro-ids.org/bro/ticket/860 From bro at tracker.bro-ids.org Mon Jul 30 08:34:13 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 30 Jul 2012 15:34:13 -0000 Subject: [Bro-Dev] #860: Rotation trouble In-Reply-To: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> References: <047.bbd2553a90b315642fa35187a16b3c23@tracker.bro-ids.org> Message-ID: <062.fb5bd3ae434b411f78d04a8be35a28e4@tracker.bro-ids.org> #860: Rotation trouble -----------------------------+------------------------ Reporter: robin | Owner: Type: Merge Request | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied Comment: Replying to [comment:5 robin]: > I went ahead and did a mix of the two: On the backend side, there are two methods but both are called FinishRotation (because even the 2nd version isn't necessarily "failed"); plus there's code to check that at least one indeed gets called. On the fronted/mgr side there's only one with a {{{success}}} parameter. See what you think. Looks like an improvement, thanks. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Mon Jul 30 09:21:19 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 30 Jul 2012 09:21:19 -0700 Subject: [Bro-Dev] Beta status? Message-ID: <20120730162119.GG86049@icir.org> I think we're almost there ... The one remaining issue is the reopened #842, which looks like a new problem to me. I'm planing to roll another "test beta tarball" once we have fixed this. Or is there anything else I'm missing? Anybody seeing any test failures? #836 doesn't seem to be ready yet? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From hlin33 at illinois.edu Mon Jul 30 09:46:33 2012 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Mon, 30 Jul 2012 11:46:33 -0500 Subject: [Bro-Dev] Hui Lin_Merging DNP3 analyzer Message-ID: Hi, Robin, I think the DNP3 analyzer is ready to be merged. The only concern now is that I still left very little Debug codes. Do u want me to remove them all? I don't have a tracker account. Can you create a ticket for me and give me a basic idea how to do the merging? Best, Hui -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120730/4b7c44b5/attachment.html From bro at tracker.bro-ids.org Mon Jul 30 11:18:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 30 Jul 2012 18:18:10 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro In-Reply-To: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> References: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> Message-ID: <063.53da896733d0dd86ab1f1e20fe690ec3@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: reopened Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: beta ----------------------+------------------------ Comment (by jsiwek): In [7b2c3db4881dd8acb3836c91c6d9da0895578405/bro]: {{{ #!CommitTicketReference repository="bro" revision="7b2c3db4881dd8acb3836c91c6d9da0895578405" Improve log filter compatibility with remote logging. If a log filter attempts to write to a path for which a writer is already instantiated due to remote logging, it will re-use the writer as long as the fields of the filter and writer are compatible, else the filter path will be auto-adjusted to not conflict with existing writer's. Conflicts between two local filters are still always auto-adjusted even if field types agree (since they could still be semantically different). Addresses #842. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Mon Jul 30 11:54:00 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 30 Jul 2012 11:54:00 -0700 Subject: [Bro-Dev] Beta status? In-Reply-To: <20120730162119.GG86049@icir.org> References: <20120730162119.GG86049@icir.org> Message-ID: <20120730185400.GA6389@icir.org> On Mon, Jul 30, 2012 at 09:21 -0700, I wrote: > I think we're almost there ... The one remaining issue is the reopened > #842, which looks like a new problem to me. Thanks for fixing this, Jon. Looks like no other stoppers anymore? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Mon Jul 30 11:58:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 30 Jul 2012 18:58:52 -0000 Subject: [Bro-Dev] #842: Adding a logging filter without a path hangs bro In-Reply-To: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> References: <048.66857bca15bdb986a757cf28916e7258@tracker.bro-ids.org> Message-ID: <063.90858f7ad9bf6a3e74cd1bf2da2ab7e0@tracker.bro-ids.org> #842: Adding a logging filter without a path hangs bro ----------------------+------------------------ Reporter: amannb | Owner: robin Type: Problem | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: beta ----------------------+------------------------ Changes (by robin): * owner: => robin * status: reopened => closed * resolution: => fixed Comment: In [de3eba7062dd0f2fc1d0ca5211bc5b0ab958df9f/bro]: {{{ #!CommitTicketReference repository="bro" revision="de3eba7062dd0f2fc1d0ca5211bc5b0ab958df9f" Merge remote-tracking branch 'origin/fastpath' * origin/fastpath: Improve log filter compatibility with remote logging. Closes #842. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From slagell at illinois.edu Mon Jul 30 12:01:08 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Mon, 30 Jul 2012 19:01:08 +0000 Subject: [Bro-Dev] Beta status? In-Reply-To: <20120730185400.GA6389@icir.org> References: <20120730162119.GG86049@icir.org> <20120730185400.GA6389@icir.org> Message-ID: <558D23D33781EF45A69229CDAC6BF15110993873@CITESMBX6.ad.uillinois.edu> On Jul 30, 2012, at 1:54 PM, Robin Sommer wrote: > > On Mon, Jul 30, 2012 at 09:21 -0700, I wrote: > >> I think we're almost there ... The one remaining issue is the reopened >> #842, which looks like a new problem to me. > > Thanks for fixing this, Jon. > > Looks like no other stoppers anymore? I think not. So yay, we can release before the workshop hopefully. From bro at tracker.bro-ids.org Mon Jul 30 20:45:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 31 Jul 2012 03:45:46 -0000 Subject: [Bro-Dev] #852: Update cert bundle from mozilla before release In-Reply-To: <046.03be5949fa44efa71d66c15d8d09104f@tracker.bro-ids.org> References: <046.03be5949fa44efa71d66c15d8d09104f@tracker.bro-ids.org> Message-ID: <061.25ffc399462c0d66b63d85d5e3730da5@tracker.bro-ids.org> #852: Update cert bundle from mozilla before release -----------------------------+------------------------ Reporter: seth | Owner: seth Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by seth): * status: new => closed * resolution: => Solved/Applied Comment: This ticket was completed through fastpath a while ago and there have been no further updates to Mozilla's CA list since. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Mon Jul 30 21:51:54 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 30 Jul 2012 21:51:54 -0700 Subject: [Bro-Dev] Hui Lin_Merging DNP3 analyzer In-Reply-To: References: Message-ID: <20120731045154.GH14438@icir.org> On Mon, Jul 30, 2012 at 11:46 -0500, you wrote: > I think the DNP3 analyzer is ready to be merged. The only concern now is > that I still left very little Debug codes. Do u want me to remove them all? Yes, generally, that should probably be removed. Take a look at the DBG_LOG macro though (if you aren't already using it), it it's a good way to keep some debugging information in. > I don't have a tracker account. Can you create a ticket for me and give me > a basic idea how to do the merging? I'll do the merging; once you're ready just create a ticket and set it to merge request. I'll create you an account on the tracker tomorrow. However, as this is something for Bro 2.2, it'll take a bit until I'll do the merge; we're in feature freeze mode right now. :) Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Tue Jul 31 05:32:16 2012 From: seth at icir.org (Seth Hall) Date: Tue, 31 Jul 2012 08:32:16 -0400 Subject: [Bro-Dev] Beta status? In-Reply-To: <20120730162119.GG86049@icir.org> References: <20120730162119.GG86049@icir.org> Message-ID: <96648348-10B1-49E3-AA22-B7D42872678B@icir.org> On Jul 30, 2012, at 12:21 PM, Robin Sommer wrote: > #836 doesn't seem to be ready yet? Yeah, sorry about that. I'll try and get that finished tonight (unless someone else feels like tackling it today). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Tue Jul 31 09:26:29 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 31 Jul 2012 09:26:29 -0700 Subject: [Bro-Dev] [Bro] Version: 2.0-907 -- Bro manager memory exhaustion In-Reply-To: <20120731151128.GI31136@icir.org> References: <20120731151128.GI31136@icir.org> Message-ID: <20120731162629.GC34987@icir.org> (Taking to bro-dev). On Tue, Jul 31, 2012 at 08:11 -0700, I wrote: > That's not a good sign for the manager ... It's possible that we have > a memory leak in there. I just reran our leak tests and they didn't report anything (which is good, but doesn't completely rule out any leaks). I did see this though from valgrind: Object at 0x94e3410 of 68 bytes from an IgnoreObject() has disappeared Does anybody know what valgrind it trying to tell me with that? Is it a problem? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From vladg at cmu.edu Tue Jul 31 09:59:56 2012 From: vladg at cmu.edu (Vlad Grigorescu) Date: Tue, 31 Jul 2012 16:59:56 +0000 Subject: [Bro-Dev] [Bro] Version: 2.0-907 -- Bro manager memory exhaustion In-Reply-To: <11805_1343751994_q6VGQWNh020043_20120731162629.GC34987@icir.org> Message-ID: I've been running 2.0-905 for ~25-26 hours. The manager's memory usage has slowly crept up to 13 GB. One thing of note - I'm using the ElasticSearch log writer. I see 3 possible scenarios for this memleak: 1) There is indeed a leak in master, potentially only triggered by specific traffic, 2) There is a leak in the ElasticSearch log writer, 3) My ElasticSearch server can't keep up with the load, and the manager is receiving logs faster than it can send them to the writer, so they just queue up. Has anyone else tried the current code over an extended period on live traffic? Also, if anyone has any ideas to try to figure out where this leak is occurring, please let me know. I'm going to switch back to ASCII logs for a bit, and see what that's like. --Vlad On 7/31/12 12:26 PM, "Robin Sommer" wrote: (Taking to bro-dev). On Tue, Jul 31, 2012 at 08:11 -0700, I wrote: > That's not a good sign for the manager ... It's possible that we have > a memory leak in there. I just reran our leak tests and they didn't report anything (which is good, but doesn't completely rule out any leaks). I did see this though from valgrind: Object at 0x94e3410 of 68 bytes from an IgnoreObject() has disappeared Does anybody know what valgrind it trying to tell me with that? Is it a problem? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org _______________________________________________ bro-dev mailing list bro-dev at bro-ids.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From tritium.cat at gmail.com Tue Jul 31 16:21:10 2012 From: tritium.cat at gmail.com (Tritium Cat) Date: Tue, 31 Jul 2012 23:21:10 +0000 Subject: [Bro-Dev] [Bro] Version: 2.0-907 -- Bro manager memory exhaustion In-Reply-To: References: <11805_1343751994_q6VGQWNh020043_20120731162629.GC34987@icir.org> Message-ID: I've came to the conclusion that Bro cannot currently scale up to 100 workers for a single manager, threaded or not. For now I've reverted back to the 2.0-stable build and have segmented the cluster into five parts, one per server with 20 workers on each. That has been stable, minus the occasional process stuck at 100% CPU. I'm going to try switching back to the dev track again with this segmented approach and see how that works. --TC On Tue, Jul 31, 2012 at 4:59 PM, Vlad Grigorescu wrote: > I've been running 2.0-905 for ~25-26 hours. The manager's memory usage has > slowly crept up to 13 GB. > > One thing of note - I'm using the ElasticSearch log writer. I see 3 > possible scenarios for this memleak: > > 1) There is indeed a leak in master, potentially only triggered by > specific traffic, > 2) There is a leak in the ElasticSearch log writer, > 3) My ElasticSearch server can't keep up with the load, and the manager is > receiving logs faster than it can send them to the writer, so they just > queue up. > > Has anyone else tried the current code over an extended period on live > traffic? Also, if anyone has any ideas to try to figure out where this > leak is occurring, please let me know. I'm going to switch back to ASCII > logs for a bit, and see what that's like. > > --Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120731/c1c9e879/attachment.html From vladg at cmu.edu Tue Jul 31 18:26:06 2012 From: vladg at cmu.edu (Vlad Grigorescu) Date: Wed, 1 Aug 2012 01:26:06 +0000 Subject: [Bro-Dev] [Bro] Version: 2.0-907 -- Bro manager memory exhaustion In-Reply-To: <5702_1343754005_q6VH040q008906_CC3D8551.58F7%vladg@andrew.cmu.edu> Message-ID: This seems to me to just be an issue of my ElasticSearch server not keeping up with the load. I ran master with just the Ascii logs for a few hours, and saw no evidence of a memleak. Valgrind also came back cleanly (and actually, insertion rate into ElasticSearch was about the same with/without valgrind - I was expecting more of a performance hit). --Vlad On 7/31/12 12:59 PM, "Vlad Grigorescu" wrote: I've been running 2.0-905 for ~25-26 hours. The manager's memory usage has slowly crept up to 13 GB. One thing of note - I'm using the ElasticSearch log writer. I see 3 possible scenarios for this memleak: 1) There is indeed a leak in master, potentially only triggered by specific traffic, 2) There is a leak in the ElasticSearch log writer, 3) My ElasticSearch server can't keep up with the load, and the manager is receiving logs faster than it can send them to the writer, so they just queue up. Has anyone else tried the current code over an extended period on live traffic? Also, if anyone has any ideas to try to figure out where this leak is occurring, please let me know. I'm going to switch back to ASCII logs for a bit, and see what that's like. --Vlad On 7/31/12 12:26 PM, "Robin Sommer" wrote: (Taking to bro-dev). On Tue, Jul 31, 2012 at 08:11 -0700, I wrote: > That's not a good sign for the manager ... It's possible that we have > a memory leak in there. I just reran our leak tests and they didn't report anything (which is good, but doesn't completely rule out any leaks). I did see this though from valgrind: Object at 0x94e3410 of 68 bytes from an IgnoreObject() has disappeared Does anybody know what valgrind it trying to tell me with that? Is it a problem? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org _______________________________________________ bro-dev mailing list bro-dev at bro-ids.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev _______________________________________________ bro-dev mailing list bro-dev at bro-ids.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From mcholste at gmail.com Tue Jul 31 20:43:45 2012 From: mcholste at gmail.com (Martin Holste) Date: Tue, 31 Jul 2012 22:43:45 -0500 Subject: [Bro-Dev] [Bro] Version: 2.0-907 -- Bro manager memory exhaustion In-Reply-To: References: <5702_1343754005_q6VH040q008906_CC3D8551.58F7%vladg@andrew.cmu.edu> Message-ID: What was the logging rate for ES and raw ASCII? On Tue, Jul 31, 2012 at 8:26 PM, Vlad Grigorescu wrote: > This seems to me to just be an issue of my ElasticSearch server not > keeping up with the load. > > I ran master with just the Ascii logs for a few hours, and saw no evidence > of a memleak. Valgrind also came back cleanly (and actually, insertion > rate into ElasticSearch was about the same with/without valgrind - I was > expecting more of a performance hit). > > --Vlad > > On 7/31/12 12:59 PM, "Vlad Grigorescu" wrote: > > I've been running 2.0-905 for ~25-26 hours. The manager's memory usage has > slowly crept up to 13 GB. > > One thing of note - I'm using the ElasticSearch log writer. I see 3 > possible scenarios for this memleak: > > 1) There is indeed a leak in master, potentially only triggered by > specific traffic, > 2) There is a leak in the ElasticSearch log writer, > 3) My ElasticSearch server can't keep up with the load, and the manager is > receiving logs faster than it can send them to the writer, so they just > queue up. > > Has anyone else tried the current code over an extended period on live > traffic? Also, if anyone has any ideas to try to figure out where this > leak is occurring, please let me know. I'm going to switch back to ASCII > logs for a bit, and see what that's like. > > --Vlad > > On 7/31/12 12:26 PM, "Robin Sommer" wrote: > > (Taking to bro-dev). > > On Tue, Jul 31, 2012 at 08:11 -0700, I wrote: > >> That's not a good sign for the manager ... It's possible that we have >> a memory leak in there. > > I just reran our leak tests and they didn't report anything (which is > good, but doesn't completely rule out any leaks). > > I did see this though from valgrind: > > Object at 0x94e3410 of 68 bytes from an IgnoreObject() has disappeared > > Does anybody know what valgrind it trying to tell me with that? Is it > a problem? > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From vladg at cmu.edu Tue Jul 31 20:58:09 2012 From: vladg at cmu.edu (Vlad Grigorescu) Date: Wed, 1 Aug 2012 03:58:09 +0000 Subject: [Bro-Dev] [Bro] Version: 2.0-907 -- Bro manager memory exhaustion In-Reply-To: <28426_1343792652_q713iB7m002259_CANpnLHjdtSrZBgzOTeQ-oPVJ4EPHVWiveHrjU1_D-_H0hthLOA@mail.gmail.com> Message-ID: An average of 9000 logs/second. --Vlad On 7/31/12 11:43 PM, "Martin Holste" wrote: What was the logging rate for ES and raw ASCII? On Tue, Jul 31, 2012 at 8:26 PM, Vlad Grigorescu wrote: > This seems to me to just be an issue of my ElasticSearch server not > keeping up with the load. > > I ran master with just the Ascii logs for a few hours, and saw no >evidence > of a memleak. Valgrind also came back cleanly (and actually, insertion > rate into ElasticSearch was about the same with/without valgrind - I was > expecting more of a performance hit). > > --Vlad > > On 7/31/12 12:59 PM, "Vlad Grigorescu" wrote: > > I've been running 2.0-905 for ~25-26 hours. The manager's memory usage >has > slowly crept up to 13 GB. > > One thing of note - I'm using the ElasticSearch log writer. I see 3 > possible scenarios for this memleak: > > 1) There is indeed a leak in master, potentially only triggered by > specific traffic, > 2) There is a leak in the ElasticSearch log writer, > 3) My ElasticSearch server can't keep up with the load, and the manager >is > receiving logs faster than it can send them to the writer, so they just > queue up. > > Has anyone else tried the current code over an extended period on live > traffic? Also, if anyone has any ideas to try to figure out where this > leak is occurring, please let me know. I'm going to switch back to ASCII > logs for a bit, and see what that's like. > > --Vlad > > On 7/31/12 12:26 PM, "Robin Sommer" wrote: > > (Taking to bro-dev). > > On Tue, Jul 31, 2012 at 08:11 -0700, I wrote: > >> That's not a good sign for the manager ... It's possible that we have >> a memory leak in there. > > I just reran our leak tests and they didn't report anything (which is > good, but doesn't completely rule out any leaks). > > I did see this though from valgrind: > > Object at 0x94e3410 of 68 bytes from an IgnoreObject() has >disappeared > > Does anybody know what valgrind it trying to tell me with that? Is it > a problem? > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev _______________________________________________ bro-dev mailing list bro-dev at bro-ids.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev