From noreply at bro-ids.org Wed Feb 1 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 1 Feb 2012 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202010800.q11802IB004120@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 764 [1] | amannb | | Normal | Change software framework interface pysubnettree | 750 [2] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 4a6a9fe | Daniel Thayer | 2012-01-30 | Fix sorting of lines in Brofiler coverage.log [3] [1] #764: http://tracker.bro-ids.org/bro/ticket/764 [2] #750: http://tracker.bro-ids.org/bro/ticket/750 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/4a6a9fe9f274b32250e2507e2b03f31057dc1e9f/bro From noreply at bro-ids.org Thu Feb 2 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 2 Feb 2012 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202020800.q12802Hs006273@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 764 [1] | amannb | | Normal | Change software framework interface pysubnettree | 750 [2] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 4a6a9fe | Daniel Thayer | 2012-01-30 | Fix sorting of lines in Brofiler coverage.log [3] [1] #764: http://tracker.bro-ids.org/bro/ticket/764 [2] #750: http://tracker.bro-ids.org/bro/ticket/750 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/4a6a9fe9f274b32250e2507e2b03f31057dc1e9f/bro From noreply at bro-ids.org Fri Feb 3 00:00:01 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 3 Feb 2012 00:00:01 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202030800.q13801W1027524@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 764 [1] | amannb | | Normal | Change software framework interface pysubnettree | 750 [2] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 1d417a3 | Daniel Thayer | 2012-02-02 | Fix minor typos in documentation [3] bro | 4a6a9fe | Daniel Thayer | 2012-01-30 | Fix sorting of lines in Brofiler coverage.log [4] [1] #764: http://tracker.bro-ids.org/bro/ticket/764 [2] #750: http://tracker.bro-ids.org/bro/ticket/750 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/1d417a3e234856bc0de1cd2247c36848a4090ab7/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/4a6a9fe9f274b32250e2507e2b03f31057dc1e9f/bro From bro at tracker.bro-ids.org Fri Feb 3 02:01:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 03 Feb 2012 10:01:21 -0000 Subject: [Bro-Dev] #764: Change software framework interface In-Reply-To: <048.fb7e2a77a0625ef0ffbe9e3ccb92f857@tracker.bro-ids.org> References: <048.fb7e2a77a0625ef0ffbe9e3ccb92f857@tracker.bro-ids.org> Message-ID: <063.1308923cba03c4f306179b61fa7bb24d@tracker.bro-ids.org> #764: Change software framework interface ----------------------------+------------------------ Reporter: amannb | Owner: seth Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => seth * status: new => assigned Comment: Seth, can you take a look at this and either merge it in or ack me doing so. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 3 05:52:53 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 03 Feb 2012 13:52:53 -0000 Subject: [Bro-Dev] #764: Change software framework interface In-Reply-To: <048.fb7e2a77a0625ef0ffbe9e3ccb92f857@tracker.bro-ids.org> References: <048.fb7e2a77a0625ef0ffbe9e3ccb92f857@tracker.bro-ids.org> Message-ID: <063.19357f1e00e71dca3b47ac814cbc13c5@tracker.bro-ids.org> #764: Change software framework interface ----------------------------+------------------------ Reporter: amannb | Owner: seth Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by seth): > Seth, can you take a look at this and either merge it in or ack me doing > so. I'll check it out and merge it now. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 3 13:20:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 03 Feb 2012 21:20:34 -0000 Subject: [Bro-Dev] #764: Change software framework interface In-Reply-To: <048.fb7e2a77a0625ef0ffbe9e3ccb92f857@tracker.bro-ids.org> References: <048.fb7e2a77a0625ef0ffbe9e3ccb92f857@tracker.bro-ids.org> Message-ID: <063.66d556b66a16269d1aa5bfeb8fe52e39@tracker.bro-ids.org> #764: Change software framework interface -----------------------------+------------------------ Reporter: amannb | Owner: seth Type: Task | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by seth): * status: assigned => closed * type: Merge Request => Task * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 3 14:08:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 03 Feb 2012 22:08:38 -0000 Subject: [Bro-Dev] #451: Remove DNS options for skipping auth/addl events In-Reply-To: <047.8ae817995be2255322e68a394e45df3e@tracker.bro-ids.org> References: <047.8ae817995be2255322e68a394e45df3e@tracker.bro-ids.org> Message-ID: <062.0a68d278cca142f8d9bbdeb356e18e6b@tracker.bro-ids.org> #451: Remove DNS options for skipping auth/addl events ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by seth): Actually after thinking about this more that won't work. The auth/addl data comes in through the same events as the rest of the DNS responses. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Feb 4 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 4 Feb 2012 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202040800.q148038h020301@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ pysubnettree | 750 [1] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #750: http://tracker.bro-ids.org/bro/ticket/750 From noreply at bro-ids.org Sun Feb 5 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 5 Feb 2012 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202050800.q158024p008541@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ pysubnettree | 750 [1] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #750: http://tracker.bro-ids.org/bro/ticket/750 From vallentin at icir.org Sun Feb 5 14:27:30 2012 From: vallentin at icir.org (Matthias Vallentin) Date: Sun, 5 Feb 2012 14:27:30 -0800 Subject: [Bro-Dev] Fwd: [ACCEPTED] Pull request #5 for birkenfeld/pygments-main: Add lexer for the Bro language In-Reply-To: <20120205121952.11916.25642@bitbucket02.managed.contegix.com> References: <20120205121952.11916.25642@bitbucket02.managed.contegix.com> Message-ID: The pygments syntax highlighter master branch now includes support for Bro scripts. In other words, brogments [1] is now officially part of pygments. This means that github will have Bro syntax highlighting soon, because their syntax highlighter regularly pulls from the pygments main branch. Matthias [1] https://github.com/mavam/brogments ---------- Forwarded message ---------- From: Bitbucket Date: Sun, Feb 5, 2012 at 4:19 AM Subject: [ACCEPTED] Pull request #5 for birkenfeld/pygments-main: Add lexer for the Bro language To: vallentin at icir.org Pull request #5 has been accepted by Georg Brandl. Changes in mavam/pygments-main have been pulled into birkenfeld/pygments-main. https://bitbucket.org/birkenfeld/pygments-main/pull-request/5/add-lexer-for-the-bro-language -- This is an issue notification from bitbucket.org. You are receiving this either because you are the participating in a pull request, or you are following it. From robin at icir.org Sun Feb 5 22:50:51 2012 From: robin at icir.org (Robin Sommer) Date: Sun, 5 Feb 2012 22:50:51 -0800 Subject: [Bro-Dev] Fwd: [ACCEPTED] Pull request #5 for birkenfeld/pygments-main: Add lexer for the Bro language In-Reply-To: References: <20120205121952.11916.25642@bitbucket02.managed.contegix.com> Message-ID: <20120206065051.GC70452@icir.org> Very cool! Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Mon Feb 6 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 6 Feb 2012 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202060800.q16802MB016050@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ pysubnettree | 750 [1] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #750: http://tracker.bro-ids.org/bro/ticket/750 From seth at icir.org Mon Feb 6 19:33:44 2012 From: seth at icir.org (Seth Hall) Date: Mon, 6 Feb 2012 22:33:44 -0500 Subject: [Bro-Dev] change to PacketFilter framework for 2.1 Message-ID: Just so everyone know, I'm going to be making some modifications to the packet filtering framework's API for 2.1. That PacketFilter::all_packets variable in 2.0 has been driving me crazy because it's a fairly hidden variable that you have to set in order to make other configuration settings work. I'm going to go the direction that restriction filters will be the primary method of filtering going forward. The idea being that we want to watch everything, but we inform the packet filter as we learn that we don't want to see certain traffic anymore. I will add a configuration variable to enable the old behavior of dynamically creating a capture_filter. The default capture_filter for 2.1 will probably have a single entry of "ip or not ip" to capture everything. This should help a lot with the reaction framework (as a fallback method to "shunt" traffic), the BPF based load balancing code, and the load levels framework when that comes back. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Mon Feb 6 23:22:40 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 07 Feb 2012 07:22:40 -0000 Subject: [Bro-Dev] #451: Remove DNS options for skipping auth/addl events In-Reply-To: <047.8ae817995be2255322e68a394e45df3e@tracker.bro-ids.org> References: <047.8ae817995be2255322e68a394e45df3e@tracker.bro-ids.org> Message-ID: <062.ca1e2d0c17931a73de3ffdcd4f898824@tracker.bro-ids.org> #451: Remove DNS options for skipping auth/addl events ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by vern): D'oh, yeah. I should've figured/remembered there was a reason the scripts didn't adhere to the usual paradigm in the first place. Well, it is important to be able to turn this processing off, it can add up to a lot of performance load for something that may or may not be of any interest. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Feb 7 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 7 Feb 2012 00:00:04 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202070800.q17804D3001894@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ pysubnettree | 750 [1] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #750: http://tracker.bro-ids.org/bro/ticket/750 From bro at tracker.bro-ids.org Tue Feb 7 05:32:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 07 Feb 2012 13:32:02 -0000 Subject: [Bro-Dev] #451: Remove DNS options for skipping auth/addl events In-Reply-To: <047.8ae817995be2255322e68a394e45df3e@tracker.bro-ids.org> References: <047.8ae817995be2255322e68a394e45df3e@tracker.bro-ids.org> Message-ID: <062.0134b8389a3007a1b6de8b208265a3d4@tracker.bro-ids.org> #451: Remove DNS options for skipping auth/addl events ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by seth): > Well, it is > important to be able to turn this processing off, it can add up to a lot > of performance load for something that may or may not be of any interest. Agreed. I want to refocus on the events that are generated by the DNS analyzer soon too, I think we could improve those events. At the very least I want to be able to fully support DNSSEC which I think there was a problem with from the last time I looked. There are actually some other query types that we can't support correctly right now too (like the type for storing SSH fingerprints in DNS): http://tools.ietf.org/html/rfc4255 -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Feb 8 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 8 Feb 2012 00:00:04 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202080800.q18804w5022074@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ pysubnettree | 750 [1] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 9ab5180 | Jon Siwek | 2012-02-07 | Fix compiler warning about Brofiler ctor init list order. [2] [1] #750: http://tracker.bro-ids.org/bro/ticket/750 [2] fastpath: http://tracker.bro-ids.org/bro/changeset/9ab5180aa9f77f09441cf341ec3052b9d075333a/bro From bro at tracker.bro-ids.org Wed Feb 8 08:52:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 08 Feb 2012 16:52:38 -0000 Subject: [Bro-Dev] #765: topic/jsiwek/detect-webapps-fix Message-ID: <048.c9c15578215ca7707f37d82f4f69744c@tracker.bro-ids.org> #765: topic/jsiwek/detect-webapps-fix ---------------------------+------------------------ Reporter: jsiwek | Owner: seth Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch in `bro` fixes a "missing field value" error that the `bro- testing` repo's long test now shows in reporter.log. Also, the software.log baselines for tests in `bro-testing` and `bro- testing-private` weren't updated after the changes to the software framework that make it track port numbers. I didn't fix that in a branch anywhere, I figured Seth could just do that directly after reviewing the changes. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Feb 8 09:09:19 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 8 Feb 2012 09:09:19 -0800 Subject: [Bro-Dev] [Fwd] [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/topic/bernhard/software' (2cd88ee) Message-ID: <20120208170919.GC66576@icir.org> Both external test suites are failing in software.bro due to this merge (I'm guessing). Robin ----- Forwarded message from Seth Hall ----- Date: Fri, 3 Feb 2012 13:18:47 -0800 From: Seth Hall To: bro-commits at bro-ids.org Subject: [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/topic/bernhard/software' (2cd88ee) Message-Id: <201202032118.q13LIld4019875 at bro-ids.icir.org> Reply-To: bro-dev at bro-ids.org Repository : ssh://git at bro-ids.icir.org/bro On branch : master Link : http://tracker.bro-ids.org/bro/changeset/2cd88ee4f693657e8d880f88ef67744bebfadba9/bro -------- -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Wed Feb 8 09:28:27 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 08 Feb 2012 17:28:27 -0000 Subject: [Bro-Dev] #765: topic/jsiwek/detect-webapps-fix In-Reply-To: <048.c9c15578215ca7707f37d82f4f69744c@tracker.bro-ids.org> References: <048.c9c15578215ca7707f37d82f4f69744c@tracker.bro-ids.org> Message-ID: <063.d02ebf02edb1be66511abe8a7224e07a@tracker.bro-ids.org> #765: topic/jsiwek/detect-webapps-fix ----------------------------+------------------------ Reporter: jsiwek | Owner: seth Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): On Wed, Feb 08, 2012 at 16:52 -0000, you wrote: > Also, the software.log baselines for tests in `bro-testing` and `bro- > testing-private` weren't updated after the changes to the software > framework that make it track port numbers. Coincidentally, I just noticed that as well. Yes, Seth, please make sure to update the baselines. -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Wed Feb 8 09:31:33 2012 From: seth at icir.org (Seth Hall) Date: Wed, 8 Feb 2012 12:31:33 -0500 Subject: [Bro-Dev] [Fwd] [Bro-Commits] [git/bro] master: Merge remote-tracking branch 'origin/topic/bernhard/software' (2cd88ee) In-Reply-To: <20120208170919.GC66576@icir.org> References: <20120208170919.GC66576@icir.org> Message-ID: <0A1DA321-4137-4E98-B371-BDED36020308@icir.org> On Feb 8, 2012, at 12:09 PM, Robin Sommer wrote: > Both external test suites are failing in software.bro due to this > merge (I'm guessing). I'm going to merge Jon's branch in a few minutes. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From jsiwek at illinois.edu Wed Feb 8 15:02:12 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 8 Feb 2012 23:02:12 +0000 Subject: [Bro-Dev] new IPv6 code Message-ID: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> In topic/v6-addr, I finished switching Bro to enable IPv6 support by default and changing the internal representation to use classes. I tried to do some initial rough benchmarking to compare it versus master and master with --enable-brov6. Tests were run on a 16GB pcap (no IPv6 traffic) with local.bro loaded. master 120574kb vsize, 207.65s user, 20.36s system master --enable-brov6 126458kb vsize, 307.38s user, 20.87s system +5% mem, +44% cpu topic/v6-addr 121142kb vsize, 309.33s user, 20.62s system +0.5%mem, +45% cpu About 65% of the time difference in either the IPv6-supporting configurations looks to be due to the fact that the size of Conn::Key exceeds UHASH_KEY_SIZE and so falls back on the slower HMAC/MD5 when creating a HashKey of the data, which happens for every packet. The negligible memory usage difference surprised me since they differ from previous results here: http://tracker.bro-ids.org/bro/ticket/68. Maybe one possibility is that the scripts have changed so much since those benchmarks were run such that there's less state being tracked that involves addrs. Or the pcap I used was likely not as robust. Thoughts on how to improve the hashing or other plans to investigate memory usage? And does it seem like a good idea to merge topic/v6-addr into master at this point so that we get more users exercising that code and so we can start working in separate branches for more IPv6 enhancements? All the unit tests pass, but the external baselines need another set of eyes to sanity check them. +Jon From seth at icir.org Wed Feb 8 18:59:23 2012 From: seth at icir.org (Seth Hall) Date: Wed, 8 Feb 2012 21:59:23 -0500 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> Message-ID: <8A8132E5-AEA7-4D91-9DF0-DF33DBCD38B4@icir.org> On Feb 8, 2012, at 6:02 PM, Siwek, Jonathan Luke wrote: > topic/v6-addr > 121142kb vsize, 309.33s user, 20.62s system > +0.5%mem, +45% cpu The memory looks awesome, but that CPU increase really bothers me. > About 65% of the time difference in either the IPv6-supporting configurations looks to be due to the fact that the size of Conn::Key exceeds UHASH_KEY_SIZE and so falls back on the slower HMAC/MD5 when creating a HashKey of the data, which happens for every packet. Can we increase the size of the H3 hash keys? I don't know enough about it to know what the ramifications of it would be or if it's even possible (but it seems to me like it should be possible from a quick read of h3.h). > Maybe one possibility is that the scripts have changed so much since those benchmarks were run such that there's less state being tracked that involves addrs. I suspect that's the case. > And does it seem like a good idea to merge topic/v6-addr into master I'd like to get it in there asap as long as there aren't any huge problems (the cpu utilization does still bother me a bit though). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From noreply at bro-ids.org Thu Feb 9 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 9 Feb 2012 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202090800.q198022b008122@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 765 [1] | jsiwek | seth | Normal | topic/jsiwek/detect-webapps-fix [2] pysubnettree | 750 [3] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | a28e671 | Daniel Thayer | 2012-02-08 | Fix minor typos in the documentation [4] bro | 9ab5180 | Jon Siwek | 2012-02-07 | Fix compiler warning about Brofiler ctor init list order. [5] [1] #765: http://tracker.bro-ids.org/bro/ticket/765 [2] detect-webapps-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/detect-webapps-fix [3] #750: http://tracker.bro-ids.org/bro/ticket/750 [4] fastpath: http://tracker.bro-ids.org/bro/changeset/a28e671f8d6e9f7582c508020565159b11a2d78f/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/9ab5180aa9f77f09441cf341ec3052b9d075333a/bro From gregor at majordomus.org Wed Feb 8 19:26:10 2012 From: gregor at majordomus.org (Gregor Maier) Date: Wed, 08 Feb 2012 19:26:10 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> Message-ID: <4F333CD2.5080001@majordomus.org> > The negligible memory usage difference surprised me since they differ from previous results here: http://tracker.bro-ids.org/bro/ticket/68. Maybe one possibility is that the scripts have changed so much since those benchmarks were run such that there's less state being tracked that involves addrs. Or the pcap I used was likely not as robust. > I think it's quite likely that we just keep more script-level state in general so that the addresses don't have as much an effect. If you want to verify that memory usage is indeed going to stay low, I would run master and topic/v6-addr on the same live traffic for a day or so. cu Gregor From robin at icir.org Thu Feb 9 00:31:52 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 9 Feb 2012 00:31:52 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> Message-ID: <20120209083152.GB86135@icir.org> Very interesting, thanks! I need to look more closely but a quick note: most of the IPAddr methods should probably be inlined to avoid the function call overhead. That might help a bit with the CPU. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Thu Feb 9 00:35:01 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 9 Feb 2012 00:35:01 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> Message-ID: <20120209083501.GC86135@icir.org> On Wed, Feb 08, 2012 at 23:02 +0000, you wrote: > And does it seem like a good idea to merge topic/v6-addr into master > at this point so that we get more users exercising that code and so we > can start working in separate branches for more IPv6 enhancements? Another quick note: I'd also like to to merge this in asap, but if you want to start working on other IPv6 code already, just branch from the topic branch. That should then merge into master easily later as well. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Thu Feb 9 05:34:01 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 09 Feb 2012 13:34:01 -0000 Subject: [Bro-Dev] #765: topic/jsiwek/detect-webapps-fix In-Reply-To: <048.c9c15578215ca7707f37d82f4f69744c@tracker.bro-ids.org> References: <048.c9c15578215ca7707f37d82f4f69744c@tracker.bro-ids.org> Message-ID: <063.254e0c67ecd096f428b662148d12c164@tracker.bro-ids.org> #765: topic/jsiwek/detect-webapps-fix -----------------------------+------------------------ Reporter: jsiwek | Owner: seth Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by seth): * status: new => closed * resolution: => Solved/Applied Comment: Forgot to close this ticket. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 9 09:58:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 09 Feb 2012 17:58:56 -0000 Subject: [Bro-Dev] #766: isipv6 function Message-ID: <049.eb8395c8d5ec1175e2ac4748ced7026b@tracker.bro-ids.org> #766: isipv6 function ---------------------+----------------------------- Reporter: aashish | Type: Feature Request Status: new | Priority: Normal Milestone: Bro2.1 | Component: Bro Version: 2.0 | Keywords: ---------------------+----------------------------- While you are developing IPv6 capabilities, it would be nice to see a function which can return if an addr is IPv6 or IPv4. Reasoning: during initial implementation of IPv6, I might want to audit more on IPv6 traffic as well as what may be normal for IPv4 may not necessarily be ok in v6 world. Thanks -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 9 10:45:07 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 09 Feb 2012 18:45:07 -0000 Subject: [Bro-Dev] #766: isipv6 function In-Reply-To: <049.eb8395c8d5ec1175e2ac4748ced7026b@tracker.bro-ids.org> References: <049.eb8395c8d5ec1175e2ac4748ced7026b@tracker.bro-ids.org> Message-ID: <064.b021f4ddd70b362ae4accf9d7d406442@tracker.bro-ids.org> #766: isipv6 function ------------------------------+-------------------- Reporter: aashish | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: 2.0 Resolution: | Keywords: ipv6 ------------------------------+-------------------- Changes (by robin): * keywords: => ipv6 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 9 10:51:28 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 09 Feb 2012 18:51:28 -0000 Subject: [Bro-Dev] #766: isipv6 function In-Reply-To: <049.eb8395c8d5ec1175e2ac4748ced7026b@tracker.bro-ids.org> References: <049.eb8395c8d5ec1175e2ac4748ced7026b@tracker.bro-ids.org> Message-ID: <064.d15fbacda133b5a3afe67bc09ace7a5c@tracker.bro-ids.org> #766: isipv6 function ------------------------------+-------------------- Reporter: aashish | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: 2.0 Resolution: | Keywords: ipv6 ------------------------------+-------------------- Comment (by jsiwek): In `topic/v6-addr` I've added `is_v4_addr` and `is_v6_addr` builtin- functions. -- Ticket URL: Bro Tracker Bro Issue Tracker From jsiwek at illinois.edu Thu Feb 9 11:29:36 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Thu, 9 Feb 2012 19:29:36 +0000 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120209083152.GB86135@icir.org> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> Message-ID: On Feb 9, 2012, at 2:31 AM, Robin Sommer wrote: > I need to look more closely but a quick note: most of the IPAddr > methods should probably be inlined to avoid the function call > overhead. That might help a bit with the CPU. Yes, that closed the difference between --enable-brov6 and topic/v6-addr. On Feb 8, 2012, at 8:59 PM, Seth Hall wrote: > Can we increase the size of the H3 hash keys? I don't know enough about it to know what the ramifications of it would be or if it's even possible (but it seems to me like it should be possible from a quick read of h3.h). Yeah, I think the 32 byte data size threshold for switching algorithms is arbitrary. Increasing to 36 bytes so that ConnID::Key doesn't go through HMAC/MD5 helped a lot. With both those changes I get a +17% cpu time difference versus the default master configuration (down from +45%). About a fourth of that looks like it's still the result of having to hash/lookup larger ConnID::Key data. Accounting for the other difference is pretty spread out among other routines -- I'm thinking that's from now having to always deal with 16 bytes instead of 4 when working with addrs. Thoughts on next steps? I'm thinking we could probably optimize even more so IPv4 addrs perform better, but maybe this is acceptable enough to be considered for merging into master? +Jon From noreply at bro-ids.org Fri Feb 10 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 10 Feb 2012 00:00:04 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202100800.q1A804YN026596@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ pysubnettree | 750 [1] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | a28e671 | Daniel Thayer | 2012-02-08 | Fix minor typos in the documentation [2] bro | 9ab5180 | Jon Siwek | 2012-02-07 | Fix compiler warning about Brofiler ctor init list order. [3] [1] #750: http://tracker.bro-ids.org/bro/ticket/750 [2] fastpath: http://tracker.bro-ids.org/bro/changeset/a28e671f8d6e9f7582c508020565159b11a2d78f/bro [3] fastpath: http://tracker.bro-ids.org/bro/changeset/9ab5180aa9f77f09441cf341ec3052b9d075333a/bro From robin at icir.org Fri Feb 10 01:24:13 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 10 Feb 2012 01:24:13 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> Message-ID: <20120210092412.GA18051@icir.org> On Thu, Feb 09, 2012 at 19:29 +0000, you wrote: > With both those changes I get a +17% cpu time difference versus the > default master configuration (down from +45%). Cool, that sounds much better. Let me play with it a bit next week and if I see similar results on our traffic here, I'll merge it in and we can then see if we can tune it further. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Sat Feb 11 00:00:07 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 11 Feb 2012 00:00:07 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202110800.q1B807CE011324@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ pysubnettree | 750 [1] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #750: http://tracker.bro-ids.org/bro/ticket/750 From hlin33 at illinois.edu Sat Feb 11 13:18:12 2012 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Sat, 11 Feb 2012 15:18:12 -0600 Subject: [Bro-Dev] Hui Lin_Design principle under broccoli Message-ID: Hi, During my experiment of my DNP3 analyzer, I found some phenomena. Perhaps in-depth understanding of the broccoli can help me to explain this. I build my experiment based on ping-pong example codes. The experiment is like this DNP3 analyzer is in machine 1, periodically sending pong event (not depends on the receiving of ping event but depends on other DNP3 event ) to broccoli client in machine 2. (machine 1 and machine 2 are on the same Ethernet). When the broccoli receives pong event, the registered bro_pong function execute some other matlab module which takes some time of computations. I find that if the periodically pong event is sent frequent enough, when the first pong event is still processed by bro_pong function, the second pong event will be received again. I am wondering how broccoli client handle this type of situation? It simply drops the second pong events, or store them into some buffer of the broccoli client and drops the event until the buffer is full? I am catching a deadline next Thursday, so quick response is really appreciated. Best, Hui However -- Hui Lin Research Assistant DEPEND Research Group, ECE Department University of Illinois at Urbana-Champaign hlin33 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120211/4c80509f/attachment.html From seth at icir.org Sat Feb 11 17:56:06 2012 From: seth at icir.org (Seth Hall) Date: Sat, 11 Feb 2012 20:56:06 -0500 Subject: [Bro-Dev] Hui Lin_Design principle under broccoli In-Reply-To: References: Message-ID: <7F0B5B2C-1B29-47A7-A80C-29E8F9E3BAB4@icir.org> On Feb 11, 2012, at 4:18 PM, Hui Lin (Hugo) wrote: > I am wondering how broccoli client handle this type of situation? It simply drops the second pong events, or store them into some buffer of the broccoli client and drops the event until the buffer is full? Someone else will probably give a better answer, but I believe that if there are additional events sent before you check for events again they will be buffered in the socket buffer. If you expect to have large numbers of buffered events you should be able to increase your OS receive socket buffer size. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From noreply at bro-ids.org Sun Feb 12 00:00:06 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 12 Feb 2012 00:00:06 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202120800.q1C8069S019251@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ pysubnettree | 750 [1] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #750: http://tracker.bro-ids.org/bro/ticket/750 From robin at icir.org Sun Feb 12 21:15:47 2012 From: robin at icir.org (Robin Sommer) Date: Sun, 12 Feb 2012 21:15:47 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120210092412.GA18051@icir.org> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> Message-ID: <20120213051547.GA50682@icir.org> On Fri, Feb 10, 2012 at 01:24 -0800, I wrote: > Cool, that sounds much better. Let me play with it a bit next week I've now run it on about 9GB of traffic. The surprising thing is that I don't see much of a difference at all between non-v6 master and the branch: master wo/ --enable-brov6 4m43.56s real 4m27.10s user 15.32s sys 243020 maximum resident set size master w/ --enable-brov6 4m55.97s real 4m42.04s user 12.71s sys 249376 maximum resident set size topic/v6-addr 4m46.32s real 4m30.37s user 14.77s sys 244148 maximum resident set size This is on an x86_64 Linux. Jon, what platform did you run your comparision on? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Mon Feb 13 00:00:07 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 13 Feb 2012 00:00:07 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202130800.q1D8072s025287@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ pysubnettree | 750 [1] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #750: http://tracker.bro-ids.org/bro/ticket/750 From seth at icir.org Mon Feb 13 05:50:52 2012 From: seth at icir.org (Seth Hall) Date: Mon, 13 Feb 2012 08:50:52 -0500 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120213051547.GA50682@icir.org> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> Message-ID: <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> On Feb 13, 2012, at 12:15 AM, Robin Sommer wrote: > This is on an x86_64 Linux. Jon, what platform did you run your > comparision on? Did you load the local.bro script? Jon said that he did and that should make a big difference in memory usage and performance. Also, we need to compare if you two populated the Site::local_nets variable since that should make an impact as well. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Mon Feb 13 06:49:54 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 13 Feb 2012 06:49:54 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> Message-ID: <20120213144954.GA61942@icir.org> On Mon, Feb 13, 2012 at 08:50 -0500, you wrote: > Did you load the local.bro script? No, I didn't. I'll try that. (But still, if the base config doesn't change at all, that's good! Means that at least internally we won't have much of a problem.) > usage and performance. Also, we need to compare if you two > populated the Site::local_nets variable since that should make an > impact as well. Didn't do that either. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From jsiwek at illinois.edu Mon Feb 13 07:50:47 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Mon, 13 Feb 2012 15:50:47 +0000 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120213051547.GA50682@icir.org> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> Message-ID: <5B32F7D3-A877-49FB-A2F4-FDBF4B741555@illinois.edu> > This is on an x86_64 Linux. Jon, what platform did you run your > comparision on? Also x86_64 Linux, with local.bro loaded and nothing added to Site::local_nets. +Jon From robin at icir.org Mon Feb 13 07:52:25 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 13 Feb 2012 07:52:25 -0800 Subject: [Bro-Dev] Hui Lin_Design principle under broccoli In-Reply-To: <7F0B5B2C-1B29-47A7-A80C-29E8F9E3BAB4@icir.org> References: <7F0B5B2C-1B29-47A7-A80C-29E8F9E3BAB4@icir.org> Message-ID: <20120213155225.GH61942@icir.org> On Sat, Feb 11, 2012 at 20:56 -0500, you wrote: > Someone else will probably give a better answer, but I believe that if > there are additional events sent before you check for events again > they will be buffered in the socket buffer. Correct, it's all going over a TCP connection so nothing will be dropped (but things may block eventually if buffers are filled). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Mon Feb 13 19:56:45 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 13 Feb 2012 19:56:45 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120213144954.GA61942@icir.org> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> <20120213144954.GA61942@icir.org> Message-ID: <20120214035645.GB78396@icir.org> On Mon, Feb 13, 2012 at 06:49 -0800, I wrote: > No, I didn't. I'll try that. Even with local.bro and setting local networks I don't see much of a difference in terms of CPU time and memory. That sounds almost too good to be true ... Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Tue Feb 14 00:00:05 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 14 Feb 2012 00:00:05 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202140800.q1E805Cr028670@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ pysubnettree | 750 [1] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #750: http://tracker.bro-ids.org/bro/ticket/750 From robin at icir.org Tue Feb 14 06:26:39 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 14 Feb 2012 06:26:39 -0800 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/bernhard/input-threads: It works. Even including all unit tests. (88233ef) In-Reply-To: <201202140632.q1E6WHko011577@bro-ids.icir.org> References: <201202140632.q1E6WHko011577@bro-ids.icir.org> Message-ID: <20120214142639.GB82204@icir.org> On Mon, Feb 13, 2012 at 22:32 -0800, Bernhard Amann wrote: > But - it works :) Very cool! Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Tue Feb 14 06:37:55 2012 From: seth at icir.org (Seth Hall) Date: Tue, 14 Feb 2012 09:37:55 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/bernhard/input-threads: It works. Even including all unit tests. (88233ef) In-Reply-To: <20120214142639.GB82204@icir.org> References: <201202140632.q1E6WHko011577@bro-ids.icir.org> <20120214142639.GB82204@icir.org> Message-ID: On Feb 14, 2012, at 9:26 AM, Robin Sommer wrote: > > On Mon, Feb 13, 2012 at 22:32 -0800, Bernhard Amann wrote: > >> But - it works :) > > Very cool! Yes! Now we just need the API additions that were waiting on multithreading and I can integrate my Unified2 parser. ;) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Tue Feb 14 11:19:55 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 14 Feb 2012 11:19:55 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120214035645.GB78396@icir.org> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> <20120213144954.GA61942@icir.org> <20120214035645.GB78396@icir.org> Message-ID: <20120214191955.GA1162@icir.org> On Mon, Feb 13, 2012 at 19:56 -0800, I wrote: > Even with local.bro and setting local networks I don't see much of a > difference in terms of CPU time and memory. So should I go ahead and merge it into master? Or, Seth, do you want to give it a try first as well? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Tue Feb 14 11:58:43 2012 From: seth at icir.org (Seth Hall) Date: Tue, 14 Feb 2012 14:58:43 -0500 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120214191955.GA1162@icir.org> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> <20120213144954.GA61942@icir.org> <20120214035645.GB78396@icir.org> <20120214191955.GA1162@icir.org> Message-ID: On Feb 14, 2012, at 2:19 PM, Robin Sommer wrote: > Or, Seth, do you want to give it a try first as well? I have been playing with it quietly and haven't seen any trouble. Unfortunately I haven't run on high volume live traffic yet. I say merge away. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From noreply at bro-ids.org Wed Feb 15 00:00:05 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 15 Feb 2012 00:00:05 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202150800.q1F805i5020810@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ pysubnettree | 750 [1] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #750: http://tracker.bro-ids.org/bro/ticket/750 From bro at tracker.bro-ids.org Wed Feb 15 06:27:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 15 Feb 2012 14:27:06 -0000 Subject: [Bro-Dev] #767: Active copies of scripts not hidden Message-ID: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> #767: Active copies of scripts not hidden ------------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Keywords: | ------------------------+------------------------ I'm not sure when this happened, but I thought that the files that broctl copies were put into a hidden dot directory. That doesn't seem to be the case anymore since the scripts are copied to spool/policy which is plainly visible. We've already had one case where someone was editing their local.bro script in there and lost the changes when they did "install". My suggestion for remediation of this ticket is: - Move the spool/policy directory to a name like spool/.active_scripts_dont_touch -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 15 07:27:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 15 Feb 2012 15:27:02 -0000 Subject: [Bro-Dev] #768: Inline monitoring of modified scripts. Message-ID: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> #768: Inline monitoring of modified scripts. ------------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Keywords: | ------------------------+------------------------ We need to train users to do check, install, restart through broctl better. I'd like to reduce the barrier to entry a bit more and if broctl can coach new users through the process better and remind existing users of the process it would be great. Here are my suggestions for what I think needs to be done: - Track hashes for all copied scripts (maybe in broctl.dat?) and watch for changes to notify the user. I think it would be ok to only notify the user when they are in broctl but I can see that people may want that to also check and occasionally email from broctl cron (let's save emailing for later though, inline notification in broctl may be enough). - Track hashes for scripts that have been "checked" because then we can coach people about what step in the process they are at. If someone has already run "check" on the current scripts we can recommend that they need to - Create variables to turn off various suggestions. I think the various suggestions would be "need to check scripts", "need to install scripts", and "ready to restart" or something along those lines. I'm not even sure I like this idea though. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 15 07:49:07 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 15 Feb 2012 15:49:07 -0000 Subject: [Bro-Dev] #768: Inline monitoring of modified scripts. In-Reply-To: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> References: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> Message-ID: <061.c088d2f1a46b69d1d2be210746b6106b@tracker.bro-ids.org> #768: Inline monitoring of modified scripts. -------------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by robin): I like this. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 15 07:58:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 15 Feb 2012 15:58:02 -0000 Subject: [Bro-Dev] #768: Inline monitoring of modified scripts. In-Reply-To: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> References: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> Message-ID: <061.2d2761fd664bedd82aa30fd06ad09539@tracker.bro-ids.org> #768: Inline monitoring of modified scripts. -------------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by seth): Oh yeah, I wanted to include what I thought a common example might look like... The scenario here is that a user has made modifications to their scripts before starting broctl. They forgot and don't pay attention to the WARNING and restart their cluster. BroControl is warning them each time before it writes out the shell prompt that they have uninstalled scripts. Also, notice the lack of the need to run "check". It's integrated into the install command, I think that we should be fine doing it if we are tracking which scripts have been checked already. We could continue to maintain the "check" command since it has use beyond the install command and the install command would just call it internally anyway. Removing the *need* to run check would help get people up to speed with broctl a lot faster I think. I cleaned up some of the display and wording too. I think the "install" command was dumping more information on users than it needed to. Thoughts or further refinement? {{{ [user at localhost]$ sudo /bro/bin/broctl Password:*********** Welcome to BroControl 1.1 Type "help" for help. WARNING: There are modified scripts that haven't been installed. [BroControl] > restart stopping: worker-1 not running worker-2 not running proxy-1 not running manager not running starting: manager ... proxy-1 ... worker-1 ... worker-2 ... WARNING: There are modified scripts that haven't been installed. [BroControl] > install checking scripts ... done. generating cluster-layout.bro ... done. generating local-networks.bro ... done. generating broctl-config.bro ... done. clearing old scripts ... done. install new scripts... done. copying to nodes...done. [BroControl] > restart stopping: worker-1 ... worker-2 ... proxy-1 ... manager ... starting: manager ... proxy-1 ... worker-1 ... worker-2 ... [BroControl] > quit }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 15 08:00:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 15 Feb 2012 16:00:19 -0000 Subject: [Bro-Dev] #767: Active copies of scripts not hidden In-Reply-To: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> References: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> Message-ID: <061.77dc9df5b96215f2fac5dad53395c2ec@tracker.bro-ids.org> #767: Active copies of scripts not hidden -------------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by seth): This is super easy, the default setting for spooldir just needs to change to spool/.active_scripts_dont_touch -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 15 08:36:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 15 Feb 2012 16:36:02 -0000 Subject: [Bro-Dev] #769: Detect non-caching recursive resolvers Message-ID: <046.010eb37acf4ecf1127b244c4355cbd20@tracker.bro-ids.org> #769: Detect non-caching recursive resolvers -----------------------------+------------------------ Reporter: seth | Owner: seth Type: Feature Request | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Keywords: | -----------------------------+------------------------ Two steps to this... - Detect recursive resolvers. This should probably be added to the intelligence framework so that it could be autodetected and people could add their own locally known information to it. We should be able to detect them by watching for lots of authoritative requests but there are probably other indicators we could use as well. - Occasionally grab or define certain host names with reasonably long TTLs (a day?) and watch for the same recursive resolver to make a request for that same hostname within the TTL. This should identify if resolvers aren't caching results which is frequently an interesting data point or at least something to go and fix. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 15 09:27:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 15 Feb 2012 17:27:10 -0000 Subject: [Bro-Dev] #770: topic/v6-addr Message-ID: <048.44b3da34753272b3bb7e6a731b786048@tracker.bro-ids.org> #770: topic/v6-addr ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch enables IPv6 support in the default Bro configuration. Notable changes are in commits: [b3f1f45082ef8ad0e13574f0c0a2702769cf820f/bro] [eca32610779dc6c707fb734e49cc0be379c6cfd5/bro] [4cb6a279f58b775ab464c7a7539a3d1c5896d86a/bro] [31565d6987e3bd15f2e4fbd3f1365d29deebd9eb/bro] [f945f3c5183dcc9fd3a0f6030e2522692123c01e/bro] [086f747bc14a9fbfdbb68e645287a9303b42c4fe/bro] [0f207c243c9e4ebfcca227b71add3ac2e8af1cc4/bro] External baseline tests have not been updated anywhere. I have looked them over and didn't think there were any problems, but it would be good to have someone else sanity check. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 15 09:28:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 15 Feb 2012 17:28:14 -0000 Subject: [Bro-Dev] #355: lookup_addr() only supports IPv4 addresses In-Reply-To: <047.6c096d333fd37f7e2ad084ea1c108762@tracker.bro-ids.org> References: <047.6c096d333fd37f7e2ad084ea1c108762@tracker.bro-ids.org> Message-ID: <062.425374d266ba7679934259913d09c86f@tracker.bro-ids.org> #355: lookup_addr() only supports IPv4 addresses ----------------------+-------------------- Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ipv6 ----------------------+-------------------- Comment (by jsiwek): This was addressed by [0f207c243c9e4ebfcca227b71add3ac2e8af1cc4/bro], not yet in master, pending merge request in #770. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 15 09:29:55 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 15 Feb 2012 17:29:55 -0000 Subject: [Bro-Dev] #771: topic/jsiwek/v6-dns-name-lookup Message-ID: <048.b7ae8177f147dea2c149ee24034bfffc@tracker.bro-ids.org> #771: topic/jsiwek/v6-dns-name-lookup ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch enabled DNS name resolutions performed by Bro to query both A and AAAA records (previously just A was done). -- Ticket URL: Bro Tracker Bro Issue Tracker From hlin33 at illinois.edu Wed Feb 15 11:59:20 2012 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Wed, 15 Feb 2012 13:59:20 -0600 Subject: [Bro-Dev] Hui Lin_synchronization problem with Bro Message-ID: Hi, I had confusions on how Bro behaves when a single instance is listening on two ethernet interfaces. Here is what I encounter. I still based on my code in broping. I install a Bro on the same machine on this broping app. This broping does two things in order: sending ping events to loopback interface and then send TCPs request to another machine and get response. I just use broping to repeat this again and again and I don't insert any latency between each iteration. With Bro monitoring on these events from two ethernet interface, I find that many ping events are received by Bro after the TCP request/responses and this happens a lot. So Bro sees events in different orders than they are sent. Any comment on this phenomenon? Is that anyway, that I can handle situation? Best, Hui -- Hui Lin Research Assistant DEPEND Research Group, ECE Department University of Illinois at Urbana-Champaign hlin33 at illinois.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120215/2bcc77a0/attachment.html From robin at icir.org Wed Feb 15 13:09:15 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 15 Feb 2012 13:09:15 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120214191955.GA1162@icir.org> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> <20120213144954.GA61942@icir.org> <20120214035645.GB78396@icir.org> <20120214191955.GA1162@icir.org> Message-ID: <20120215210915.GZ37020@icir.org> I'm going through the v6-addr branch. Good job with that! A few questions: - in BroValUnion, 'addr_val' is of type pointer type IPAddr*. I suppose that's to keep the union size as small as possible, which makes sense. I'd still be curious though how allocating the addresses dynamically compares against storing the instance directly inside the union. Have you tried that? I realize it's not an easy change to do that, so fine if not. - The DNS binpac analyzer generates both dns_a6_reply and dns_aaaa_reply, but the standard analyzer only the latter (for both resource record types). What is correct? - In the RPC code, there's this: is_mapped_dce_rpc_endpoint(const ConnID* id, TransportProto proto) ????????{ ????????if ( id->dst_addr.family() == IPAddr::IPv6 ) ????????????????return false; ... } Does the protocol not support IPv6 at all or is that a "todo"? - in DPM.cc, the ExpectedConn class: should it now store IPAddr instead of uint32[4] for orig and resp? - Expr.cc, BinaryExpr::AddrFold: why still the uint32[4] here? If it's just to make the macro work, I'd just remove that and use comparision operators instead. - IP.h, IP_Hdr: Do we need the new src_addr/dst_addr attributes? Why not construct on the fly out of ip4/ip6? - PktSrc.cc: - protocol = (data[3] << 24) + (data[2] << 16) + (data[1] << 8) + data[0]; + protocol = (data[0] << 24) + (data[1] << 16) + (data[2] << 8) + data[3]; Does that mean this was buggy before? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Wed Feb 15 13:12:32 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 15 Feb 2012 13:12:32 -0800 Subject: [Bro-Dev] IPv6 testing Message-ID: <20120215211232.GA46781@icir.org> I don't recall what we said about testing IPv6. Do we have traces that we can include as part of the public test-suite? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Wed Feb 15 13:19:27 2012 From: seth at icir.org (Seth Hall) Date: Wed, 15 Feb 2012 16:19:27 -0500 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120215210915.GZ37020@icir.org> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> <20120213144954.GA61942@icir.org> <20120214035645.GB78396@icir.org> <20120214191955.GA1162@icir.org> <20120215210915.GZ37020@icir.org> Message-ID: <3222B6E0-5A55-4BD9-8EF2-C7AE0366C4FC@icir.org> On Feb 15, 2012, at 4:09 PM, Robin Sommer wrote: > - PktSrc.cc: > > - protocol = (data[3] << 24) + (data[2] << 16) + (data[1] << 8) + data[0]; > + protocol = (data[0] << 24) + (data[1] << 16) + (data[2] << 8) + data[3]; > > Does that mean this was buggy before? (I did that). It seems like it was, but Jon pointed out that it may still be wrong if that 32bit int is supposed to be in a defined byte order. It was broken for me on Mac OS X at least with some IPv6 packets I had that used the loopback interface. Jon did also point out that it probably shouldn't have been fixed in the v6 branch which made sense to me after I had already committed it. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Wed Feb 15 13:20:14 2012 From: seth at icir.org (Seth Hall) Date: Wed, 15 Feb 2012 16:20:14 -0500 Subject: [Bro-Dev] IPv6 testing In-Reply-To: <20120215211232.GA46781@icir.org> References: <20120215211232.GA46781@icir.org> Message-ID: <878E3A2E-B883-4C9E-A540-15798B11B438@icir.org> On Feb 15, 2012, at 4:12 PM, Robin Sommer wrote: > I don't recall what we said about testing IPv6. Do we have traces that > we can include as part of the public test-suite? Daniel has some for EPRT and EPSV I created for him that are in his branch where he's fixing/testing FTP over IPv6. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From jsiwek at illinois.edu Wed Feb 15 14:33:42 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 15 Feb 2012 22:33:42 +0000 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120215210915.GZ37020@icir.org> References: <8984D883-4872-46D3-AEBD-18A6CC5E87E9@illinois.edu> <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> <20120213144954.GA61942@icir.org> <20120214035645.GB78396@icir.org> <20120214191955.GA1162@icir.org> <20120215210915.GZ37020@icir.org> Message-ID: > - in BroValUnion, 'addr_val' is of type pointer type IPAddr*. I > suppose that's to keep the union size as small as possible, which > makes sense. I'd still be curious though how allocating the > addresses dynamically compares against storing the instance directly > inside the union. Have you tried that? I realize it's not an easy > change to do that, so fine if not. You mean having a IPAddr versus IPAddr* member in the union? I had started out that way, until the compiler told me union members cannot have non-trivial constructors. And I figured going with a pointer is better for size anyway as you mentioned, although the largest member in the old --enable-brov6 code was the subnets with five uint32's and that did not seem to increase memory as much in the tests as I expected. > > - The DNS binpac analyzer generates both dns_a6_reply and > dns_aaaa_reply, but the standard analyzer only the latter (for both > resource record types). What is correct? My thought was to generate both depending on the type, but I guess I forgot to do it for the non-binpac one. I'll fix it. > > - In the RPC code, there's this: > > is_mapped_dce_rpc_endpoint(const ConnID* id, TransportProto proto) > ????????{ > ????????if ( id->dst_addr.family() == IPAddr::IPv6 ) > ????????????????return false; > ... > } > > Does the protocol not support IPv6 at all or is that a "todo"? I'm not sure -- the old --enable-brov6 switch also bailed out for IPv6. So yeah, I suppose the todo is to determine how much more of a todo there is. > > - in DPM.cc, the ExpectedConn class: should it now store IPAddr > instead of uint32[4] for orig and resp? Yes, seems like it could do that. > > - Expr.cc, BinaryExpr::AddrFold: why still the uint32[4] here? If it's > just to make the macro work, I'd just remove that and use > comparision operators instead. > Ok, will do that. > - IP.h, IP_Hdr: Do we need the new src_addr/dst_addr attributes? Why > not construct on the fly out of ip4/ip6? I guess not. Hadn't thought of that as I was just replicating the face that the old --enable-brov6 code also added those. Will fix. +Jon From bro at tracker.bro-ids.org Wed Feb 15 14:41:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 15 Feb 2012 22:41:02 -0000 Subject: [Bro-Dev] #770: topic/v6-addr In-Reply-To: <048.44b3da34753272b3bb7e6a731b786048@tracker.bro-ids.org> References: <048.44b3da34753272b3bb7e6a731b786048@tracker.bro-ids.org> Message-ID: <063.8dc1728fca383652ef02b0270631ca7c@tracker.bro-ids.org> #770: topic/v6-addr ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by jsiwek): * owner: => jsiwek * status: new => assigned * type: Merge Request => Task Comment: Some suggestions from Robin that I'll fix {{{ - The DNS binpac analyzer generates both dns_a6_reply and dns_aaaa_reply, but the standard analyzer only the latter (for both resource record types). What is correct? - in DPM.cc, the ExpectedConn class: should it now store IPAddr instead of uint32[4] for orig and resp? - Expr.cc, BinaryExpr::AddrFold: why still the uint32[4] here? If it's just to make the macro work, I'd just remove that and use comparision operators instead. - IP.h, IP_Hdr: Do we need the new src_addr/dst_addr attributes? Why not construct on the fly out of ip4/ip6? }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 15 14:42:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 15 Feb 2012 22:42:20 -0000 Subject: [Bro-Dev] #771: topic/jsiwek/v6-dns-name-lookup In-Reply-To: <048.b7ae8177f147dea2c149ee24034bfffc@tracker.bro-ids.org> References: <048.b7ae8177f147dea2c149ee24034bfffc@tracker.bro-ids.org> Message-ID: <063.6c76dd64747e4958963f1fa841c665c1@tracker.bro-ids.org> #771: topic/jsiwek/v6-dns-name-lookup ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Description changed by jsiwek: Old description: > This branch enabled DNS name resolutions performed by Bro to query both A > and AAAA records (previously just A was done). New description: This branch enables DNS name resolutions performed by Bro to query both A and AAAA records (previously just A was done). This merge depends on the one in #770 first. -- -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Feb 15 14:42:53 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 15 Feb 2012 14:42:53 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: References: <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> <20120213144954.GA61942@icir.org> <20120214035645.GB78396@icir.org> <20120214191955.GA1162@icir.org> <20120215210915.GZ37020@icir.org> Message-ID: <20120215224253.GB47348@icir.org> On Wed, Feb 15, 2012 at 22:33 +0000, you wrote: > You mean having a IPAddr versus IPAddr* member in the union? > I had started out that way, until the compiler told me union members > cannot have non-trivial constructors. Ah, good point. Then it's mood anyway. Thanks, Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Wed Feb 15 19:06:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 03:06:38 -0000 Subject: [Bro-Dev] #769: Detect non-caching recursive resolvers In-Reply-To: <046.010eb37acf4ecf1127b244c4355cbd20@tracker.bro-ids.org> References: <046.010eb37acf4ecf1127b244c4355cbd20@tracker.bro-ids.org> Message-ID: <061.31a5ccd0c520a215f7b2a00fae31b5a7@tracker.bro-ids.org> #769: Detect non-caching recursive resolvers ------------------------------+------------------------ Reporter: seth | Owner: seth Type: Feature Request | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Comment (by gregor): > - Detect recursive resolvers. This should probably be added to the > intelligence framework so that it could be autodetected and people could > add their own locally known information to it. We should be able to > detect them by watching for lots of authoritative requests but there are > probably other indicators we could use as well. The RD (recursion desired) flag is another indicator. It should be off for recursive resolvers. cu Gregor -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Feb 16 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 16 Feb 2012 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202160800.q1G803bg032275@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 771 [1] | jsiwek | | Normal | topic/jsiwek/v6-dns-name-lookup [2] pysubnettree | 750 [3] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #771: http://tracker.bro-ids.org/bro/ticket/771 [2] v6-dns-name-lookup: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/v6-dns-name-lookup [3] #750: http://tracker.bro-ids.org/bro/ticket/750 From bro at tracker.bro-ids.org Thu Feb 16 07:11:31 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 15:11:31 -0000 Subject: [Bro-Dev] #772: Problem with $path in Log filters Message-ID: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> #772: Problem with $path in Log filters ---------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ I finally wrote a generic path_func and it doesn't work. The 'path' variable that is passed into the $path_func field when it's called is not filled out. This should be an easy fix. {{{ module Log; export { ## A generic log path function that can be used in any filter if the record associated ## with the stream has a field named 'id' of type :bro:type:`conn_id` to split the log ## records into different files names based on if the connection was originated locally ## or not. global directional_path_func: function(id: Log::ID, path: string, rec: record {id: conn_id;}): string; } function directional_path_func(id: Log::ID, path: string, rec: record { id: conn_id; }): string { local direction: string; local orig_local = Site::is_local_addr(rec$id$orig_h); local resp_local = Site::is_local_addr(rec$id$resp_h); if ( orig_local ) direction = resp_local ? "localonly" : "outbound"; else direction = resp_local ? "inbound" : "remoteonly"; return fmt("%s_%s", path, direction); } event bro_init() { Log::remove_default_filter(DNS::LOG); Log::add_filter(DNS::LOG, [$name = "directional_split", $path_func = directional_path_func]); Log::remove_default_filter(HTTP::LOG); Log::add_filter(HTTP::LOG, [$name = "directional_split", $path_func = directional_path_func]); } }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 16 07:11:43 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 15:11:43 -0000 Subject: [Bro-Dev] #772: Problem with $path_func in Log filters (was: Problem with $path in Log filters) In-Reply-To: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> References: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> Message-ID: <061.dc77f1ba6fba4c591649d7ac7107a043@tracker.bro-ids.org> #772: Problem with $path_func in Log filters ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 16 07:52:04 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 15:52:04 -0000 Subject: [Bro-Dev] #772: Problem with $path_func in Log filters In-Reply-To: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> References: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> Message-ID: <061.82493554563710297c67917997a16e12@tracker.bro-ids.org> #772: Problem with $path_func in Log filters ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): The value of the `path` parameter in the `path_func` is tricky. Here's the documentation for it: {{{ ## path: A suggested path value, which may be either the filter's ## ``path`` if defined, else a previous result from the function. ## If no ``path`` is defined for the filter, then the first call ## to the function will contain an empty string. }}} So does setting `$path = "dns"` and `$path = "http"` in the filters do what you want? -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Thu Feb 16 07:55:27 2012 From: seth at icir.org (Seth Hall) Date: Thu, 16 Feb 2012 10:55:27 -0500 Subject: [Bro-Dev] #772: Problem with $path_func in Log filters In-Reply-To: <061.82493554563710297c67917997a16e12@tracker.bro-ids.org> References: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> <061.82493554563710297c67917997a16e12@tracker.bro-ids.org> Message-ID: > So does setting `$path = "dns"` and `$path = "http"` in the filters do > what you want? Hm, I suppose it would. There should be the default path that is set though, right? From bro at tracker.bro-ids.org Thu Feb 16 07:55:30 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 15:55:30 -0000 Subject: [Bro-Dev] #772: Problem with $path_func in Log filters In-Reply-To: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> References: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> Message-ID: <061.39542a9fcbe1f035569cc33774359405@tracker.bro-ids.org> #772: Problem with $path_func in Log filters ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by seth): > So does setting `$path = "dns"` and `$path = "http"` in the filters do > what you want? Hm, I suppose it would. There should be the default path that is set though, right? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 16 08:22:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 16:22:03 -0000 Subject: [Bro-Dev] #772: Problem with $path_func in Log filters In-Reply-To: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> References: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> Message-ID: <061.be0de1418f0da824909c91e7985b544f@tracker.bro-ids.org> #772: Problem with $path_func in Log filters ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): > > So does setting `$path = "dns"` and `$path = "http"` in the filters do > > what you want? > > Hm, I suppose it would. There should be the default path that is set though, right? I don't think that gets decided internally anymore: it's entirely up to the `default_path_func`, which you're now replacing. Maybe a better way than explicitly setting a filter $path is to find a way to refactor/re-use the code in `default_path_func` that transforms the Log::ID into a default path and then call out to it from the new path_func. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Thu Feb 16 08:22:57 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 16 Feb 2012 08:22:57 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <3222B6E0-5A55-4BD9-8EF2-C7AE0366C4FC@icir.org> References: <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> <20120213144954.GA61942@icir.org> <20120214035645.GB78396@icir.org> <20120214191955.GA1162@icir.org> <20120215210915.GZ37020@icir.org> <3222B6E0-5A55-4BD9-8EF2-C7AE0366C4FC@icir.org> Message-ID: <20120216162257.GE32670@icir.org> On Wed, Feb 15, 2012 at 16:19 -0500, you wrote: > Jon did also point out that it probably shouldn't have been fixed in the v6 branch which made sense to me after I had already committed it. :) Yeah. :) Sounds like I should leave this out when merging? Not much help to go from broken to broken differently ... Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Thu Feb 16 08:27:09 2012 From: seth at icir.org (Seth Hall) Date: Thu, 16 Feb 2012 11:27:09 -0500 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120216162257.GE32670@icir.org> References: <20120209083152.GB86135@icir.org> <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> <20120213144954.GA61942@icir.org> <20120214035645.GB78396@icir.org> <20120214191955.GA1162@icir.org> <20120215210915.GZ37020@icir.org> <3222B6E0-5A55-4BD9-8EF2-C7AE0366C4FC@icir.org> <20120216162257.GE32670@icir.org> Message-ID: On Feb 16, 2012, at 11:22 AM, Robin Sommer wrote: > Sounds like I should leave this out when merging? Not much help to go > from broken to broken differently ... Yep, it's pretty low priority anyway since it's only going to come into play when sniffing loopback. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Thu Feb 16 08:34:13 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 16:34:13 -0000 Subject: [Bro-Dev] #773: DCE_RPC IPv6 support Message-ID: <048.4dd4f1bdcaff64eb4d2323749ab5b206@tracker.bro-ids.org> #773: DCE_RPC IPv6 support ---------------------+------------------------ Reporter: jsiwek | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: ipv6 | ---------------------+------------------------ In DCE_RPC.cc, the `is_mapped_dce_rpc_endpoint` function (after changes in #770) has: {{{ if ( id->dst_addr.family() == IPAddr::IPv6 ) return false; }}} This is directly ported from the old --enable-brov6 version of the code, but we should find out if that guard is really necessary. (this function doesn't currently appear to be called from anywhere, though) -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 16 08:54:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 16:54:46 -0000 Subject: [Bro-Dev] #774: IPv6 in signatures Message-ID: <046.e3fc459c715e558f55a8d0d7807eaeac@tracker.bro-ids.org> #774: IPv6 in signatures ---------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ IPv6 addresses aren't supported in signatures. This isn't huge priority since addresses are very rarely used in signatures, but I wanted to document the broken-ness. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 16 08:59:08 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 16:59:08 -0000 Subject: [Bro-Dev] #774: IPv6 in signatures In-Reply-To: <046.e3fc459c715e558f55a8d0d7807eaeac@tracker.bro-ids.org> References: <046.e3fc459c715e558f55a8d0d7807eaeac@tracker.bro-ids.org> Message-ID: <061.41ce0571355b2d42b82165a703b36f2b@tracker.bro-ids.org> #774: IPv6 in signatures ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------+------------------------ Changes (by jsiwek): * keywords: => ipv6 -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Thu Feb 16 09:01:51 2012 From: seth at icir.org (Seth Hall) Date: Thu, 16 Feb 2012 12:01:51 -0500 Subject: [Bro-Dev] #772: Problem with $path_func in Log filters In-Reply-To: <061.be0de1418f0da824909c91e7985b544f@tracker.bro-ids.org> References: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> <061.be0de1418f0da824909c91e7985b544f@tracker.bro-ids.org> Message-ID: <482E425D-0CA5-4E9F-B00D-30F8115DACA8@icir.org> > Maybe a better way > than explicitly setting a filter $path is to find a way to refactor/re-use > the code in `default_path_func` that transforms the Log::ID into a default > path and then call out to it from the new path_func. Ah! You're right. I'll fix it. From bro at tracker.bro-ids.org Thu Feb 16 09:01:53 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 17:01:53 -0000 Subject: [Bro-Dev] #772: Problem with $path_func in Log filters In-Reply-To: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> References: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> Message-ID: <061.cdbe7998b6cc80f7c351fd15fef18339@tracker.bro-ids.org> #772: Problem with $path_func in Log filters ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by seth): > Maybe a better way > than explicitly setting a filter $path is to find a way to refactor/re- use > the code in `default_path_func` that transforms the Log::ID into a default > path and then call out to it from the new path_func. Ah! You're right. I'll fix it. -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Thu Feb 16 09:11:42 2012 From: seth at icir.org (Seth Hall) Date: Thu, 16 Feb 2012 12:11:42 -0500 Subject: [Bro-Dev] #772: Problem with $path_func in Log filters In-Reply-To: <482E425D-0CA5-4E9F-B00D-30F8115DACA8@icir.org> References: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> <061.be0de1418f0da824909c91e7985b544f@tracker.bro-ids.org> <482E425D-0CA5-4E9F-B00D-30F8115DACA8@icir.org> Message-ID: <873BDFB4-39D1-4E06-BEF0-35B0456C94D8@icir.org> On Feb 16, 2012, at 12:01 PM, Seth Hall wrote: > Ah! You're right. I'll fix it. Not sure what my problem is today, I don't seem to be able to remove the bro-dev list when I send mail to the tracker! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Thu Feb 16 09:27:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 17:27:38 -0000 Subject: [Bro-Dev] #770: topic/v6-addr In-Reply-To: <048.44b3da34753272b3bb7e6a731b786048@tracker.bro-ids.org> References: <048.44b3da34753272b3bb7e6a731b786048@tracker.bro-ids.org> Message-ID: <063.d88d3e8616b549a18224bc73333bc3db@tracker.bro-ids.org> #770: topic/v6-addr ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by jsiwek): In [93fa1167381f3b450c05426867577ffbe47eb32b/bro]: {{{ #!CommitTicketReference repository="bro" revision="93fa1167381f3b450c05426867577ffbe47eb32b" Various tweaks/refactor of new IPAddr class usages or IPv6 related code. - non-binpac DNS analyzer now also generates dns_a6_reply event - ExpectedConn class refactored to use IPAddr's - BinaryExpr::AddrFold simplified - IP_Hdr src/dst address accessor methods changed to construct IPAddr objects on the fly from ip4/ip6 members. Addresses #770. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 16 09:28:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 17:28:48 -0000 Subject: [Bro-Dev] #770: topic/v6-addr In-Reply-To: <048.44b3da34753272b3bb7e6a731b786048@tracker.bro-ids.org> References: <048.44b3da34753272b3bb7e6a731b786048@tracker.bro-ids.org> Message-ID: <063.6da1e7062aced245f673c476d0bc67d9@tracker.bro-ids.org> #770: topic/v6-addr ----------------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * type: Task => Merge Request Old description: > This branch enables IPv6 support in the default Bro configuration. > > Notable changes are in commits: > > [b3f1f45082ef8ad0e13574f0c0a2702769cf820f/bro] > [eca32610779dc6c707fb734e49cc0be379c6cfd5/bro] > [4cb6a279f58b775ab464c7a7539a3d1c5896d86a/bro] > [31565d6987e3bd15f2e4fbd3f1365d29deebd9eb/bro] > [f945f3c5183dcc9fd3a0f6030e2522692123c01e/bro] > [086f747bc14a9fbfdbb68e645287a9303b42c4fe/bro] > [0f207c243c9e4ebfcca227b71add3ac2e8af1cc4/bro] > > External baseline tests have not been updated anywhere. I have looked > them over and didn't think there were any problems, but it would be good > to have someone else sanity check. New description: This branch enables IPv6 support in the default Bro configuration. Notable changes are in commits: [b3f1f45082ef8ad0e13574f0c0a2702769cf820f/bro] [4cb6a279f58b775ab464c7a7539a3d1c5896d86a/bro] [31565d6987e3bd15f2e4fbd3f1365d29deebd9eb/bro] [f945f3c5183dcc9fd3a0f6030e2522692123c01e/bro] [086f747bc14a9fbfdbb68e645287a9303b42c4fe/bro] [0f207c243c9e4ebfcca227b71add3ac2e8af1cc4/bro] External baseline tests have not been updated anywhere. I have looked them over and didn't think there were any problems, but it would be good to have someone else sanity check. -- -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 16 09:31:11 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 16 Feb 2012 17:31:11 -0000 Subject: [Bro-Dev] #448: Broccoli does not support IPv6 addresses In-Reply-To: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> References: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> Message-ID: <062.fcefd3ceb7ca6462a6cd69103aa48874@tracker.bro-ids.org> #448: Broccoli does not support IPv6 addresses -----------------------+---------------------- Reporter: robin | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: Resolution: | Keywords: ipv6 -----------------------+---------------------- Changes (by jsiwek): * owner: kreibich => jsiwek * status: reopened => assigned -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Thu Feb 16 12:10:39 2012 From: seth at icir.org (Seth Hall) Date: Thu, 16 Feb 2012 15:10:39 -0500 Subject: [Bro-Dev] tunnel decapsulators? Message-ID: <2CB49470-D736-4DC0-8126-EE8B4C7548E2@icir.org> Does anyone have it on their todo list to work on the IPv6 tunnel decapsulators? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From slagell at illinois.edu Thu Feb 16 12:23:08 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Thu, 16 Feb 2012 20:23:08 +0000 Subject: [Bro-Dev] tunnel decapsulators? In-Reply-To: <2CB49470-D736-4DC0-8126-EE8B4C7548E2@icir.org> References: <2CB49470-D736-4DC0-8126-EE8B4C7548E2@icir.org> Message-ID: It is on the horizon for us, but no one specifically is working on it yet. On Feb 16, 2012, at 2:10 PM, Seth Hall wrote: > Does anyone have it on their todo list to work on the IPv6 tunnel decapsulators? > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > ------ Adam J. Slagell, CISO, CISSP Chief Information Security Officer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info 217.244.8965 "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From gregor at majordomus.org Thu Feb 16 12:49:18 2012 From: gregor at majordomus.org (Gregor Maier) Date: Thu, 16 Feb 2012 12:49:18 -0800 Subject: [Bro-Dev] tunnel decapsulators? In-Reply-To: <2CB49470-D736-4DC0-8126-EE8B4C7548E2@icir.org> References: <2CB49470-D736-4DC0-8126-EE8B4C7548E2@icir.org> Message-ID: <4F3D6BCE.5060709@majordomus.org> On 2/16/12 12:10 , Seth Hall wrote: > Does anyone have it on their todo list to work on the IPv6 tunnel decapsulators? see topic/gregor/tunnel I was planning to port it to the current master but just didn't have the time yet. If somebody wants to give it a shot go ahead. Should be fairly straight-forward to adopt to the current v6 addresses. BTW, It's branched off from master around Jul / Aug. cu gregor From gregor at icir.org Thu Feb 16 12:50:07 2012 From: gregor at icir.org (Gregor Maier) Date: Thu, 16 Feb 2012 12:50:07 -0800 Subject: [Bro-Dev] tunnel decapsulators? In-Reply-To: <2CB49470-D736-4DC0-8126-EE8B4C7548E2@icir.org> References: <2CB49470-D736-4DC0-8126-EE8B4C7548E2@icir.org> Message-ID: <4F3D6BFF.1090802@icir.org> On 2/16/12 12:10 , Seth Hall wrote: > Does anyone have it on their todo list to work on the IPv6 tunnel decapsulators? see topic/gregor/tunnel I was planning to port it to the current master but just didn't have the time yet. If somebody wants to give it a shot go ahead. Should be fairly straight-forward to adopt to the current v6 addresses. BTW, It's branched off from master around Jul / Aug. cu gregor From seth at icir.org Thu Feb 16 13:16:03 2012 From: seth at icir.org (Seth Hall) Date: Thu, 16 Feb 2012 16:16:03 -0500 Subject: [Bro-Dev] tunnel decapsulators? In-Reply-To: <4F3D6BCE.5060709@majordomus.org> References: <2CB49470-D736-4DC0-8126-EE8B4C7548E2@icir.org> <4F3D6BCE.5060709@majordomus.org> Message-ID: <54E5E954-C8F1-4DEC-844D-6F89DAB746A9@icir.org> On Feb 16, 2012, at 3:49 PM, Gregor Maier wrote: > I was planning to port it to the current master but just didn't have the time yet. If somebody wants to give it a shot go ahead. Should be fairly straight-forward to adopt to the current v6 addresses. Exactly why I was asking. I'll email if I start working on it, but I wouldn't complain if someone else started on it. :) Thanks, .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Thu Feb 16 14:40:05 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 16 Feb 2012 14:40:05 -0800 Subject: [Bro-Dev] tunnel decapsulators? In-Reply-To: <4F3D6BFF.1090802@icir.org> References: <2CB49470-D736-4DC0-8126-EE8B4C7548E2@icir.org> <4F3D6BFF.1090802@icir.org> Message-ID: <20120216224005.GC37681@icir.org> On Thu, Feb 16, 2012 at 12:50 -0800, you wrote: > see topic/gregor/tunnel I havne't actually following that too closely. Can you briefly summarize what the code is doing? We also might need to do a bit more brainstorming on how to generally handle tunnels. It's not really clear yet to me at least. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From gregor at icir.org Thu Feb 16 15:30:36 2012 From: gregor at icir.org (Gregor Maier) Date: Thu, 16 Feb 2012 15:30:36 -0800 Subject: [Bro-Dev] tunnel decapsulators? In-Reply-To: <20120216224005.GC37681@icir.org> References: <2CB49470-D736-4DC0-8126-EE8B4C7548E2@icir.org> <4F3D6BFF.1090802@icir.org> <20120216224005.GC37681@icir.org> Message-ID: <4F3D919C.5040101@icir.org> On 2/16/12 14:40 , Robin Sommer wrote: > On Thu, Feb 16, 2012 at 12:50 -0800, you wrote: > >> see topic/gregor/tunnel > > I havne't actually following that too closely. Can you briefly > summarize what the code is doing? from my tunnel.bro script ;-) ##! Handle tunneled connections. ##! ##! Bro can decapsulate IPinIP and IPinUDP tunnels, were "IP" can be either ##! IPv4 or IPv6. The most common case will be decapsulating Teredo, 6to4, ##! 6in4, and AYIYA. When this script is loaded, decapsulation will be ##! enabled. "tunnel.log" will log the "parent" for each tunneled ##! connection. The identity (and existence) of the tunnel connection ##! is otherwise lost. ##! ##! Currently handles: ##! ##! * IP6 in IP{4,6}. (IP4 in IP is easy to add, but omitted due to lack ##! of test cases. ##! * IP{4,6} in UDP. This decapsulates e.g., standard *Teredo* packets ##! (without authentication or origin indicator) ##! * IP{4,6} in AYIYA ##! * Only checks for UDP tunnels on Teredo's and AYIYA's default ##! ports. See :bro:id:`udp_tunnel_ports` and ##! :bro:id:`udp_tunnel_allports` ##! ##! Decapsulation happens early in a packets processing, right after IP ##! defragmentation but before there is a connection context. The tunnel ##! headers are stripped from packet and the identity of the parent is ##! is stored as the ``tunnel_parent`` member of :bro:type:`connection`, ##! which is of type :bro:type:`Tunnel::Parent`. ##! ##! *Limitation:* The decapsulated packets are not fed through the ##! defragmenter again and decapsulation happens only on the primary ##! path, i.e., it's not available for the secondary path. ##! ##! Tunnel decapsulation can be enabled/disabled for IP and UDP separately. For UDP you can specify the set of UDP ports that should be considered. For UDP tunnels without glue headers (i.e., the inner IP header starts right after the UDP header) we probabilistically check if the UDP payload is indeed a valid IP header. This is similar to the old UDP tunnel code. > We also might need to do a bit more brainstorming on how to generally > handle tunnels. It's not really clear yet to me at least. ACK. My code is simple and works but definitely has limitations. The right thing TODO would be to make the IP handling in Bro part of the analyzer tree. Then tunnel decapsulation can just be an analyzer that feeds the decapsulated packets back into IP processing. cu Gregor From seth at icir.org Thu Feb 16 18:26:50 2012 From: seth at icir.org (Seth Hall) Date: Thu, 16 Feb 2012 21:26:50 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/dnthayer/ftp-ipv6: Add a test for FTP over IPv6 (278704f) In-Reply-To: <201202162126.q1GLQpbx031403@bro-ids.icir.org> References: <201202162126.q1GLQpbx031403@bro-ids.icir.org> Message-ID: On Feb 16, 2012, at 4:26 PM, Daniel Thayer wrote: > +1329327787.396984 UWkUyAuUGXf 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49185 2001:470:4867:99::21 21 anonymous test RETR ftp://2001:470:4867:99::21/robots.txt - - 77 226 Transfer complete. - - Could you make sure that the code that generates URLs for FTP sessions puts brackets around the address for ipv6 addresses? We need to do the same thing for HTTP too. The URL should look like this: ftp://[2001:470:4867:99::21]/robots.txt Thanks! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Thu Feb 16 21:00:02 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 16 Feb 2012 21:00:02 -0800 Subject: [Bro-Dev] new IPv6 code In-Reply-To: <20120215224253.GB47348@icir.org> References: <20120210092412.GA18051@icir.org> <20120213051547.GA50682@icir.org> <99B32B80-4E64-426C-95B7-CD1783650CFB@icir.org> <20120213144954.GA61942@icir.org> <20120214035645.GB78396@icir.org> <20120214191955.GA1162@icir.org> <20120215210915.GZ37020@icir.org> <20120215224253.GB47348@icir.org> Message-ID: <20120217050002.GA49977@icir.org> Thanks for working on my previous points. I've made a pass over the code and stream-lined a few things further, see the recent commit message into my new branch. I didn't get to finishing it anymore, there's still some bug in the new serialization methods (I think). I have one more request: I'd like to get rid of as many GetBytes()/CopyIPv6() calls as possible. Using those seems generally error-prone and I think we should avoid that by moving code into IPAddr.* where we can can. I think some of their uses is just an artifact of how the old code was structured. Mind going through and see what can be done differently? I've done a bit along these lines for the serialization code already, but another example is the various hashing code. Can we add methods to IPAddr to call from the hashing code so that it doesn't need to care about the internal representation? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Fri Feb 17 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 17 Feb 2012 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202170800.q1H803rG002603@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 770 [1] | jsiwek | jsiwek | Normal | topic/v6-addr [2] Bro | 771 [3] | jsiwek | | Normal | topic/jsiwek/v6-dns-name-lookup [4] pysubnettree | 750 [5] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #770: http://tracker.bro-ids.org/bro/ticket/770 [2] v6-addr: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/v6-addr [3] #771: http://tracker.bro-ids.org/bro/ticket/771 [4] v6-dns-name-lookup: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/v6-dns-name-lookup [5] #750: http://tracker.bro-ids.org/bro/ticket/750 From seth at icir.org Fri Feb 17 09:00:28 2012 From: seth at icir.org (Seth Hall) Date: Fri, 17 Feb 2012 12:00:28 -0500 Subject: [Bro-Dev] broctl process tracking problems Message-ID: <7D56A480-82D3-44CF-9F6C-23F27E73A0D6@icir.org> Has anyone else ever had trouble with broctl getting confused about the status of a process? I just ran into it a little bit ago where broctl thought that all of my workers were dead when I tried to do a restart command. They failed when they were trying to start again because with the myricom sniffer drivers you can only sniff the interface once. We need to do some debugging on this, but it happens sporadically enough that it might be tough. I sort of wonder if there are issues with broctl.dat being written, I've run into problems in that file before where things wouldn't be written right. Would it make sense to maybe even move away from broctl.dat (which tracks cluster state) and toward something like an SQLite database? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Fri Feb 17 09:21:13 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 17 Feb 2012 17:21:13 -0000 Subject: [Bro-Dev] #767: Active copies of scripts not hidden In-Reply-To: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> References: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> Message-ID: <061.37d3e7b6f742b27e3572bd48c27bb6ee@tracker.bro-ids.org> #767: Active copies of scripts not hidden -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: accepted Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Changes (by dnthayer): * owner: => dnthayer * status: new => accepted -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 17 10:00:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 17 Feb 2012 18:00:56 -0000 Subject: [Bro-Dev] #767: Active copies of scripts not hidden In-Reply-To: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> References: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> Message-ID: <061.8fbfb5ee22d3a24a8fb2f20de65e402b@tracker.bro-ids.org> #767: Active copies of scripts not hidden -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: accepted Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by dnthayer): In [0c69c582f09c0d8b2b8040ad73a37886e09f291a/broctl]: {{{ #!CommitTicketReference repository="broctl" revision="0c69c582f09c0d8b2b8040ad73a37886e09f291a" Rename the spool/policy directory so it is less visible Addresses #767. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From asharma at lbl.gov Fri Feb 17 10:08:07 2012 From: asharma at lbl.gov (Aashish Sharma) Date: Fri, 17 Feb 2012 10:08:07 -0800 Subject: [Bro-Dev] broctl process tracking problems In-Reply-To: <7D56A480-82D3-44CF-9F6C-23F27E73A0D6@icir.org> References: <7D56A480-82D3-44CF-9F6C-23F27E73A0D6@icir.org> Message-ID: <20120217180806.GA29541@yaksha.lbl.gov> Yes. Incidently, I had same issue 3 days back. broctl analysis scan showed scan was disabled but cluster was still dropping host on scan. So I started looking at broctl status which said cluster not running while tail -f conn.log was growing. Ended up kill -s 9 on all bro worker nodes and restart with broctl. After which it has been fine. I was quite unusual. Aashish On Fri, Feb 17, 2012 at 12:00:28PM -0500, Seth Hall wrote: > Has anyone else ever had trouble with broctl getting confused about the status of a process? I just ran into it a little bit ago where broctl thought that all of my workers were dead when I tried to do a restart command. They failed when they were trying to start again because with the myricom sniffer drivers you can only sniff the interface once. > > We need to do some debugging on this, but it happens sporadically enough that it might be tough. I sort of wonder if there are issues with broctl.dat being written, I've run into problems in that file before where things wouldn't be written right. Would it make sense to maybe even move away from broctl.dat (which tracks cluster state) and toward something like an SQLite database? > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Information Technology Division Lawrence Berkeley National Laboratory http://www.lbl.gov/cyber/pgp-aashish.txt Office: (510)-495-2680 Cell: (510)-457-1525 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120217/1712c78a/attachment.bin From robin at icir.org Fri Feb 17 10:08:21 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 17 Feb 2012 10:08:21 -0800 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/v6-addr-merge: Fix IPAddr/IPPrefix serialization bugs. (all unit tests pass) (06e59e1) In-Reply-To: <201202171803.q1HI3xSO003472@bro-ids.icir.org> References: <201202171803.q1HI3xSO003472@bro-ids.icir.org> Message-ID: <20120217180820.GM63037@icir.org> On Fri, Feb 17, 2012 at 10:03 -0800, Jonathan Siwek wrote: > Fix IPAddr/IPPrefix serialization bugs. (all unit tests pass) Great, thanks! Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From baxterw3232 at gmail.com Fri Feb 17 10:23:35 2012 From: baxterw3232 at gmail.com (Will) Date: Fri, 17 Feb 2012 12:23:35 -0600 Subject: [Bro-Dev] broctl process tracking problems In-Reply-To: <20120217180806.GA29541@yaksha.lbl.gov> References: <7D56A480-82D3-44CF-9F6C-23F27E73A0D6@icir.org> <20120217180806.GA29541@yaksha.lbl.gov> Message-ID: I had the same issue a time or two. Running 'broctl ps.bro' right after 'broctl status' has become part of my new ritual before stopping/starting or just restarting any of my clusters. Will On Feb 17, 2012 12:08 PM, "Aashish Sharma" wrote: > Yes. Incidently, I had same issue 3 days back. broctl analysis scan > showed scan was disabled but cluster was still dropping host on scan. > > So I started looking at broctl status which said cluster not running > while tail -f conn.log was growing. > > Ended up kill -s 9 on all bro worker nodes and restart with broctl. > After which it has been fine. I was quite unusual. > > Aashish > > On Fri, Feb 17, 2012 at 12:00:28PM -0500, Seth Hall wrote: > > Has anyone else ever had trouble with broctl getting confused about the > status of a process? I just ran into it a little bit ago where broctl > thought that all of my workers were dead when I tried to do a restart > command. They failed when they were trying to start again because with the > myricom sniffer drivers you can only sniff the interface once. > > > > We need to do some debugging on this, but it happens sporadically enough > that it might be tough. I sort of wonder if there are issues with > broctl.dat being written, I've run into problems in that file before where > things wouldn't be written right. Would it make sense to maybe even move > away from broctl.dat (which tracks cluster state) and toward something like > an SQLite database? > > > > .Seth > > > > -- > > Seth Hall > > International Computer Science Institute > > (Bro) because everyone has a network > > http://www.bro-ids.org/ > > > > > > _______________________________________________ > > bro-dev mailing list > > bro-dev at bro-ids.org > > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > -- > Aashish Sharma (asharma at lbl.gov) > Cyber Security, Information Technology Division > Lawrence Berkeley National Laboratory > http://www.lbl.gov/cyber/pgp-aashish.txt > Office: (510)-495-2680 Cell: (510)-457-1525 > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120217/0e1f7d18/attachment.html From seth at icir.org Fri Feb 17 10:53:41 2012 From: seth at icir.org (Seth Hall) Date: Fri, 17 Feb 2012 13:53:41 -0500 Subject: [Bro-Dev] broctl process tracking problems In-Reply-To: References: <7D56A480-82D3-44CF-9F6C-23F27E73A0D6@icir.org> <20120217180806.GA29541@yaksha.lbl.gov> Message-ID: <0EF482AD-75B7-4522-A7A9-CFFDBF5B97EC@icir.org> On Feb 17, 2012, at 1:23 PM, Will wrote: > I had the same issue a time or two. Running 'broctl ps.bro' right after 'broctl status' has become part of my new ritual before stopping/starting or just restarting any of my clusters. That's actually perfect! Sometime could you make a copy of your spool/broctl.dat before you restart? I'd like two copies of that file. One before a restart and one after the restart (assuming the restart fails). It could point out if there is corruption entering the broctl.dat file at some point. This problem drives me crazy and I'd love to fix it. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From asharma at lbl.gov Fri Feb 17 11:02:23 2012 From: asharma at lbl.gov (Aashish Sharma) Date: Fri, 17 Feb 2012 11:02:23 -0800 Subject: [Bro-Dev] broctl process tracking problems In-Reply-To: <0EF482AD-75B7-4522-A7A9-CFFDBF5B97EC@icir.org> References: <7D56A480-82D3-44CF-9F6C-23F27E73A0D6@icir.org> <20120217180806.GA29541@yaksha.lbl.gov> <0EF482AD-75B7-4522-A7A9-CFFDBF5B97EC@icir.org> Message-ID: <20120217190222.GF29541@yaksha.lbl.gov> > > I had the same issue a time or two. Running 'broctl ps.bro' right after 'broctl status' has become part of my new ritual before stopping/starting or just restarting any of my clusters. Yes, but there could be other situations you can probably look for when the bro process is running: - Your network interfaces may not see the data - Another possibility is that data is coming on the interfaces and bro is running but not processing the incoming data (hopefully won't happen) so you might want to also check if the logs are growing regularly. Additional complexities: Your logs may be growing but above situation happens only on 1 worker-node ( in which case you'd see manager logs growing but won't know that one node is consistently missing logs) .... and so on for various other failures (I must say these are rare but on a production system you have to account for them and over period of time this has happened) We have a cron script which watches for these conditions. Aashish On Fri, Feb 17, 2012 at 01:53:41PM -0500, Seth Hall wrote: > > On Feb 17, 2012, at 1:23 PM, Will wrote: > > > I had the same issue a time or two. Running 'broctl ps.bro' right after 'broctl status' has become part of my new ritual before stopping/starting or just restarting any of my clusters. > > That's actually perfect! Sometime could you make a copy of your spool/broctl.dat before you restart? > > I'd like two copies of that file. One before a restart and one after the restart (assuming the restart fails). It could point out if there is corruption entering the broctl.dat file at some point. This problem drives me crazy and I'd love to fix it. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Information Technology Division Lawrence Berkeley National Laboratory http://www.lbl.gov/cyber/pgp-aashish.txt Office: (510)-495-2680 Cell: (510)-457-1525 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120217/1681b324/attachment-0001.bin From bro at tracker.bro-ids.org Fri Feb 17 13:23:43 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 17 Feb 2012 21:23:43 -0000 Subject: [Bro-Dev] #775: IP address parsing more strict with new IPv6 code Message-ID: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> #775: IP address parsing more strict with new IPv6 code ---------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Keywords: ipv6 | ---------------------+------------------------ Given this scripts: {{{ print to_addr("5.01.04.05"); print to_addr("5.1.4.5"); }}} 2.0: {{{ bro ./a.bro 5.1.4.5 5.1.4.5 }}} topic/v6-addr (and soon master): {{{ error: Bad IP address: 5.01.04.05 :: 5.1.4.5 }}} Not sure how important this is but it triggers a test-suite difference, and generally be liberal in accepting input is preferable.. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 17 15:05:41 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 17 Feb 2012 23:05:41 -0000 Subject: [Bro-Dev] #776: DNS not logging some replies on trace Message-ID: <047.c145c45ba409da87364da0d7029b2420@tracker.bro-ids.org> #776: DNS not logging some replies on trace ---------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ I'm attaching a DNS session (extracted from the test suite) for which Bro does not log all the replies. dns.log looks like this: {{{ 1258563890.835277 JFmrS5rE7re 192.168.1.103 51228 192.168.1.1 53 udp 55939 h.zedo.com 1 C_INTERNET 1 A 0 NOERROR F F F T T 0 63.211.147.11 7200.000000 }}} However, when running the test suite on the full trace, it logs them all: {{{ 1258563890.835277 LEDZLphhTIg 192.168.1.103 51228 192.168.1.1 53 udp 55939 h.zedo.com 1 C_INTERNET 1 A 0 NOERROR F F F T T 0 63.211.147.11 7200.000000 pdns4.ultradns.org,pdns1.ultradns.net,pdns5.ultradns.info,pdns2.ultradns.net,pdns3.ultradns.org,pdns6.ultradns.co.uk 199.7.69.1,204.74.114.1,204.74.115.1,199.7.68.1,2001:502:4612::1 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 17 15:20:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 17 Feb 2012 23:20:06 -0000 Subject: [Bro-Dev] #776: DNS not logging replies on trace (was: DNS not logging some replies on trace) In-Reply-To: <047.c145c45ba409da87364da0d7029b2420@tracker.bro-ids.org> References: <047.c145c45ba409da87364da0d7029b2420@tracker.bro-ids.org> Message-ID: <062.bac3be47af54bf9f70bb5badadf4fb7f@tracker.bro-ids.org> #776: DNS not logging replies on trace ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Description changed by robin: Old description: > I'm attaching a DNS session (extracted from the test suite) for which Bro > does not log all the replies. dns.log looks like this: > > {{{ > 1258563890.835277 JFmrS5rE7re 192.168.1.103 51228 > 192.168.1.1 53 udp 55939 h.zedo.com 1 > C_INTERNET 1 A 0 NOERROR F F F > T T 0 63.211.147.11 7200.000000 > }}} > > However, when running the test suite on the full trace, it logs them all: > > {{{ > 1258563890.835277 LEDZLphhTIg 192.168.1.103 51228 > 192.168.1.1 53 udp 55939 h.zedo.com 1 > C_INTERNET 1 A 0 NOERROR F F F > T T 0 63.211.147.11 7200.000000 > pdns4.ultradns.org,pdns1.ultradns.net,pdns5.ultradns.info,pdns2.ultradns.net,pdns3.ultradns.org,pdns6.ultradns.co.uk > 199.7.69.1,204.74.114.1,204.74.115.1,199.7.68.1,2001:502:4612::1 > }}} New description: I'm attaching a DNS session (extracted from the test suite) for which Bro does not log the replies. dns.log looks like this: {{{ 1258563890.835277 JFmrS5rE7re 192.168.1.103 51228 192.168.1.1 53 udp 55939 h.zedo.com 1 C_INTERNET 1 A 0 NOERROR F F F T T 0 63.211.147.11 7200.000000 }}} However, when running the test suite on the full trace, it logs them all: {{{ 1258563890.835277 LEDZLphhTIg 192.168.1.103 51228 192.168.1.1 53 udp 55939 h.zedo.com 1 C_INTERNET 1 A 0 NOERROR F F F T T 0 63.211.147.11 7200.000000 pdns4.ultradns.org,pdns1.ultradns.net,pdns5.ultradns.info,pdns2.ultradns.net,pdns3.ultradns.org,pdns6.ultradns.co.uk 199.7.69.1,204.74.114.1,204.74.115.1,199.7.68.1,2001:502:4612::1 }}} -- -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Feb 18 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 18 Feb 2012 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202180800.q1I802OI016922@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 770 [1] | jsiwek | jsiwek | Normal | topic/v6-addr [2] Bro | 771 [3] | jsiwek | | Normal | topic/jsiwek/v6-dns-name-lookup [4] pysubnettree | 750 [5] | robin | robin | Normal | Patch adding IPv6 support for pysubnettree [1] #770: http://tracker.bro-ids.org/bro/ticket/770 [2] v6-addr: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/v6-addr [3] #771: http://tracker.bro-ids.org/bro/ticket/771 [4] v6-dns-name-lookup: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/v6-dns-name-lookup [5] #750: http://tracker.bro-ids.org/bro/ticket/750 From bro at tracker.bro-ids.org Sat Feb 18 14:35:43 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 18 Feb 2012 22:35:43 -0000 Subject: [Bro-Dev] #750: Patch adding IPv6 support for pysubnettree In-Reply-To: <047.e6d50f498ca5a5aca3d4d44bb09a0d18@tracker.bro-ids.org> References: <047.e6d50f498ca5a5aca3d4d44bb09a0d18@tracker.bro-ids.org> Message-ID: <062.5c6ac0c4112707139a2215ce4c45a96d@tracker.bro-ids.org> #750: Patch adding IPv6 support for pysubnettree ---------------------------+---------------------- Reporter: robin | Owner: Type: Patch | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: pysubnettree | Version: Resolution: | Keywords: ipv6 ---------------------------+---------------------- Changes (by robin): * owner: robin => * type: Merge Request => Patch -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Sat Feb 18 14:36:30 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 18 Feb 2012 22:36:30 -0000 Subject: [Bro-Dev] #770: topic/v6-addr In-Reply-To: <048.44b3da34753272b3bb7e6a731b786048@tracker.bro-ids.org> References: <048.44b3da34753272b3bb7e6a731b786048@tracker.bro-ids.org> Message-ID: <063.234413c3657b8f88be81a0eb82ce6127@tracker.bro-ids.org> #770: topic/v6-addr -----------------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: assigned => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Sat Feb 18 14:36:41 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 18 Feb 2012 22:36:41 -0000 Subject: [Bro-Dev] #771: topic/jsiwek/v6-dns-name-lookup In-Reply-To: <048.b7ae8177f147dea2c149ee24034bfffc@tracker.bro-ids.org> References: <048.b7ae8177f147dea2c149ee24034bfffc@tracker.bro-ids.org> Message-ID: <063.e2d319aaf2d7f24e335ca6a8f96fc30c@tracker.bro-ids.org> #771: topic/jsiwek/v6-dns-name-lookup -----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Sat Feb 18 19:40:00 2012 From: seth at icir.org (Seth Hall) Date: Sat, 18 Feb 2012 22:40:00 -0500 Subject: [Bro-Dev] New IPv6 code on live cluster Message-ID: <0D1BCBA9-2CD6-4342-9954-C5FF1F2F3B71@icir.org> The new IPv6 code that seems to mostly be merged into master is running in production now (on a large cluster) and it seems to be fine as far as I can tell. The only thing I needed was one final merge from topic/robin/v6-addr-merge because there are a few changes there that are very much needed. I just wanted to point out that the code is in live testing now and should be ready to wider testing for those interested once the next merge from Robin's branch happens. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Sun Feb 19 08:38:33 2012 From: robin at icir.org (Robin Sommer) Date: Sun, 19 Feb 2012 08:38:33 -0800 Subject: [Bro-Dev] New IPv6 code on live cluster In-Reply-To: <0D1BCBA9-2CD6-4342-9954-C5FF1F2F3B71@icir.org> References: <0D1BCBA9-2CD6-4342-9954-C5FF1F2F3B71@icir.org> Message-ID: <20120219163832.GA68930@icir.org> On Sat, Feb 18, 2012 at 22:40 -0500, you wrote: > as far as I can tell. The only thing I needed was one final merge > from topic/robin/v6-addr-merge because there are a few changes there > that are very much needed. That was an oversight, it's now all merged into master and ready for testing. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Sun Feb 19 08:57:17 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sun, 19 Feb 2012 16:57:17 -0000 Subject: [Bro-Dev] #777: Leak in DNS manager Message-ID: <047.4c03b3759c6980611a3da450aecd9407@tracker.bro-ids.org> #777: Leak in DNS manager ---------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ core.leaks.dns reports a leak: {{{ Total: 4 objects 2 50.0% 50.0% 4 100.0% DNS_Mgr::AddResult DNS_Mgr.cc:706 2 50.0% 100.0% 2 50.0% copy_string util.cc:77 0 0.0% 100.0% 4 100.0% DNS_Mgr::Process DNS_Mgr.cc:1169 0 0.0% 100.0% 2 50.0% DNS_Mapping::DNS_Mapping DNS_Mgr.cc:171 0 0.0% 100.0% 4 100.0% __libc_start_main ??:0 0 0.0% 100.0% 4 100.0% main main.cc:1022 0 0.0% 100.0% 4 100.0% DNS_Mgr::DoProcess DNS_Mgr.cc:1217 0 0.0% 100.0% 4 100.0% net_run Net.cc:444 0 0.0% 100.0% 4 100.0% _start ??:0 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Sun Feb 19 19:51:43 2012 From: seth at icir.org (Seth Hall) Date: Sun, 19 Feb 2012 22:51:43 -0500 Subject: [Bro-Dev] IPv6 with Bro Message-ID: <67A4D810-6F87-4D15-BDDF-3749A4665E15@icir.org> I was looking through logs looking for evidence of IPv6 traffic and I found a host that appears to have run traceroute6 against the network so I thought I'd show an example of how traceroutes look over IPv6 in the Bro conn.log? #ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes #time string addr port addr port enum string interval count count string bool count string count count count count 1329660378.154848 Jfusxx0UPU 2001:470:9:babe::3 37990 33460 udp - - - - S0 F 0 D 1 80 0 0 1329660378.158040 brhRmsVxFb9 2001:470:9:babe::3 57120 33463 udp - - - - S0 F 0 D 1 80 0 0 1329660378.172236 YMi0UUbrhie 2001:470:9:babe::3 44529 33468 udp - - - - S0 F 0 D 1 80 0 0 1329660378.157556 4FdCveTHPR1 2001:470:9:babe::3 51427 33462 udp - - - - S0 F 0 D 1 80 0 0 1329660378.172791 iwezQsY3bK8 2001:470:9:babe::3 39695 33467 udp - - - - S0 F 0 D 1 80 0 0 1329660378.174321 OQdvzk0kKy8 2001:470:9:babe::3 51260 33469 udp - - - - S0 F 0 D 1 80 0 0 1329660378.155912 yqa4zrwi3x 2001:470:9:babe::3 39514 33461 udp - - - - S0 F 0 D 1 80 0 0 1329660378.152268 4yVVIrTprM3 2001:470:9:babe::3 45626 33458 udp - - - - S0 F 0 D 1 80 0 0 1329660378.154758 iMSZfgKHYbk 2001:470:9:babe::3 35970 33459 udp - - - - S0 F 0 D 1 80 0 0 1329660378.169734 3hRVQFTFaKl 2001:470:9:babe::3 58529 33464 udp - - - - S0 F 0 D 1 80 0 0 1329660378.170493 pDVhGxwtIX8 2001:470:9:babe::3 43646 33465 udp - - - - S0 F 0 D 1 80 0 0 1329660378.171148 lGOET1T2DZa 2001:470:9:babe::3 53249 33466 udp - - - - S0 F 0 D 1 80 0 0 1329660453.213549 groeGNpiJoe 2001:470:9:babe::3 42299 33476 udp - - - - S0 F 0 D 1 80 0 0 1329660453.212178 laZDPNSTdc8 2001:470:9:babe::3 37647 33471 udp - - - - S0 F 0 D 1 80 0 0 1329660453.216245 XYBwZnFA7u7 2001:470:9:babe::3 39810 33480 udp - - - - S0 F 0 D 1 80 0 0 1329660453.215838 gFNZL587Byi 2001:470:9:babe::3 42812 33481 udp - - - - S0 F 0 D 1 80 0 0 1329660453.217636 VW7azawZ3jg 2001:470:9:babe::3 56608 33485 udp - - - - S0 F 0 D 1 80 0 0 1329660453.217262 nMEvDi845rf 2001:470:9:babe::3 34063 33484 udp - - - - S0 F 0 D 1 80 0 0 1329660453.213996 c7BGJ9pcyqe 2001:470:9:babe::3 36350 33477 udp - - - - S0 F 0 D 1 80 0 0 1329660453.214985 EWUPH0p7Chk 2001:470:9:babe::3 49498 33479 udp - - - - S0 F 0 D 1 80 0 0 1329660453.213572 MtY4WlA0glb 2001:470:9:babe::3 56689 33473 udp - - - - S0 F 0 D 1 80 0 0 1329660453.218169 MoXEoERSMt1 2001:470:9:babe::3 33510 33483 udp - - - - S0 F 0 D 1 80 0 0 1329660453.213684 8B71hDNHG09 2001:470:9:babe::3 46544 33475 udp - - - - S0 F 0 D 1 80 0 0 1329660453.213145 HbnJHMRmCCi 2001:470:9:babe::3 52538 33472 udp - - - - S0 F 0 D 1 80 0 0 1329660453.217201 Dnu86Fujgrd 2001:470:9:babe::3 37475 33482 udp - - - - S0 F 0 D 1 80 0 0 1329660453.210887 wIRlKplOj28 2001:470:9:babe::3 41090 33470 udp - - - - S0 F 0 D 1 80 0 0 1329660453.214113 9ElDnNEyE5c 2001:470:9:babe::3 56094 33474 udp - - - - S0 F 0 D 1 80 0 0 1329660453.215381 D6jSe4y1E0h 2001:470:9:babe::3 39121 33478 udp - - - - S0 F 0 D 1 80 0 0 1329660528.253395 OJtSePR8BU2 2001:470:9:babe::3 33845 33492 udp - - - - S0 F 0 D 1 80 0 0 1329660528.251139 njFLGFwddG 2001:470:9:babe::3 44546 33487 udp - - - - S0 F 0 D 1 80 0 0 1329660528.253073 0k8dJwiYv0b 2001:470:9:babe::3 52688 33490 udp - - - - S0 F 0 D 1 80 0 0 1329660528.251681 TYP32iKDjm3 2001:470:9:babe::3 35294 33489 udp - - - - S0 F 0 D 1 80 0 0 1329660528.251192 342AxR3jVee 2001:470:9:babe::3 40207 33488 udp - - - - S0 F 0 D 1 80 0 0 1329660528.252991 3o6WUxcV6qj 2001:470:9:babe::3 53110 33493 udp - - - - S0 F 0 D 1 80 0 0 1329660528.253406 G2tidXRpZob 2001:470:9:babe::3 60440 33491 udp - - - - S0 F 0 D 1 80 0 0 1329660528.249881 OLqgFalrBAg 2001:470:9:babe::3 60518 33486 udp - - - - S0 F 0 D 1 80 0 0 From bro at tracker.bro-ids.org Mon Feb 20 07:56:08 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 20 Feb 2012 15:56:08 -0000 Subject: [Bro-Dev] #776: DNS not logging replies on trace In-Reply-To: <047.c145c45ba409da87364da0d7029b2420@tracker.bro-ids.org> References: <047.c145c45ba409da87364da0d7029b2420@tracker.bro-ids.org> Message-ID: <062.44dfb355438a89ade2c6a101c20ce711@tracker.bro-ids.org> #776: DNS not logging replies on trace ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): I think by default it just logs the answer section, which is the first result you show. And the second one logs the authority and additional sections of the reply because the test suite is loading the `policy/protocols/dns/auth-addl.bro` script. Here's the result I get when it's loaded for the single DNS session that you extracted: {{{ $ bro -r 2009-M57-day11-18.trace.gz.LEDZLphhTIg protocols/dns/auth- addl.bro $ tail -n1 dns.log 1258563890.835277 n9yOrUVn8g1 192.168.1.103 51228 192.168.1.1 53 udp 55939 h.zedo.com 1 C_INTERNET 1 A NOERROR F F F T T 0 63.211.147.11 7200.000000 pdns4.ultradns.org,pdns5.ultradns.info,pdns6.ultradns.co.uk,pdns3.ultradns.org,pdns1.ultradns.net,pdns2.ultradns.net 2001:502:4612::1,204.74.115.1,199.7.69.1,199.7.68.1,204.74.114.1 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 20 08:58:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 20 Feb 2012 16:58:38 -0000 Subject: [Bro-Dev] #776: DNS not logging replies on trace In-Reply-To: <047.c145c45ba409da87364da0d7029b2420@tracker.bro-ids.org> References: <047.c145c45ba409da87364da0d7029b2420@tracker.bro-ids.org> Message-ID: <062.52a5d4b966adc9e14aa0eb4b78f7b6a2@tracker.bro-ids.org> #776: DNS not logging replies on trace -----------------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied Comment: Makes sense, closing ticket. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Mon Feb 20 09:16:13 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 20 Feb 2012 09:16:13 -0800 Subject: [Bro-Dev] broctl process tracking problems In-Reply-To: <20120217190222.GF29541@yaksha.lbl.gov> References: <7D56A480-82D3-44CF-9F6C-23F27E73A0D6@icir.org> <20120217180806.GA29541@yaksha.lbl.gov> <0EF482AD-75B7-4522-A7A9-CFFDBF5B97EC@icir.org> <20120217190222.GF29541@yaksha.lbl.gov> Message-ID: <20120220171613.GE30899@icir.org> For the record, coincidentally I saw the same once early last week. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Mon Feb 20 09:57:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 20 Feb 2012 17:57:00 -0000 Subject: [Bro-Dev] #777: Leak in DNS manager In-Reply-To: <047.4c03b3759c6980611a3da450aecd9407@tracker.bro-ids.org> References: <047.4c03b3759c6980611a3da450aecd9407@tracker.bro-ids.org> Message-ID: <062.859b2a85606d883fa5da5623a62fb48c@tracker.bro-ids.org> #777: Leak in DNS manager ----------------------+------------------------ Reporter: robin | Owner: jsiwek Type: Problem | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------+------------------------ Changes (by jsiwek): * owner: => jsiwek * status: new => closed * resolution: => fixed Comment: In [1f7bfbb83c6b320d52f4ff91b6fdc426cb03b59a/bro]: {{{ #!CommitTicketReference repository="bro" revision="1f7bfbb83c6b320d52f4ff91b6fdc426cb03b59a" Fix memory leak in DNS manager (fixes #777). }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 20 12:35:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 20 Feb 2012 20:35:02 -0000 Subject: [Bro-Dev] #775: IP address parsing more strict with new IPv6 code In-Reply-To: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> References: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> Message-ID: <062.c0a4b445abb5327c034ee655adc5ca11@tracker.bro-ids.org> #775: IP address parsing more strict with new IPv6 code ----------------------+------------------------ Reporter: robin | Owner: jsiwek Type: Problem | Status: closed Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: fixed | Keywords: ipv6 ----------------------+------------------------ Changes (by jsiwek): * owner: => jsiwek * status: new => closed * resolution: => fixed Comment: In [b66b74e5dc0ce96bb1e58cfec591e1a41240d5e0/bro]: {{{ #!CommitTicketReference repository="bro" revision="b66b74e5dc0ce96bb1e58cfec591e1a41240d5e0" Decrease strictness of parsing IPv4 strings into addrs. (fixes #775) IPv4 strings in dotted-decimal format with decimal parts containing leading zeroes now parse better. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Feb 21 00:00:08 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 21 Feb 2012 00:00:08 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202210800.q1L808Jr023493@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | b66b74e | Jon Siwek | 2012-02-20 | Decrease strictness of parsing IPv4 strings into addrs. (fixes #775) [1] bro | 1f7bfbb | Jon Siwek | 2012-02-20 | Fix memory leak in DNS manager (fixes #777). [2] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/b66b74e5dc0ce96bb1e58cfec591e1a41240d5e0/bro [2] fastpath: http://tracker.bro-ids.org/bro/changeset/1f7bfbb83c6b320d52f4ff91b6fdc426cb03b59a/bro From robin at icir.org Tue Feb 21 07:18:06 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 21 Feb 2012 07:18:06 -0800 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/bernhard/postgresql: PostgreSQL writer for Logging framework. (5b805ae) In-Reply-To: <201202210838.q1L8cqJL026669@bro-ids.icir.org> References: <201202210838.q1L8cqJL026669@bro-ids.icir.org> Message-ID: <20120221151806.GC73852@icir.org> On Tue, Feb 21, 2012 at 00:38 -0800, Bernhard Amann wrote: > PostgreSQL writer for Logging framework. Woo-hoo! Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Tue Feb 21 09:52:22 2012 From: seth at icir.org (Seth Hall) Date: Tue, 21 Feb 2012 12:52:22 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/bernhard/postgresql: PostgreSQL writer for Logging framework. (5b805ae) In-Reply-To: <201202210838.q1L8cqJL026669@bro-ids.icir.org> References: <201202210838.q1L8cqJL026669@bro-ids.icir.org> Message-ID: On Feb 21, 2012, at 3:38 AM, Bernhard Amann wrote: > PostgreSQL writer for Logging framework. I would like to reiterate what Robin said, woohoo! Now it's time for me to install postgresql and start playing. .Seth From bro at tracker.bro-ids.org Tue Feb 21 09:55:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 21 Feb 2012 17:55:34 -0000 Subject: [Bro-Dev] #778: topic/dnthayer/ftp-ipv6 Message-ID: <050.50e0df68adc54ee6d3e3d9db13052fff@tracker.bro-ids.org> #778: topic/dnthayer/ftp-ipv6 ---------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch fixes the code that handles the FTP EPSV and EPRT commands (previously, there would be error messages in reporter.log because IPv6 addresses were not being extracted correctly). Also, square brackets are now added around IPv6 addresses in FTP URLs that appear in ftp.log. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Feb 21 12:25:42 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 21 Feb 2012 20:25:42 -0000 Subject: [Bro-Dev] #779: missing values cause bro to crash when used inside of a 'when' statement. Message-ID: <048.d7eccddacf988dcfe011e1dac3c48b28@tracker.bro-ids.org> #779: missing values cause bro to crash when used inside of a 'when' statement. ------------------------+--------------------------------------- Reporter: justin | Type: Problem Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: when InterpreterException ------------------------+--------------------------------------- Simplest test case: {{{ event bro_init() { local loc: geo_location; when (local hostname = lookup_addr(127.0.0.1)){ print "Location", loc$country_code; print "ok"; terminate(); } } }}} gives: {{{ terminate called after throwing an instance of 'InterpreterException' }}} outside of the when block, reporter.log would get: {{{ Reporter::ERROR field value missing [loc$country_code] }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Feb 21 12:34:54 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 21 Feb 2012 20:34:54 -0000 Subject: [Bro-Dev] #780: file extraction trunctation. Message-ID: <048.841e7736f715e5a45f63422aac975ed4@tracker.bro-ids.org> #780: file extraction trunctation. ------------------------+--------------------------------------- Reporter: justin | Type: Problem Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: file extraction truncated ------------------------+--------------------------------------- Tracked this down a while ago but forgot to submit it. I was running into an issue with the http file extraction. The files were being extracted properly, but then the files were being truncated to zero length. The reason turned out to be that the file object was being serialized, and then when deserialized, it was being re-opened and truncated. This also leaks file handles. putting a breakpoint on Open found this: {{{ #0 BroFile::Open (this=0x39108a0, file=0x0) at /spare/tmp/workshop/bro/src/File.cc:176 #1 0x00000000005939b3 in BroFile::Unserialize (info=0x7fff08c75170) at /spare/tmp/workshop/bro/src/File.cc:853 #2 0x00000000006602cb in Val::DoUnserialize (this=0x3910760, info=0x7fff08c75170) at /spare/tmp/workshop/bro/src/Val.cc:397 #3 0x0000000000627ee8 in SerialObj::Unserialize (info=0x7fff08c75170, type=34817) at /spare/tmp/workshop/bro/src/SerialObj.cc:220 #4 0x000000000065fbe8 in Val::Unserialize (info=0x39108a0, type=TYPE_ANY, exact_type=0x0) at /spare/tmp/workshop/bro/src/Val.cc:101 #5 0x0000000000660749 in Val::Unserialize (this=0x356e320, info=0x7fff08c75170) at /spare/tmp/workshop/bro/src/Val.h:332 #6 RecordVal::DoUnserialize (this=0x356e320, info=0x7fff08c75170) at /spare/tmp/workshop/bro/src/Val.cc:3067 #7 0x0000000000627ee8 in SerialObj::Unserialize (info=0x7fff08c75170, type=34817) at /spare/tmp/workshop/bro/src/SerialObj.cc:220 #8 0x000000000065fbe8 in Val::Unserialize (info=0x39108a0, type=TYPE_RECORD, exact_type=0x0) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Feb 22 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 22 Feb 2012 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202220800.q1M803HQ022030@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 778 [1] | dnthayer | | Normal | topic/dnthayer/ftp-ipv6 [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | c0839cb | Seth Hall | 2012-02-21 | GeoIP installation documentation update. [3] bro | b66b74e | Jon Siwek | 2012-02-20 | Decrease strictness of parsing IPv4 strings into addrs. (fixes #775) [4] bro | 1f7bfbb | Jon Siwek | 2012-02-20 | Fix memory leak in DNS manager (fixes #777). [5] [1] #778: http://tracker.bro-ids.org/bro/ticket/778 [2] ftp-ipv6: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/ftp-ipv6 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/c0839cb945f2daa43a800063319c6ed688f16e74/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/b66b74e5dc0ce96bb1e58cfec591e1a41240d5e0/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/1f7bfbb83c6b320d52f4ff91b6fdc426cb03b59a/bro From bro at tracker.bro-ids.org Wed Feb 22 05:29:29 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 13:29:29 -0000 Subject: [Bro-Dev] #775: IP address parsing more strict with new IPv6 code In-Reply-To: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> References: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> Message-ID: <062.3909377cb9344389f234a6a4e9e95c3a@tracker.bro-ids.org> #775: IP address parsing more strict with new IPv6 code ----------------------+------------------------ Reporter: robin | Owner: jsiwek Type: Problem | Status: reopened Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------+------------------------ Changes (by robin): * status: closed => reopened * resolution: fixed => Comment: I'm merging this in, but the CanonIPv4() function looks overly complex/expensive. Would it be better to just parse the IP address ourselves (rather than splitting the string, then reassemble, to then parse via the C library? Or, alternatively, we could build at least a fast path into CanonIPv4() which triggers splitting only when there are actually any leading zeros (that would be a quick string search). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 07:10:23 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 15:10:23 -0000 Subject: [Bro-Dev] #781: Case sensitive (non-normalized) HTTP header names Message-ID: <048.1cb22f83281a8d29460fe152fe9d53ce@tracker.bro-ids.org> #781: Case sensitive (non-normalized) HTTP header names ------------------------+----------------------------- Reporter: sconzo | Type: Feature Request Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: ------------------------+----------------------------- There are a lot of usecases to have access to the normalized HTTP header names, I'm asking for an extension that will allow access to the non- normalized version. This can be used to find non-standard/suspicious HTTP connections. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 08:02:08 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 16:02:08 -0000 Subject: [Bro-Dev] #775: IP address parsing more strict with new IPv6 code In-Reply-To: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> References: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> Message-ID: <062.0d2cbfd99795ad9d2221ab53ca08567a@tracker.bro-ids.org> #775: IP address parsing more strict with new IPv6 code ----------------------+------------------------ Reporter: robin | Owner: jsiwek Type: Problem | Status: reopened Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------+------------------------ Comment (by jsiwek): > I'm merging this in, but the CanonIPv4() function looks overly complex/expensive. Would it be better to just parse the IP address ourselves Yeah, just doing the parsing directly sounds better. I'll change that. -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Wed Feb 22 09:54:18 2012 From: seth at icir.org (Seth Hall) Date: Wed, 22 Feb 2012 12:54:18 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/bernhard/input-threads: raw input reader for seth, which can simply read a file into string-events given a line separator. (7e5f733) In-Reply-To: <201202221745.q1MHjVmA011966@bro-ids.icir.org> References: <201202221745.q1MHjVmA011966@bro-ids.icir.org> Message-ID: On Feb 22, 2012, at 12:45 PM, Bernhard Amann wrote: > raw input reader for seth, which can simply read a file into string-events given a line separator. Thanks Bernhard! I think this may turn out to be a surprisingly cool feature. ;) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Wed Feb 22 10:52:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 18:52:46 -0000 Subject: [Bro-Dev] #782: to_subnet BiF Message-ID: <046.afd33cbc4f7aed0c8c691b3622478d62@tracker.bro-ids.org> #782: to_subnet BiF -----------------------------+------------------------ Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | -----------------------------+------------------------ Would it make sense to have a bif to convert from a string to a subnet val? We could implement it at the script layer and it might make more sense there, but I wanted to someone else's feedback because it would be a little odd looking in the scripting language. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 11:21:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 19:21:20 -0000 Subject: [Bro-Dev] #777: Leak in DNS manager In-Reply-To: <047.4c03b3759c6980611a3da450aecd9407@tracker.bro-ids.org> References: <047.4c03b3759c6980611a3da450aecd9407@tracker.bro-ids.org> Message-ID: <062.4f74f7c1b0e72931375ecdbdc7d598c9@tracker.bro-ids.org> #777: Leak in DNS manager ----------------------+------------------------ Reporter: robin | Owner: jsiwek Type: Problem | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------+------------------------ Comment (by robin): In [d887eb3178c9f50a8808fae6ffe838e52e0a8f72/bro]: {{{ #!CommitTicketReference repository="bro" revision="d887eb3178c9f50a8808fae6ffe838e52e0a8f72" Merge remote-tracking branch 'origin/fastpath' * origin/fastpath: GeoIP installation documentation update. Decrease strictness of parsing IPv4 strings into addrs. (fixes #775) Fix memory leak in DNS manager (fixes #777). Closes #777. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 11:21:20 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 19:21:20 -0000 Subject: [Bro-Dev] #775: IP address parsing more strict with new IPv6 code In-Reply-To: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> References: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> Message-ID: <062.12c88f62f9cdb597b0dfad4b7b8212c3@tracker.bro-ids.org> #775: IP address parsing more strict with new IPv6 code ----------------------+------------------------ Reporter: robin | Owner: jsiwek Type: Problem | Status: closed Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: fixed | Keywords: ipv6 ----------------------+------------------------ Changes (by robin): * status: reopened => closed * resolution: => fixed Comment: In [d887eb3178c9f50a8808fae6ffe838e52e0a8f72/bro]: {{{ #!CommitTicketReference repository="bro" revision="d887eb3178c9f50a8808fae6ffe838e52e0a8f72" Merge remote-tracking branch 'origin/fastpath' * origin/fastpath: GeoIP installation documentation update. Decrease strictness of parsing IPv4 strings into addrs. (fixes #775) Fix memory leak in DNS manager (fixes #777). Closes #777. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 11:22:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 19:22:34 -0000 Subject: [Bro-Dev] #775: IP address parsing more strict with new IPv6 code In-Reply-To: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> References: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> Message-ID: <062.ebd7ee3dadff61f70db895e7829ac314@tracker.bro-ids.org> #775: IP address parsing more strict with new IPv6 code ----------------------+------------------------ Reporter: robin | Owner: jsiwek Type: Problem | Status: reopened Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------+------------------------ Changes (by robin): * status: closed => reopened * resolution: fixed => -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 11:46:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 19:46:03 -0000 Subject: [Bro-Dev] #782: to_subnet BiF In-Reply-To: <046.afd33cbc4f7aed0c8c691b3622478d62@tracker.bro-ids.org> References: <046.afd33cbc4f7aed0c8c691b3622478d62@tracker.bro-ids.org> Message-ID: <061.e93e29244b1a8c228490b9e52ea5121a@tracker.bro-ids.org> #782: to_subnet BiF ------------------------------+------------------------ Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Comment (by jsiwek): I'd say use a bif since other script-layer types use bifs to convert from strings. There's already a `SubNetVal(const char* text)` ctor, but doesn't look like it's used anywhere. Also at a glance, I think it might be buggy, but straightforward to fix if that's the case. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 11:50:57 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 19:50:57 -0000 Subject: [Bro-Dev] #352: Make --enable-ipv6 default In-Reply-To: <047.752eeb1e1fab85f95a271a2ddf98f038@tracker.bro-ids.org> References: <047.752eeb1e1fab85f95a271a2ddf98f038@tracker.bro-ids.org> Message-ID: <062.82af8b1b6d5d540cd1ad4a6a401d4259@tracker.bro-ids.org> #352: Make --enable-ipv6 default -----------------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: Resolution: Solved/Applied | Keywords: ipv6 -----------------------------+-------------------- Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied Comment: IPv6 support is enabled by default in the master branch now. The comment about Broccoli needing v6 serialization support is covered by #448, which I hope to get to soon. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 11:53:24 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 19:53:24 -0000 Subject: [Bro-Dev] #766: isipv6 function In-Reply-To: <049.eb8395c8d5ec1175e2ac4748ced7026b@tracker.bro-ids.org> References: <049.eb8395c8d5ec1175e2ac4748ced7026b@tracker.bro-ids.org> Message-ID: <064.7f2d9fb11173b859c22244671ebbcb82@tracker.bro-ids.org> #766: isipv6 function ------------------------------+-------------------- Reporter: aashish | Owner: Type: Feature Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: 2.0 Resolution: Solved/Applied | Keywords: ipv6 ------------------------------+-------------------- Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied Comment: The `is_v4_addr` and `is_v6_addr` functions are now available in the master branch. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 11:59:26 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 19:59:26 -0000 Subject: [Bro-Dev] #355: lookup_addr() only supports IPv4 addresses In-Reply-To: <047.6c096d333fd37f7e2ad084ea1c108762@tracker.bro-ids.org> References: <047.6c096d333fd37f7e2ad084ea1c108762@tracker.bro-ids.org> Message-ID: <062.223fa2b2e1c8d239bed06f472a22c43f@tracker.bro-ids.org> #355: lookup_addr() only supports IPv4 addresses -----------------------------+-------------------- Reporter: robin | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: Solved/Applied | Keywords: ipv6 -----------------------------+-------------------- Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 12:04:42 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 20:04:42 -0000 Subject: [Bro-Dev] #307: Complete IPv6 support In-Reply-To: <046.efb05b525c02d279854217ce367553cf@tracker.bro-ids.org> References: <046.efb05b525c02d279854217ce367553cf@tracker.bro-ids.org> Message-ID: <061.86f008aabf22973b0c7c5846bdb79070@tracker.bro-ids.org> #307: Complete IPv6 support ------------------------+-------------------- Reporter: seth | Owner: Type: Task | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: Resolution: Duplicate | Keywords: ipv6 ------------------------+-------------------- Changes (by jsiwek): * status: new => closed * resolution: => Duplicate Comment: Probably more helpful to track this goal with more granularity in individual tickets tagged with "ipv6" keyword (as already exist). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 12:15:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 20:15:38 -0000 Subject: [Bro-Dev] #68: BroV6 support -- memory usage In-Reply-To: <048.a7ea7982ce037ed963ed20f3871939b0@tracker.bro-ids.org> References: <048.a7ea7982ce037ed963ed20f3871939b0@tracker.bro-ids.org> Message-ID: <063.16798e910b473e635e26367df2c1c864@tracker.bro-ids.org> #68: BroV6 support -- memory usage -----------------------------+------------------------------------------ Reporter: gregor | Owner: Type: Task | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: 1.5.2 Resolution: Solved/Applied | Keywords: BroV6, binpac++, HILTI, IPv6 -----------------------------+------------------------------------------ Changes (by jsiwek): * status: seen => closed * resolution: => Solved/Applied Comment: The same results (higher memory usage after enabling IPv6 support) were no longer observed with Bro 2.0 or after the change to default IPv6 support via the new `IPAddr` class, possibly due to the large overhaul of Bro's scripts. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 12:19:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 20:19:51 -0000 Subject: [Bro-Dev] #783: Update IPv6-related website documentation Message-ID: <048.b6917f135ad80ce9768ff7e76b549342@tracker.bro-ids.org> #783: Update IPv6-related website documentation --------------------+------------------------ Reporter: jsiwek | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: ipv6 | --------------------+------------------------ There's at least these two pages I know of that are now outdated: http://www.bro-ids.org/development/ipv6.html http://www.bro-ids.org/development/projects/ipv6.html -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 12:31:26 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 20:31:26 -0000 Subject: [Bro-Dev] #453: Connection compressor doesn't handle IPv6 In-Reply-To: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> References: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> Message-ID: <062.b80a9bf9b5bbca92d69f58e218fdef1b@tracker.bro-ids.org> #453: Connection compressor doesn't handle IPv6 ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------+------------------------ Comment (by jsiwek): There was talk (and a general agreement) of removing the connection compressor code ("Connection Compressor" thread in bro-dev list from August 2011). Is that still true? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 12:41:07 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 20:41:07 -0000 Subject: [Bro-Dev] #744: Subnets types broken in logging framework with ipv6 enabled In-Reply-To: <046.a1ca177adc1b775d95f1066898382676@tracker.bro-ids.org> References: <046.a1ca177adc1b775d95f1066898382676@tracker.bro-ids.org> Message-ID: <061.9ec2dc054d5f8dc9dc69ea1acf6dd48a@tracker.bro-ids.org> #744: Subnets types broken in logging framework with ipv6 enabled -----------------------------+-------------------- Reporter: seth | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: Solved/Applied | Keywords: ipv6 -----------------------------+-------------------- Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied Comment: The default IPv6 code in master using the new `IPPrefix` class doesn't have this problem anymore. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 13:11:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 21:11:46 -0000 Subject: [Bro-Dev] #784: topic/jsiwek/ipaddr-refactoring Message-ID: <048.4015aae978a118ff2be0f7557207639c@tracker.bro-ids.org> #784: topic/jsiwek/ipaddr-refactoring ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ I did a review of IPAdd::GetBytes/CopyIPv6 usages and tried to clean some up. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 22 13:46:27 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 22 Feb 2012 21:46:27 -0000 Subject: [Bro-Dev] #775: IP address parsing more strict with new IPv6 code In-Reply-To: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> References: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> Message-ID: <062.a7d1001262867ea74d74c51d2c960a62@tracker.bro-ids.org> #775: IP address parsing more strict with new IPv6 code ----------------------+------------------------ Reporter: robin | Owner: jsiwek Type: Problem | Status: closed Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: fixed | Keywords: ipv6 ----------------------+------------------------ Changes (by jsiwek): * status: reopened => closed * resolution: => fixed Comment: In [c84394d07f30aa856ca4283ade98c08d63d37b4b/bro]: {{{ #!CommitTicketReference repository="bro" revision="c84394d07f30aa856ca4283ade98c08d63d37b4b" Refactor IPAddr v4 initialization from string. (fixes #775) Revived code from old dotted_to_addr function to parse the dotted address string directly instead of canonicalizing and passing to inet_pton. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Wed Feb 22 17:11:55 2012 From: seth at icir.org (Seth Hall) Date: Wed, 22 Feb 2012 20:11:55 -0500 Subject: [Bro-Dev] ipv6 fragment reassembling Message-ID: <165CBD62-EB10-4D08-ABE6-4DF58A4D3D51@icir.org> Here is a great article about ipv6 fragment handling. http://blog.si6networks.com/2012/02/ipv6-nids-evasion-and-improvements-in.html The article concludes by point out that it looks like the IETF is converging on RFCs that forbid overlapping fragments which should make fragment reassembly much clearer for us. Current operating systems are of course all over the map in terms of what they actually support of course. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From noreply at bro-ids.org Thu Feb 23 00:00:07 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 23 Feb 2012 00:00:07 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202230800.q1N807pn018300@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 778 [1] | dnthayer | | Normal | topic/dnthayer/ftp-ipv6 [2] Bro | 784 [3] | jsiwek | | Normal | topic/jsiwek/ipaddr-refactoring [4] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | c84394d | Jon Siwek | 2012-02-22 | Refactor IPAddr v4 initialization from string. (fixes #775) [5] [1] #778: http://tracker.bro-ids.org/bro/ticket/778 [2] ftp-ipv6: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/ftp-ipv6 [3] #784: http://tracker.bro-ids.org/bro/ticket/784 [4] ipaddr-refactoring: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipaddr-refactoring [5] fastpath: http://tracker.bro-ids.org/bro/changeset/c84394d07f30aa856ca4283ade98c08d63d37b4b/bro From bro at tracker.bro-ids.org Thu Feb 23 08:18:50 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 23 Feb 2012 16:18:50 -0000 Subject: [Bro-Dev] #785: SMB processid field Message-ID: <055.6069bd68783308f444cbf30d7b311792@tracker.bro-ids.org> #785: SMB processid field ---------------------------+-------------------- Reporter: JulienSentier | Type: Patch Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: smb ---------------------------+-------------------- This patch adds the field processid in the smb_hdr structure. There is a test in this patch with a capture showing an exploitation of CVE-2009-3103. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 23 08:21:16 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 23 Feb 2012 16:21:16 -0000 Subject: [Bro-Dev] #786: Minor bugfixes Message-ID: <055.31e8fc347475856631bd8ecd15bf1b98@tracker.bro-ids.org> #786: Minor bugfixes ---------------------------+-------------------- Reporter: JulienSentier | Type: Patch Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: ---------------------------+-------------------- Some small leaks, dead code, correction of operators priorities... -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 23 08:51:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 23 Feb 2012 16:51:51 -0000 Subject: [Bro-Dev] #787: CMake 2.6.3 minimum requirement Message-ID: <048.babdd0e99efcd087e3d5f85d24fc8e44@tracker.bro-ids.org> #787: CMake 2.6.3 minimum requirement ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ The `cmake_minimum_required()` for all the repos is 2.6 and documentation also says 2.6 or greater is required, but some CMake scripts now do things that are incompatible with versions less than 2.6.3. That requirement is generally fulfilled by package repositories of distros we support, so I'd like to increase the minimum to 2.6.3. Since this involves minor changes to all the repos, I'll do it directly in master branches unless you'd like me to make topic branches, Robin. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 23 09:56:58 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 23 Feb 2012 17:56:58 -0000 Subject: [Bro-Dev] #787: CMake 2.6.3 minimum requirement In-Reply-To: <048.babdd0e99efcd087e3d5f85d24fc8e44@tracker.bro-ids.org> References: <048.babdd0e99efcd087e3d5f85d24fc8e44@tracker.bro-ids.org> Message-ID: <063.67624d64db058718cdcfe0d5393b7fc8@tracker.bro-ids.org> #787: CMake 2.6.3 minimum requirement ----------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): On Thu, Feb 23, 2012 at 16:51 -0000, you wrote: > generally fulfilled by package repositories of distros we support, so I'd > like to increase the minimum to 2.6.3. Since this involves minor changes > to all the repos, I'll do it directly in master branches Makes sense, please go aehad. Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From asharma at lbl.gov Thu Feb 23 11:21:44 2012 From: asharma at lbl.gov (Aashish Sharma) Date: Thu, 23 Feb 2012 11:21:44 -0800 Subject: [Bro-Dev] bro-2.1 IPv6, headers, evasion and other fun things Message-ID: <20120223192122.GA14682@yaksha.lbl.gov> Hello brodevs: Since effort to get bro IPv6 ready is ongoing, I thought there are some IPv6 security issues to bring to attention. May be you already have plans to handle these or otherwise I think these are somethings to pay attention to. Offcourse, I realize all this is easier said then done: One thing I would really like is ability to log optional headers at minimum. Here is some situations why it is important. 1) Evasion attack: Routing header can result in bro looking only at the IP header and not the routing header which dictates where the packet goes next. So IDS things that packet is going to host A but ends up going to host B and then C which are included in the routing header. So unless we parse these headers we shall not know what actually is the ultimate destination of the packet. 2) Evasion attack using Extension header size: could be 128K per header, and RFC doesn't dictate the sequence of the headers so these can be out of order. Bro needs to address the problem of parsing and processing these extension headers, taking in to account the full sized extension headers, else miscreants might put malicious contents at the end of these headers and evade the detection. Even tcpdump only looks at the next header only and not the routing header unless you specify protochain 6 in the filter. 3) Mobility header issue: With this one can use your internal IP address anywhere else in the world. RFC dictates to use IPSec but there is still trust issues - check out RFC4487. An Application of such attack is subverting geoip blocking - example youtube not allowing content viewing for IP's in Canada etc or avoid roaming charges when using a mobile phone. 4) IDS connection caching issue: Since IPv6 doesn't support fragmentation, a Sender has to keep a copy of the packet in the IP stack because application doesn?t know if routers are going to fragment or not and sender has to account for a possible ICMP error with frag bit set. IDS has to be able to account for such situations and re-transmissions. 5) Neighbor discovery attack: ICMP6 can broadcast address for someone else. It assumes that network is secure. Also given ICMP6 messages are routable where as with ARP this was not a problem because ARP is/was local to a subnet only. 6) Address Priority issues: It is possible that part of attack uses a link local address and other part uses the site address. BRO needs to maintain a state which can account for all the IP's assigned and used by the host to assemble a connection/attack etc. 7) IPv6 allows multiple routers allocate multiple address - one from each router. Can we flag on rogue router broadcasts or new address allocation other then the ones which are predefined in the networks.cfg 8) IPv4 and IPv6 co-existence issues: Its not only that hosts have affinity to IPv6, but applications also do. Crome prefers using v6 (it tries connecting on v6 for 300mS and then falls back to v4) while safari prefers to pick up the IP from OS X stack. Depending on the latency and/or other situation host and applications may be using different addresses - Crome can use v6 while safari v4. How do we know both of these connections are part of the same attack ? 9) DNS resolution - Any IPv6 host takes upon itself to resolve DNS for any query which it sees. So if an attacker chooses to use broadcast address ff02::1 for DNS query resolution all hosts are going to resolve. 10) Collusion attack - because of the vastness of IPv6 address space, it is possible for two hosts to use a new IP address for every packet to communicate between them. 11) IPv6 also brings isolation problem: Address scheme allows assigning a Unicast local IP which doesn't have to be globally routable to a network device such as printer. However when a work station gets both local and global address, an external IP can bounce off this work station to access the network device which has unicast local IP. Likewise the network device can use a workstation (with dual address) as proxy to communicate to the outside world. 12) Offcourse Teredo which encapsulates protocol and provides globally reachable IPv6 address - enabled by win7 and win8.rc2 by default but only works when ipv4 address is not available. Aashish -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Information Technology Division Lawrence Berkeley National Laboratory http://www.lbl.gov/cyber/pgp-aashish.txt Office: (510)-495-2680 Cell: (510)-457-1525 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20120223/e5418dd5/attachment.bin From jmellander at lbl.gov Thu Feb 23 11:33:42 2012 From: jmellander at lbl.gov (Jim Mellander) Date: Thu, 23 Feb 2012 11:33:42 -0800 Subject: [Bro-Dev] bro-2.1 IPv6, headers, evasion and other fun things In-Reply-To: <20120223192122.GA14682@yaksha.lbl.gov> References: <20120223192122.GA14682@yaksha.lbl.gov> Message-ID: Thanks Aashish, I might add an additional thing to think about: Per RFC 5722, overlapping fragments are explicitly forbidden in IPv6 (more info at http://blog.si6networks.com/2012/02/ipv6-nids-evasion-and-improvements-in.html ) It probably would be good to flag such attempts as hostile. I don't know what the rfcs say about ipv6 fragments that are retransmitted with differing contents, but those also would likely be hostile. We've thought about dropping packets containing extension headers on the floor, at least until more is known about the threat surface. Jim On Thu, Feb 23, 2012 at 11:21 AM, Aashish Sharma wrote: > Hello brodevs: > > Since effort to get bro IPv6 ready is ongoing, I thought there are some > IPv6 security issues to bring to attention. May be you already have > plans to ?handle these or otherwise I think these are somethings to pay > attention to. Offcourse, I realize all this is easier said then done: > > One thing I would really like is ability to log optional headers at > minimum. Here is some situations why it is important. > > 1) Evasion attack: Routing header can result in bro looking only > at the IP header and not the routing header which dictates where the > packet goes next. So IDS things that packet is going to host A but ends > up going to host B and then C which are included in the routing header. > So unless we parse these headers we shall not know what actually is the > ultimate destination of the packet. > > 2) Evasion attack using Extension header size: ?could be 128K per > header, and RFC doesn't dictate the sequence of the headers so these can > be out of order. Bro needs to address the problem of parsing and > processing these extension headers, taking in to account the full sized > extension headers, else miscreants might put malicious contents at the > end of these headers and evade the detection. > > Even tcpdump only looks at the next header only and not the routing > header unless ?you specify protochain 6 in the filter. > > 3) Mobility header issue: With this one can use your internal IP address > anywhere else in the world. RFC dictates to use IPSec but there is still > trust issues - check out RFC4487. > > An Application of such attack is subverting geoip blocking - example > youtube not allowing content viewing for IP's in Canada etc or avoid > roaming charges when using a mobile phone. > > 4) IDS connection caching issue: Since IPv6 doesn't support > fragmentation, a Sender has to keep a copy of the packet in the IP stack > because ?application doesn?t know if routers are going to fragment or > not and sender has to account for a possible ICMP error with frag bit > set. IDS has to be able to account for such situations and > re-transmissions. > > 5) Neighbor discovery attack: ICMP6 can broadcast address for someone > else. It assumes that network is secure. ?Also given ICMP6 messages are > routable where as with ?ARP this was not a problem because ARP is/was > local to a subnet only. > > 6) Address Priority issues: It is possible that part of attack uses a > link local address and other part uses the site address. BRO needs to > maintain a ?state which can account for all the IP's assigned and used > by the host to assemble a connection/attack etc. > > 7) IPv6 allows multiple routers allocate multiple address - one from > each router. Can we flag on rogue router broadcasts or new address > allocation other then the ones which are predefined in the networks.cfg > > > 8) IPv4 and IPv6 co-existence issues: Its not only that hosts have > affinity to IPv6, but applications also do. ?Crome prefers using v6 > (it tries connecting on v6 for 300mS and then falls back to v4) while > safari prefers to pick up the IP from OS X stack. Depending on the latency > and/or other situation host and applications may be using different > addresses - Crome can use v6 while safari v4. How do we know both of > these connections are part of the same attack ? > > 9) DNS resolution - Any IPv6 host takes upon itself to resolve DNS for > any query which it sees. So if an attacker chooses to use broadcast > address ff02::1 for DNS query resolution all hosts are going to resolve. > > 10) Collusion attack - because of the vastness of IPv6 address space, it > is possible for two hosts to use a new IP address for every packet to > communicate between them. > > 11) IPv6 also brings isolation problem: Address scheme allows assigning a > Unicast local IP which doesn't have to be globally routable to a network > device such as printer. However when a work station gets both local and > global address, an external IP can bounce off this work station to > access the network device which has unicast local IP. Likewise the > network device can use a workstation (with dual address) as proxy to > communicate to the outside world. > > 12) Offcourse Teredo which encapsulates protocol and provides globally > reachable IPv6 address - enabled by win7 and win8.rc2 by default but > only works when ipv4 address is not available. > > Aashish > > > -- > Aashish Sharma ?(asharma at lbl.gov) > Cyber Security, Information Technology Division > Lawrence Berkeley National Laboratory > http://www.lbl.gov/cyber/pgp-aashish.txt > Office: (510)-495-2680 ?Cell: (510)-457-1525 > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > From bro at tracker.bro-ids.org Thu Feb 23 15:02:05 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 23 Feb 2012 23:02:05 -0000 Subject: [Bro-Dev] #448: Broccoli does not support IPv6 addresses In-Reply-To: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> References: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> Message-ID: <062.182eb21aa420c1a5833f8d6404382cf4@tracker.bro-ids.org> #448: Broccoli does not support IPv6 addresses -----------------------+---------------------- Reporter: robin | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: Resolution: | Keywords: ipv6 -----------------------+---------------------- Comment (by jsiwek): In [cb37a5116283e567667e0a8f7a8f9056b61da8b8/broccoli]: {{{ #!CommitTicketReference repository="broccoli" revision="cb37a5116283e567667e0a8f7a8f9056b61da8b8" Update broccoli library to handle IPv6 addrs/subnets (addresses #448) Addresses now use a new BroAddr struct to hold the address data and BroSubnet changed to use a BroAddr member instead of a single uint32 to represent the address. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 23 15:02:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 23 Feb 2012 23:02:21 -0000 Subject: [Bro-Dev] #448: Broccoli does not support IPv6 addresses In-Reply-To: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> References: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> Message-ID: <062.3b603482356d696bcc4d43b28953227f@tracker.bro-ids.org> #448: Broccoli does not support IPv6 addresses -----------------------+---------------------- Reporter: robin | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: Resolution: | Keywords: ipv6 -----------------------+---------------------- Comment (by jsiwek): In [00a73f01b34355da23493bf78d33bcd3e8134434/broccoli-python]: {{{ #!CommitTicketReference repository="broccoli-python" revision="00a73f01b34355da23493bf78d33bcd3e8134434" Update broccoli-python for IPv6 addr/subnet support (addresses #448) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 23 15:02:32 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 23 Feb 2012 23:02:32 -0000 Subject: [Bro-Dev] #448: Broccoli does not support IPv6 addresses In-Reply-To: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> References: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> Message-ID: <062.198673c59ac60aed89fd712579c87509@tracker.bro-ids.org> #448: Broccoli does not support IPv6 addresses -----------------------+---------------------- Reporter: robin | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: Resolution: | Keywords: ipv6 -----------------------+---------------------- Comment (by jsiwek): In [29e36a03636ac8a0013323435466ad60335e16a4/broccoli-ruby]: {{{ #!CommitTicketReference repository="broccoli-ruby" revision="29e36a03636ac8a0013323435466ad60335e16a4" Update broccoli-ruby for IPv6 addr/subnet support (addresses #448) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 23 15:02:55 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 23 Feb 2012 23:02:55 -0000 Subject: [Bro-Dev] #448: Broccoli does not support IPv6 addresses In-Reply-To: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> References: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> Message-ID: <062.892b53927281e9437241871b82a014a7@tracker.bro-ids.org> #448: Broccoli does not support IPv6 addresses -----------------------+---------------------- Reporter: robin | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: Resolution: | Keywords: ipv6 -----------------------+---------------------- Comment (by jsiwek): In [14ccd6436ff8db03fef181a8f56c86162bb6af53/bro]: {{{ #!CommitTicketReference repository="bro" revision="14ccd6436ff8db03fef181a8f56c86162bb6af53" Update/add tests for broccoli IPv6 addr/subnet support (addresses #448) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From gregor at majordomus.org Thu Feb 23 15:20:32 2012 From: gregor at majordomus.org (Gregor Maier) Date: Thu, 23 Feb 2012 15:20:32 -0800 Subject: [Bro-Dev] ipv6 fragment reassembling In-Reply-To: <165CBD62-EB10-4D08-ABE6-4DF58A4D3D51@icir.org> References: <165CBD62-EB10-4D08-ABE6-4DF58A4D3D51@icir.org> Message-ID: <4F46C9C0.5010105@majordomus.org> On 2/22/12 17:11 , Seth Hall wrote: > Here is a great article about ipv6 fragment handling. > > http://blog.si6networks.com/2012/02/ipv6-nids-evasion-and-improvements-in.html > > The article concludes by point out that it looks like the IETF is converging on RFCs that forbid overlapping fragments which should make fragment reassembly much clearer for us. Current operating systems are of course all over the map in terms of what they actually support of course. :) I don't get what all this fuss is about. IPv4 has exactly the same issues. Even with a "standard" way of handling overlaps the IDS has no way of knowing if the monitored systems actually implement the standard correctly. So you are still in the same position you were before. And if it's not fragments that you can still have overlapping TCP segments, TTL-tuning, hiding ports in subsequent fragments, etc., etc.) cu Gregor From noreply at bro-ids.org Fri Feb 24 00:00:06 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 24 Feb 2012 00:00:06 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202240800.q1O805gn012525@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 778 [1] | dnthayer | | Normal | topic/dnthayer/ftp-ipv6 [2] Bro | 784 [3] | jsiwek | | Normal | topic/jsiwek/ipaddr-refactoring [4] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | c84394d | Jon Siwek | 2012-02-22 | Refactor IPAddr v4 initialization from string. (fixes #775) [5] [1] #778: http://tracker.bro-ids.org/bro/ticket/778 [2] ftp-ipv6: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/ftp-ipv6 [3] #784: http://tracker.bro-ids.org/bro/ticket/784 [4] ipaddr-refactoring: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipaddr-refactoring [5] fastpath: http://tracker.bro-ids.org/bro/changeset/c84394d07f30aa856ca4283ade98c08d63d37b4b/bro From bro at tracker.bro-ids.org Fri Feb 24 02:26:28 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 10:26:28 -0000 Subject: [Bro-Dev] #788: Good analysis of unidirectional DNS flows Message-ID: <055.77b175353d85b253152a74f5499c191a@tracker.bro-ids.org> #788: Good analysis of unidirectional DNS flows ---------------------------+-------------------- Reporter: JulienSentier | Type: Patch Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: ---------------------------+-------------------- Some use port udp 53 as a source port for dns requests. And sometimes, we can miss the DNS request. In this case, we can rely on the DNS field QR to identify the direction of the flow. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 03:20:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 11:20:56 -0000 Subject: [Bro-Dev] #789: An example of a BiF : computing Conficker P2P ports Message-ID: <055.f5ce34b401942705c08aa57548263d6f@tracker.bro-ids.org> #789: An example of a BiF : computing Conficker P2P ports ---------------------------+-------------------- Reporter: JulienSentier | Type: Patch Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: ---------------------------+-------------------- The goal of this patch is to show how to make your own BiF rather than being actually used. The BiF file contains a function which computes the Conficker P2P ports for an IP address. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 03:35:23 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 11:35:23 -0000 Subject: [Bro-Dev] #790: Event connection_rejected with parameter is_orig Message-ID: <055.b70f73fdaff19bad90b76acf34271c1d@tracker.bro-ids.org> #790: Event connection_rejected with parameter is_orig ---------------------------+-------------------- Reporter: JulienSentier | Type: Patch Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: ---------------------------+-------------------- So that we can know if the client or the server rejected the connection -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 09:32:07 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 17:32:07 -0000 Subject: [Bro-Dev] #448: Broccoli does not support IPv6 addresses In-Reply-To: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> References: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> Message-ID: <062.4cca65aa7e212f3ffc556441a0304d57@tracker.bro-ids.org> #448: Broccoli does not support IPv6 addresses ----------------------------+---------------------- Reporter: robin | Owner: jsiwek Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: Resolution: | Keywords: ipv6 ----------------------------+---------------------- Changes (by jsiwek): * type: Problem => Merge Request Comment: Support has been added in the `topic/jsiwek/broccoli-ipv6` branch of bro, broccoli, broccoli-python, and broccoli-ruby repos. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 10:40:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 18:40:02 -0000 Subject: [Bro-Dev] #782: to_subnet BiF In-Reply-To: <046.afd33cbc4f7aed0c8c691b3622478d62@tracker.bro-ids.org> References: <046.afd33cbc4f7aed0c8c691b3622478d62@tracker.bro-ids.org> Message-ID: <061.2454be1dc015637f5f68df5c1cf0a177@tracker.bro-ids.org> #782: to_subnet BiF ------------------------------+------------------------ Reporter: seth | Owner: jsiwek Type: Feature Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ------------------------------+------------------------ Changes (by jsiwek): * owner: => jsiwek * status: new => closed * resolution: => fixed Comment: In [32aabe843245afa28492450f7fc1f49ceb758d66/bro]: {{{ #!CommitTicketReference repository="bro" revision="32aabe843245afa28492450f7fc1f49ceb758d66" Add to_subnet bif (fixes #782). Also fix IPAddr::Mask/ReverseMask not allowing argument of 0. And clarified return value of to_addr bif when the input string does not parse into a valid IP address. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Fri Feb 24 11:03:26 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 24 Feb 2012 11:03:26 -0800 Subject: [Bro-Dev] ipv6 fragment reassembling In-Reply-To: <4F46C9C0.5010105@majordomus.org> References: <165CBD62-EB10-4D08-ABE6-4DF58A4D3D51@icir.org> <4F46C9C0.5010105@majordomus.org> Message-ID: <20120224190326.GF59422@icir.org> On Thu, Feb 23, 2012 at 15:20 -0800, you wrote: > Even with a "standard" way of handling overlaps the IDS has no way of > knowing if the monitored systems actually implement the standard > correctly. Yeah, that's right. In particular given that this is a recent change. What we could see is if overlapping fragements are rare enough that it's worth alarming on. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Fri Feb 24 11:15:29 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 19:15:29 -0000 Subject: [Bro-Dev] #787: CMake 2.6.3 minimum requirement In-Reply-To: <048.babdd0e99efcd087e3d5f85d24fc8e44@tracker.bro-ids.org> References: <048.babdd0e99efcd087e3d5f85d24fc8e44@tracker.bro-ids.org> Message-ID: <063.7697e6c052efb8823fb10ec82b046053@tracker.bro-ids.org> #787: CMake 2.6.3 minimum requirement -----------------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Fri Feb 24 11:19:13 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 24 Feb 2012 11:19:13 -0800 Subject: [Bro-Dev] [security] bro-2.1 IPv6, headers, evasion and other fun things In-Reply-To: <20120223192122.GA14682@yaksha.lbl.gov> References: <20120223192122.GA14682@yaksha.lbl.gov> Message-ID: <20120224191912.GI59422@icir.org> Cool, thanks Aashish, that's good food for thought and I'll make sure we have it at hand as we work on the code. The first step is making sure we're extracting the right information at the protocol level and for that, such sample application are very helpful to have in mind. Everbody else, please feel free to chime in with more thoughts (thanks Jim!). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Fri Feb 24 11:27:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 19:27:48 -0000 Subject: [Bro-Dev] #453: Connection compressor doesn't handle IPv6 In-Reply-To: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> References: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> Message-ID: <062.fd2e59e173ad250900ab694d8d543c7d@tracker.bro-ids.org> #453: Connection compressor doesn't handle IPv6 ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------+------------------------ Comment (by robin): On Wed, Feb 22, 2012 at 20:31 -0000, you wrote: > There was talk (and a general agreement) of removing the connection > compressor code ("Connection Compressor" thread in bro-dev list from > August 2011). Is that still true? Yes, I would still like to remove it. But Seth I believe had some concerns recently? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 12:07:24 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 20:07:24 -0000 Subject: [Bro-Dev] #780: file extraction trunctation. In-Reply-To: <048.841e7736f715e5a45f63422aac975ed4@tracker.bro-ids.org> References: <048.841e7736f715e5a45f63422aac975ed4@tracker.bro-ids.org> Message-ID: <063.96ced97ba55f2c8c36dda751a86178f4@tracker.bro-ids.org> #780: file extraction trunctation. ----------------------+--------------------------------------- Reporter: justin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: file extraction truncated ----------------------+--------------------------------------- Changes (by robin): * milestone: => Bro2.1 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 12:07:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 20:07:46 -0000 Subject: [Bro-Dev] #779: missing values cause bro to crash when used inside of a 'when' statement. In-Reply-To: <048.d7eccddacf988dcfe011e1dac3c48b28@tracker.bro-ids.org> References: <048.d7eccddacf988dcfe011e1dac3c48b28@tracker.bro-ids.org> Message-ID: <063.04b00c99efdb9b6bf342902c1581c3b2@tracker.bro-ids.org> #779: missing values cause bro to crash when used inside of a 'when' statement. ----------------------+--------------------------------------- Reporter: justin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: when InterpreterException ----------------------+--------------------------------------- Changes (by robin): * milestone: => Bro2.1 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 12:11:37 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 20:11:37 -0000 Subject: [Bro-Dev] #453: Connection compressor doesn't handle IPv6 In-Reply-To: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> References: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> Message-ID: <062.d20f65a052edc6ee432cb039516968a8@tracker.bro-ids.org> #453: Connection compressor doesn't handle IPv6 ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------+------------------------ Comment (by seth): > Yes, I would still like to remove it. But Seth I believe had some > concerns recently? I don't know if they are grounded concerns or not. I was only basing it on the emails from the users list about large numbers of single packet scans coming across a network at one instant and resulting in packet loss. Maybe the connection compressor wouldn't even help with this? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 13:04:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 21:04:00 -0000 Subject: [Bro-Dev] #453: Connection compressor doesn't handle IPv6 In-Reply-To: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> References: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> Message-ID: <062.e58fadba1b5fdbafaa76c9c9c9ad5cc9@tracker.bro-ids.org> #453: Connection compressor doesn't handle IPv6 ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------+------------------------ Comment (by robin): On Fri, Feb 24, 2012 at 20:11 -0000, you wrote: > I don't know if they are grounded concerns or not. I was only basing it > on the emails from the users list about large numbers of single packet > scans coming across a network at one instant and resulting in packet loss. > Maybe the connection compressor wouldn't even help with this? In principle it could, with specifics hard to say without more information/testing. But given that we don't fully understand that and that we've already deactivated it, I'm in favor of removing as otherwise it'll be just more code not being tested and maintained (in particualr with the IPv6 changes now). We can always bring it back later if we really want to, it's all there somewhere in the repository. -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 15:56:28 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 23:56:28 -0000 Subject: [Bro-Dev] #789: An example of a BiF : computing Conficker P2P ports In-Reply-To: <055.f5ce34b401942705c08aa57548263d6f@tracker.bro-ids.org> References: <055.f5ce34b401942705c08aa57548263d6f@tracker.bro-ids.org> Message-ID: <070.0abe32a49199591039f50cf6558c5e6c@tracker.bro-ids.org> #789: An example of a BiF : computing Conficker P2P ports ----------------------------+------------------------ Reporter: JulienSentier | Owner: Type: Patch | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Hmm... I'm reluctant to apply this as it's a very specific bif. Could this be done at the scripting layer instead? Also, from the ticket description I'm not sure if you mean it mainly as a demonstration of writing a bif? If so, it might be more appropriate for the documentation. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 15:58:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 24 Feb 2012 23:58:00 -0000 Subject: [Bro-Dev] #790: Event connection_rejected with parameter is_orig In-Reply-To: <055.b70f73fdaff19bad90b76acf34271c1d@tracker.bro-ids.org> References: <055.b70f73fdaff19bad90b76acf34271c1d@tracker.bro-ids.org> Message-ID: <070.67ada90cefdaeea419968b9e8394519f@tracker.bro-ids.org> #790: Event connection_rejected with parameter is_orig ----------------------------+------------------------ Reporter: JulienSentier | Owner: Type: Patch | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): The event should actually be only generated when the server rejects. Do you have an example where that's not the case? -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Fri Feb 24 16:02:47 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 24 Feb 2012 16:02:47 -0800 Subject: [Bro-Dev] Sorting test output? Message-ID: <20120225000247.GB90478@icir.org> I keep getting bogus test diffs for a number of tests because of line order changes. I'm wondering if we could just generally sort all output before diffing. Does anybody see a problem with that? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Fri Feb 24 16:38:29 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 25 Feb 2012 00:38:29 -0000 Subject: [Bro-Dev] #786: Minor bugfixes In-Reply-To: <055.31e8fc347475856631bd8ecd15bf1b98@tracker.bro-ids.org> References: <055.31e8fc347475856631bd8ecd15bf1b98@tracker.bro-ids.org> Message-ID: <070.272670386a985eb6bca4fe6a9ae4d2d7@tracker.bro-ids.org> #786: Minor bugfixes ----------------------------+------------------------ Reporter: JulienSentier | Owner: Type: Patch | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Applied these (with a few further tweaks). Thanks! Out of curiosity, how did you find these? The one change I don't understand is this: {{{ --- a/src/SMTP.cc +++ b/src/SMTP.cc @@ -353,7 +353,6 @@ void SMTP_Analyzer::ProcessLine(int length, const char* line, bool orig) int ext_len; get_word(end_of_line - line, line, ext_len, ext); - line = skip_whitespace(line + ext_len, end_of_line); ProcessExtension(ext_len, ext); } } }}} Why is that dead code? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 16:40:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 25 Feb 2012 00:40:06 -0000 Subject: [Bro-Dev] #785: SMB processid field In-Reply-To: <055.6069bd68783308f444cbf30d7b311792@tracker.bro-ids.org> References: <055.6069bd68783308f444cbf30d7b311792@tracker.bro-ids.org> Message-ID: <070.b066e67426a81de6b073786ffe1f55dd@tracker.bro-ids.org> #785: SMB processid field ----------------------------+------------------------ Reporter: JulienSentier | Owner: seth Type: Patch | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: smb ----------------------------+------------------------ Changes (by robin): * owner: => seth * status: new => assigned * milestone: => Bro2.1 Comment: Seth, can you take a look at this? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 16:41:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 25 Feb 2012 00:41:21 -0000 Subject: [Bro-Dev] #784: topic/jsiwek/ipaddr-refactoring In-Reply-To: <048.4015aae978a118ff2be0f7557207639c@tracker.bro-ids.org> References: <048.4015aae978a118ff2be0f7557207639c@tracker.bro-ids.org> Message-ID: <063.789f348d4e6ea157a6f48324845d406b@tracker.bro-ids.org> #784: topic/jsiwek/ipaddr-refactoring ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [33236927713252520b42f8375d509a7be94a636e/bro]: {{{ #!CommitTicketReference repository="bro" revision="33236927713252520b42f8375d509a7be94a636e" Merge remote-tracking branch 'origin/topic/jsiwek/ipaddr-refactoring' * origin/topic/jsiwek/ipaddr-refactoring: Refactoring various usages of new IPAddr class. Conflicts: src/bro.bif Closes #784. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 16:41:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 25 Feb 2012 00:41:21 -0000 Subject: [Bro-Dev] #784: topic/jsiwek/ipaddr-refactoring In-Reply-To: <048.4015aae978a118ff2be0f7557207639c@tracker.bro-ids.org> References: <048.4015aae978a118ff2be0f7557207639c@tracker.bro-ids.org> Message-ID: <063.3a2b4111c45c852e5e4507dcf8f3ceee@tracker.bro-ids.org> #784: topic/jsiwek/ipaddr-refactoring ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Comment (by robin): In [2eeac548574addeb9b307794713ccbeb019eabba/bro]: {{{ #!CommitTicketReference repository="bro" revision="2eeac548574addeb9b307794713ccbeb019eabba" Merge remote-tracking branch 'origin/fastpath' * origin/fastpath: Add to_subnet bif (fixes #782). Refactor IPAddr v4 initialization from string. (fixes #775) Closes #782. Closes #775. Closes #784. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 16:41:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 25 Feb 2012 00:41:21 -0000 Subject: [Bro-Dev] #782: to_subnet BiF In-Reply-To: <046.afd33cbc4f7aed0c8c691b3622478d62@tracker.bro-ids.org> References: <046.afd33cbc4f7aed0c8c691b3622478d62@tracker.bro-ids.org> Message-ID: <061.62050a3bee7afe62f50448280f720879@tracker.bro-ids.org> #782: to_subnet BiF ------------------------------+------------------------ Reporter: seth | Owner: jsiwek Type: Feature Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ------------------------------+------------------------ Comment (by robin): In [2eeac548574addeb9b307794713ccbeb019eabba/bro]: {{{ #!CommitTicketReference repository="bro" revision="2eeac548574addeb9b307794713ccbeb019eabba" Merge remote-tracking branch 'origin/fastpath' * origin/fastpath: Add to_subnet bif (fixes #782). Refactor IPAddr v4 initialization from string. (fixes #775) Closes #782. Closes #775. Closes #784. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 16:41:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 25 Feb 2012 00:41:21 -0000 Subject: [Bro-Dev] #775: IP address parsing more strict with new IPv6 code In-Reply-To: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> References: <047.807b45143b2c46004f034f1b76b0143a@tracker.bro-ids.org> Message-ID: <062.faf85420c5f7db00b379feb2509a8c98@tracker.bro-ids.org> #775: IP address parsing more strict with new IPv6 code ----------------------+------------------------ Reporter: robin | Owner: jsiwek Type: Problem | Status: closed Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: fixed | Keywords: ipv6 ----------------------+------------------------ Comment (by robin): In [2eeac548574addeb9b307794713ccbeb019eabba/bro]: {{{ #!CommitTicketReference repository="bro" revision="2eeac548574addeb9b307794713ccbeb019eabba" Merge remote-tracking branch 'origin/fastpath' * origin/fastpath: Add to_subnet bif (fixes #782). Refactor IPAddr v4 initialization from string. (fixes #775) Closes #782. Closes #775. Closes #784. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 16:41:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 25 Feb 2012 00:41:21 -0000 Subject: [Bro-Dev] #778: topic/dnthayer/ftp-ipv6 In-Reply-To: <050.50e0df68adc54ee6d3e3d9db13052fff@tracker.bro-ids.org> References: <050.50e0df68adc54ee6d3e3d9db13052fff@tracker.bro-ids.org> Message-ID: <065.3a55f2f3e455d028405478e581b09684@tracker.bro-ids.org> #778: topic/dnthayer/ftp-ipv6 ----------------------------+------------------------ Reporter: dnthayer | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [4ef8607e60230a68ce2ed4d4ee361bff89be60f6/bro]: {{{ #!CommitTicketReference repository="bro" revision="4ef8607e60230a68ce2ed4d4ee361bff89be60f6" Merge remote-tracking branch 'origin/topic/dnthayer/ftp-ipv6' * origin/topic/dnthayer/ftp-ipv6: Add test case for FTP over IPv4 Fix IPv6 URLs Add a test for FTP over IPv6 Update FTP EPSV response processing for IPv6 Fix parsing of FTP EPRT command and EPSV response Conflicts: src/bro.bif Closes #778. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Fri Feb 24 18:02:01 2012 From: seth at icir.org (Seth Hall) Date: Fri, 24 Feb 2012 21:02:01 -0500 Subject: [Bro-Dev] Sorting test output? In-Reply-To: <20120225000247.GB90478@icir.org> References: <20120225000247.GB90478@icir.org> Message-ID: <8DE15BAB-94C0-40E1-819E-C02C41FC9778@icir.org> On Feb 24, 2012, at 7:02 PM, Robin Sommer wrote: > I keep getting bogus test diffs for a number of tests because of line > order changes. I'm wondering if we could just generally sort all > output before diffing. Does anybody see a problem with that? It seems alright to me. Do you have any clue what could be causing the order changes in the lines? It seems like we have some code paths that are still somewhat non-deterministic. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Fri Feb 24 18:30:57 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 25 Feb 2012 02:30:57 -0000 Subject: [Bro-Dev] #785: SMB processid field In-Reply-To: <055.6069bd68783308f444cbf30d7b311792@tracker.bro-ids.org> References: <055.6069bd68783308f444cbf30d7b311792@tracker.bro-ids.org> Message-ID: <070.b1f30abead28aaf794c294b3c3e5cd69@tracker.bro-ids.org> #785: SMB processid field ----------------------------+------------------------ Reporter: JulienSentier | Owner: seth Type: Patch | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Invalid | Keywords: smb ----------------------------+------------------------ Changes (by seth): * status: assigned => closed * resolution: => Invalid Comment: This patch isn't actually correct. The uint16 it's extracting is the high 16 bits of the 32bit process id. It's split because they made the process id a 32bit value after defining this message and they used the formerly reserved bytes to get the other 16bits. It's already fixed in the topic/seth/smb-smb2-work branch along with a lot of other issues. I'm going to close the ticket, but Julien, let me know if you have specific goals with SMB+SMB2. I'll make sure and commit my current work to the branch really soon. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 18:45:07 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 25 Feb 2012 02:45:07 -0000 Subject: [Bro-Dev] #785: SMB processid field In-Reply-To: <055.6069bd68783308f444cbf30d7b311792@tracker.bro-ids.org> References: <055.6069bd68783308f444cbf30d7b311792@tracker.bro-ids.org> Message-ID: <070.c6f0b5e33b839697c169ae72038be4b5@tracker.bro-ids.org> #785: SMB processid field ----------------------------+------------------------ Reporter: JulienSentier | Owner: seth Type: Patch | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Invalid | Keywords: smb ----------------------------+------------------------ Comment (by seth): CVE-2009-3103 also appears to be a vulnerability in SMB2, is that the intended CVE? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Feb 24 18:47:26 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 25 Feb 2012 02:47:26 -0000 Subject: [Bro-Dev] #453: Connection compressor doesn't handle IPv6 In-Reply-To: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> References: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> Message-ID: <062.5718c6b4bd0554e726e188de9f5f229c@tracker.bro-ids.org> #453: Connection compressor doesn't handle IPv6 ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------+------------------------ Comment (by seth): > But given that we don't fully understand that and that we've already > deactivated it, I'm in favor of removing as otherwise it'll be just > more code not being tested and maintained (in particualr with the IPv6 > changes now). We can always bring it back later if we really want to, > it's all there somewhere in the repository. True, let's do it. :) -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Feb 25 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 25 Feb 2012 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202250800.q1P803sj031587@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Broccoli | 448 [1] | robin | jsiwek | Normal | Broccoli does not support IPv6 addresses [1] #448: http://tracker.bro-ids.org/bro/ticket/448 From noreply at bro-ids.org Mon Feb 27 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 27 Feb 2012 00:00:04 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201202270800.q1R804wg016007@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Broccoli | 448 [1] | robin | jsiwek | Normal | Broccoli does not support IPv6 addresses [1] #448: http://tracker.bro-ids.org/bro/ticket/448 From bro at tracker.bro-ids.org Mon Feb 27 01:31:24 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 27 Feb 2012 09:31:24 -0000 Subject: [Bro-Dev] #789: An example of a BiF : computing Conficker P2P ports In-Reply-To: <055.f5ce34b401942705c08aa57548263d6f@tracker.bro-ids.org> References: <055.f5ce34b401942705c08aa57548263d6f@tracker.bro-ids.org> Message-ID: <070.913e3472519e09ebb1b547f729eea682@tracker.bro-ids.org> #789: An example of a BiF : computing Conficker P2P ports ----------------------------+------------------------ Reporter: JulienSentier | Owner: Type: Patch | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by JulienSentier): Yes, I mean it mainly as a demonstration of writing a bif. It is not to be applied in bro main distribution. I thought it could be useful to you though. You can include it in a documentation if you wish. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 01:35:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 27 Feb 2012 09:35:02 -0000 Subject: [Bro-Dev] #786: Minor bugfixes In-Reply-To: <055.31e8fc347475856631bd8ecd15bf1b98@tracker.bro-ids.org> References: <055.31e8fc347475856631bd8ecd15bf1b98@tracker.bro-ids.org> Message-ID: <070.32d45b08be6943d3c0827c8bc8173899@tracker.bro-ids.org> #786: Minor bugfixes ----------------------------+------------------------ Reporter: JulienSentier | Owner: Type: Patch | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by JulienSentier): These were found with a static analysis tool. This is dead code because the variable line is not used after being changed on line 353. Maybe, this line should be useful and a loop has be forgotten around it. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 02:00:31 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 27 Feb 2012 10:00:31 -0000 Subject: [Bro-Dev] #790: Event connection_rejected with parameter is_orig In-Reply-To: <055.b70f73fdaff19bad90b76acf34271c1d@tracker.bro-ids.org> References: <055.b70f73fdaff19bad90b76acf34271c1d@tracker.bro-ids.org> Message-ID: <070.ba7d8d2f192004ebb6f5e8ff3df22de0@tracker.bro-ids.org> #790: Event connection_rejected with parameter is_orig ----------------------------+------------------------ Reporter: JulienSentier | Owner: Type: Patch | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by JulienSentier): I had assumed first that this event meant that a connection was closed with a RST packet, either from client or server. So this patch may make no sense. Still, there is this case where the event is raised with is_orig == true. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 09:37:58 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 27 Feb 2012 17:37:58 -0000 Subject: [Bro-Dev] #559: Remove conn compressor In-Reply-To: <047.44f48ecee506dc9ee47f21a82ca03af5@tracker.bro-ids.org> References: <047.44f48ecee506dc9ee47f21a82ca03af5@tracker.bro-ids.org> Message-ID: <062.90633cf5a3289497cbea15648bfee1e4@tracker.bro-ids.org> #559: Remove conn compressor ---------------------+------------------------ Reporter: robin | Owner: robin Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: beta ---------------------+------------------------ Comment (by jsiwek): In [e07470c7f137fbcca4ffd578266f3fd25fefaa26/bro]: {{{ #!CommitTicketReference repository="bro" revision="e07470c7f137fbcca4ffd578266f3fd25fefaa26" Remove connection compressor (addresses #559). }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 09:40:12 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 27 Feb 2012 17:40:12 -0000 Subject: [Bro-Dev] #559: Remove conn compressor In-Reply-To: <047.44f48ecee506dc9ee47f21a82ca03af5@tracker.bro-ids.org> References: <047.44f48ecee506dc9ee47f21a82ca03af5@tracker.bro-ids.org> Message-ID: <062.b418a11fb6887e920e1343c1a57bef26@tracker.bro-ids.org> #559: Remove conn compressor ----------------------------+------------------------ Reporter: robin | Owner: robin Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * keywords: beta => * type: Task => Merge Request Comment: `topic/jsiwek/remove-conn-compressor` is ready for merge. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 09:42:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 27 Feb 2012 17:42:09 -0000 Subject: [Bro-Dev] #453: Connection compressor doesn't handle IPv6 In-Reply-To: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> References: <047.842c6855ebae98e521587ca01b1978b0@tracker.bro-ids.org> Message-ID: <062.7cd066700b390f82b0b1cfd41d954d88@tracker.bro-ids.org> #453: Connection compressor doesn't handle IPv6 -----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: Rejected | Keywords: ipv6 -----------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Rejected Comment: See #559 for the connection compressor removal followup. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 10:29:13 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 27 Feb 2012 18:29:13 -0000 Subject: [Bro-Dev] #532: IP_Hdr class constructor possible problem In-Reply-To: <048.0fe3b43b12692907fc98a90a54e00944@tracker.bro-ids.org> References: <048.0fe3b43b12692907fc98a90a54e00944@tracker.bro-ids.org> Message-ID: <063.8006cbdebf2c548946d42f4b2d2c8436@tracker.bro-ids.org> #532: IP_Hdr class constructor possible problem ----------------------+------------------------ Reporter: gregor | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: IPv6 ----------------------+------------------------ Comment (by jsiwek): In [dfad686d7c1b819dc0677807f47cd8ba018e4194/bro]: {{{ #!CommitTicketReference repository="bro" revision="dfad686d7c1b819dc0677807f47cd8ba018e4194" Refactor IP_Hdr class ctors (addresses #532). They now take an explicit flag argument toggling whether the other pointer argument needs to be released on destruction. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 10:30:25 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 27 Feb 2012 18:30:25 -0000 Subject: [Bro-Dev] #532: IP_Hdr class constructor possible problem In-Reply-To: <048.0fe3b43b12692907fc98a90a54e00944@tracker.bro-ids.org> References: <048.0fe3b43b12692907fc98a90a54e00944@tracker.bro-ids.org> Message-ID: <063.df0e171715a43c239ac2885b645b7c11@tracker.bro-ids.org> #532: IP_Hdr class constructor possible problem ----------------------------+------------------------ Reporter: gregor | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: IPv6 ----------------------------+------------------------ Changes (by jsiwek): * type: Problem => Merge Request Comment: In `topic/jsiwek/iphdr-ctor` -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Mon Feb 27 11:43:56 2012 From: seth at icir.org (Seth Hall) Date: Mon, 27 Feb 2012 14:43:56 -0500 Subject: [Bro-Dev] persistent storage location Message-ID: The input framework is about to shake up some assumptions that we make in the logging framework now. The assumption now is that our logs are dead and only write once then ignore. The input framework makes it possible to use logs as a persistent storage mechanism but in my opinion much better than the old &persistent attribute because it gives us a way to provide an interface between the storage mechanism (database, log file, etc) and the way the data is stored in Bro. It gets around the problems I've had with iterative development of Bro scripts and safety of maintaining state which led me to avoid &persistent like the plague. I don't like the model of keeping persistent state (that's how I'll refer to files/databases used with the input framework and logging framework) in the CWD in a hidden .state directory either. Ultimately that ends up putting persistent state in the spool/ directory when using broctl which seems very wrong and unsafe to me since "spool" implies log spooling intended for eventual rotation. I propose we add another field to logging filters which indicates what "type" of log is being written. The default type could be "LOG" which would do the normal rotation and write to the normal logging location (whatever that means for the plugin being used for writing). Optionally we could use the "PERSISTENT_STATE" type (better names?) which would store to whatever output plugin is configured for the filter in a more appropriate location and not do the normal file rotation and other log maintenance. Ultimately being able to store persistent state with a script level defined interface on how to write to the store and read from it using the logging framework and input framework we could pull off a lot of stuff that is now either difficult or impossible. Thoughts on changes to the logging framework to fit this model better? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Mon Feb 27 13:25:04 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 27 Feb 2012 21:25:04 -0000 Subject: [Bro-Dev] #522: Event to report non TCP/UDP/ICMP packets In-Reply-To: <048.9578ab814d281f028dba6ef156f882b0@tracker.bro-ids.org> References: <048.9578ab814d281f028dba6ef156f882b0@tracker.bro-ids.org> Message-ID: <063.64b5714d4f0eb79ec70b3221bee8ba0d@tracker.bro-ids.org> #522: Event to report non TCP/UDP/ICMP packets ----------------------+------------------------ Reporter: gregor | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: IPv6 ----------------------+------------------------ Changes (by jsiwek): * owner: => jsiwek * status: new => assigned -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 13:25:16 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 27 Feb 2012 21:25:16 -0000 Subject: [Bro-Dev] #531: Handle IPv6 protocol chains In-Reply-To: <048.a1036225e7e49822f38fb91c4b12ccff@tracker.bro-ids.org> References: <048.a1036225e7e49822f38fb91c4b12ccff@tracker.bro-ids.org> Message-ID: <063.c6b399d66e3067c136e6708ca7ed167c@tracker.bro-ids.org> #531: Handle IPv6 protocol chains ----------------------+------------------------ Reporter: gregor | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: IPv6 ----------------------+------------------------ Changes (by jsiwek): * owner: => jsiwek * status: new => assigned -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Mon Feb 27 13:52:05 2012 From: seth at icir.org (Seth Hall) Date: Mon, 27 Feb 2012 16:52:05 -0500 Subject: [Bro-Dev] bro-2.1 IPv6, headers, evasion and other fun things In-Reply-To: <20120223192122.GA14682@yaksha.lbl.gov> References: <20120223192122.GA14682@yaksha.lbl.gov> Message-ID: <363A1EA0-530C-47F7-92C2-B7EA5F4DEA6D@icir.org> On Feb 23, 2012, at 2:21 PM, Aashish Sharma wrote: > 1) Evasion attack: Routing header can result in bro looking only As far as I know, this extension has effectively been deprecated. Once people realized that it was bad for all routers to obey this header all of the router vendors disabled handling of it by default. > 2) Evasion attack using Extension header size: could be 128K per > header, and RFC doesn't dictate the sequence of the headers so these can > be out of order. Bro needs to address the problem of parsing and > processing these extension headers, taking in to account the full sized > extension headers, else miscreants might put malicious contents at the > end of these headers and evade the detection. Hm, not sure I understand this. > 3) Mobility header issue: > An Application of such attack is subverting geoip blocking - example > youtube not allowing content viewing for IP's in Canada etc or avoid > roaming charges when using a mobile phone. I don't think we can really worry about subverting geoip blocking as a concern since there are already a tons of ways to avoid it. It works when it works, otherwise it doesn't. > 4) IDS connection caching issue: Since IPv6 doesn't support > fragmentation, a Sender has to keep a copy of the packet in the IP stack > because application doesn?t know if routers are going to fragment or > not and sender has to account for a possible ICMP error with frag bit > set. IDS has to be able to account for such situations and > re-transmissions. Hm. IPv6 does support fragmentation but routers won't perform the fragmentation, it has to be done by the end point after doing PMTU with ICMPv6 (which we are going to support). Could you explain more about what you meant? > 5) Neighbor discovery attack: ICMP6 can broadcast address for someone > else. It assumes that network is secure. Also given ICMP6 messages are > routable where as with ARP this was not a problem because ARP is/was > local to a subnet only. We'll be able to detect this with the ICMPv6 analyzer. I suppose there could be some issue with this if people are monitoring outside of their router and we start doing scripts that do things with neighbor discovery packets. With the interest in internal network monitoring it wouldn't surprise me if eventually we get to the point where the location of each monitoring node in a cluster is configured so that we can deal with packets that are internal or external to the network since something could conceivably be spoofed on the outside of your network (your address space as the source for an externally sourced packet). > 6) Address Priority issues: It is possible that part of attack uses a > link local address and other part uses the site address. BRO needs to > maintain a state which can account for all the IP's assigned and used > by the host to assemble a connection/attack etc. You should only be able to see link local addresses if you are monitoring within a broadcast domain but even in that case I expect that over time people will figure out how to start profiling hosts and compiling lists of addresses that are all used by the same host. > 7) IPv6 allows multiple routers allocate multiple address - one from > each router. Can we flag on rogue router broadcasts or new address > allocation other then the ones which are predefined in the networks.cfg That should just be a script that needs to be written once we integrate the ICMPv6 analyzer. > Crome can use v6 while safari v4. How do we know both of > these connections are part of the same attack ? Could give a more concrete scenario that you are thinking of? > 9) DNS resolution - Any IPv6 host takes upon itself to resolve DNS for > any query which it sees. So if an attacker chooses to use broadcast > address ff02::1 for DNS query resolution all hosts are going to resolve. Not sure I follow here. Presumably only hosts providing recursion will resolve. > 10) Collusion attack - because of the vastness of IPv6 address space, it > is possible for two hosts to use a new IP address for every packet to > communicate between them. This is mostly a script level issue in my opinion. We would have to watch for sudden increases in address utilization for internal subnets. > 11) IPv6 also brings isolation problem That doesn't sound like a Bro issue (although it's going to be a huge pain in the ass for a lot of teams I'm sure). > 12) Offcourse Teredo which encapsulates protocol and provides globally > reachable IPv6 address - enabled by win7 and win8.rc2 by default but > only works when ipv4 address is not available. We have a decapsulator and are discussing how to fit it into the analysis model. >From the Bro perspective, I don't see anything here that is a huge obstacle. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Mon Feb 27 14:05:01 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 27 Feb 2012 14:05:01 -0800 Subject: [Bro-Dev] persistent storage location In-Reply-To: References: Message-ID: <20120227220500.GG67961@icir.org> On Mon, Feb 27, 2012 at 14:43 -0500, you wrote: > I propose we add another field to logging filters which indicates what > "type" of log is being written. Yeah, I agree we need some different model here. What I'm not sure is that hardcoding use-cases (like LOG and PERSISTENT_STATE) is the best way to go. How far would it get us if we could just specify destination paths outside of the spool directory somehow? (We can already disable rotation etc. with the current filters.). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Mon Feb 27 21:35:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 28 Feb 2012 05:35:06 -0000 Subject: [Bro-Dev] #532: IP_Hdr class constructor possible problem In-Reply-To: <048.0fe3b43b12692907fc98a90a54e00944@tracker.bro-ids.org> References: <048.0fe3b43b12692907fc98a90a54e00944@tracker.bro-ids.org> Message-ID: <063.1545c5e2593f80ea8033fe9e1f2f7570@tracker.bro-ids.org> #532: IP_Hdr class constructor possible problem ----------------------------+------------------------ Reporter: gregor | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: IPv6 ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [885647686bba9c6d297597b93d2166a68549a0a5/bro]: {{{ #!CommitTicketReference repository="bro" revision="885647686bba9c6d297597b93d2166a68549a0a5" Merge remote-tracking branch 'origin/topic/jsiwek/iphdr-ctor' * origin/topic/jsiwek/iphdr-ctor: Refactor IP_Hdr class ctors (addresses #532). Closes #532. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 21:35:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 28 Feb 2012 05:35:06 -0000 Subject: [Bro-Dev] #559: Remove conn compressor In-Reply-To: <047.44f48ecee506dc9ee47f21a82ca03af5@tracker.bro-ids.org> References: <047.44f48ecee506dc9ee47f21a82ca03af5@tracker.bro-ids.org> Message-ID: <062.c7de982256460dad164aa4318a8629e7@tracker.bro-ids.org> #559: Remove conn compressor ----------------------------+------------------------ Reporter: robin | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => fixed Comment: In [59773db8a99575b1708b9c533bc052c9ec73a7c2/bro]: {{{ #!CommitTicketReference repository="bro" revision="59773db8a99575b1708b9c533bc052c9ec73a7c2" Merge remote-tracking branch 'origin/topic/jsiwek/remove-conn-compressor' * origin/topic/jsiwek/remove-conn-compressor: Remove connection compressor (addresses #559). Conflicts: src/ConnCompressor.cc Closes #559. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 21:35:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 28 Feb 2012 05:35:06 -0000 Subject: [Bro-Dev] #448: Broccoli does not support IPv6 addresses In-Reply-To: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> References: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> Message-ID: <062.bf1bb2c708125f13e18b26a77d151234@tracker.bro-ids.org> #448: Broccoli does not support IPv6 addresses ----------------------------+---------------------- Reporter: robin | Owner: jsiwek Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: Resolution: | Keywords: ipv6 ----------------------------+---------------------- Comment (by robin): In [36d46efa68c2e9071f19a6106e4e72ba94ee7854/bro]: {{{ #!CommitTicketReference repository="bro" revision="36d46efa68c2e9071f19a6106e4e72ba94ee7854" Merge remote-tracking branch 'origin/topic/jsiwek/broccoli-ipv6' * origin/topic/jsiwek/broccoli-ipv6: Update/add tests for broccoli IPv6 addr/subnet support (addresses #448) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 21:35:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 28 Feb 2012 05:35:19 -0000 Subject: [Bro-Dev] #448: Broccoli does not support IPv6 addresses In-Reply-To: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> References: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> Message-ID: <062.7565e3635730dacca360bbf1f42b6b15@tracker.bro-ids.org> #448: Broccoli does not support IPv6 addresses ----------------------------+-------------------- Reporter: robin | Owner: jsiwek Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: Resolution: fixed | Keywords: ipv6 ----------------------------+-------------------- Changes (by robin): * status: assigned => closed * resolution: => fixed Comment: In [d6e36c95e0335f7cc081191c8612085bd12706f9/broccoli]: {{{ #!CommitTicketReference repository="broccoli" revision="d6e36c95e0335f7cc081191c8612085bd12706f9" Merge remote-tracking branch 'origin/topic/jsiwek/broccoli-ipv6' * origin/topic/jsiwek/broccoli-ipv6: Update broconn broccoli test for IPv6 addrs. Update broccoli library to handle IPv6 addrs/subnets (addresses #448) Closes #448. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 21:35:26 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 28 Feb 2012 05:35:26 -0000 Subject: [Bro-Dev] #448: Broccoli does not support IPv6 addresses In-Reply-To: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> References: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> Message-ID: <062.e120a430da6e4b805dd79317d7d4b275@tracker.bro-ids.org> #448: Broccoli does not support IPv6 addresses ----------------------------+-------------------- Reporter: robin | Owner: jsiwek Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: Resolution: fixed | Keywords: ipv6 ----------------------------+-------------------- Comment (by robin): In [e8d483c949a5bc54d90ac78077eb86f38005fc6d/broccoli-python]: {{{ #!CommitTicketReference repository="broccoli-python" revision="e8d483c949a5bc54d90ac78077eb86f38005fc6d" Merge remote-tracking branch 'origin/topic/jsiwek/broccoli-ipv6' * origin/topic/jsiwek/broccoli-ipv6: Update broccoli-python for IPv6 addr/subnet support (addresses #448) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 27 21:35:32 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 28 Feb 2012 05:35:32 -0000 Subject: [Bro-Dev] #448: Broccoli does not support IPv6 addresses In-Reply-To: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> References: <047.24dcf4ee9ba479e14bf9dc723cf7b498@tracker.bro-ids.org> Message-ID: <062.83763e9b4c0b6bd9dd078070172f6ff7@tracker.bro-ids.org> #448: Broccoli does not support IPv6 addresses ----------------------------+-------------------- Reporter: robin | Owner: jsiwek Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: Resolution: fixed | Keywords: ipv6 ----------------------------+-------------------- Comment (by robin): In [f0a7f425ba6a9b83cab590b2a93bfae232afa463/broccoli-ruby]: {{{ #!CommitTicketReference repository="broccoli-ruby" revision="f0a7f425ba6a9b83cab590b2a93bfae232afa463" Merge remote-tracking branch 'origin/topic/jsiwek/broccoli-ipv6' * origin/topic/jsiwek/broccoli-ipv6: Fix count/enum being treated same as addr. Update broccoli-ruby for IPv6 addr/subnet support (addresses #448) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From gc355804 at ohio.edu Tue Feb 28 07:24:27 2012 From: gc355804 at ohio.edu (G. Clark) Date: Tue, 28 Feb 2012 10:24:27 -0500 Subject: [Bro-Dev] persistent storage location In-Reply-To: References: Message-ID: <4F4CF1AB.4020800@ohio.edu> I would suggest that, rather than trying to attach this functionality to the logging framework, we wrap the input / output frameworks into a single unified set of script functions that handles generic K/V data I/O, e.g.: ds_create(datastore_name, datastore_key, datastore_value) ds_read(datastore_name, datastore_key) ds_update(datastore_name, datastore_key, datastore_value) ds_delete(datastore_name, datastore_key) Unfortunately, logging currently only does create, and input only does read, so we have update and delete to somehow work into the mix. For store operations, I believe those two operations are relatively important. Since not every log writer is suited for CRUD, it might make sense to support C & R on all formats, and only support U & D on those formats meant to be used for store operations (e.g. redis, memcache, whatever). Anyway, I'd imagine the logging framework would need support for U & D thrown in to really make this model work well. We'd also need to ensure that ds_update / ds_delete failed in a sane way for formats on which it was not supported. Just a thought. --Gilbert On 2/27/12 2:43 PM, Seth Hall wrote: > The input framework is about to shake up some assumptions that we make in the logging framework now. The assumption now is that our logs are dead and only write once then ignore. The input framework makes it possible to use logs as a persistent storage mechanism but in my opinion much better than the old&persistent attribute because it gives us a way to provide an interface between the storage mechanism (database, log file, etc) and the way the data is stored in Bro. It gets around the problems I've had with iterative development of Bro scripts and safety of maintaining state which led me to avoid&persistent like the plague. > > I don't like the model of keeping persistent state (that's how I'll refer to files/databases used with the input framework and logging framework) in the CWD in a hidden .state directory either. Ultimately that ends up putting persistent state in the spool/ directory when using broctl which seems very wrong and unsafe to me since "spool" implies log spooling intended for eventual rotation. > > I propose we add another field to logging filters which indicates what "type" of log is being written. The default type could be "LOG" which would do the normal rotation and write to the normal logging location (whatever that means for the plugin being used for writing). Optionally we could use the "PERSISTENT_STATE" type (better names?) which would store to whatever output plugin is configured for the filter in a more appropriate location and not do the normal file rotation and other log maintenance. > > Ultimately being able to store persistent state with a script level defined interface on how to write to the store and read from it using the logging framework and input framework we could pull off a lot of stuff that is now either difficult or impossible. > > Thoughts on changes to the logging framework to fit this model better? > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro-ids.org/ > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From seth at icir.org Tue Feb 28 07:43:43 2012 From: seth at icir.org (Seth Hall) Date: Tue, 28 Feb 2012 10:43:43 -0500 Subject: [Bro-Dev] persistent storage location In-Reply-To: <4F4CF1AB.4020800@ohio.edu> References: <4F4CF1AB.4020800@ohio.edu> Message-ID: <550D5A68-358C-4D98-98E1-C2E67068862F@icir.org> On Feb 28, 2012, at 10:24 AM, G. Clark wrote: > unified set of script functions that handles generic K/V data I/O, e.g.: I've been thinking about this a lot too and we may be getting to that point before long. > Unfortunately, logging currently only does create, and input only does > read, so we have update and delete to somehow work into the mix. I'm not really sure that update and delete can even fit into the model generally as it is. It may turn out that a future IO framework might consume the input framework but the logging framework would remain standalone although it would probably be able to reuse code internally from the IO framework. This may be a job for 2.2. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Wed Feb 29 01:16:17 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 29 Feb 2012 09:16:17 -0000 Subject: [Bro-Dev] #786: Minor bugfixes In-Reply-To: <055.31e8fc347475856631bd8ecd15bf1b98@tracker.bro-ids.org> References: <055.31e8fc347475856631bd8ecd15bf1b98@tracker.bro-ids.org> Message-ID: <070.bfbb2eb2368aeaac1eca69fcdb9d7f4a@tracker.bro-ids.org> #786: Minor bugfixes ----------------------------+------------------------ Reporter: JulienSentier | Owner: Type: Patch | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by JulienSentier): It seems to me that there is more dead code : The variables ssl_verify_certificates, ssl_store_certificates, ssl_store_cert_path are no longer used with the new ssl scripts. But they are still in NetVar.cc -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 29 06:42:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 29 Feb 2012 14:42:09 -0000 Subject: [Bro-Dev] #786: Minor bugfixes In-Reply-To: <055.31e8fc347475856631bd8ecd15bf1b98@tracker.bro-ids.org> References: <055.31e8fc347475856631bd8ecd15bf1b98@tracker.bro-ids.org> Message-ID: <070.dcafad5b0e7d66e355133e0ecb172b2f@tracker.bro-ids.org> #786: Minor bugfixes ----------------------------+------------------------ Reporter: JulienSentier | Owner: Type: Patch | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by seth): Robin, it looks like if don't remove that line from the SMTP analyzer that Julien pointed out, Bro segfaults on SMTP traffic. I recommend removing it. :) Also, Julien, I almost have a branch done that removes those unused variables you pointed out and some other remaining vestiges of the old SSL analyzers. Thanks! -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 29 06:48:49 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 29 Feb 2012 14:48:49 -0000 Subject: [Bro-Dev] #791: Cleaning up dead code from old SSL analyzers Message-ID: <046.8aed53d4d889994b6f18e135c925dc54@tracker.bro-ids.org> #791: Cleaning up dead code from old SSL analyzers ---------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch removes some crud that was still hanging around from the previous SSL analyzers. 'topic/seth/ssl-cleanup' is ready for merging. -- Ticket URL: Bro Tracker Bro Issue Tracker From gregor at icir.org Wed Feb 29 08:32:08 2012 From: gregor at icir.org (Gregor Maier) Date: Wed, 29 Feb 2012 08:32:08 -0800 Subject: [Bro-Dev] bro-2.1 IPv6, headers, evasion and other fun things In-Reply-To: <363A1EA0-530C-47F7-92C2-B7EA5F4DEA6D@icir.org> References: <20120223192122.GA14682@yaksha.lbl.gov> <363A1EA0-530C-47F7-92C2-B7EA5F4DEA6D@icir.org> Message-ID: <4F4E5308.2090609@icir.org> >> 4) IDS connection caching issue: Since IPv6 doesn't support >> fragmentation, a Sender has to keep a copy of the packet in the IP stack >> because application doesn?t know if routers are going to fragment or >> not and sender has to account for a possible ICMP error with frag bit >> set. IDS has to be able to account for such situations and >> re-transmissions. > > Hm. IPv6 does support fragmentation but routers won't perform the fragmentation, it has to be done by the end point after doing PMTU with ICMPv6 (which we are going to support). Could you explain more about what you meant? The stack doesn't have to cache the packet. Does it really do that? For TCP the stack can just rely on TCP retransmissions. Other IP packets are best effort anyways. And, as Seth pointed out, the sender should do PMTU first anyways. >> 5) Neighbor discovery attack: ICMP6 can broadcast address for someone >> else. It assumes that network is secure. Also given ICMP6 messages are >> routable where as with ARP this was not a problem because ARP is/was >> local to a subnet only. Neighbor discovery is limited to the the local subnet only as well. The multicast address used has link-local scope. Routers should never forward these packages. So it should be just as (in-)secure as ARP. OTOH, forged router advertisement message can become a problem, since it allows for an incredible easy way to redirect traffic. However, this too is a link-local problem. >> 6) Address Priority issues: It is possible that part of attack uses a >> link local address and other part uses the site address. BRO needs to >> maintain a state which can account for all the IP's assigned and used >> by the host to assemble a connection/attack etc. > > You should only be able to see link local addresses if you are monitoring within a broadcast domain but even in that case I expect that over time people will figure out how to start profiling hosts and compiling lists of addresses that are all used by the same host. Well, you can have multiple globally routable IPv6 addresses per host (cf. router advertisements). But in this case the 64 host bits uniquely identify a host even across prefixes (since they are based on the MAC address). (However, I can't recall how the privacy extension for creating the host part work -- they might change that) In any ways, this will only affect attacks that span multiple connections since you can't change the destination IP within a connection. cu Gregor From bro at tracker.bro-ids.org Wed Feb 29 08:34:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 29 Feb 2012 16:34:00 -0000 Subject: [Bro-Dev] #792: Signatures with Site::local_nets Message-ID: <055.14b45880c30ae19e160c331cfa04ef0e@tracker.bro-ids.org> #792: Signatures with Site::local_nets ---------------------------+-------------------- Reporter: JulienSentier | Type: Patch Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: ---------------------------+-------------------- Currently, the signature engine does not support variables with a module in their namespace. And now, local_nets has become Site::local_nets. This small patch allows to define signature with dst-ip == Site::local_nets Maybe it is not the best way to do it as I am not aware of all the consequences over Bro of this little change. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Feb 29 08:48:26 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 29 Feb 2012 08:48:26 -0800 Subject: [Bro-Dev] persistent storage location In-Reply-To: <550D5A68-358C-4D98-98E1-C2E67068862F@icir.org> References: <4F4CF1AB.4020800@ohio.edu> <550D5A68-358C-4D98-98E1-C2E67068862F@icir.org> Message-ID: <20120229164826.GM79009@icir.org> Talked with Seth about this yesterday: I don't really want to create an indendepent persistence framework in parallel but I believe that we could create a key/value interface on top of the current input/output framework, potentially even purely in script-land. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Wed Feb 29 17:28:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 01 Mar 2012 01:28:46 -0000 Subject: [Bro-Dev] #793: istate.broccoli-ipv6 fails on FreeBSD Message-ID: <047.06a7b6c1d5b8682c072c3076abd9a768@tracker.bro-ids.org> #793: istate.broccoli-ipv6 fails on FreeBSD ---------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ With current master. {{{ istate.broccoli-ipv6 ... failed % 'btest-diff broccoli/.stdout' failed unexpectedly (exit code 1) % cat .diag == File =============================== == Diff =============================== --- /tmp/test-diff.43124.broccoli..stdout.baseline.tmp 2012-03-01 00:52:02.000000000 +0000 +++ /tmp/test-diff.43124.broccoli..stdout.tmp 2012-03-01 00:52:02.000000000 +0000 @@ -1,6 +0,0 @@ -Connected to Bro instance at: localhost:47757 -Received bro_addr(1.2.3.4) -Received bro_subnet(10.0.0.0/16) -Received bro_addr(2607:f8b0:4009:802::1014) -Received bro_subnet(2607:f8b0::/32) -Terminating ======================================= }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Feb 29 17:54:21 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 29 Feb 2012 17:54:21 -0800 Subject: [Bro-Dev] Problems parallelizing btests Message-ID: <20120301015421.GH19963@icir.org> btest:topic/robin/parallel has a version of btest that can run tests in parallel. That works pretty well, except for two issues with Bro's standard tests: - The coverage analysis doesn't like running in parallel, it messes up the state file. Jon, do you think we could get that to work somehow? - The communication tests can't be parallelized because they use the same port. The btest in the branch supports groups of tests that are executed sequentially, which solves the problem. But unfortunately that comes at the expenses of losing much of the speed-up that parallelizing would otherwise be able to achieve (because the communication tests take the longest). I'm wondering if we could randomize the ports being used in some form. But not sure how that would look like. Other than these, it's actually pretty cool to run the tests in parallel. :) Btw, that branch also has a new option to rerun only tests that failed last time. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Wed Feb 29 18:21:05 2012 From: seth at icir.org (Seth Hall) Date: Wed, 29 Feb 2012 21:21:05 -0500 Subject: [Bro-Dev] Problems parallelizing btests In-Reply-To: <20120301015421.GH19963@icir.org> References: <20120301015421.GH19963@icir.org> Message-ID: <29E10600-E3D0-419E-A29D-34D5E986E26F@icir.org> On Feb 29, 2012, at 8:54 PM, Robin Sommer wrote: > I'm wondering if we could randomize the ports being used in some > form. But not sure how that would look like. How about we read in the port to use as an environment variable? Btest could just set that before running each test (maybe we could limit it to only set it for communication tests?). > Other than these, it's actually pretty cool to run the tests in > parallel. :) Looking forward to playing with it! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/