From noreply at bro-ids.org Thu Mar 1 00:02:27 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 1 Mar 2012 00:02:27 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203010802.q2182Q2S019669@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 791 [1] | seth | robin | Normal | Cleaning up dead code from old SSL analyzers > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 87ad77c | Seth Hall | 2012-02-29 | Standardized on the &default function for SSL constants. [2] [1] #791: http://tracker.bro-ids.org/bro/ticket/791 [2] fastpath: http://tracker.bro-ids.org/bro/changeset/87ad77cc100697659c72429386b5264ae67051dc/bro From slagell at illinois.edu Thu Mar 1 07:26:50 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Thu, 1 Mar 2012 15:26:50 +0000 Subject: [Bro-Dev] Problems parallelizing btests In-Reply-To: <29E10600-E3D0-419E-A29D-34D5E986E26F@icir.org> References: <20120301015421.GH19963@icir.org> <29E10600-E3D0-419E-A29D-34D5E986E26F@icir.org> Message-ID: <373773D8-5A52-423F-9CF4-F0A51C0534C6@illinois.edu> On Feb 29, 2012, at 8:21 PM, Seth Hall wrote: >> I'm wondering if we could randomize the ports being used in some >> form. But not sure how that would look like. > > How about we read in the port to use as an environment variable? Btest could just set that before running each test (maybe we could limit it to only set it for communication tests?). Do they really need to be random or just unique for each test? If the latter, maybe the port could be derived from the test names which themselves could be numbered. ------ Adam J. Slagell, CISO, CISSP Chief Information Security Officer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info 217.244.8965 "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From jsiwek at illinois.edu Thu Mar 1 08:07:58 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Thu, 1 Mar 2012 16:07:58 +0000 Subject: [Bro-Dev] Problems parallelizing btests In-Reply-To: <20120301015421.GH19963@icir.org> References: <20120301015421.GH19963@icir.org> Message-ID: > - The coverage analysis doesn't like running in parallel, it messes > up the state file. Jon, do you think we could get that to work > somehow? Yeah, what I'm thinking is to have Brofiler.cc pass BRO_PROFILER_FILE through mkstemp() instead of fopen() and then change that env. var. in the btest.cfg files to use some .XXXXX suffix so that each bro instance writes coverage state to a unique file. Then I've already got a script in testing/scripts/coverage-calc that glues coverage files together. I'll go ahead and try that real quick and commit if it works out. +Jon From robin at icir.org Thu Mar 1 08:18:00 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 1 Mar 2012 08:18:00 -0800 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/seth/smb-smb2-work: Checkpoint for SMB support. (ca51999) In-Reply-To: <201203010739.q217dNTS003834@bro-ids.icir.org> References: <201203010739.q217dNTS003834@bro-ids.icir.org> Message-ID: <20120301161800.GC51011@icir.org> On Wed, Feb 29, 2012 at 23:39 -0800, Seth Hall wrote: > Checkpoint for SMB support. Very cool stuff! Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Thu Mar 1 08:35:25 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 1 Mar 2012 08:35:25 -0800 Subject: [Bro-Dev] Compiler error on MacOS Message-ID: <20120301163524.GD51011@icir.org> All of a sudden my Mac says this when compiling master (Linux and FreeBSD work fine): In file included from /Users/robin/bro/master/build/src/netflow_pac.cc:3: /Users/robin/bro/master/build/src/netflow_pac.h:13: error: expected initializer before ?*? token /Users/robin/bro/master/build/src/netflow_pac.cc: In member function ?bool binpac::NetFlow::NetFlow_Flow::deliver_v5_header(binpac::uint16, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint8, binpac::uint8, binpac::uint16)?: /Users/robin/bro/master/build/src/netflow_pac.cc:158: error: ?mgr? was not declared in this scope /Users/robin/bro/master/build/src/netflow_pac.cc: In member function ?bool binpac::NetFlow::NetFlow_Flow::deliver_v5_record(binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint16, binpac::uint16, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint16, binpac::uint16, binpac::uint8, binpac::uint8, binpac::uint8, binpac::uint16, binpac::uint16, binpac::uint8, binpac::uint8)?: /Users/robin/bro/master/build/src/netflow_pac.cc:225: error: ?mgr? was not declared in this scope I'm pretty sure that must be something local to my setup as others here are using Mac too. Does anybody happen to have an idea what could be causing this? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Thu Mar 1 08:36:16 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 01 Mar 2012 16:36:16 -0000 Subject: [Bro-Dev] #791: Cleaning up dead code from old SSL analyzers In-Reply-To: <046.8aed53d4d889994b6f18e135c925dc54@tracker.bro-ids.org> References: <046.8aed53d4d889994b6f18e135c925dc54@tracker.bro-ids.org> Message-ID: <061.f15ae18645fa5e50dc4e0e4509f93fd2@tracker.bro-ids.org> #791: Cleaning up dead code from old SSL analyzers -----------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Thu Mar 1 08:44:46 2012 From: seth at icir.org (Seth Hall) Date: Thu, 1 Mar 2012 11:44:46 -0500 Subject: [Bro-Dev] Compiler error on MacOS In-Reply-To: <20120301163524.GD51011@icir.org> References: <20120301163524.GD51011@icir.org> Message-ID: On Mar 1, 2012, at 11:35 AM, Robin Sommer wrote: > I'm pretty sure that must be something local to my setup as others > here are using Mac too. Does anybody happen to have an idea what could > be causing this? You don't happen to have my broken 64bit binpac branch checked out do you? I think I had that error when I was working on it. Speaking of that I need to dig back into that and fix it. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Thu Mar 1 08:46:14 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 1 Mar 2012 08:46:14 -0800 Subject: [Bro-Dev] Problems parallelizing btests In-Reply-To: <29E10600-E3D0-419E-A29D-34D5E986E26F@icir.org> References: <20120301015421.GH19963@icir.org> <20120301015421.GH19963@icir.org> <29E10600-E3D0-419E-A29D-34D5E986E26F@icir.org> <373773D8-5A52-423F-9CF4-F0A51C0534C6@illinois.edu> <20120301015421.GH19963@icir.org> <29E10600-E3D0-419E-A29D-34D5E986E26F@icir.org> Message-ID: <20120301164614.GF51011@icir.org> On Wed, Feb 29, 2012 at 21:21 -0500, Seth wrote: > How about we read in the port to use as an environment variable? > Btest could just set that before running each test On Thu, Mar 01, 2012 at 15:26 +0000, Adam wrote: > Do they really need to be random or just unique for each test? If the > latter, maybe the port could be derived from the test names which > themselves could be numbered. Just unique is indeed fine. How about a combination of the two: broctl numbers all tests internally and passes the current test's number on via an environment variable. The test can then derive a port from that, and it's a bit more general in that we might end up using the number for other purposes too. The remaining piece is then using the environment variable to configure the port for Bro and Broccoli. Would be nice if we could do that centrally somehow, not manually in each test needing it. On Thu, Mar 01, 2012 at 16:07 +0000, Jon wrote: > files together. I'll go ahead and try that real quick and commit if > it works out. Sounds good, thanks! Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From jsiwek at illinois.edu Thu Mar 1 08:47:35 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Thu, 1 Mar 2012 16:47:35 +0000 Subject: [Bro-Dev] Compiler error on MacOS In-Reply-To: <20120301163524.GD51011@icir.org> References: <20120301163524.GD51011@icir.org> Message-ID: <46A04CA8-B585-4AA7-8499-992CF64A1A81@illinois.edu> > All of a sudden my Mac says this when compiling master (Linux and > FreeBSD work fine): > > In file included from /Users/robin/bro/master/build/src/netflow_pac.cc:3: > /Users/robin/bro/master/build/src/netflow_pac.h:13: error: expected initializer before ?*? token > /Users/robin/bro/master/build/src/netflow_pac.cc: In member function ?bool binpac::NetFlow::NetFlow_Flow::deliver_v5_header(binpac::uint16, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint8, binpac::uint8, binpac::uint16)?: > /Users/robin/bro/master/build/src/netflow_pac.cc:158: error: ?mgr? was not declared in this scope > /Users/robin/bro/master/build/src/netflow_pac.cc: In member function ?bool binpac::NetFlow::NetFlow_Flow::deliver_v5_record(binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint16, binpac::uint16, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint16, binpac::uint16, binpac::uint8, binpac::uint8, binpac::uint8, binpac::uint16, binpac::uint16, binpac::uint8, binpac::uint8)?: > /Users/robin/bro/master/build/src/netflow_pac.cc:225: error: ?mgr? was not declared in this scope > > I'm pretty sure that must be something local to my setup as others > here are using Mac too. Does anybody happen to have an idea what could > be causing this? Looks similar to what happened in this thread: http://mailman.icsi.berkeley.edu/pipermail/bro/2011-November/005185.html +Jon From robin at icir.org Thu Mar 1 08:49:10 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 1 Mar 2012 08:49:10 -0800 Subject: [Bro-Dev] Compiler error on MacOS In-Reply-To: References: <20120301163524.GD51011@icir.org> Message-ID: <20120301164910.GG51011@icir.org> On Thu, Mar 01, 2012 at 11:44 -0500, you wrote: > You don't happen to have my broken 64bit binpac branch checked out do No, don't think so. It's a clean master (even did a fresh clone to be sure) and I don't think it picks up another binpac. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From slagell at illinois.edu Thu Mar 1 08:52:25 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Thu, 1 Mar 2012 16:52:25 +0000 Subject: [Bro-Dev] [VIRUS PM SCAN ERROR] Compiler error on MacOS In-Reply-To: <20120301163524.GD51011@icir.org> References: <20120301163524.GD51011@icir.org> Message-ID: I haven't tried a build since the new xCode. But that totally screwed up MacPorts. On Mar 1, 2012, at 10:35 AM, Robin Sommer wrote: > All of a sudden my Mac says this when compiling master (Linux and > FreeBSD work fine): > > In file included from /Users/robin/bro/master/build/src/netflow_pac.cc:3: > /Users/robin/bro/master/build/src/netflow_pac.h:13: error: expected initializer before ?*? token > /Users/robin/bro/master/build/src/netflow_pac.cc: In member function ?bool binpac::NetFlow::NetFlow_Flow::deliver_v5_header(binpac::uint16, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint8, binpac::uint8, binpac::uint16)?: > /Users/robin/bro/master/build/src/netflow_pac.cc:158: error: ?mgr? was not declared in this scope > /Users/robin/bro/master/build/src/netflow_pac.cc: In member function ?bool binpac::NetFlow::NetFlow_Flow::deliver_v5_record(binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint16, binpac::uint16, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint32, binpac::uint16, binpac::uint16, binpac::uint8, binpac::uint8, binpac::uint8, binpac::uint16, binpac::uint16, binpac::uint8, binpac::uint8)?: > /Users/robin/bro/master/build/src/netflow_pac.cc:225: error: ?mgr? was not declared in this scope > > I'm pretty sure that must be something local to my setup as others > here are using Mac too. Does anybody happen to have an idea what could > be causing this? > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > ------ Adam J. Slagell, CISO, CISSP Chief Information Security Officer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info 217.244.8965 "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From robin at icir.org Thu Mar 1 08:53:17 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 1 Mar 2012 08:53:17 -0800 Subject: [Bro-Dev] Compiler error on MacOS In-Reply-To: <46A04CA8-B585-4AA7-8499-992CF64A1A81@illinois.edu> References: <20120301163524.GD51011@icir.org> <46A04CA8-B585-4AA7-8499-992CF64A1A81@illinois.edu> Message-ID: <20120301165316.GH51011@icir.org> On Thu, Mar 01, 2012 at 16:47 +0000, you wrote: > http://mailman.icsi.berkeley.edu/pipermail/bro/2011-November/005185.html Ah, interesting. Except that the suggested fix seems to be applied to master already. But that may get me on the right track, I'll look at it more closely later. Thanks, Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Thu Mar 1 11:29:04 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 01 Mar 2012 19:29:04 -0000 Subject: [Bro-Dev] #794: topic/jsiwek/coverage-tweaks Message-ID: <048.fc6ddb0752fbc5bb8867d98fcc506f0c@tracker.bro-ids.org> #794: topic/jsiwek/coverage-tweaks ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch is in bro, bro-testing, and bro-testing-private repos. It allows BRO_PROFILER_FILE to use a template filename to pass to mkstemp so the test suites can now use it to have each Bro instance write coverage information to a unique state file. There's also Makefile rearrangements/cleanup. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Thu Mar 1 16:33:10 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 1 Mar 2012 16:33:10 -0800 Subject: [Bro-Dev] Testing threaded logging Message-ID: <20120302003310.GJ76119@icir.org> I'm looking for testers for the new threaded logging code in topic/robin/log-threads. It's working fine for me now but a few more eyes (and OSs) would be good. Ideally, you won't see any difference to current master except that Bro now spawns a number of threads (which show up in top if you press H). Thanks, Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Fri Mar 2 00:00:01 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 2 Mar 2012 00:00:01 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203020800.q22801Aq018187@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 794 [1] | jsiwek | | Normal | topic/jsiwek/coverage-tweaks [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 87ad77c | Seth Hall | 2012-02-29 | Standardized on the &default function for SSL constants. [3] [1] #794: http://tracker.bro-ids.org/bro/ticket/794 [2] coverage-tweaks: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/coverage-tweaks [3] fastpath: http://tracker.bro-ids.org/bro/changeset/87ad77cc100697659c72429386b5264ae67051dc/bro From bro at tracker.bro-ids.org Fri Mar 2 08:15:43 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 02 Mar 2012 16:15:43 -0000 Subject: [Bro-Dev] #768: Inline monitoring of modified scripts. In-Reply-To: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> References: <046.1ca40d01be659875b87df303d68539c8@tracker.bro-ids.org> Message-ID: <061.571b24163695e34f6bef7f676dc6f7c5@tracker.bro-ids.org> #768: Inline monitoring of modified scripts. -------------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by justin): Additionally, it would be really great if broctl could somehow tell if a full restart is needed, or just an update. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Fri Mar 2 09:11:37 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 2 Mar 2012 09:11:37 -0800 Subject: [Bro-Dev] Deleting branches In-Reply-To: <20120130233518.GF29720@icir.org> References: <20120130233518.GF29720@icir.org> Message-ID: <20120302171137.GE720@icir.org> On Mon, Jan 30, 2012 at 15:35 -0800, I wrote: > For the first run, however, I'd like to double-check that it would be > doing the right thing. Below is the list of deletes, please shout if > there's any branch in there that you do *not* want to be deleted. Ok, I threatened deleting tons of branches but never followed through. But now for real: below's an updated list and I'll be deleting these soon if nobody complains. Robin %%%%%% . push origin :topic/appleman/unittests push origin :topic/bif_cleanup push origin :topic/dist-cleanup push origin :topic/dnthayer/dns-ipv6 push origin :topic/dnthayer/ftp-ipv6 push origin :topic/gilbert/ascii-header push origin :topic/gilbert/rand-pool push origin :topic/jsiwek/ascii-log-rotate-fix push origin :topic/jsiwek/bro-log-suffix push origin :topic/jsiwek/broccoli-ipv6 push origin :topic/jsiwek/brofiler push origin :topic/jsiwek/broxygen-cleanup push origin :topic/jsiwek/comphash-func-determinism2 push origin :topic/jsiwek/compiler-warnings push origin :topic/jsiwek/complex-record-indices push origin :topic/jsiwek/custom-b64-alphabet push origin :topic/jsiwek/debug-flags push origin :topic/jsiwek/doc-framework push origin :topic/jsiwek/dynamic-example-install push origin :topic/jsiwek/filter-rotation push origin :topic/jsiwek/findpcap_pfring push origin :topic/jsiwek/fix-dns-double-free push origin :topic/jsiwek/http-1xx-replies push origin :topic/jsiwek/http-multipart-byteranges push origin :topic/jsiwek/index-opt-record push origin :topic/jsiwek/ipaddr-refactoring push origin :topic/jsiwek/iphdr-ctor push origin :topic/jsiwek/irc-orig push origin :topic/jsiwek/leak-fixes push origin :topic/jsiwek/local-node-order push origin :topic/jsiwek/local-table-init push origin :topic/jsiwek/mask_addr_rval push origin :topic/jsiwek/misc-doc-fixes push origin :topic/jsiwek/nested-record-coerce-fix push origin :topic/jsiwek/openbsd-support push origin :topic/jsiwek/parallel-make-recursion push origin :topic/jsiwek/path-func-record-demote push origin :topic/jsiwek/pybroccoli-fixes push origin :topic/jsiwek/pybroccoli-float-repr push origin :topic/jsiwek/raw_output push origin :topic/jsiwek/record-coerce-default push origin :topic/jsiwek/remote-log-peer push origin :topic/jsiwek/remove-conn-compressor push origin :topic/jsiwek/remove-refined-type push origin :topic/jsiwek/reporter-fatal-bif push origin :topic/jsiwek/require-libmagic-libz push origin :topic/jsiwek/ruby push origin :topic/jsiwek/sftp-pp push origin :topic/jsiwek/snaplen push origin :topic/jsiwek/update-restdoc-target push origin :topic/jsiwek/v6-dns-name-lookup push origin :topic/logging-framework push origin :topic/policy-scripts-new push origin :topic/robin/broccoli-connrec push origin :topic/robin/cleanup-active-mapping push origin :topic/robin/cleanup-dfa-cache push origin :topic/robin/comm-ssl push origin :topic/robin/conn-ids push origin :topic/robin/extend-records push origin :topic/robin/interpreter-exceptions push origin :topic/robin/logging-fix push origin :topic/robin/optional-fields push origin :topic/robin/parallel-btest push origin :topic/robin/record-table-default push origin :topic/robin/reporting push origin :topic/robin/rotation-pp push origin :topic/robin/v6-addr-merge push origin :topic/robin/work push origin :topic/script-load-changes push origin :topic/script-reference push origin :topic/seth/dns-updates push origin :topic/seth/notice-email-delay push origin :topic/seth/notice-suppression push origin :topic/seth/ssh-script-fix push origin :topic/seth/ssl-binpac push origin :topic/seth/ssl-cleanup push origin :topic/seth/ssl-improvements push origin :topic/seth/ssl-updates-for-2.0 push origin :topic/seth/syslog-analyzer push origin :topic/seth/weird-updates push origin :topic/v6-addr %%%%%% /home/robin/bro/master/aux/binpac push origin :topic/dist-cleanup push origin :topic/jsiwek/CMake-IDE-tweaks push origin :topic/jsiwek/cmake-rpath push origin :topic/jsiwek/debug-flags push origin :topic/jsiwek/parallel-make-recursion push origin :topic/robin/cleanup %%%%%% /home/robin/bro/master/aux/binpac/cmake push origin :topic/jsiwek/find-pythondev push origin :topic/jsiwek/openbsd-support %%%%%% /home/robin/bro/master/aux/bro-aux push origin :topic/dist-cleanup push origin :topic/jsiwek/cmake-rpath push origin :topic/jsiwek/compiler-warnings push origin :topic/jsiwek/debug-flags push origin :topic/jsiwek/findpcap_pfring push origin :topic/jsiwek/openbsd-support push origin :topic/jsiwek/parallel-make-recursion push origin :topic/mozilla-ca-list push origin :topic/robin/bro-cut push origin :topic/robin/cleanup %%%%%% /home/robin/bro/master/aux/bro-aux/cmake push origin :topic/jsiwek/find-pythondev push origin :topic/jsiwek/openbsd-support %%%%%% /home/robin/bro/master/aux/broccoli push origin :topic/broccoli-manual-rest push origin :topic/christian/broccoli-connrec push origin :topic/dist-cleanup push origin :topic/jsiwek/64bit-val-fix push origin :topic/jsiwek/CMake-IDE-tweaks push origin :topic/jsiwek/broccoli-ipv6 push origin :topic/jsiwek/cmake-rpath push origin :topic/jsiwek/compiler-warnings push origin :topic/jsiwek/debug-flags push origin :topic/jsiwek/dynamic-example-install push origin :topic/jsiwek/findpcap_pfring push origin :topic/jsiwek/istate-tests-update push origin :topic/jsiwek/openbsd-support push origin :topic/jsiwek/parallel-make-recursion push origin :topic/jsiwek/remove-refined-type push origin :topic/robin/cleanup %%%%%% /home/robin/bro/master/aux/broccoli/bindings/broccoli-python push origin :topic/dist-cleanup push origin :topic/jsiwek/broccoli-ipv6 push origin :topic/jsiwek/cmake-rpath push origin :topic/jsiwek/compiler-warnings push origin :topic/jsiwek/debug-flags push origin :topic/jsiwek/find-pythondev push origin :topic/jsiwek/istate-tests-update push origin :topic/jsiwek/pybroccoli-fixes push origin :topic/jsiwek/pybroccoli-float-repr push origin :topic/remove-tabs push origin :topic/robin/cleanup %%%%%% /home/robin/bro/master/aux/broccoli/bindings/broccoli-python/cmake push origin :topic/jsiwek/find-pythondev push origin :topic/jsiwek/openbsd-support %%%%%% /home/robin/bro/master/aux/broccoli/bindings/broccoli-ruby push origin :topic/jsiwek/broccoli-ipv6 push origin :topic/jsiwek/compiler-warnings push origin :topic/jsiwek/opt-ruby-bindings %%%%%% /home/robin/bro/master/aux/broccoli/bindings/broccoli-ruby/cmake push origin :topic/jsiwek/find-pythondev push origin :topic/jsiwek/openbsd-support %%%%%% /home/robin/bro/master/aux/broccoli/cmake push origin :topic/jsiwek/find-pythondev push origin :topic/jsiwek/openbsd-support %%%%%% /home/robin/bro/master/aux/broctl push origin :topic/dist-cleanup push origin :topic/jsiwek/abs-interp-path push origin :topic/jsiwek/broctl-cluster-fixes push origin :topic/jsiwek/broctl-tweaks push origin :topic/jsiwek/cmake-rpath push origin :topic/jsiwek/dynamic-example-install push origin :topic/jsiwek/local-node-order push origin :topic/jsiwek/openbsd-support push origin :topic/jsiwek/parallel-make-recursion push origin :topic/jsiwek/pfring-configure-check push origin :topic/jsiwek/ticket658 push origin :topic/policy-scripts-new push origin :topic/robin/plugins %%%%%% /home/robin/bro/master/aux/broctl/aux/capstats push origin :topic/dist-cleanup push origin :topic/jsiwek/cmake-rpath push origin :topic/jsiwek/debug-flags push origin :topic/jsiwek/findpcap_pfring push origin :topic/jsiwek/parallel-make-recursion push origin :topic/robin/cleanup %%%%%% /home/robin/bro/master/aux/broctl/aux/capstats/cmake push origin :topic/jsiwek/find-pythondev push origin :topic/jsiwek/openbsd-support %%%%%% /home/robin/bro/master/aux/broctl/aux/pysubnettree push origin :topic/dist-cleanup push origin :topic/jsiwek/cmake-rpath push origin :topic/jsiwek/compiler-warnings push origin :topic/jsiwek/debug-flags push origin :topic/jsiwek/find-pythondev push origin :topic/robin/cleanup %%%%%% /home/robin/bro/master/aux/broctl/aux/pysubnettree/cmake push origin :topic/jsiwek/find-pythondev push origin :topic/jsiwek/openbsd-support %%%%%% /home/robin/bro/master/aux/broctl/aux/trace-summary push origin :topic/dist-cleanup push origin :topic/jsiwek/cmake-rpath push origin :topic/robin/cleanup %%%%%% /home/robin/bro/master/aux/broctl/aux/trace-summary/cmake push origin :topic/jsiwek/find-pythondev push origin :topic/jsiwek/openbsd-support %%%%%% /home/robin/bro/master/aux/broctl/cmake push origin :topic/jsiwek/find-pythondev push origin :topic/jsiwek/openbsd-support %%%%%% /home/robin/bro/master/aux/btest push origin :topic/dist-cleanup push origin :topic/jsiwek/brofiler %%%%%% /home/robin/bro/master/cmake push origin :topic/jsiwek/find-pythondev push origin :topic/jsiwek/openbsd-support -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Fri Mar 2 09:44:26 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 02 Mar 2012 17:44:26 -0000 Subject: [Bro-Dev] #794: topic/jsiwek/coverage-tweaks In-Reply-To: <048.fc6ddb0752fbc5bb8867d98fcc506f0c@tracker.bro-ids.org> References: <048.fc6ddb0752fbc5bb8867d98fcc506f0c@tracker.bro-ids.org> Message-ID: <063.8420e6dc1e31ecf768bb3df0abf4ea72@tracker.bro-ids.org> #794: topic/jsiwek/coverage-tweaks ----------------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => jsiwek * status: new => assigned Comment: I tried merging this but a number of tests fail now with `Reporter::ERROR Failed to generate unique file name from BRO_PROFILER_FILE: /da/home/robin/bro/master/testing/btest/.tmp/script-coverage.XXXX\x0a`. Also, with the parallel btest (`btest -j 5`) almost all tests fail. Before the merge, I had tried removing the BRO_PROFILE_FILE from btest.cfg and then `-j 5` worked fine. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Fri Mar 2 09:45:34 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 2 Mar 2012 09:45:34 -0800 Subject: [Bro-Dev] Problems parallelizing btests In-Reply-To: <20120301015421.GH19963@icir.org> References: <20120301015421.GH19963@icir.org> Message-ID: <20120302174534.GA5316@icir.org> On Wed, Feb 29, 2012 at 17:54 -0800, I wrote: > btest:topic/robin/parallel has a version of btest that can run tests > in parallel. This branch could use some testing as well, btw. I've also restructured things internally a bit. README isn't updated yet but the new options are: -j THREADS, --jobs=THREADS number of threads to run tests in simultaniously; 0 disables threading -g GROUP, --group=GROUP execute only test of given group, or '-' for those without any group -r, --rerun Execute commands for tests that failed last time (For the Bro tests, one currently needs to remove the BRO_PROFILER_FILE variable from btest.cfg to make it work.) Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Fri Mar 2 10:45:33 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 02 Mar 2012 18:45:33 -0000 Subject: [Bro-Dev] #794: topic/jsiwek/coverage-tweaks In-Reply-To: <048.fc6ddb0752fbc5bb8867d98fcc506f0c@tracker.bro-ids.org> References: <048.fc6ddb0752fbc5bb8867d98fcc506f0c@tracker.bro-ids.org> Message-ID: <063.525bbb0899a1e0b6f7a9012bdbba5817@tracker.bro-ids.org> #794: topic/jsiwek/coverage-tweaks ----------------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by jsiwek): In [fef671e4a6480605e2500e970093d4999ee9de31/bro]: {{{ #!CommitTicketReference repository="bro" revision="fef671e4a6480605e2500e970093d4999ee9de31" Fix a BRO_PROFILER_FILE/mkstemp portability issue. (addresses #794) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 2 11:00:15 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 02 Mar 2012 19:00:15 -0000 Subject: [Bro-Dev] #794: topic/jsiwek/coverage-tweaks In-Reply-To: <048.fc6ddb0752fbc5bb8867d98fcc506f0c@tracker.bro-ids.org> References: <048.fc6ddb0752fbc5bb8867d98fcc506f0c@tracker.bro-ids.org> Message-ID: <063.add8f16dd35a643c6e7927009aaeeb87@tracker.bro-ids.org> #794: topic/jsiwek/coverage-tweaks ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * owner: jsiwek => robin Comment: > I tried merging this but a number of tests fail Let me know if that last change doesn't fix it for you. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 2 18:22:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 03 Mar 2012 02:22:10 -0000 Subject: [Bro-Dev] #523: event new_packet does not support IPv6 In-Reply-To: <048.6a57705fb7303481ddab2ff96d7bc20e@tracker.bro-ids.org> References: <048.6a57705fb7303481ddab2ff96d7bc20e@tracker.bro-ids.org> Message-ID: <063.26f8801a2886a27f86463f8d8e71ac7d@tracker.bro-ids.org> #523: event new_packet does not support IPv6 ----------------------+------------------------ Reporter: gregor | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: IPv6 ----------------------+------------------------ Comment (by jsiwek): In [eb9f686bb20fc1fe5021cd0b92eea3b5a147a1cd/bro]: {{{ #!CommitTicketReference repository="bro" revision="eb9f686bb20fc1fe5021cd0b92eea3b5a147a1cd" Add handling for IPv6 extension header chains (addresses #531) - The script-layer 'pkt_hdr' type is extended with a new 'ip6' field representing the full IPv6 header chain. - The 'new_packet' event is now raised for IPv6 packets (addresses #523) - A new event called 'ipv6_ext_header' is raised for any IPv6 packet containing extension headers. - A new event called 'esp_packet' is raised for any packets using ESP ('new_packet' and 'ipv6_ext_header' events provide connection info, but that info can't be provided here since the upper-layer payload is encrypted). - The 'unknown_protocol' weird is now raised more reliably when Bro sees a transport protocol or IPv6 extension header it can't handle. (addresses #522) Still need to do IPv6 fragment reassembly and needs more testing. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 2 18:22:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 03 Mar 2012 02:22:10 -0000 Subject: [Bro-Dev] #531: Handle IPv6 protocol chains In-Reply-To: <048.a1036225e7e49822f38fb91c4b12ccff@tracker.bro-ids.org> References: <048.a1036225e7e49822f38fb91c4b12ccff@tracker.bro-ids.org> Message-ID: <063.e2a5cac73ac92ef5d359e39cd269af85@tracker.bro-ids.org> #531: Handle IPv6 protocol chains ----------------------+------------------------ Reporter: gregor | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: IPv6 ----------------------+------------------------ Comment (by jsiwek): In [eb9f686bb20fc1fe5021cd0b92eea3b5a147a1cd/bro]: {{{ #!CommitTicketReference repository="bro" revision="eb9f686bb20fc1fe5021cd0b92eea3b5a147a1cd" Add handling for IPv6 extension header chains (addresses #531) - The script-layer 'pkt_hdr' type is extended with a new 'ip6' field representing the full IPv6 header chain. - The 'new_packet' event is now raised for IPv6 packets (addresses #523) - A new event called 'ipv6_ext_header' is raised for any IPv6 packet containing extension headers. - A new event called 'esp_packet' is raised for any packets using ESP ('new_packet' and 'ipv6_ext_header' events provide connection info, but that info can't be provided here since the upper-layer payload is encrypted). - The 'unknown_protocol' weird is now raised more reliably when Bro sees a transport protocol or IPv6 extension header it can't handle. (addresses #522) Still need to do IPv6 fragment reassembly and needs more testing. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 2 18:22:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 03 Mar 2012 02:22:10 -0000 Subject: [Bro-Dev] #522: Event to report non TCP/UDP/ICMP packets In-Reply-To: <048.9578ab814d281f028dba6ef156f882b0@tracker.bro-ids.org> References: <048.9578ab814d281f028dba6ef156f882b0@tracker.bro-ids.org> Message-ID: <063.0ca8227b6e9b20cef0026d14b74ba459@tracker.bro-ids.org> #522: Event to report non TCP/UDP/ICMP packets ----------------------+------------------------ Reporter: gregor | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: IPv6 ----------------------+------------------------ Comment (by jsiwek): In [eb9f686bb20fc1fe5021cd0b92eea3b5a147a1cd/bro]: {{{ #!CommitTicketReference repository="bro" revision="eb9f686bb20fc1fe5021cd0b92eea3b5a147a1cd" Add handling for IPv6 extension header chains (addresses #531) - The script-layer 'pkt_hdr' type is extended with a new 'ip6' field representing the full IPv6 header chain. - The 'new_packet' event is now raised for IPv6 packets (addresses #523) - A new event called 'ipv6_ext_header' is raised for any IPv6 packet containing extension headers. - A new event called 'esp_packet' is raised for any packets using ESP ('new_packet' and 'ipv6_ext_header' events provide connection info, but that info can't be provided here since the upper-layer payload is encrypted). - The 'unknown_protocol' weird is now raised more reliably when Bro sees a transport protocol or IPv6 extension header it can't handle. (addresses #522) Still need to do IPv6 fragment reassembly and needs more testing. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Mar 3 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 3 Mar 2012 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203030800.q23802mN014801@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 794 [1] | jsiwek | robin | Normal | topic/jsiwek/coverage-tweaks [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 87ad77c | Seth Hall | 2012-02-29 | Standardized on the &default function for SSL constants. [3] [1] #794: http://tracker.bro-ids.org/bro/ticket/794 [2] coverage-tweaks: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/coverage-tweaks [3] fastpath: http://tracker.bro-ids.org/bro/changeset/87ad77cc100697659c72429386b5264ae67051dc/bro From noreply at bro-ids.org Sun Mar 4 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 4 Mar 2012 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203040800.q24802HJ013444@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 794 [1] | jsiwek | robin | Normal | topic/jsiwek/coverage-tweaks [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 87ad77c | Seth Hall | 2012-02-29 | Standardized on the &default function for SSL constants. [3] [1] #794: http://tracker.bro-ids.org/bro/ticket/794 [2] coverage-tweaks: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/coverage-tweaks [3] fastpath: http://tracker.bro-ids.org/bro/changeset/87ad77cc100697659c72429386b5264ae67051dc/bro From noreply at bro-ids.org Mon Mar 5 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 5 Mar 2012 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203050800.q25802xR032166@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 794 [1] | jsiwek | robin | Normal | topic/jsiwek/coverage-tweaks [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 87ad77c | Seth Hall | 2012-02-29 | Standardized on the &default function for SSL constants. [3] [1] #794: http://tracker.bro-ids.org/bro/ticket/794 [2] coverage-tweaks: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/coverage-tweaks [3] fastpath: http://tracker.bro-ids.org/bro/changeset/87ad77cc100697659c72429386b5264ae67051dc/bro From robin at icir.org Mon Mar 5 08:17:25 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 5 Mar 2012 08:17:25 -0800 Subject: [Bro-Dev] Problems parallelizing btests In-Reply-To: <4F54E6A8.6090804@illinois.edu> References: <20120301015421.GH19963@icir.org> <20120302174534.GA5316@icir.org> <4F54E6A8.6090804@illinois.edu> Message-ID: <20120305161724.GC7606@icir.org> On Mon, Mar 05, 2012 at 10:15 -0600, you wrote: > It appears that OutputHandlers.py isn't getting installed. When I run > btest (after doing "python setup.py install"), I see this error: Oops, yeah. I always run it directly from the source directory, which is why I didn't notice. Will fix. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From dnthayer at illinois.edu Mon Mar 5 08:15:36 2012 From: dnthayer at illinois.edu (Daniel Thayer) Date: Mon, 5 Mar 2012 10:15:36 -0600 Subject: [Bro-Dev] Problems parallelizing btests In-Reply-To: <20120302174534.GA5316@icir.org> References: <20120301015421.GH19963@icir.org> <20120302174534.GA5316@icir.org> Message-ID: <4F54E6A8.6090804@illinois.edu> On 03/02/2012 11:45 AM, Robin Sommer wrote: > > On Wed, Feb 29, 2012 at 17:54 -0800, I wrote: > >> btest:topic/robin/parallel has a version of btest that can run tests >> in parallel. > > This branch could use some testing as well, btw. I've also > restructured things internally a bit. README isn't updated yet but the > new options are: > > -j THREADS, --jobs=THREADS > number of threads to run tests in simultaniously; 0 > disables threading > -g GROUP, --group=GROUP > execute only test of given group, or '-' for those > without any group > -r, --rerun Execute commands for tests that failed last time > > > (For the Bro tests, one currently needs to remove the > BRO_PROFILER_FILE variable from btest.cfg to make it work.) > > > Robin > It appears that OutputHandlers.py isn't getting installed. When I run btest (after doing "python setup.py install"), I see this error: ImportError: No module named OutputHandlers From bro at tracker.bro-ids.org Mon Mar 5 17:26:25 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 06 Mar 2012 01:26:25 -0000 Subject: [Bro-Dev] #794: topic/jsiwek/coverage-tweaks In-Reply-To: <048.fc6ddb0752fbc5bb8867d98fcc506f0c@tracker.bro-ids.org> References: <048.fc6ddb0752fbc5bb8867d98fcc506f0c@tracker.bro-ids.org> Message-ID: <063.55ced720b9c567f47ba493826a4519f6@tracker.bro-ids.org> #794: topic/jsiwek/coverage-tweaks ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Solved, thanks. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Mar 5 17:26:36 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 06 Mar 2012 01:26:36 -0000 Subject: [Bro-Dev] #794: topic/jsiwek/coverage-tweaks In-Reply-To: <048.fc6ddb0752fbc5bb8867d98fcc506f0c@tracker.bro-ids.org> References: <048.fc6ddb0752fbc5bb8867d98fcc506f0c@tracker.bro-ids.org> Message-ID: <063.da9a0332f93376fe5c10d77748c8253e@tracker.bro-ids.org> #794: topic/jsiwek/coverage-tweaks ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * status: assigned => closed * resolution: => fixed Comment: In [035de0216e2239071f6faa16f96a177ec6280971/bro]: {{{ #!CommitTicketReference repository="bro" revision="035de0216e2239071f6faa16f96a177ec6280971" Merge remote-tracking branch 'origin/topic/jsiwek/coverage-tweaks' * origin/topic/jsiwek/coverage-tweaks: Changes to how script coverage integrates with test suites. Closes #794. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Mon Mar 5 17:45:07 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 5 Mar 2012 17:45:07 -0800 Subject: [Bro-Dev] Proxy problems Message-ID: <20120306014507.GC30213@icir.org> On a cluster running current git, I'm seeing reproducible proxy crashes: CPU goes up to 100% within seconds after the workers connect, both for child and parent processes. The proxy then eventually terminates due to overload. Anyone else seeing something like that? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From dnthayer at illinois.edu Tue Mar 6 14:48:29 2012 From: dnthayer at illinois.edu (Daniel Thayer) Date: Tue, 6 Mar 2012 16:48:29 -0600 Subject: [Bro-Dev] Problems parallelizing btests In-Reply-To: <20120302174534.GA5316@icir.org> References: <20120301015421.GH19963@icir.org> <20120302174534.GA5316@icir.org> Message-ID: <4F56943D.6060802@illinois.edu> On 03/02/2012 11:45 AM, Robin Sommer wrote: > > On Wed, Feb 29, 2012 at 17:54 -0800, I wrote: > >> btest:topic/robin/parallel has a version of btest that can run tests >> in parallel. > > This branch could use some testing as well, btw. I've also > restructured things internally a bit. README isn't updated yet but the > new options are: > > -j THREADS, --jobs=THREADS > number of threads to run tests in simultaniously; 0 > disables threading > -g GROUP, --group=GROUP > execute only test of given group, or '-' for those > without any group > -r, --rerun Execute commands for tests that failed last time > > > (For the Bro tests, one currently needs to remove the > BRO_PROFILER_FILE variable from btest.cfg to make it work.) > > > Robin > It seems that using the "-f" option (without "-b" or "-v") now prevents the status message for each test from being output. The following patch should fix this bug: --- a/btest +++ b/btest @@ -927,11 +927,9 @@ if Options.diagfile: if Options.verbose: output_handlers += [Verbose(Options, )] - -if Options.brief: +elif Options.brief: output_handlers += [Brief(Options, )] - -if not output_handlers: +else: output_handlers += [Standard(Options, )] output_handler = Forwarder(Options, output_handlers) From jsiwek at illinois.edu Wed Mar 7 11:19:33 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 7 Mar 2012 19:19:33 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix some IPv6 header related bugs. (6530776) In-Reply-To: <201203071845.q27Ijoan016789@bro-ids.icir.org> References: <201203071845.q27Ijoan016789@bro-ids.icir.org> Message-ID: <9CE382F6-7003-4EB3-9BCB-099741EEAB4D@illinois.edu> > Fix some IPv6 header related bugs. > > - IPv6 payload length calculation didn't count main 40 byte IPv6 header. Er, I realize that sounds confusing: it shouldn't (doesn't) count the main header as part of the payload. What I meant was I accidentally subtracted the main header length from the payload length field when that field doesn't count the main header in the first place. +Jon From dnthayer at illinois.edu Thu Mar 8 10:35:51 2012 From: dnthayer at illinois.edu (Daniel Thayer) Date: Thu, 8 Mar 2012 12:35:51 -0600 Subject: [Bro-Dev] unused broctl directory? Message-ID: <4F58FC07.2080906@illinois.edu> In the file /spool/broctl-config.sh (where is the bro install directory), there is a line: policydirbroctl="/spool/policy/broctl" However, it seems that the directory "broctl" is never created, and after searching the source code, I don't see it being used anywhere (as of Bro 2.0 and broctl 1.0). Therefore, I plan to remove this line, unless I hear from someone that it is needed for some reason. -Daniel From bro at tracker.bro-ids.org Thu Mar 8 13:56:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 08 Mar 2012 21:56:51 -0000 Subject: [Bro-Dev] #795: topic/jsiwek/ipv6-ext-headers Message-ID: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> #795: topic/jsiwek/ipv6-ext-headers ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch adds handling of IPv6 extension header chains (for header types defined by RFC 2460) and IPv6 fragment reassembly. Some more details on significant changes are in the commit messages of: [eb9f686bb20fc1fe5021cd0b92eea3b5a147a1cd/bro] [0b32c980bf6117d3149d4ce8d41aa46df11c27e4/bro] Tickets #531, #522, and #523 are addressed by this branch. The only (correct) difference in baselines for external tests I saw was the long test in `bro-testing` has a weird.log now reporting some unknown_protocol 2 and 58's. -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Thu Mar 8 18:37:41 2012 From: seth at icir.org (Seth Hall) Date: Thu, 8 Mar 2012 21:37:41 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/log-threads: Finetuning communication CPU usage. (51009b7) In-Reply-To: <201203090213.q292DaxJ027484@bro-ids.icir.org> References: <201203090213.q292DaxJ027484@bro-ids.icir.org> Message-ID: On Mar 8, 2012, at 9:13 PM, Robin Sommer wrote: > --- a/scripts/base/frameworks/cluster/setup-connections.bro > +++ b/scripts/base/frameworks/cluster/setup-connections.bro > @@ -44,7 +44,7 @@ event bro_init() &priority=9 > { > if ( n$node_type == WORKER && n$proxy == node ) > Communication::nodes[i] = > - [$host=n$ip, $connect=F, $class=i, $sync=T, $auth=T, $events=worker2proxy_events]; > + [$host=n$ip, $connect=F, $class=i, $sync=F, $auth=T, $events=worker2proxy_events]; Was this an accident? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Thu Mar 8 20:34:06 2012 From: robin at icir.org (Robin Sommer) Date: Thu, 8 Mar 2012 20:34:06 -0800 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/log-threads: Finetuning communication CPU usage. (51009b7) In-Reply-To: References: <201203090213.q292DaxJ027484@bro-ids.icir.org> Message-ID: <20120309043406.GR53405@icir.org> On Thu, Mar 08, 2012 at 21:37 -0500, you wrote: > Was this an accident? Yes and no. I didn't want to commit it but I needed it to solve my proxy problem: turns out that's the change between 2.0 and git master that drives it into crashing. Not sure what's going on, need to investigate. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Fri Mar 9 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 9 Mar 2012 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203090800.q29802lx007374@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 795 [1] | jsiwek | | Normal | topic/jsiwek/ipv6-ext-headers [2] [1] #795: http://tracker.bro-ids.org/bro/ticket/795 [2] ipv6-ext-headers: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-ext-headers From robin at icir.org Fri Mar 9 08:04:55 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 9 Mar 2012 08:04:55 -0800 Subject: [Bro-Dev] unused broctl directory? In-Reply-To: <4F58FC07.2080906@illinois.edu> References: <4F58FC07.2080906@illinois.edu> Message-ID: <20120309160455.GI85217@icir.org> On Thu, Mar 08, 2012 at 12:35 -0600, you wrote: > policydirbroctl="/spool/policy/broctl" Yeah, I believe that's indeed no longer used with all the scripts now moved into frameworks/cluster. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From jsiwek at illinois.edu Fri Mar 9 09:01:39 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 9 Mar 2012 17:01:39 +0000 Subject: [Bro-Dev] IPv6 literal addr constants Message-ID: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> Representing compressed-hex IPv6 addresses (replacing consecutive fields of zeros with ::) in scripts as literal constants can be ambiguous with identifiers that use "::" as a namespace resolver. For example, the lexer will treat "aaaa::bbbb" as the "bbbb" identifier in the "aaaa" namespace/module. Specifically, this is the case where someone tries to write an address that uses only the first and last 16-bit fields and the first nibble of both fields is a letter. I think this would be uncommon, but also maybe not obvious to figure out when someone actually runs into it, though it's easy to workaround once you know what's going on. Any ideas for fixing the ambiguity or does it seem reasonable to just have it documented? +Jon From dnthayer at illinois.edu Fri Mar 9 09:27:52 2012 From: dnthayer at illinois.edu (Daniel Thayer) Date: Fri, 9 Mar 2012 11:27:52 -0600 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> References: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> Message-ID: <4F5A3D98.7050600@illinois.edu> On 03/09/2012 11:01 AM, Siwek, Jonathan Luke wrote: > Representing compressed-hex IPv6 addresses (replacing consecutive fields of zeros with ::) in scripts as literal constants can be ambiguous with identifiers that use "::" as a namespace resolver. > > For example, the lexer will treat "aaaa::bbbb" as the "bbbb" identifier in the "aaaa" namespace/module. Specifically, this is the case where someone tries to write an address that uses only the first and last 16-bit fields and the first nibble of both fields is a letter. > > I think this would be uncommon, but also maybe not obvious to figure out when someone actually runs into it, though it's easy to workaround once you know what's going on. Any ideas for fixing the ambiguity or does it seem reasonable to just have it documented? > > +Jon The example that I found yesterday was a607:f8b0::/32 (I get an error message from bro, "unknown identifier a607"). If I write it as a607:f8b0::0:0:0:0:0/32, then I still get the same error message. Writing it without a double colon a607:f8b0:0:0:0:0:0:0/32 seems to work. If the first digit is in the range 0-9 (and not in the range a-f), then bro does not complain (such as 2607:f8b0::/32). -Daniel From jsiwek at illinois.edu Fri Mar 9 09:44:28 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 9 Mar 2012 17:44:28 +0000 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <4F5A3D98.7050600@illinois.edu> References: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> <4F5A3D98.7050600@illinois.edu> Message-ID: > The example that I found yesterday was a607:f8b0::/32 (I get an error message from bro, "unknown identifier a607"). If I write it > as a607:f8b0::0:0:0:0:0/32, then I still get the same > error message. Writing it without a double colon > a607:f8b0:0:0:0:0:0:0/32 seems to work. > > If the first digit is in the range 0-9 (and not in > the range a-f), then bro does not complain (such > as 2607:f8b0::/32). Right, the current rule in scan.l for compressed hex notation looks for the first nibble to be a digit and not a letter. That's fixable, but as I was testing more potential address formats I ran into the ambiguity I mentioned before which doesn't look like it's so easy to work around. +Jon From bro at tracker.bro-ids.org Fri Mar 9 10:45:49 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 09 Mar 2012 18:45:49 -0000 Subject: [Bro-Dev] #753: Remove the match statement In-Reply-To: <046.7542a9a4cc041d43889cad73e3ab0b83@tracker.bro-ids.org> References: <046.7542a9a4cc041d43889cad73e3ab0b83@tracker.bro-ids.org> Message-ID: <061.e0e2f385d49785fa48c11480e6dc6933@tracker.bro-ids.org> #753: Remove the match statement ----------------------------+------------------------ Reporter: seth | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * type: Task => Merge Request Comment: In `topic/jsiwek/remove-match` in bro and broccoli repos. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 9 15:54:33 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 09 Mar 2012 23:54:33 -0000 Subject: [Bro-Dev] #795: topic/jsiwek/ipv6-ext-headers In-Reply-To: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> References: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> Message-ID: <063.3d87151ff0a274feb257f06603f68a27@tracker.bro-ids.org> #795: topic/jsiwek/ipv6-ext-headers ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Unit tests for the new functionality (IPv6 frag reassembly, new events) would be good. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Mar 10 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 10 Mar 2012 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203100800.q2A803qY008656@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 753 [1] | seth | | Normal | Remove the match statement Bro | 795 [2] | jsiwek | | Normal | topic/jsiwek/ipv6-ext-headers [3] [1] #753: http://tracker.bro-ids.org/bro/ticket/753 [2] #795: http://tracker.bro-ids.org/bro/ticket/795 [3] ipv6-ext-headers: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-ext-headers From vern at icir.org Sat Mar 10 16:39:46 2012 From: vern at icir.org (Vern Paxson) Date: Sat, 10 Mar 2012 16:39:46 -0800 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: (Fri, 09 Mar 2012 17:44:28 GMT). Message-ID: <20120311003946.D13AD2C4002@rock.ICSI.Berkeley.EDU> > Right, the current rule in scan.l for compressed hex notation looks for > the first nibble to be a digit and not a letter. That's fixable ... I don't recall the genesis of this rule (which I probably added a long time ago), but it could be that starting with a digit is intentional, because otherwise examples like the one given earlier of aaaa::bbbb are fully ambiguous. With the rule, you can write 0aaaa::bbbb and it will (should!) parse correctly. Maybe that's too ugly. I'm not sure there's a better fix, however. Vern From seth at icir.org Sat Mar 10 18:02:26 2012 From: seth at icir.org (Seth Hall) Date: Sat, 10 Mar 2012 21:02:26 -0500 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <20120311003946.D13AD2C4002@rock.ICSI.Berkeley.EDU> References: <20120311003946.D13AD2C4002@rock.ICSI.Berkeley.EDU> Message-ID: <3A3531F0-C6DB-4941-AB73-D8E979C0778F@icir.org> On Mar 10, 2012, at 7:39 PM, Vern Paxson wrote: > With the rule, you can write 0aaaa::bbbb and it will > (should!) parse correctly. Maybe that's too ugly. I'm not sure there's > a better fix, however. Could we just disallow module names that could be interpreted as addresses? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From vern at icir.org Sat Mar 10 21:05:22 2012 From: vern at icir.org (Vern Paxson) Date: Sat, 10 Mar 2012 21:05:22 -0800 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <3A3531F0-C6DB-4941-AB73-D8E979C0778F@icir.org> (Sat, 10 Mar 2012 21:02:26 EST). Message-ID: <20120311050522.C01E42C4002@rock.ICSI.Berkeley.EDU> > Could we just disallow module names that could be interpreted as addresses? Yeee-uck! "Identifiers begin with a letter followed by zero or more digits or letters. However, they must include at least one letter in the range g-z." Vern From noreply at bro-ids.org Sun Mar 11 00:00:01 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 11 Mar 2012 00:00:01 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203110800.q2B801bu032741@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 753 [1] | seth | | Normal | Remove the match statement Bro | 795 [2] | jsiwek | | Normal | topic/jsiwek/ipv6-ext-headers [3] [1] #753: http://tracker.bro-ids.org/bro/ticket/753 [2] #795: http://tracker.bro-ids.org/bro/ticket/795 [3] ipv6-ext-headers: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-ext-headers From leres at ee.lbl.gov Sun Mar 11 15:52:21 2012 From: leres at ee.lbl.gov (Craig Leres) Date: Sun, 11 Mar 2012 15:52:21 -0700 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> References: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> Message-ID: <4F5D2CA5.8030502@ee.lbl.gov> How about enclosing IPv6 literals in brackets, e.g. [aaaa::bbbb]? As with URLs this would also allow IPv6 addresses with ports, e.g. [2620:83:8000:102::c9]:22. Craig From noreply at bro-ids.org Mon Mar 12 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 12 Mar 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203120700.q2C702f2016632@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 753 [1] | seth | | Normal | Remove the match statement Bro | 795 [2] | jsiwek | | Normal | topic/jsiwek/ipv6-ext-headers [3] [1] #753: http://tracker.bro-ids.org/bro/ticket/753 [2] #795: http://tracker.bro-ids.org/bro/ticket/795 [3] ipv6-ext-headers: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-ext-headers From robin at icir.org Mon Mar 12 09:00:57 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 12 Mar 2012 09:00:57 -0700 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <20120311003946.D13AD2C4002@rock.ICSI.Berkeley.EDU> References: <20120311003946.D13AD2C4002@rock.ICSI.Berkeley.EDU> Message-ID: <20120312160057.GL27202@icir.org> On Sat, Mar 10, 2012 at 16:39 -0800, you wrote: > (should!) parse correctly. Maybe that's too ugly. I'm not sure there's > a better fix, however. I think the right fix would be not having the lexer make the decision, but do it later when we can say whether there's an identifier with that name. But that's not easy to do. :-( Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Mon Mar 12 09:01:34 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 12 Mar 2012 09:01:34 -0700 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <4F5D2CA5.8030502@ee.lbl.gov> References: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> <4F5D2CA5.8030502@ee.lbl.gov> Message-ID: <20120312160134.GM27202@icir.org> On Sun, Mar 11, 2012 at 15:52 -0700, you wrote: > How about enclosing IPv6 literals in brackets, e.g. [aaaa::bbbb]? I like that, that's a well defined standard syntax. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Mon Mar 12 09:09:29 2012 From: seth at icir.org (Seth Hall) Date: Mon, 12 Mar 2012 12:09:29 -0400 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <20120312160134.GM27202@icir.org> References: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> <4F5D2CA5.8030502@ee.lbl.gov> <20120312160134.GM27202@icir.org> Message-ID: On Mar 12, 2012, at 12:01 PM, Robin Sommer wrote: > On Sun, Mar 11, 2012 at 15:52 -0700, you wrote: > >> How about enclosing IPv6 literals in brackets, e.g. [aaaa::bbbb]? > > I like that, that's a well defined standard syntax. I like it too, but won't it be yet another problem for differentiating between record definition, tuple definition, and now IPv6 definition? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From slagell at illinois.edu Mon Mar 12 10:31:46 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Mon, 12 Mar 2012 17:31:46 +0000 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <20120312160134.GM27202@icir.org> References: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> <4F5D2CA5.8030502@ee.lbl.gov> <20120312160134.GM27202@icir.org> Message-ID: <3C185C94-4E17-4B1C-9AE2-53BEF4BC6984@illinois.edu> On Mar 12, 2012, at 11:01 AM, Robin Sommer wrote: > > On Sun, Mar 11, 2012 at 15:52 -0700, you wrote: > >> How about enclosing IPv6 literals in brackets, e.g. [aaaa::bbbb]? > > I like that, that's a well defined standard syntax. Me too. ------ Adam J. Slagell, CISO, CISSP Chief Information Security Officer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info 217.244.8965 "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From gregor at icir.org Mon Mar 12 11:37:26 2012 From: gregor at icir.org (Gregor Maier) Date: Mon, 12 Mar 2012 11:37:26 -0700 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <20120312160057.GL27202@icir.org> References: <20120311003946.D13AD2C4002@rock.ICSI.Berkeley.EDU> <20120312160057.GL27202@icir.org> Message-ID: <4F5E4266.2090205@icir.org> On 3/12/12 9:00 , Robin Sommer wrote: > > On Sat, Mar 10, 2012 at 16:39 -0800, you wrote: > >> (should!) parse correctly. Maybe that's too ugly. I'm not sure there's >> a better fix, however. > > I think the right fix would be not having the lexer make the decision, > but do it later when we can say whether there's an identifier with > that name. But that's not easy to do. :-( That's a bad idea as well. if you have a typo in your identifier, bro won't complain anymore and assume it's an IPv6 literal. cu gregor From gregor at icir.org Mon Mar 12 11:38:55 2012 From: gregor at icir.org (Gregor Maier) Date: Mon, 12 Mar 2012 11:38:55 -0700 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: References: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> <4F5D2CA5.8030502@ee.lbl.gov> <20120312160134.GM27202@icir.org> Message-ID: <4F5E42BF.1090902@icir.org> On 3/12/12 9:09 , Seth Hall wrote: > > On Mar 12, 2012, at 12:01 PM, Robin Sommer wrote: > >> On Sun, Mar 11, 2012 at 15:52 -0700, you wrote: >> >>> How about enclosing IPv6 literals in brackets, e.g. [aaaa::bbbb]? >> >> I like that, that's a well defined standard syntax. +1 From robin at icir.org Mon Mar 12 21:30:41 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 12 Mar 2012 21:30:41 -0700 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: References: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> <4F5D2CA5.8030502@ee.lbl.gov> <20120312160134.GM27202@icir.org> Message-ID: <20120313043041.GC61817@icir.org> On Mon, Mar 12, 2012 at 12:09 -0400, you wrote: > >> How about enclosing IPv6 literals in brackets, e.g. [aaaa::bbbb]? > > > > I like that, that's a well defined standard syntax. > > I like it too, but won't it be yet another problem for differentiating > between record definition, tuple definition, and now IPv6 definition? That is a good point. I'm not fully sure but I believe it should be less of a problem than with the current syntax. Most of the other usages have some characters in there that aren't valid inside an address (or the other way round: the address' ':' and '::' aren't valid in them). But the trick is to define the lexer so that it picks the addresses but leaves the other ones alones ... Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Tue Mar 13 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 13 Mar 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203130700.q2D702d2001292@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 753 [1] | seth | | Normal | Remove the match statement Bro | 795 [2] | jsiwek | | Normal | topic/jsiwek/ipv6-ext-headers [3] [1] #753: http://tracker.bro-ids.org/bro/ticket/753 [2] #795: http://tracker.bro-ids.org/bro/ticket/795 [3] ipv6-ext-headers: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-ext-headers From jsiwek at illinois.edu Tue Mar 13 10:09:43 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Tue, 13 Mar 2012 17:09:43 +0000 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <20120313043041.GC61817@icir.org> References: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> <4F5D2CA5.8030502@ee.lbl.gov> <20120312160134.GM27202@icir.org> <20120313043041.GC61817@icir.org> Message-ID: <5D5EE407-43E3-4C59-81DA-E7752B2B0A07@illinois.edu> > That is a good point. I'm not fully sure but I believe it should be > less of a problem than with the current syntax. Most of the other > usages have some characters in there that aren't valid inside an > address (or the other way round: the address' ':' and '::' aren't > valid in them). But the trick is to define the lexer so that it picks > the addresses but leaves the other ones alones ... Yeah, I think the patterns for bracketed IPv6 literals are going to be specific enough to not be ambiguous with the other uses of brackets. Sounds like enough people like that syntax so I I'll add it, but what to do with the old syntax for IPv6 literals? Should it be removed at this time, deprecated until 2.2, or kept indefinitely? +Jon From robin at icir.org Tue Mar 13 10:23:03 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 13 Mar 2012 10:23:03 -0700 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <5D5EE407-43E3-4C59-81DA-E7752B2B0A07@illinois.edu> References: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> <4F5D2CA5.8030502@ee.lbl.gov> <20120312160134.GM27202@icir.org> <20120313043041.GC61817@icir.org> <5D5EE407-43E3-4C59-81DA-E7752B2B0A07@illinois.edu> Message-ID: <20120313172303.GQ78256@icir.org> On Tue, Mar 13, 2012 at 17:09 +0000, you wrote: > Sounds like enough people like that syntax so I I'll add it, but what > to do with the old syntax for IPv6 literals? Should it be removed at > this time, deprecated until 2.2, or kept indefinitely? I vote for just removing. Now is the one time where we can break IPv6 stuff to make it better. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Tue Mar 13 12:02:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 13 Mar 2012 19:02:03 -0000 Subject: [Bro-Dev] #796: topic/jsiwek/ipv6-literals Message-ID: <048.17aab3d0f533b77779b3211ad8047e4f@tracker.bro-ids.org> #796: topic/jsiwek/ipv6-literals ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: ipv6 | ---------------------------+------------------------ This branch changes the syntax of IPv6 literal addr constants in scripts to require being encased in square brackets. It is in bro, broccoli, and broccoli-python repos. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Mar 13 15:30:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 13 Mar 2012 22:30:38 -0000 Subject: [Bro-Dev] #795: topic/jsiwek/ipv6-ext-headers In-Reply-To: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> References: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> Message-ID: <063.9c5b57125ec4a77c5ec8bcd61415455a@tracker.bro-ids.org> #795: topic/jsiwek/ipv6-ext-headers ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): I'm going to commit feedback into the branch, marked with '[Robin]' (just grep for it or use "git diff -w" (-w to suppress some white space differences that I did before ealizing that I'll wait with the merge.) I like how you factored out the IPHdr code! (But see comments in there.) -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Mar 13 15:39:13 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 13 Mar 2012 22:39:13 -0000 Subject: [Bro-Dev] #796: topic/jsiwek/ipv6-literals In-Reply-To: <048.17aab3d0f533b77779b3211ad8047e4f@tracker.bro-ids.org> References: <048.17aab3d0f533b77779b3211ad8047e4f@tracker.bro-ids.org> Message-ID: <063.e42a63303402a2a5f67f555088b7b5c6@tracker.bro-ids.org> #796: topic/jsiwek/ipv6-literals ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ipv6 ----------------------------+------------------------ Comment (by robin): Merged. The one thing that gives me pause is that now "print " produces a different representation than the parser expects. But I don't see a way around that. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Mar 13 16:28:22 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 13 Mar 2012 23:28:22 -0000 Subject: [Bro-Dev] #796: topic/jsiwek/ipv6-literals In-Reply-To: <048.17aab3d0f533b77779b3211ad8047e4f@tracker.bro-ids.org> References: <048.17aab3d0f533b77779b3211ad8047e4f@tracker.bro-ids.org> Message-ID: <063.1d50a76de64ba8673fb069db2d4c273d@tracker.bro-ids.org> #796: topic/jsiwek/ipv6-literals ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ipv6 ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [b4239de4a3dfaa836a1a0dab46ae41e4065be771/bro]: {{{ #!CommitTicketReference repository="bro" revision="b4239de4a3dfaa836a1a0dab46ae41e4065be771" Updating NEWS. Previous commit closes #796. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Mar 13 16:28:22 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 13 Mar 2012 23:28:22 -0000 Subject: [Bro-Dev] #786: Minor bugfixes In-Reply-To: <055.31e8fc347475856631bd8ecd15bf1b98@tracker.bro-ids.org> References: <055.31e8fc347475856631bd8ecd15bf1b98@tracker.bro-ids.org> Message-ID: <070.b71655d20fe9843bb9bfb1f8942c9baf@tracker.bro-ids.org> #786: Minor bugfixes ----------------------------+------------------------ Reporter: JulienSentier | Owner: robin Type: Patch | Status: closed Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [cba160c8ac23560a2c663e75a9e916e1a0b9f2f4/bro]: {{{ #!CommitTicketReference repository="bro" revision="cba160c8ac23560a2c663e75a9e916e1a0b9f2f4" Removing a line of dead code. Found by Julien Sentier. Closes #786. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Mar 13 16:28:22 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 13 Mar 2012 23:28:22 -0000 Subject: [Bro-Dev] #792: Signatures with Site::local_nets In-Reply-To: <055.14b45880c30ae19e160c331cfa04ef0e@tracker.bro-ids.org> References: <055.14b45880c30ae19e160c331cfa04ef0e@tracker.bro-ids.org> Message-ID: <070.edd842f1a1f4c2733f4cc5eb3a04b006@tracker.bro-ids.org> #792: Signatures with Site::local_nets ----------------------------+------------------------ Reporter: JulienSentier | Owner: robin Type: Patch | Status: closed Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [a4f8b2ccbee913cd782e2244508b5b6d2b4c64ac/bro]: {{{ #!CommitTicketReference repository="bro" revision="a4f8b2ccbee913cd782e2244508b5b6d2b4c64ac" Changing the regular expression to allow Site::local_nets in signatures Previous commit closes #792. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Tue Mar 13 16:32:54 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 13 Mar 2012 16:32:54 -0700 Subject: [Bro-Dev] #795: topic/jsiwek/ipv6-ext-headers In-Reply-To: <063.9c5b57125ec4a77c5ec8bcd61415455a@tracker.bro-ids.org> References: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> <063.9c5b57125ec4a77c5ec8bcd61415455a@tracker.bro-ids.org> Message-ID: <20120313233254.GX78256@icir.org> On Tue, Mar 13, 2012 at 22:30 -0000, you wrote: > I'm going to commit feedback into the branch, marked with '[Robin]' I'm still trying to figure out a good way to provide feedback to individual pieces of code. Not sure this is great, let me know if you have a better idea for next time. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Wed Mar 14 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 14 Mar 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203140700.q2E702ff010272@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 753 [1] | seth | | Normal | Remove the match statement Bro | 795 [2] | jsiwek | | Normal | topic/jsiwek/ipv6-ext-headers [3] [1] #753: http://tracker.bro-ids.org/bro/ticket/753 [2] #795: http://tracker.bro-ids.org/bro/ticket/795 [3] ipv6-ext-headers: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-ext-headers From bro at tracker.bro-ids.org Wed Mar 14 08:19:45 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 14 Mar 2012 15:19:45 -0000 Subject: [Bro-Dev] #753: Remove the match statement In-Reply-To: <046.7542a9a4cc041d43889cad73e3ab0b83@tracker.bro-ids.org> References: <046.7542a9a4cc041d43889cad73e3ab0b83@tracker.bro-ids.org> Message-ID: <061.94f54a140e1892345353819ea856ba88@tracker.bro-ids.org> #753: Remove the match statement -----------------------------+------------------------ Reporter: seth | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From jsiwek at illinois.edu Wed Mar 14 11:36:38 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 14 Mar 2012 18:36:38 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Updating baseline. (9dd63ac) In-Reply-To: <201203132328.q2DNSV78015658@bro-ids.icir.org> References: <201203132328.q2DNSV78015658@bro-ids.icir.org> Message-ID: > Updating baseline. > > Is that a platform-specific difference? > > -::0.0.255.255 > +::ffff > -aaaa:bbbb:cccc:dddd:eeee:ffff::2222 > +aaaa:bbbb:cccc:dddd:eeee:ffff:0:2222 Yes, looks like inet_ntop()'s give different results depending on where you're at. How about taking FreeBSD's implementation of inet_ntop() as a starting point for canonicalization? Looks like it follows RFC 5952 well. +Jon From robin at icir.org Wed Mar 14 11:51:53 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 14 Mar 2012 11:51:53 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Updating baseline. (9dd63ac) In-Reply-To: References: <201203132328.q2DNSV78015658@bro-ids.icir.org> Message-ID: <20120314185153.GA15663@icir.org> On Wed, Mar 14, 2012 at 18:36 +0000, you wrote: > Yes, looks like inet_ntop()'s give different results depending on > where you're at. This was on Linux. > How about taking FreeBSD's implementation of inet_ntop() as a starting > point for canonicalization? Do you mean using putting their function directly into Bro, or something else? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From jsiwek at illinois.edu Wed Mar 14 12:09:49 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 14 Mar 2012 19:09:49 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Updating baseline. (9dd63ac) In-Reply-To: <20120314185153.GA15663@icir.org> References: <201203132328.q2DNSV78015658@bro-ids.icir.org> <20120314185153.GA15663@icir.org> Message-ID: >> How about taking FreeBSD's implementation of inet_ntop() as a starting >> point for canonicalization? > > Do you mean using putting their function directly into Bro, or > something else? Yeah, taking their code as a starting point, but renaming to something like bro_inet_ntop and using that everywhere instead. +Jon From gregor at majordomus.org Wed Mar 14 13:27:21 2012 From: gregor at majordomus.org (Gregor Maier) Date: Wed, 14 Mar 2012 13:27:21 -0700 Subject: [Bro-Dev] IPv6 literal addr constants In-Reply-To: <20120313172303.GQ78256@icir.org> References: <1C107DD8-B77F-48CD-9E32-6812F9168AE2@illinois.edu> <4F5D2CA5.8030502@ee.lbl.gov> <20120312160134.GM27202@icir.org> <20120313043041.GC61817@icir.org> <5D5EE407-43E3-4C59-81DA-E7752B2B0A07@illinois.edu> <20120313172303.GQ78256@icir.org> Message-ID: <4F60FF29.2020105@majordomus.org> On 3/13/12 10:23 , Robin Sommer wrote: > > On Tue, Mar 13, 2012 at 17:09 +0000, you wrote: > >> Sounds like enough people like that syntax so I I'll add it, but what >> to do with the old syntax for IPv6 literals? Should it be removed at >> this time, deprecated until 2.2, or kept indefinitely? > > I vote for just removing. Now is the one time where we can break IPv6 > stuff to make it better. ACK. let's break it now! From bro at tracker.bro-ids.org Wed Mar 14 13:50:49 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 14 Mar 2012 20:50:49 -0000 Subject: [Bro-Dev] #531: Handle IPv6 protocol chains In-Reply-To: <048.a1036225e7e49822f38fb91c4b12ccff@tracker.bro-ids.org> References: <048.a1036225e7e49822f38fb91c4b12ccff@tracker.bro-ids.org> Message-ID: <063.a333d7f83f7cdb061eb6fc7916535fb6@tracker.bro-ids.org> #531: Handle IPv6 protocol chains ----------------------+------------------------ Reporter: gregor | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: IPv6 ----------------------+------------------------ Comment (by robin): In [79948c79741e005565ab0aa29006249014428905/bro]: {{{ #!CommitTicketReference repository="bro" revision="79948c79741e005565ab0aa29006249014428905" Merge remote-tracking branch 'origin/topic/jsiwek/ipv6-ext-headers' * origin/topic/jsiwek/ipv6-ext-headers: Update PacketFilter/Discarder code for IP version independence. Add a few comments to IP.h Fix some IPv6 header related bugs. Add IPv6 fragment reassembly. Add handling for IPv6 extension header chains (addresses #531) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Mar 14 22:03:43 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 14 Mar 2012 22:03:43 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] master: Updating baseline. (9dd63ac) In-Reply-To: References: <201203132328.q2DNSV78015658@bro-ids.icir.org> <20120314185153.GA15663@icir.org> Message-ID: <20120315050343.GA33182@icir.org> On Wed, Mar 14, 2012 at 19:09 +0000, you wrote: > Yeah, taking their code as a starting point, but renaming to something > like bro_inet_ntop and using that everywhere instead. Sounds good. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Thu Mar 15 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 15 Mar 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203150700.q2F702Bs014742@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 795 [1] | jsiwek | | Normal | topic/jsiwek/ipv6-ext-headers [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 94864da | Jon Siwek | 2012-03-14 | Update documentation for new syntax of IPv6 literals. [3] [1] #795: http://tracker.bro-ids.org/bro/ticket/795 [2] ipv6-ext-headers: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-ext-headers [3] fastpath: http://tracker.bro-ids.org/bro/changeset/94864da465a134bea251461adf588442e2d6d2bd/bro From bro at tracker.bro-ids.org Thu Mar 15 07:10:11 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 15 Mar 2012 14:10:11 -0000 Subject: [Bro-Dev] #795: topic/jsiwek/ipv6-ext-headers In-Reply-To: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> References: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> Message-ID: <063.d462760fb217676a25a3521b6e470ce0@tracker.bro-ids.org> #795: topic/jsiwek/ipv6-ext-headers ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by robin): * owner: => jsiwek * status: new => assigned * type: Merge Request => Task -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Mar 15 15:08:17 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 15 Mar 2012 22:08:17 -0000 Subject: [Bro-Dev] #797: broctl doesn't load policy specified in the SitePolicyPath and defaults to local.bro Message-ID: <049.10ac0122cad154cc2df5706bda12aac8@tracker.bro-ids.org> #797: broctl doesn't load policy specified in the SitePolicyPath and defaults to local.bro ---------------------+--------------------- Reporter: aashish | Type: Problem Status: new | Priority: Normal Milestone: | Component: Bro Version: 2.0 | Keywords: ---------------------+--------------------- broctl config shows: sitepolicymanager = local-manager.bro sitepolicypath = /usr/local/mysite/policies:/usr/local/bro/share/bro sitepolicystandalone = mysite.bro sitepolicyworker = local-worker.bro Unless I add mysite.bro to /usr/local/bro/share/bro/site/local.bro; bro won't actually load mysite.bro. Even though broctl config shows the sitepolicystandalone as mysite.bro it doesn't get loaded if the path is other than ../share/bro/site/ -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Mar 16 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 16 Mar 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203160700.q2G703hV013731@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 94864da | Jon Siwek | 2012-03-14 | Update documentation for new syntax of IPv6 literals. [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/94864da465a134bea251461adf588442e2d6d2bd/bro From seth at icir.org Fri Mar 16 06:22:53 2012 From: seth at icir.org (Seth Hall) Date: Fri, 16 Mar 2012 09:22:53 -0400 Subject: [Bro-Dev] open a pipe? Message-ID: How hard would it be to modify files so that we could use them like a pipe to another program? I'll give an example? global myfile = open("| shasum > output"); print myfile, "hello"; close(myfile); Then the output file would have the sha1 value for "hello". I don't think it would be a major change to files to support this, but I've been wrong before. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Fri Mar 16 08:34:34 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 16 Mar 2012 08:34:34 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/bernhard/input-threads: make raw reading work. (367c4b4) In-Reply-To: <201203161454.q2GEs50Z012086@bro-ids.icir.org> References: <201203161454.q2GEs50Z012086@bro-ids.icir.org> Message-ID: <20120316153434.GM58991@icir.org> On Fri, Mar 16, 2012 at 07:54 -0700, Bernhard Amann wrote: > apparently there was a crash in the reader plugin, but main bro > did not notice but waited for eternity for it do to something. Would be nice if main Bro noticed and just continued orderly. Something I haven't tried with the logging framework either yet ... Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Fri Mar 16 08:36:38 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 16 Mar 2012 08:36:38 -0700 Subject: [Bro-Dev] open a pipe? In-Reply-To: References: Message-ID: <20120316153638.GN58991@icir.org> On Fri, Mar 16, 2012 at 09:22 -0400, you wrote: > global myfile = open("| shasum > output"); I'm very reluctant to allow that without having script-level async I/O. What if the pipe blocks? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Fri Mar 16 08:55:45 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 16 Mar 2012 15:55:45 -0000 Subject: [Bro-Dev] #797: broctl doesn't load policy specified in the SitePolicyPath and defaults to local.bro In-Reply-To: <049.10ac0122cad154cc2df5706bda12aac8@tracker.bro-ids.org> References: <049.10ac0122cad154cc2df5706bda12aac8@tracker.bro-ids.org> Message-ID: <064.d3b122ac17a35ba12ffa6dde11864939@tracker.bro-ids.org> #797: broctl doesn't load policy specified in the SitePolicyPath and defaults to local.bro ----------------------+-------------------- Reporter: aashish | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: 2.0 Resolution: | Keywords: ----------------------+-------------------- Changes (by robin): * milestone: => Bro2.1 -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Fri Mar 16 09:05:31 2012 From: seth at icir.org (Seth Hall) Date: Fri, 16 Mar 2012 12:05:31 -0400 Subject: [Bro-Dev] open a pipe? In-Reply-To: <20120316153638.GN58991@icir.org> References: <20120316153638.GN58991@icir.org> Message-ID: <37D8733D-51F4-4D8F-B052-E4571B397126@icir.org> On Mar 16, 2012, at 11:36 AM, Robin Sommer wrote: > On Fri, Mar 16, 2012 at 09:22 -0400, you wrote: > >> global myfile = open("| shasum > output"); > > I'm very reluctant to allow that without having script-level async > I/O. What if the pipe blocks? Explosions and fire as you'd expect. :) Don't we already theoretically have this problem with the print statement? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Fri Mar 16 09:14:05 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 16 Mar 2012 09:14:05 -0700 Subject: [Bro-Dev] open a pipe? In-Reply-To: <37D8733D-51F4-4D8F-B052-E4571B397126@icir.org> References: <20120316153638.GN58991@icir.org> <37D8733D-51F4-4D8F-B052-E4571B397126@icir.org> Message-ID: <20120316161405.GA79764@icir.org> On Fri, Mar 16, 2012 at 12:05 -0400, you wrote: > Don't we already theoretically have this problem with the print statement? Yes, but it's harder to make that block. :) One we have the pipe, the next thing you'll be doing is printing to netcat. :) Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Fri Mar 16 09:21:38 2012 From: seth at icir.org (Seth Hall) Date: Fri, 16 Mar 2012 12:21:38 -0400 Subject: [Bro-Dev] open a pipe? In-Reply-To: <20120316161405.GA79764@icir.org> References: <20120316153638.GN58991@icir.org> <37D8733D-51F4-4D8F-B052-E4571B397126@icir.org> <20120316161405.GA79764@icir.org> Message-ID: On Mar 16, 2012, at 12:14 PM, Robin Sommer wrote: > One we have the pipe, the next thing you'll be doing is printing to > netcat. :) Dammit, that *is* one of my use cases. (not the only one though) You know me too well! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bernhard at ICSI.Berkeley.EDU Fri Mar 16 11:04:52 2012 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Fri, 16 Mar 2012 11:04:52 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/bernhard/input-threads: make raw reading work. (367c4b4) In-Reply-To: <20120316153434.GM58991@icir.org> References: <201203161454.q2GEs50Z012086@bro-ids.icir.org> <20120316153434.GM58991@icir.org> Message-ID: <8AD04971-22AB-4794-996D-64CA6B8D0296@icsi.berkeley.edu> On Mar 16, 2012, at 8:34 AM, Robin Sommer wrote: > > On Fri, Mar 16, 2012 at 07:54 -0700, Bernhard Amann wrote: > >> apparently there was a crash in the reader plugin, but main bro >> did not notice but waited for eternity for it do to something. > > Would be nice if main Bro noticed and just continued orderly. > Something I haven't tried with the logging framework either yet ? An error message would also be kind of nice :). The behavior I saw was that it was just "stuck" - bro didn't do anything, debug output didn't show anything and once one pressed ctr+c it shut down. But - when attaching gdb, one could see the crash (null pointer dereference) - which then was quite easy to fix. Bernhard From vallentin at icir.org Fri Mar 16 12:46:23 2012 From: vallentin at icir.org (Matthias Vallentin) Date: Fri, 16 Mar 2012 12:46:23 -0700 Subject: [Bro-Dev] The BPAN project Message-ID: We've been talking a bit about our Bro community script archive BPAN [1] quite a bit lately. Below, I summarize how I envision this project to manifest. Feedback would be much appreciated. Matthias Objectives - Facilitate sharing Bro scripts in the community - Facilitate the integration of user-provided scripts - Specify a set of repositories (or paths inside) to load Bro scripts Requirements: - Sharing scripts should be simple: one should be able to point someone to a remote repository and the integration happens automatically. - Organization by topic tags: similar to package managers, we need a way to categorize scripts and bundle them. For example - Separation between officially supported and user/experimental scripts. For the official ones, we offer support and make sure they'll always work with the latest/current version of Bro. We do not make such a guarantee for external scripts. - Integration with the Input/Output framework. That is, users should be able to provide custom data sources and sinks. Implementation details: - github's popular fork-pull-merge model is an excellent fit for sharing community repositories. We may want to use https://github.com/broids as base for our community scripts. Users that want to extend the base simply perform the changes, open a pull request, and we can review it. The nice thing is that people can comment on it and follow-up changes are nicely reflected. Moreover, github automatically lists contributors to a certain file, so this encourages coders to share, participate, and get "reputation"---akin to stackoverflow. - Unlike homebrew, I believe it does not make sense for Bro to have single huge repository that contains all scripts and that users continuously branch from. Bro users probably want to organize their scripts into different repositories and combine multiple repositories ad libitum, because some repositories may contain sensitive elements and should not be public. Yet, private repositories can still be easily integrated with this new framework. Having separate repositories also keeps the repository size under control. Although the input framework will be the main mechanism to source external data, I could imagine some folks would like to stuff some file-based intelligence in their repositories. However, separate repositories require integration efforts, but this is manageable in my eyes and outlined below. - Directory structure: the directory structure provides implicit naming and some meta data. If the directly structure is violated, Bro will not be able to automatically load scripts. These are the rules: * Sub-directories in the repository represent "addressable units," e.g., names that users can refer to. I call these from now on a "rebro". Each rebro must have at least one file called '__load__.bro'. This is the entry point for Bro. Example: - repo := https://www.github.com/joe-user/bromising/ - repo/ repo/a/ repo/a/__load__.bro repo/a/aa.bro repo/a/ab.bro repo/b/__load__.bro/ repo/b/x/__load__.bro repo/b/x/xx1.bro repo/b/x/xx2.bro repo/b/y/yy1.bro In this example, we have the following rebros: repo/a repo/b repo/b/x By placing __load__.bro files judiciously, users can explicitly decide what parts of the repo may be interpreted as a rebro. We may also allow putting a __load__.bro file in the top-level directory. * Each __load__.bro file contains additional meta data. Here is an example ## @version: 0.42 ## @author: Joe Bro ## ## @license: BSD-style (should be the default unless specified) ## ## @bro: 2.1 ## @dependencies: b/x, https://my-repo.git/foo ## ## @tags: tls, x509, browser - The @-prefix inside comments may be used as a specific marker for parsing. But any other character would do as well. - @author: An optional field that identifies the script author's name and email address. If not given and the author hosts its repository on github, we may extract this data from github. - @license: An optional field to name the license under which this rebro should go - @bro: A mandatory field that specifies the Bro version this rebro is compatible with. Examples: * =major.minor exact version match, e.g., =2.1 * >=major.minor minimum version, e.g., >=2.0 * major.minor-major.minor range of comptabile versions, e.g., 2.0-2.1 * major.minor,major.minor list of compatible versions, e.g., 1.5, 2.0beta - @dependencies: An optional field of internal and external rebro dependencies. Internal dependencies are specified relative to the repository top-level directory and external dependencies need a full-qualified rebro path. - @tags: An optional field that imposes a logical structure over rebros. Or we just keep it an unordered list of tags that users can combine at will. E.g., @tags: tls, x509, browser would associate three tags with this rebro. We may pre-define a set of categories under which users can label their rebros, e.g., @tags: webapp/facebook, webapp/facebook would categorize the rebro hierarchically. - Let the union of all rebros be the 'universe'. We, the Bro team, should provide a global view of the universe by encouraging users to register their rebro's with us. The point of the universe is that it allows a global search across all existing rebros out there. One should be able to search the universe by tags, names, versions, etc. We may cache this meta data in a separate repository. A script may re-generate the meta data whenever a new rebro is added to it. - Configuration: Bro needs to be told which script repositories to integrate. I suggest all repositories sit in a one directory, that is controllable via BroControl.cfg, e.g., Community = $PREFIX/share/bro/community Inside that directory, we have two subdirectories: one for all the repositories and one for the universe, e.g., $Community/repositories/git.bro-ids.org/skype-analyzer $Community/repositories/github.com/git-user/experimental-scripts $Community/repositories/github.com/git-user/popular-stuff $Community/universe/... - One should be able to use BroControl to add new repositories at runtime and reload Bro with these scripts. Here are some examples that I will explain below > rebro add https://git.bro-ids.org/skype-analyzer [alias] > rebro add git-user/experimental-scripts [testing] > rebro rm testing > rebro load alias/decryptor > rebro load -r alias/spit-injector > rebro list > rebro list -u > rebro list -u -t > rebro pull > rebro deps alias > rebro deps alias/decryptor > rebro license alias Commands: * add: downloads a remote repository and register make all rebros inside it loadable * rm: removes a remote repository. * load: loads a specific rebro. The command fails unless all dependency requirements are met. --recursive|-r: downloads dependencies automatically * list: lists all available loadable rebros. --universe|-u: performs a global search in the universe --tags|-t: lists rebro's by tags * deps: lists dependencies of a rebro. * license: displays licencing information about a rebro. [1] Here are some other naming suggestions instead of BPAN: - Bropane (Bro Policy script Archive NEtwork; inflammable) - Brofuse (profuse: exuberantly plentiful) - Browl (A bowl full of Bro scripts) - Broil (Cooking with the community) - Brodder (Bro protestants) - Brovine (Grass-fed, no growth hormones) From noreply at bro-ids.org Sat Mar 17 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 17 Mar 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203170700.q2H702t2007609@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 94864da | Jon Siwek | 2012-03-14 | Update documentation for new syntax of IPv6 literals. [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/94864da465a134bea251461adf588442e2d6d2bd/bro From noreply at bro-ids.org Sun Mar 18 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 18 Mar 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203180700.q2I703ul007064@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 94864da | Jon Siwek | 2012-03-14 | Update documentation for new syntax of IPv6 literals. [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/94864da465a134bea251461adf588442e2d6d2bd/bro From noreply at bro-ids.org Mon Mar 19 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 19 Mar 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203190700.q2J703No017804@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 94864da | Jon Siwek | 2012-03-14 | Update documentation for new syntax of IPv6 literals. [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/94864da465a134bea251461adf588442e2d6d2bd/bro From slagell at illinois.edu Mon Mar 19 08:57:32 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Mon, 19 Mar 2012 15:57:32 +0000 Subject: [Bro-Dev] The BPAN project In-Reply-To: References: Message-ID: <618953DA-0162-4768-8CF3-4B908072EA48@illinois.edu> On Mar 16, 2012, at 2:46 PM, Matthias Vallentin wrote: > > > Requirements: > > - Sharing scripts should be simple: one should be able to point > someone to a remote repository and the integration happens > automatically. Dependencies will likely arise. Particularly problematic are extension scripts which require changes in base scripts that we ship. I don't think we can allow new scripts that require tweaks to base scripts. > > - Organization by topic tags: similar to package managers, we need a > way to categorize scripts and bundle them. For example > > - Separation between officially supported and user/experimental > scripts. For the official ones, we offer support and make sure > they'll always work with the latest/current version of Bro. We do > not make such a guarantee for external scripts. I think the clear line between supported/not supported is what we ship with our tarballs. > > [1] Here are some other naming suggestions instead of BPAN: > > - Bropane (Bro Policy script Archive NEtwork; inflammable) +1 > - Brofuse (profuse: exuberantly plentiful) > - Browl (A bowl full of Bro scripts) > - Broil (Cooking with the community) > - Brodder (Bro protestants) > - Brovine (Grass-fed, no growth hormones) > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > ------ Adam J. Slagell, CISO, CISSP Chief Information Security Officer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info 217.244.8965 "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From jsiwek at illinois.edu Mon Mar 19 09:12:19 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Mon, 19 Mar 2012 16:12:19 +0000 Subject: [Bro-Dev] The BPAN project In-Reply-To: References: Message-ID: > * Each __load__.bro file contains additional meta data. Here is > an example > > ## @version: 0.42 > ## @author: Joe Bro > ## > ## @license: BSD-style (should be the default unless specified) > ## > ## @bro: 2.1 > ## @dependencies: b/x, https://my-repo.git/foo > ## > ## @tags: tls, x509, browser Maybe also a short paragraph description of the repo's purpose/goal/function could be included here. And assuming that, for the officially supported repo, we'll also want auto-generated documentation, note that there's currently some scripting that occurs to take reST markup from README files in the same directory as __load__.bro files and display that in the "index of all packages" along with the link that lets you drill down into individual scripts within. We didn't actually get around to using that anywhere, so it's probably not a big deal to change to using __load__.bro instead, but my main point is that the metadata should get specified in a way that can be rendered in to the docs. > > - The @-prefix inside comments may be used as a specific > marker for parsing. But any other character would do as > well. Maybe we could make this reST-compatible markup to start with for reasons above? +Jon From bro at tracker.bro-ids.org Mon Mar 19 09:19:07 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 19 Mar 2012 16:19:07 -0000 Subject: [Bro-Dev] #798: topic/jsiwek/destdir-fix Message-ID: <048.3dd86a4e6c3dec62810e5f6c2e0d1dda@tracker.bro-ids.org> #798: topic/jsiwek/destdir-fix ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch (in `cmake` repo only) teaches some CMake scripts that run at install-time to respect the DESTDIR environment variable. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Mar 19 09:37:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 19 Mar 2012 16:37:19 -0000 Subject: [Bro-Dev] #799: topic/jsiwek/bro_inet_ntop Message-ID: <048.0b8f09afe39f7390d27121e1960938a0@tracker.bro-ids.org> #799: topic/jsiwek/bro_inet_ntop ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch in the `bro` repo normalizes some uses of `inet_ntop` to use FreeBSD's implementation (renamed to `bro_inet_ntop`) so that results are more consistent across platforms. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Mar 20 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 20 Mar 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203200700.q2K703q1026145@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 798 [1] | jsiwek | | Normal | topic/jsiwek/destdir-fix [2] Bro | 799 [3] | jsiwek | | Normal | topic/jsiwek/bro_inet_ntop [4] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 94864da | Jon Siwek | 2012-03-14 | Update documentation for new syntax of IPv6 literals. [5] [1] #798: http://tracker.bro-ids.org/bro/ticket/798 [2] destdir-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/destdir-fix [3] #799: http://tracker.bro-ids.org/bro/ticket/799 [4] bro_inet_ntop: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/bro_inet_ntop [5] fastpath: http://tracker.bro-ids.org/bro/changeset/94864da465a134bea251461adf588442e2d6d2bd/bro From vallentin at icir.org Tue Mar 20 12:36:53 2012 From: vallentin at icir.org (Matthias Vallentin) Date: Tue, 20 Mar 2012 12:36:53 -0700 Subject: [Bro-Dev] IPv4 and IPv6 addresses in Broccoli Message-ID: Broccoli uses the following structure to represent addresses: typedef struct bro_addr { uint32 addr[4]; /**< IP address in network byte order */ int size; /**< Number of 4-byte words occupied in addr */ } BroAddr; Why do we need the second field? That is, why don't we use a single 16 byte array, where the version distinction occurs implicitly through the IPv4 mapped prefix (as in Bro)? Matthias From bro at tracker.bro-ids.org Tue Mar 20 13:50:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 20 Mar 2012 20:50:51 -0000 Subject: [Bro-Dev] #795: topic/jsiwek/ipv6-ext-headers In-Reply-To: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> References: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> Message-ID: <063.de5e88dfe8ab7f2823d1746402dbc4a9@tracker.bro-ids.org> #795: topic/jsiwek/ipv6-ext-headers ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by jsiwek): In [1c1d6570395432b9bfef8fb9ab1c27f02491f754/bro]: {{{ #!CommitTicketReference repository="bro" revision="1c1d6570395432b9bfef8fb9ab1c27f02491f754" Changes to IPv6 ext. header parsing (addresses #795). In response to feedback from Robin: - rename "ip_hdr" to "ip4_hdr" - pkt_hdr$ip6 is now of type "ip6_hdr" instead of "ip6_hdr_chain" - "ip6_hdr_chain" no longer contains an "ip6_hdr" field, instead it's the other way around, "ip6_hdr" contains an "ip6_hdr_chain" - other internal refactoring }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Mar 20 13:58:07 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 20 Mar 2012 20:58:07 -0000 Subject: [Bro-Dev] #795: topic/jsiwek/ipv6-ext-headers In-Reply-To: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> References: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> Message-ID: <063.b872eebe75d1036517522b7115b2b926@tracker.bro-ids.org> #795: topic/jsiwek/ipv6-ext-headers ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * owner: jsiwek => robin * type: Task => Merge Request Comment: Replying to [comment:2 robin]: > I'm going to commit feedback into the branch, marked with '[Robin]' I responded to them all in the commit above by either removing your comment and making the suggested change or by adding my own '[Jon]' comments. If it all seems reasonable, you can probably just remove all of our '[]' comments and merge, else commit some more feedback to the branch and switch it back to a task for me. -- Ticket URL: Bro Tracker Bro Issue Tracker From jsiwek at illinois.edu Tue Mar 20 14:29:33 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Tue, 20 Mar 2012 21:29:33 +0000 Subject: [Bro-Dev] IPv4 and IPv6 addresses in Broccoli In-Reply-To: References: Message-ID: > Broccoli uses the following structure to represent addresses: > > typedef struct bro_addr { > uint32 addr[4]; /**< IP address in network byte order */ > int size; /**< Number of 4-byte words occupied in addr */ > } BroAddr; > > Why do we need the second field? That is, why don't we use a single 16 byte > array, where the version distinction occurs implicitly through the IPv4 > mapped prefix (as in Bro)? I think I had done that just to be more like how the serialization process in Bro represents addresses as the length of the address in bytes followed by the address, but it could also construct the IPv4 mapped addresses like you said instead and it would be fine, too. Think it's worth changing? Should we also change the serialization to always represent addresses as 16 bytes? Seems inefficient right now, but I guess it would eventually become better once IPv6 addresses are the more frequent type. +Jon From robin at icir.org Tue Mar 20 15:20:15 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 20 Mar 2012 15:20:15 -0700 Subject: [Bro-Dev] IPv4 and IPv6 addresses in Broccoli In-Reply-To: References: Message-ID: <20120320222015.GK81568@icir.org> On Tue, Mar 20, 2012 at 21:29 +0000, you wrote: > I think I had done that just to be more like how the serialization > process in Bro represents addresses as the length of the address in I actually prefer the explicit size field for Broccoli but for a different reason: it makes it easier for applications to differentiate between IPv4 and v6. Otherwise, each client application needs to implement our v4-in-v6 mapping in same way (even if standard). With that reasoning, we could even make it more explicit: instead of giving size, give the protocol directly and use a union with separate fields for v4 and v6 address. > Should we also change the serialization to always represent addresses > as 16 bytes? Seems inefficient right now, but I guess it would > eventually become better once IPv6 addresses are the more frequent > type. (No opionion here, fine either way). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From vallentin at icir.org Tue Mar 20 17:59:50 2012 From: vallentin at icir.org (Matthias Vallentin) Date: Tue, 20 Mar 2012 17:59:50 -0700 Subject: [Bro-Dev] IPv4 and IPv6 addresses in Broccoli In-Reply-To: <20120320222015.GK81568@icir.org> References: <20120320222015.GK81568@icir.org> Message-ID: > I actually prefer the explicit size field for Broccoli but for a > different reason: it makes it easier for applications to differentiate > between IPv4 and v6. Otherwise, each client application needs to > implement our v4-in-v6 mapping in same way (even if standard). I see your point, but wouldn't it suffice to provide an additional boolean Broccoli function for that purpose? >> Should we also change the serialization to always represent addresses >> as 16 bytes? ?Seems inefficient right now, but I guess it would >> eventually become better once IPv6 addresses are the more frequent >> type. > > (No opionion here, fine either way). I agree with Jon that such a change may not yet amortize, but at some point in the future. We could still revisit it then. Matthias From vallentin at icir.org Tue Mar 20 18:12:30 2012 From: vallentin at icir.org (Matthias Vallentin) Date: Tue, 20 Mar 2012 18:12:30 -0700 Subject: [Bro-Dev] The BPAN project In-Reply-To: References: Message-ID: > Maybe we could make this reST-compatible markup to start with for reasons above? Makes perfectly sense. I updated the corresponding project page with your and Adam's feedback: http://www.bro-ids.org/development/projects/cban.html Matthias From vern at icir.org Tue Mar 20 19:24:03 2012 From: vern at icir.org (Vern Paxson) Date: Tue, 20 Mar 2012 19:24:03 -0700 Subject: [Bro-Dev] IPv4 and IPv6 addresses in Broccoli In-Reply-To: <20120320222015.GK81568@icir.org> (Tue, 20 Mar 2012 15:20:15 PDT). Message-ID: <20120321022403.7997C2C4009@rock.ICSI.Berkeley.EDU> > implement our v4-in-v6 mapping in same way (even if standard). (FYI, it is indeed standard) From noreply at bro-ids.org Wed Mar 21 00:00:05 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 21 Mar 2012 00:00:05 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203210700.q2L70531011709@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 795 [1] | jsiwek | robin | Normal | topic/jsiwek/ipv6-ext-headers [2] Bro | 798 [3] | jsiwek | | Normal | topic/jsiwek/destdir-fix [4] Bro | 799 [5] | jsiwek | | Normal | topic/jsiwek/bro_inet_ntop [6] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 94864da | Jon Siwek | 2012-03-14 | Update documentation for new syntax of IPv6 literals. [7] [1] #795: http://tracker.bro-ids.org/bro/ticket/795 [2] ipv6-ext-headers: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-ext-headers [3] #798: http://tracker.bro-ids.org/bro/ticket/798 [4] destdir-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/destdir-fix [5] #799: http://tracker.bro-ids.org/bro/ticket/799 [6] bro_inet_ntop: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/bro_inet_ntop [7] fastpath: http://tracker.bro-ids.org/bro/changeset/94864da465a134bea251461adf588442e2d6d2bd/bro From robin at icir.org Wed Mar 21 08:14:52 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 21 Mar 2012 08:14:52 -0700 Subject: [Bro-Dev] IPv4 and IPv6 addresses in Broccoli In-Reply-To: References: <20120320222015.GK81568@icir.org> Message-ID: <20120321151452.GB92677@icir.org> On Tue, Mar 20, 2012 at 17:59 -0700, you wrote: > I see your point, but wouldn't it suffice to provide an additional > boolean Broccoli function for that purpose? Sure, that would work as well. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Wed Mar 21 08:35:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 21 Mar 2012 15:35:02 -0000 Subject: [Bro-Dev] #795: topic/jsiwek/ipv6-ext-headers In-Reply-To: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> References: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> Message-ID: <063.e2323152624861e6eb6df84e86da53f8@tracker.bro-ids.org> #795: topic/jsiwek/ipv6-ext-headers ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by jsiwek): In [c765f43fe3eb6fd4cb49b2b947654881a225e145/bro]: {{{ #!CommitTicketReference repository="bro" revision="c765f43fe3eb6fd4cb49b2b947654881a225e145" Refactor script-layer IPv6 ext. header chain (addresses #795) This replaces the "ip6_hdr_chain" in the "ip6_hdr" record with a vector of "ip6_ext_hdr" to make it easier to traverse the chain. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Mar 21 08:41:33 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 21 Mar 2012 15:41:33 -0000 Subject: [Bro-Dev] #795: topic/jsiwek/ipv6-ext-headers In-Reply-To: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> References: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> Message-ID: <063.1402fcf4815c8225b6d7e33edd606ed8@tracker.bro-ids.org> #795: topic/jsiwek/ipv6-ext-headers ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by jsiwek): > In [c765f43fe3eb6fd4cb49b2b947654881a225e145/bro]: > {{{ > #!CommitTicketReference repository="bro" revision="c765f43fe3eb6fd4cb49b2b947654881a225e145" > Refactor script-layer IPv6 ext. header chain (addresses #795) > > This replaces the "ip6_hdr_chain" in the "ip6_hdr" record with a vector of > "ip6_ext_hdr" to make it easier to traverse the chain. > }}} While it does make it easier to traverse the chain in the original order of the headers, it makes it harder to directly access whether any given ext. header is present without looping over the chain. I do think this new way is cleaner and easier to understand, though, so maybe it wins out. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Mar 21 11:23:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 21 Mar 2012 18:23:09 -0000 Subject: [Bro-Dev] #800: Change Broccoli BroAddr type to a single 16 byte array Message-ID: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> #800: Change Broccoli BroAddr type to a single 16 byte array -----------------------------+------------------------ Reporter: matthias | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: git/master Keywords: | -----------------------------+------------------------ Broccoli currently uses an extra size field to distinguish IPv4 and IPv6 addresses: {{{ typedef struct bro_addr { uint32 addr[4]; /**< IP address in network byte order */ int size; /**< Number of 4-byte words occupied in addr */ } BroAddr; }}} Since there exists a standard mapping for IPv4 in IPv6 addresses, we can get rid of the extra size field. To facilitate the distinction between IPv4 and IPv6, we could provide a boolean function that tests the first 12 bytes against the mapped prefix. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Mar 22 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 22 Mar 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203220700.q2M703ws015254@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 795 [1] | jsiwek | robin | Normal | topic/jsiwek/ipv6-ext-headers [2] Bro | 798 [3] | jsiwek | | Normal | topic/jsiwek/destdir-fix [4] Bro | 799 [5] | jsiwek | | Normal | topic/jsiwek/bro_inet_ntop [6] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 94864da | Jon Siwek | 2012-03-14 | Update documentation for new syntax of IPv6 literals. [7] [1] #795: http://tracker.bro-ids.org/bro/ticket/795 [2] ipv6-ext-headers: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-ext-headers [3] #798: http://tracker.bro-ids.org/bro/ticket/798 [4] destdir-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/destdir-fix [5] #799: http://tracker.bro-ids.org/bro/ticket/799 [6] bro_inet_ntop: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/bro_inet_ntop [7] fastpath: http://tracker.bro-ids.org/bro/changeset/94864da465a134bea251461adf588442e2d6d2bd/bro From bro at tracker.bro-ids.org Thu Mar 22 12:07:57 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 22 Mar 2012 19:07:57 -0000 Subject: [Bro-Dev] #800: Change Broccoli BroAddr type to a single 16 byte array In-Reply-To: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> References: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> Message-ID: <065.85e3a1d1426945c1dcc868090b404121@tracker.bro-ids.org> #800: Change Broccoli BroAddr type to a single 16 byte array ------------------------------+------------------------ Reporter: matthias | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Comment (by jsiwek): In [ac82a5b4086cf4e0627a7b0d64dc13e5b11f38b6/broccoli]: {{{ #!CommitTicketReference repository="broccoli" revision="ac82a5b4086cf4e0627a7b0d64dc13e5b11f38b6" Change BroAddr to use standard IPv4 in IPv6 mapping (addresses #800) The size field is now removed and the bro_util_is_v4_addr() function can be used instead to check whether the BroAddr is IPv4 or not. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Mar 22 13:10:49 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 22 Mar 2012 20:10:49 -0000 Subject: [Bro-Dev] #793: istate.broccoli-ipv6 fails on FreeBSD In-Reply-To: <047.06a7b6c1d5b8682c072c3076abd9a768@tracker.bro-ids.org> References: <047.06a7b6c1d5b8682c072c3076abd9a768@tracker.bro-ids.org> Message-ID: <062.48d15e1da77092cf119a391acefb0912@tracker.bro-ids.org> #793: istate.broccoli-ipv6 fails on FreeBSD ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [f60523c24d1f889bd6bae7ebf2f1f4d15f5aeed2/broccoli]: {{{ #!CommitTicketReference repository="broccoli" revision="f60523c24d1f889bd6bae7ebf2f1f4d15f5aeed2" Add timeout to broccoli-v6addrs.c test. (addresses #793) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Mar 22 13:13:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 22 Mar 2012 20:13:21 -0000 Subject: [Bro-Dev] #800: Change Broccoli BroAddr type to a single 16 byte array In-Reply-To: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> References: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> Message-ID: <065.122af5fc55003a860cd03bfb71f84101@tracker.bro-ids.org> #800: Change Broccoli BroAddr type to a single 16 byte array ----------------------------+------------------------ Reporter: matthias | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * type: Feature Request => Merge Request Comment: This is changed in `topic/jsiwek/broccoli-v4-mapped` in broccoli, broccoli-python, and broccoli-ruby repos. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Mar 22 17:19:47 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 23 Mar 2012 00:19:47 -0000 Subject: [Bro-Dev] #801: Configure option for specifying path for ../etc/ Message-ID: <049.b7db21b4f1420f3f6902534231deb9b2@tracker.bro-ids.org> #801: Configure option for specifying path for ../etc/ ------------------------+----------------------------- Reporter: aashish | Type: Feature Request Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: ------------------------+----------------------------- It would be useful to get a configure option to specify path for ../etc and all the files within. Currently defaults to ....prefix/etc/ and has files broctl.cfg (along with node.cfg and networks.cfg) Why is this needed? Keeping broctl.cfg out of the bro-base install, allows a tighter control on version control of the files in etc folder along with the other site specific policy files which are also outside the default /usr/local/bro Thanks, -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Mar 23 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 23 Mar 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203230700.q2N70205028390@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 795 [1] | jsiwek | robin | Normal | topic/jsiwek/ipv6-ext-headers [2] Bro | 798 [3] | jsiwek | | Normal | topic/jsiwek/destdir-fix [4] Bro | 799 [5] | jsiwek | | Normal | topic/jsiwek/bro_inet_ntop [6] Broccoli | 800 [7] | matthias | | Normal | Change Broccoli BroAddr type to a single 16 byte array > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 94864da | Jon Siwek | 2012-03-14 | Update documentation for new syntax of IPv6 literals. [8] broccoli | f60523c | Jon Siwek | 2012-03-22 | Add timeout to broccoli-v6addrs.c test. (addresses #793) [9] [1] #795: http://tracker.bro-ids.org/bro/ticket/795 [2] ipv6-ext-headers: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/ipv6-ext-headers [3] #798: http://tracker.bro-ids.org/bro/ticket/798 [4] destdir-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/destdir-fix [5] #799: http://tracker.bro-ids.org/bro/ticket/799 [6] bro_inet_ntop: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/bro_inet_ntop [7] #800: http://tracker.bro-ids.org/bro/ticket/800 [8] fastpath: http://tracker.bro-ids.org/bro/changeset/94864da465a134bea251461adf588442e2d6d2bd/bro [9] fastpath: http://tracker.bro-ids.org/bro/changeset/f60523c24d1f889bd6bae7ebf2f1f4d15f5aeed2/broccoli From bro at tracker.bro-ids.org Fri Mar 23 08:19:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 23 Mar 2012 15:19:03 -0000 Subject: [Bro-Dev] #801: Configure option for specifying path for ../etc/ In-Reply-To: <049.b7db21b4f1420f3f6902534231deb9b2@tracker.bro-ids.org> References: <049.b7db21b4f1420f3f6902534231deb9b2@tracker.bro-ids.org> Message-ID: <064.be58e0bc9d3941e334b55727b86a7f01@tracker.bro-ids.org> #801: Configure option for specifying path for ../etc/ ------------------------------+------------------------ Reporter: aashish | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Changes (by robin): * milestone: => Bro2.1 Comment: Yeah, we can add that. As a work-around, a symlink should work as well. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 23 09:25:49 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 23 Mar 2012 16:25:49 -0000 Subject: [Bro-Dev] #802: A Broccoli server to send IDMEF alerts via prelude Message-ID: <055.a4daa01ad386fdf32154e3c9acaf0466@tracker.bro-ids.org> #802: A Broccoli server to send IDMEF alerts via prelude ---------------------------+---------------------- Reporter: JulienSentier | Type: Patch Status: new | Priority: Normal Milestone: | Component: Broccoli Version: git/master | Keywords: ---------------------------+---------------------- Here is an application which uses to Broccoli. It runs as a server waiting for multiple Bros to connect. It requests a certain event, and using its parameters, makes an IDMEF alert out of it and sends it via libprelude. In the archive, there is a Bro script given as an example, with a dummy alert generation. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 23 16:16:37 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 23 Mar 2012 23:16:37 -0000 Subject: [Bro-Dev] #800: Change Broccoli BroAddr type to a single 16 byte array In-Reply-To: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> References: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> Message-ID: <065.b88bcc35311d1daf6b0e4ddfa0d1a1f5@tracker.bro-ids.org> #800: Change Broccoli BroAddr type to a single 16 byte array ----------------------------+------------------------ Reporter: matthias | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Merged, but should we now also provide helper functions to convert the address value from/to in_addr/in6_addr so that we directly offer access with standard C data types? But it's not crucial, so feel free to close the ticket alternatively. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 23 17:47:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 24 Mar 2012 00:47:46 -0000 Subject: [Bro-Dev] #799: topic/jsiwek/bro_inet_ntop In-Reply-To: <048.0b8f09afe39f7390d27121e1960938a0@tracker.bro-ids.org> References: <048.0b8f09afe39f7390d27121e1960938a0@tracker.bro-ids.org> Message-ID: <063.0ca45fba9e5a8a19e31574e3c91e3e0a@tracker.bro-ids.org> #799: topic/jsiwek/bro_inet_ntop ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [80e856bbe977bd9533104df647e806153f6f34da/bro]: {{{ #!CommitTicketReference repository="bro" revision="80e856bbe977bd9533104df647e806153f6f34da" Merge remote-tracking branch 'origin/topic/jsiwek/bro_inet_ntop' * origin/topic/jsiwek/bro_inet_ntop: Adapt FreeBSD's inet_ntop implementation for internal use. Closes #799. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 23 17:47:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 24 Mar 2012 00:47:46 -0000 Subject: [Bro-Dev] #531: Handle IPv6 protocol chains In-Reply-To: <048.a1036225e7e49822f38fb91c4b12ccff@tracker.bro-ids.org> References: <048.a1036225e7e49822f38fb91c4b12ccff@tracker.bro-ids.org> Message-ID: <063.ac0fc7d4573404ae6f9e21eefb1099b9@tracker.bro-ids.org> #531: Handle IPv6 protocol chains ----------------------+------------------------ Reporter: gregor | Owner: jsiwek Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: IPv6 ----------------------+------------------------ Comment (by robin): In [02d8c52e6f4de6bfcab6b36b71e07526f7fd93ad/bro]: {{{ #!CommitTicketReference repository="bro" revision="02d8c52e6f4de6bfcab6b36b71e07526f7fd93ad" Merge branch 'topic/jsiwek/ipv6-ext-headers' * topic/jsiwek/ipv6-ext-headers: Cosmetics in preparation for merge. Removing remaining comments. Looks fine. Refactor script-layer IPv6 ext. header chain (addresses #795) Changes to IPv6 ext. header parsing (addresses #795). Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. Remove the default "tcp or udp or icmp" filter. Merge remote-tracking branch 'origin/topic/jsiwek/ipv6-ext-headers' Add unit test for IPv6 fragment reassembly. Update PacketFilter/Discarder code for IP version independence. Add a few comments to IP.h Fix some IPv6 header related bugs. Add IPv6 fragment reassembly. Add handling for IPv6 extension header chains (addresses #531) Closes #795. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 23 17:47:47 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 24 Mar 2012 00:47:47 -0000 Subject: [Bro-Dev] #795: topic/jsiwek/ipv6-ext-headers In-Reply-To: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> References: <048.146e4251fa7435f405317b81cec535c5@tracker.bro-ids.org> Message-ID: <063.db86c5b94fb13524ca87f0009f85e842@tracker.bro-ids.org> #795: topic/jsiwek/ipv6-ext-headers ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * status: assigned => closed * resolution: => fixed Comment: In [02d8c52e6f4de6bfcab6b36b71e07526f7fd93ad/bro]: {{{ #!CommitTicketReference repository="bro" revision="02d8c52e6f4de6bfcab6b36b71e07526f7fd93ad" Merge branch 'topic/jsiwek/ipv6-ext-headers' * topic/jsiwek/ipv6-ext-headers: Cosmetics in preparation for merge. Removing remaining comments. Looks fine. Refactor script-layer IPv6 ext. header chain (addresses #795) Changes to IPv6 ext. header parsing (addresses #795). Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. Remove the default "tcp or udp or icmp" filter. Merge remote-tracking branch 'origin/topic/jsiwek/ipv6-ext-headers' Add unit test for IPv6 fragment reassembly. Update PacketFilter/Discarder code for IP version independence. Add a few comments to IP.h Fix some IPv6 header related bugs. Add IPv6 fragment reassembly. Add handling for IPv6 extension header chains (addresses #531) Closes #795. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 23 17:48:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 24 Mar 2012 00:48:10 -0000 Subject: [Bro-Dev] #793: istate.broccoli-ipv6 fails on FreeBSD In-Reply-To: <047.06a7b6c1d5b8682c072c3076abd9a768@tracker.bro-ids.org> References: <047.06a7b6c1d5b8682c072c3076abd9a768@tracker.bro-ids.org> Message-ID: <062.8b51edb50610461eb3cdaf8843287642@tracker.bro-ids.org> #793: istate.broccoli-ipv6 fails on FreeBSD ----------------------+------------------------ Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): In [cb8efb0b46c164a2e753588fe18cc50164edfd8f/broccoli]: {{{ #!CommitTicketReference repository="broccoli" revision="cb8efb0b46c164a2e753588fe18cc50164edfd8f" Merge remote-tracking branch 'origin/fastpath' * origin/fastpath: Add timeout to broccoli-v6addrs.c test. (addresses #793) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 23 17:48:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 24 Mar 2012 00:48:10 -0000 Subject: [Bro-Dev] #800: Change Broccoli BroAddr type to a single 16 byte array In-Reply-To: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> References: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> Message-ID: <065.1ace5749f4b5ff3864472f019d954681@tracker.bro-ids.org> #800: Change Broccoli BroAddr type to a single 16 byte array ----------------------------+------------------------ Reporter: matthias | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): In [612e95ac62a06b32b2e9e627f30527012a89a12c/broccoli]: {{{ #!CommitTicketReference repository="broccoli" revision="612e95ac62a06b32b2e9e627f30527012a89a12c" Merge remote-tracking branch 'origin/topic/jsiwek/broccoli-v4-mapped' * origin/topic/jsiwek/broccoli-v4-mapped: Change BroAddr to use standard IPv4 in IPv6 mapping (addresses #800) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 23 20:34:57 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 24 Mar 2012 03:34:57 -0000 Subject: [Bro-Dev] #800: Change Broccoli BroAddr type to a single 16 byte array In-Reply-To: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> References: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> Message-ID: <065.a7eeb7949332201abc1a9d234b06374e@tracker.bro-ids.org> #800: Change Broccoli BroAddr type to a single 16 byte array ----------------------------+------------------------ Reporter: matthias | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by matthias): Thanks for addressing this so quickly Jon! -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Mar 24 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 24 Mar 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203240700.q2O703Nd028908@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 798 [1] | jsiwek | | Normal | topic/jsiwek/destdir-fix [2] Broccoli | 800 [3] | matthias | | Normal | Change Broccoli BroAddr type to a single 16 byte array [1] #798: http://tracker.bro-ids.org/bro/ticket/798 [2] destdir-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/destdir-fix [3] #800: http://tracker.bro-ids.org/bro/ticket/800 From noreply at bro-ids.org Sun Mar 25 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 25 Mar 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203250700.q2P703Z7011153@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 798 [1] | jsiwek | | Normal | topic/jsiwek/destdir-fix [2] Broccoli | 800 [3] | matthias | | Normal | Change Broccoli BroAddr type to a single 16 byte array [1] #798: http://tracker.bro-ids.org/bro/ticket/798 [2] destdir-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/destdir-fix [3] #800: http://tracker.bro-ids.org/bro/ticket/800 From gregor at majordomus.org Sun Mar 25 08:32:38 2012 From: gregor at majordomus.org (Gregor Maier) Date: Sun, 25 Mar 2012 08:32:38 -0700 Subject: [Bro-Dev] open a pipe? In-Reply-To: References: Message-ID: <4F6F3A96.7010404@majordomus.org> On 3/16/12 6:22 , Seth Hall wrote: > How hard would it be to modify files so that we could use them like a pipe to another program? I'll give an example? > > global myfile = open("| shasum> output"); > print myfile, "hello"; > close(myfile); Robin's comments aside, also note that you can implement a single pipe with popen() et al., but in order to add further output redirection or multiple pipes (e.g. your "> output") you need to run a shell (or a lot of work to do it without a shell) cu Gregor From noreply at bro-ids.org Mon Mar 26 00:00:08 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 26 Mar 2012 00:00:08 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203260700.q2Q708se026613@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 798 [1] | jsiwek | | Normal | topic/jsiwek/destdir-fix [2] Broccoli | 800 [3] | matthias | | Normal | Change Broccoli BroAddr type to a single 16 byte array [1] #798: http://tracker.bro-ids.org/bro/ticket/798 [2] destdir-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/destdir-fix [3] #800: http://tracker.bro-ids.org/bro/ticket/800 From bro at tracker.bro-ids.org Mon Mar 26 07:39:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 26 Mar 2012 14:39:21 -0000 Subject: [Bro-Dev] #798: topic/jsiwek/destdir-fix In-Reply-To: <048.3dd86a4e6c3dec62810e5f6c2e0d1dda@tracker.bro-ids.org> References: <048.3dd86a4e6c3dec62810e5f6c2e0d1dda@tracker.bro-ids.org> Message-ID: <063.1a3f4f7ce7a53a60bca457454e94086d@tracker.bro-ids.org> #798: topic/jsiwek/destdir-fix -----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Mar 26 07:40:42 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 26 Mar 2012 14:40:42 -0000 Subject: [Bro-Dev] #800: Change Broccoli BroAddr type to a single 16 byte array In-Reply-To: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> References: <050.48cc50fbe0c6f83c8870390701b0e2f8@tracker.bro-ids.org> Message-ID: <065.e59184b0b8c856e7b340d34caa74a17e@tracker.bro-ids.org> #800: Change Broccoli BroAddr type to a single 16 byte array -----------------------------+------------------------ Reporter: matthias | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Broccoli | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Mar 26 07:45:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 26 Mar 2012 14:45:18 -0000 Subject: [Bro-Dev] #531: Handle IPv6 protocol chains In-Reply-To: <048.a1036225e7e49822f38fb91c4b12ccff@tracker.bro-ids.org> References: <048.a1036225e7e49822f38fb91c4b12ccff@tracker.bro-ids.org> Message-ID: <063.d0884544de1700a14c5d02b3ecb179ca@tracker.bro-ids.org> #531: Handle IPv6 protocol chains -----------------------------+------------------------ Reporter: gregor | Owner: jsiwek Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: IPv6 -----------------------------+------------------------ Changes (by jsiwek): * status: assigned => closed * resolution: => Solved/Applied Comment: #795 fixes this. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Mar 26 07:48:39 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 26 Mar 2012 14:48:39 -0000 Subject: [Bro-Dev] #522: Event to report non TCP/UDP/ICMP packets In-Reply-To: <048.9578ab814d281f028dba6ef156f882b0@tracker.bro-ids.org> References: <048.9578ab814d281f028dba6ef156f882b0@tracker.bro-ids.org> Message-ID: <063.9c01d12476060171a0eb7073d737d047@tracker.bro-ids.org> #522: Event to report non TCP/UDP/ICMP packets -----------------------------+------------------------ Reporter: gregor | Owner: jsiwek Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: IPv6 -----------------------------+------------------------ Changes (by jsiwek): * status: assigned => closed * resolution: => Solved/Applied Comment: #795 fixes this by raising `unknown_protocol_%d` weirds where `%d` is the number of the protocol or extension header that Bro doesn't understand. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Mar 26 07:49:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 26 Mar 2012 14:49:34 -0000 Subject: [Bro-Dev] #523: event new_packet does not support IPv6 In-Reply-To: <048.6a57705fb7303481ddab2ff96d7bc20e@tracker.bro-ids.org> References: <048.6a57705fb7303481ddab2ff96d7bc20e@tracker.bro-ids.org> Message-ID: <063.7abed824c1e936364f0e59a059188e26@tracker.bro-ids.org> #523: event new_packet does not support IPv6 -----------------------------+------------------------ Reporter: gregor | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: IPv6 -----------------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied Comment: Fixed by the merge of #795. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Mon Mar 26 07:53:41 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 26 Mar 2012 07:53:41 -0700 Subject: [Bro-Dev] open a pipe? In-Reply-To: <4F6F3A96.7010404@majordomus.org> References: <4F6F3A96.7010404@majordomus.org> Message-ID: <20120326145341.GA29540@icir.org> > > global myfile = open("| shasum> output"); I'm coming around on this if we move the execution of the pipe into a thread. With the new thread infstrastructure, that should be mostly straight-forward and get us around all the blocking problems. Actually, I'm wondering if non-packet IOSources should generally move into threads (e.g., the DNS Mgr). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bernhard at ICSI.Berkeley.EDU Mon Mar 26 08:07:18 2012 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Mon, 26 Mar 2012 08:07:18 -0700 Subject: [Bro-Dev] open a pipe? In-Reply-To: <20120326145341.GA29540@icir.org> References: <4F6F3A96.7010404@majordomus.org> <20120326145341.GA29540@icir.org> Message-ID: On Mar 26, 2012, at 7:53 AM, Robin Sommer wrote: > >>> global myfile = open("| shasum> output"); > > I'm coming around on this if we move the execution of the pipe into a > thread. With the new thread infstrastructure, that should be mostly > straight-forward and get us around all the blocking problems. > Actually, I'm wondering if non-packet IOSources should generally move > into threads (e.g., the DNS Mgr). We just have to be a little bit careful about that, because it mixes threading and forks. Seth and me are already testing that in the input framework for a bit - and it seems to work. But it is not that easy to find straight answers if it is a good idea to call popen (which essentially does a fork and passes the command to "sh -c") in a thread - opinions on the tread-safety of popen seem to be divided. Bernhard From seth at icir.org Mon Mar 26 08:10:30 2012 From: seth at icir.org (Seth Hall) Date: Mon, 26 Mar 2012 11:10:30 -0400 Subject: [Bro-Dev] open a pipe? In-Reply-To: <20120326145341.GA29540@icir.org> References: <4F6F3A96.7010404@majordomus.org> <20120326145341.GA29540@icir.org> Message-ID: On Mar 26, 2012, at 10:53 AM, Robin Sommer wrote: >>> global myfile = open("| shasum> output"); > > I'm coming around on this if we move the execution of the pipe into a > thread. With the new thread infstrastructure, that should be mostly > straight-forward and get us around all the blocking problems. Woo. We definitely need to think about this a bit more though. I don't really like the perl-ism inherent in writing commands with pipes in the "file name". I've been looking at node.js's Stream API[1] lately. We may be able to borrow some ideas from that, but I already see some things about their API that don't like. > Actually, I'm wondering if non-packet IOSources should generally move > into threads (e.g., the DNS Mgr).  Do you have any intuitions yet on where we might run into issues with too many threads? When I was playing around a few nights ago I actually created about a hundred threads. :) 1. http://nodejs.org/api/stream.html .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Mon Mar 26 08:19:38 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 26 Mar 2012 08:19:38 -0700 Subject: [Bro-Dev] open a pipe? In-Reply-To: References: <4F6F3A96.7010404@majordomus.org> <20120326145341.GA29540@icir.org> Message-ID: <20120326151938.GH29540@icir.org> On Mon, Mar 26, 2012 at 08:07 -0700, you wrote: > But it is not that easy to find straight answers if it is a good idea > to call popen (which essentially does a fork and passes the command to > "sh -c") in a thread - opinions on the tread-safety of popen seem to > be divided. Yeah, we'd need to understand that. My intuition believes it should be doable, but I might be missing something here. Do you know a web page or something that explains where the trouble is? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Mon Mar 26 08:24:46 2012 From: robin at icir.org (Robin Sommer) Date: Mon, 26 Mar 2012 08:24:46 -0700 Subject: [Bro-Dev] open a pipe? In-Reply-To: References: <4F6F3A96.7010404@majordomus.org> <20120326145341.GA29540@icir.org> Message-ID: <20120326152446.GI29540@icir.org> On Mon, Mar 26, 2012 at 11:10 -0400, you wrote: > Woo. We definitely need to think about this a bit more though. I > don't really like the perl-ism inherent in writing commands with pipes > in the "file name". I'm open to ideas. :) The "classic" non-Perl way would obvioysly be just providing a popen() that returns an instance of type file. > Do you have any intuitions yet on where we might run into issues with > too many threads? No. My hope is that current OSs can deal well with *many* threads, in particular if they are all of low load. What we'll eventually need though is a way to cleanup threads no longer used. That's not done currently. That said, adding one thread per IOSource doesn't make much of a difference, it's the logging that creates potentially many threads. If things really don't work out, we'll need to swtich to one-thread-per-stream mode, but that gets more tricky on the backend side. I would prefer to avoid that. > When I was playing around a few nights ago I actually created about > a hundred threads. :) I'm not surprised. :) Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From vern at icir.org Mon Mar 26 10:07:21 2012 From: vern at icir.org (Vern Paxson) Date: Mon, 26 Mar 2012 10:07:21 -0700 Subject: [Bro-Dev] open a pipe? In-Reply-To: (Mon, 26 Mar 2012 11:10:30 EDT). Message-ID: <20120326170721.2D62E2C400B@rock.ICSI.Berkeley.EDU> > Woo. We definitely need to think about this a bit more though. I don't > really like the perl-ism inherent in writing commands with pipes in the > "file name". I agree. One particular concern I have is that it makes it easier to screw up and not properly escape/sanitize untrusted input that goes into the "filename", which in this case instead allows shell command injection :-(. Also, Robin, from what you sketch I'm not understanding how threading is going to help. Are you moving away from the model that script execution is atomic (other than "when" statements) and serialized? Wouldn't using "when" statements of some form better fit here? Vern From noreply at bro-ids.org Tue Mar 27 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 27 Mar 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203270700.q2R703xe024214@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 0ceca70 | Jon Siwek | 2012-03-26 | Change routing0_data_to_addrs BIF to return vector of addresses. [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/0ceca706f6d1a465bcb00b28164751e16e7ca0ff/bro From robin at icir.org Tue Mar 27 07:21:55 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 27 Mar 2012 07:21:55 -0700 Subject: [Bro-Dev] open a pipe? In-Reply-To: <20120326170721.2D62E2C400B@rock.ICSI.Berkeley.EDU> References: <20120326170721.2D62E2C400B@rock.ICSI.Berkeley.EDU> Message-ID: <20120327142155.GC54230@icir.org> On Mon, Mar 26, 2012 at 10:07 -0700, you wrote: > I agree. One particular concern I have is that it makes it easier to screw > up and not properly escape/sanitize untrusted input that goes into the > "filename", which in this case instead allows shell command injection :-(. yeah, I'm sure there are nicer interfaces, though I'm not sure we can really avoid the injection problem; in the end, we give the user the power to run a shell one way or the other (but we already do that with system()). > Also, Robin, from what you sketch I'm not understanding how threading is > going to help. Are you moving away from the model that script execution > is atomic (other than "when" statements) and serialized? Wouldn't using > "when" statements of some form better fit here? There are two different questions here: what the script-level interface looks like, and how the implementation achieves that. I was primarily talking about the latter: rather than manually interleaving reading the pipe's output with the packet processing (which gets cumbersome in particular if we need to support a potentially large number of open pipes), we can have a thread execute the command and take care of I/O. We already have the infrastructure to send results back asynchronously into the main thread, where it can turn into whatever we need. (Assuming any potential pthread/fork problems can be solved.) Regarding what the interface looks like, there are a number of options. Using "when" is one, we could indeed feed in there. But I'm not sure it's right model here: when would work best for simple one-request-one-reply style I/O but with pipes we may want more: keep writing into it, and keep reading out. That would work better with a file-like object ones prints to, and any output turning into events. But there may be still better models that that. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From vern at icir.org Tue Mar 27 08:35:45 2012 From: vern at icir.org (Vern Paxson) Date: Tue, 27 Mar 2012 08:35:45 -0700 Subject: [Bro-Dev] open a pipe? In-Reply-To: <20120327142155.GC54230@icir.org> (Tue, 27 Mar 2012 07:21:55 PDT). Message-ID: <20120327153545.D044F2C403D@rock.ICSI.Berkeley.EDU> > yeah, I'm sure there are nicer interfaces, though I'm not sure we can > really avoid the injection problem Right. My point is how *easy* it is. The issue with building piping into open() is the script writer might not even remember that the feature is there. Thus, if they construct a filename from untrusted input, it could wind up starting with '|', which was never anticipated. At least with something like popen() it's clear up-front "whoa this is running a command". > but with pipes we may want more: keep > writing into it, and keep reading out. That would work better with a > file-like object ones prints to, and any output turning into events. I see. Yeah, for that, what you sketch makes more sense. Vern From seth at icir.org Tue Mar 27 08:41:29 2012 From: seth at icir.org (Seth Hall) Date: Tue, 27 Mar 2012 11:41:29 -0400 Subject: [Bro-Dev] open a pipe? In-Reply-To: <20120327142155.GC54230@icir.org> References: <20120326170721.2D62E2C400B@rock.ICSI.Berkeley.EDU> <20120327142155.GC54230@icir.org> Message-ID: <8EC20886-C7DB-4FE2-9EDD-EE0F89569C22@icir.org> On Mar 27, 2012, at 10:21 AM, Robin Sommer wrote: > Regarding what the interface looks like, there are a number of > options. Using "when" is one, we could indeed feed in there. But I'm > not sure it's right model here: when would work best for simple > one-request-one-reply style I/O but with pipes we may want more: keep > writing into it, and keep reading out. That would work better with a > file-like object ones prints to, and any output turning into events. > But there may be still better models that that. I keep thinking that we just need to provide the connection between the input framework and sub processes. From the script-land perspective, something like this maybe? # The sub process is defined SubProcess::new("sha_hash", [$cmd="shasum"]); # STDIN is connected to a Bro file. local sha_command = SubProcess::get_stdin("sha_hash"); # STDERR and STDOUT are connected to inputs Input::add_event([$name="sha_hash", $source=SubProcess::get_stdout("sha_hash"), $fields=ShaVal, $ev=sha_line, $mode=Input::STREAM, $reader=Input::READER_RAW]); Input::add_event([$name="sha_hash", $source=SubProcess::get_stderr("sha_hash"), $fields=ShaVal, $ev=sha_line, $mode=Input::STREAM, $reader=Input::READER_RAW]); # The subprocess is actually executed. SubProcess::run("sha_hash"); # Send data to STDIN of the command. print sha_command, "some data"; # The command dies and the input streams are destroyed. close(sha_command); I think that SubProcess::get_stdout and SubProcess::get_stderr would return strings (file names) and SubProcess::get_stdin would return a file typed variable. It's all pretty verbose, but I don't think we have many options to keep things async and functional with the event loop. Thoughts? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Tue Mar 27 15:05:54 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 27 Mar 2012 22:05:54 -0000 Subject: [Bro-Dev] #803: topic/jsiwek/routing0 Message-ID: <048.e5574454d06d704eb3953e5fb4724cb5@tracker.bro-ids.org> #803: topic/jsiwek/routing0 ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch adds improved handling of packets with IPv6 routing type 0 headers. See [f4101b52659e19bea11a94c7e51fcfa501e4317c/bro] for details. It seemed important to be able to use the final destination of packets with an RH0 as the endpoint of connections in case there's still nodes out there that do RH0 processing according to RFC 2460, but I'm also feeling like it's a bit wasteful or makes the code more complex than necessary since RH0 is deprecated by RFC 5095 anyway and going to be uncommon. Any opinions? Would it make sense to just raise a weird (already do that in my changes) and stop analyzing a packet that contains an RH0 with non-zero segments left on the grounds that it's supposed to be deprecated and the analysis isn't going to be correct unless we make extra effort to understand the final destination? -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Mar 28 00:00:07 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 28 Mar 2012 00:00:07 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201203280700.q2S707Bf009037@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 803 [1] | jsiwek | | Normal | topic/jsiwek/routing0 [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 0ceca70 | Jon Siwek | 2012-03-26 | Change routing0_data_to_addrs BIF to return vector of addresses. [3] [1] #803: http://tracker.bro-ids.org/bro/ticket/803 [2] routing0: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/routing0 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/0ceca706f6d1a465bcb00b28164751e16e7ca0ff/bro From bro at tracker.bro-ids.org Wed Mar 28 08:04:08 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 28 Mar 2012 15:04:08 -0000 Subject: [Bro-Dev] #803: topic/jsiwek/routing0 In-Reply-To: <048.e5574454d06d704eb3953e5fb4724cb5@tracker.bro-ids.org> References: <048.e5574454d06d704eb3953e5fb4724cb5@tracker.bro-ids.org> Message-ID: <063.93f8a7effae078ccb0c4892197933d6d@tracker.bro-ids.org> #803: topic/jsiwek/routing0 ----------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by seth): > It seemed important to be able to use the final destination of packets > with an RH0 as the endpoint of connections in case there's still nodes out > there that do RH0 processing according to RFC 2460, but I'm also feeling > like it's a bit wasteful or makes the code more complex than necessary > since RH0 is deprecated by RFC 5095 anyway and going to be uncommon. Any > opinions? Yeah, I don't think we need the code to deal with that due to the deprecation. > Would it make sense to just raise a weird (already do that in my changes) > and stop analyzing a packet that contains an RH0 with non-zero segments > left on the grounds that it's supposed to be deprecated and the analysis > isn't going to be correct unless we make extra effort to understand the > final destination? I think that's exactly the right thing to do. In the script land we can watch for correct implementation by routers of RFC 5095 by watching for the ICMPv6 parameters problem message as specified in 5095. -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Mar 28 08:20:13 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 28 Mar 2012 08:20:13 -0700 Subject: [Bro-Dev] open a pipe? In-Reply-To: <8EC20886-C7DB-4FE2-9EDD-EE0F89569C22@icir.org> References: <20120326170721.2D62E2C400B@rock.ICSI.Berkeley.EDU> <20120327142155.GC54230@icir.org> <8EC20886-C7DB-4FE2-9EDD-EE0F89569C22@icir.org> Message-ID: <20120328152013.GF39215@icir.org> On Tue, Mar 27, 2012 at 11:41 -0400, you wrote: > It's all pretty verbose, Yeah, that's my main concern. It's pretty complex for "just" opening a pipe. Though perhaps we could hide some of the boilerplate in a function that takes care of the most common case like reading a list of lines as strings. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Wed Mar 28 08:26:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 28 Mar 2012 15:26:14 -0000 Subject: [Bro-Dev] #803: topic/jsiwek/routing0 In-Reply-To: <048.e5574454d06d704eb3953e5fb4724cb5@tracker.bro-ids.org> References: <048.e5574454d06d704eb3953e5fb4724cb5@tracker.bro-ids.org> Message-ID: <063.4b9d953355ae47a0a1bfdbbc17e55b5b@tracker.bro-ids.org> #803: topic/jsiwek/routing0 ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by jsiwek): * owner: => jsiwek * status: new => assigned * type: Merge Request => Task -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Mar 28 12:02:45 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 28 Mar 2012 19:02:45 -0000 Subject: [Bro-Dev] #804: topic/jsiwek/skip-rh0-segleft Message-ID: <048.467ce40b279ec04238f7da935a8578a3@tracker.bro-ids.org> #804: topic/jsiwek/skip-rh0-segleft ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This branch adds improved handling of packets with IPv6 routing type 0 headers, in particular by skipping such headers that have segments left. See [256cd592a7d4c0bdbf43c3f2e9c4e1cdb0fe995a/bro] for more. (There were also notice_policy.log baseline changes in bro-testing and bro-testing-private that need to be updated after merging this). For alternative approach that correctly handles RH0 headers with segments left see #803. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Mar 28 12:09:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 28 Mar 2012 19:09:10 -0000 Subject: [Bro-Dev] #803: topic/jsiwek/routing0 In-Reply-To: <048.e5574454d06d704eb3953e5fb4724cb5@tracker.bro-ids.org> References: <048.e5574454d06d704eb3953e5fb4724cb5@tracker.bro-ids.org> Message-ID: <063.79f3873b5c41e7017b2f0780ae43f4f9@tracker.bro-ids.org> #803: topic/jsiwek/routing0 -----------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Rejected | Keywords: -----------------------+------------------------ Changes (by jsiwek): * status: assigned => closed * resolution: => Rejected Comment: I'm going to close this as I did the alternative approach in a different branch indicated by #804, but I guess we might just leave this branch sitting around in case it's helpful to someone for whatever reason. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Mar 28 14:39:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 28 Mar 2012 21:39:10 -0000 Subject: [Bro-Dev] #767: Active copies of scripts not hidden In-Reply-To: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> References: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> Message-ID: <061.be64dc7726c8d159e06fbe97d45cb9f1@tracker.bro-ids.org> #767: Active copies of scripts not hidden ----------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Merge Request | Status: accepted Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by dnthayer): * type: Problem => Merge Request -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Mar 28 15:17:33 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 28 Mar 2012 22:17:33 -0000 Subject: [Bro-Dev] #767: Active copies of scripts not hidden In-Reply-To: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> References: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> Message-ID: <061.b4f1792036c9106a1bdbb3777c2633e0@tracker.bro-ids.org> #767: Active copies of scripts not hidden ----------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Merge Request | Status: accepted Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Renaming to ``installed-scripts-do-not-touch``. I don't like the dot in particular, given that it's inside spool directory I don't think we should hide it. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Mar 28 15:57:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 28 Mar 2012 22:57:18 -0000 Subject: [Bro-Dev] #804: topic/jsiwek/skip-rh0-segleft In-Reply-To: <048.467ce40b279ec04238f7da935a8578a3@tracker.bro-ids.org> References: <048.467ce40b279ec04238f7da935a8578a3@tracker.bro-ids.org> Message-ID: <063.05832536438d56ccda02199c46dbe6cf@tracker.bro-ids.org> #804: topic/jsiwek/skip-rh0-segleft ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by jsiwek): * owner: => jsiwek * status: new => assigned * type: Merge Request => Task Comment: Actually, the more complex routing header handling like in #803 for determining final destination address may be necessary for other types of routing headers anyway (e.g. the one used by Mobile IPv6). I'll study the RFCs more... or if anyone knows "the right answer" already, do let me know! -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Mar 28 16:14:07 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 28 Mar 2012 23:14:07 -0000 Subject: [Bro-Dev] #804: topic/jsiwek/skip-rh0-segleft In-Reply-To: <048.467ce40b279ec04238f7da935a8578a3@tracker.bro-ids.org> References: <048.467ce40b279ec04238f7da935a8578a3@tracker.bro-ids.org> Message-ID: <063.e76422b61a63d66912f08f2f0d12d7f8@tracker.bro-ids.org> #804: topic/jsiwek/skip-rh0-segleft ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by robin): I've actually already merged it, just running testing before pushing. I suppose getting this in doesn't hurt? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Mar 28 17:00:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 29 Mar 2012 00:00:19 -0000 Subject: [Bro-Dev] #767: Active copies of scripts not hidden In-Reply-To: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> References: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> Message-ID: <061.1052ab7693cb4eca34d950b214dbaece@tracker.bro-ids.org> #767: Active copies of scripts not hidden ----------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Merge Request | Status: accepted Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by seth): > Renaming to ``installed-scripts-do-not-touch``. > > I don't like the dot in particular, given that it's inside spool directory > I don't think we should hide it. Fair enough. As long as it's more clear that users need to leave it alone. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Mar 28 19:12:39 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 29 Mar 2012 02:12:39 -0000 Subject: [Bro-Dev] #804: topic/jsiwek/skip-rh0-segleft In-Reply-To: <048.467ce40b279ec04238f7da935a8578a3@tracker.bro-ids.org> References: <048.467ce40b279ec04238f7da935a8578a3@tracker.bro-ids.org> Message-ID: <063.4a3b30194d30227afcf5b6cc16b5da63@tracker.bro-ids.org> #804: topic/jsiwek/skip-rh0-segleft ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ---------------------+------------------------ Changes (by robin): * status: assigned => closed * resolution: => fixed Comment: In [de7300f999c576c6b873b453f610e1b7668ea926/bro]: {{{ #!CommitTicketReference repository="bro" revision="de7300f999c576c6b873b453f610e1b7668ea926" Merge remote-tracking branch 'origin/topic/jsiwek/skip-rh0-segleft' * origin/topic/jsiwek/skip-rh0-segleft: Improve handling of IPv6 Routing Type 0 headers. Closes #804. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Mar 28 19:12:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 29 Mar 2012 02:12:51 -0000 Subject: [Bro-Dev] #767: Active copies of scripts not hidden In-Reply-To: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> References: <046.7e34ff6c97a322fb558e7371cbe1b793@tracker.bro-ids.org> Message-ID: <061.7f767e21a3e15d3b877093068b1100ff@tracker.bro-ids.org> #767: Active copies of scripts not hidden ----------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * status: accepted => closed * resolution: => fixed Comment: In [58855556605bf3b1c14adf59d29c41d318b92916/broctl]: {{{ #!CommitTicketReference repository="broctl" revision="58855556605bf3b1c14adf59d29c41d318b92916" Merge remote-tracking branch 'origin/topic/dnthayer/bug767' * origin/topic/dnthayer/bug767: Remove the unused "PolicyDirBroCtl" option Rename the spool/policy directory so it is less visible Closes #767. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Mar 29 07:29:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 29 Mar 2012 14:29:18 -0000 Subject: [Bro-Dev] #804: topic/jsiwek/skip-rh0-segleft In-Reply-To: <048.467ce40b279ec04238f7da935a8578a3@tracker.bro-ids.org> References: <048.467ce40b279ec04238f7da935a8578a3@tracker.bro-ids.org> Message-ID: <063.5430dced2e1f265f7eb5bcfbe5e11b04@tracker.bro-ids.org> #804: topic/jsiwek/skip-rh0-segleft ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ---------------------+------------------------ Comment (by jsiwek): Replying to [comment:2 robin]: > I've actually already merged it, just running testing before pushing. I suppose getting this in doesn't hurt? That's fine, it doesn't break anything, but I may end up reverting it later. Sorry about that. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Mar 29 07:56:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 29 Mar 2012 14:56:00 -0000 Subject: [Bro-Dev] #750: Patch adding IPv6 support for pysubnettree In-Reply-To: <047.e6d50f498ca5a5aca3d4d44bb09a0d18@tracker.bro-ids.org> References: <047.e6d50f498ca5a5aca3d4d44bb09a0d18@tracker.bro-ids.org> Message-ID: <062.2b59f49ddacb92067a4bb813b846e54f@tracker.bro-ids.org> #750: Patch adding IPv6 support for pysubnettree ---------------------------+---------------------- Reporter: robin | Owner: dnthayer Type: Patch | Status: accepted Priority: Normal | Milestone: Bro2.1 Component: pysubnettree | Version: Resolution: | Keywords: ipv6 ---------------------------+---------------------- Changes (by dnthayer): * owner: => dnthayer * status: assigned => accepted -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 07:49:12 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 14:49:12 -0000 Subject: [Bro-Dev] #805: Make the various "weird" events stop printing to stdout Message-ID: <046.21500fb1bde0b9f1dd898fd92880d54b@tracker.bro-ids.org> #805: Make the various "weird" events stop printing to stdout ---------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ When the conn_weird, flow_weird, and net_weird events aren't being handled in scripts, the core will print the weird message to stdout. This makes Bro look very messy if you are running in "base" -b mode and doesn't really provide any benefit. -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Fri Mar 30 09:14:30 2012 From: seth at icir.org (Seth Hall) Date: Fri, 30 Mar 2012 12:14:30 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: <201203141533.q2EFXlEk012119@bro-ids.icir.org> References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> Message-ID: <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> I merged from master in a branch I'm working on and the following line is causing trouble? In file included from /Users/seth/bro/bro.merging/src/Func.cc:531: bro.bif: In function ?Val* BifFunc::bro_routing0_data_to_addrs(Frame*, val_list*)?: bro.bif:2073: error: ?IPv6? is not a member of ?IPAddr? On Mar 14, 2012, at 11:33 AM, Jonathan Siwek wrote: > --- a/src/bro.bif > +++ b/src/bro.bif > +function routing0_data_to_addrs%(s: string%): addr_set > + %{ > + IPAddr a(IPAddr::IPv6, (const uint32*) bytes, IPAddr::Network); .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Fri Mar 30 09:43:29 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 16:43:29 -0000 Subject: [Bro-Dev] #805: Make the various "weird" events stop printing to stdout In-Reply-To: <046.21500fb1bde0b9f1dd898fd92880d54b@tracker.bro-ids.org> References: <046.21500fb1bde0b9f1dd898fd92880d54b@tracker.bro-ids.org> Message-ID: <061.a513fb60e14f5da513f0fe0019d1cfd9@tracker.bro-ids.org> #805: Make the various "weird" events stop printing to stdout ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): Where would they go? Just /dev/null? -- Ticket URL: Bro Tracker Bro Issue Tracker From jsiwek at illinois.edu Fri Mar 30 09:43:59 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 30 Mar 2012 16:43:59 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> Message-ID: <4D796578-DB02-4A1D-B11E-A21B60C1E54D@illinois.edu> > I merged from master in a branch I'm working on and the following line is causing trouble? > > In file included from /Users/seth/bro/bro.merging/src/Func.cc:531: > bro.bif: In function ?Val* BifFunc::bro_routing0_data_to_addrs(Frame*, val_list*)?: > bro.bif:2073: error: ?IPv6? is not a member of ?IPAddr? That's correct for master, but I think I saw Robin may have moved that IP Family enum declaration into the global namespace in one branch (maybe log-threads). In your branch, you can probably try just removing the IPAddr:: scoping. +Jon From robin at icir.org Fri Mar 30 09:45:54 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 30 Mar 2012 09:45:54 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> Message-ID: <20120330164554.GP27503@icir.org> master compiles fine for me. Any chance that might be something specific to your bracnh? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Fri Mar 30 09:47:22 2012 From: seth at icir.org (Seth Hall) Date: Fri, 30 Mar 2012 12:47:22 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: <4D796578-DB02-4A1D-B11E-A21B60C1E54D@illinois.edu> References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> <4D796578-DB02-4A1D-B11E-A21B60C1E54D@illinois.edu> Message-ID: <6B10BFBC-AB4F-45B1-9C23-23ED6E15D8FE@icir.org> On Mar 30, 2012, at 12:43 PM, Siwek, Jonathan Luke wrote: > In your branch, you can probably try just removing the IPAddr:: scoping. I actually tried that already? /Users/seth/bro/bro.merging/src/IP.cc: In member function ?RecordVal* IPv6_Hdr::BuildRecordVal(VectorVal*) const?: /Users/seth/bro/bro.merging/src/IP.cc:77: error: no matching function for call to ?AddrVal::AddrVal(const in6_addr&)? /Users/seth/bro/bro.merging/src/Val.h:571: note: candidates are: AddrVal::AddrVal(BroType*) /Users/seth/bro/bro.merging/src/Val.h:570: note: AddrVal::AddrVal(TypeTag) /Users/seth/bro/bro.merging/src/Val.h:569: note: AddrVal::AddrVal() /Users/seth/bro/bro.merging/src/Val.h:563: note: AddrVal::AddrVal(const IPAddr&) /Users/seth/bro/bro.merging/src/Val.h:562: note: AddrVal::AddrVal(const uint32*) /Users/seth/bro/bro.merging/src/Val.h:561: note: AddrVal::AddrVal(uint32) /Users/seth/bro/bro.merging/src/Val.h:555: note: AddrVal::AddrVal(const char*) /Users/seth/bro/bro.merging/src/Val.h:553: note: AddrVal::AddrVal(const AddrVal&) /Users/seth/bro/bro.merging/src/IP.cc:78: error: no matching function for call to ?AddrVal::AddrVal(const in6_addr&)? /Users/seth/bro/bro.merging/src/Val.h:571: note: candidates are: AddrVal::AddrVal(BroType*) /Users/seth/bro/bro.merging/src/Val.h:570: note: AddrVal::AddrVal(TypeTag) /Users/seth/bro/bro.merging/src/Val.h:569: note: AddrVal::AddrVal() /Users/seth/bro/bro.merging/src/Val.h:563: note: AddrVal::AddrVal(const IPAddr&) /Users/seth/bro/bro.merging/src/Val.h:562: note: AddrVal::AddrVal(const uint32*) /Users/seth/bro/bro.merging/src/Val.h:561: note: AddrVal::AddrVal(uint32) /Users/seth/bro/bro.merging/src/Val.h:555: note: AddrVal::AddrVal(const char*) /Users/seth/bro/bro.merging/src/Val.h:553: note: AddrVal::AddrVal(const AddrVal&) /Users/seth/bro/bro.merging/src/IP.cc: In member function ?void IPv6_Hdr_Chain::Init(const ip6_hdr*, bool, uint16)?: /Users/seth/bro/bro.merging/src/IP.cc:317: error: no matching function for call to ?Reporter::Weird(IPAddr&, const in6_addr&, const char [17])? /Users/seth/bro/bro.merging/src/Reporter.h:75: note: candidates are: void Reporter::Weird(const char*) /Users/seth/bro/bro.merging/src/Reporter.h:76: note: void Reporter::Weird(Connection*, const char*, const char*) /Users/seth/bro/bro.merging/src/Reporter.h:77: note: void Reporter::Weird(Val*, const char*, const char*) /Users/seth/bro/bro.merging/src/Reporter.h:78: note: void Reporter::Weird(const IPAddr&, const IPAddr&, const char*) make[2]: *** [src/CMakeFiles/bro.dir/IP.cc.o] Error 1 make[1]: *** [src/CMakeFiles/bro.dir/all] Error 2 make: *** [all] Error 2 .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Fri Mar 30 09:48:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 16:48:18 -0000 Subject: [Bro-Dev] #805: Make the various "weird" events stop printing to stdout In-Reply-To: <046.21500fb1bde0b9f1dd898fd92880d54b@tracker.bro-ids.org> References: <046.21500fb1bde0b9f1dd898fd92880d54b@tracker.bro-ids.org> Message-ID: <061.a8823cb36f2bb0786c7574773f4bf412@tracker.bro-ids.org> #805: Make the various "weird" events stop printing to stdout ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by seth): > Where would they go? Just /dev/null? The events just wouldn't be handled and nothing would happen. It's really weird behavior for those special events to print to stdout if they aren't handled. -- Ticket URL: Bro Tracker Bro Issue Tracker From bernhard at ICSI.Berkeley.EDU Fri Mar 30 09:52:12 2012 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Fri, 30 Mar 2012 09:52:12 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: <20120330164554.GP27503@icir.org> References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> <20120330164554.GP27503@icir.org> Message-ID: <01E8B07D-76C7-46FF-99E4-F28E2635BE4D@icsi.berkeley.edu> The same thing happens when I merge master with my input branch. So - it does not seem to be branch-specific. Bernhard On Mar 30, 2012, at 9:45 AM, Robin Sommer wrote: > master compiles fine for me. Any chance that might be something > specific to your bracnh? > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > _______________________________________________ > bro-dev mailing list > bro-dev at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From seth at icir.org Fri Mar 30 09:56:32 2012 From: seth at icir.org (Seth Hall) Date: Fri, 30 Mar 2012 12:56:32 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: <20120330164554.GP27503@icir.org> References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> <20120330164554.GP27503@icir.org> Message-ID: <5B380DA0-FB25-46DE-8407-BE8B36F36310@icir.org> On Mar 30, 2012, at 12:45 PM, Robin Sommer wrote: > master compiles fine for me. Any chance that might be something > specific to your bracnh? Hm, I guess so. Maybe I have too much merged in. Thanks. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From robin at icir.org Fri Mar 30 10:03:22 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 30 Mar 2012 10:03:22 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: <5B380DA0-FB25-46DE-8407-BE8B36F36310@icir.org> References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> <20120330164554.GP27503@icir.org> <5B380DA0-FB25-46DE-8407-BE8B36F36310@icir.org> Message-ID: <20120330170322.GR27503@icir.org> On Fri, Mar 30, 2012 at 12:56 -0400, you wrote: > Hm, I guess so. Maybe I have too much merged in. Thanks. Let me merge master into my logging branch, that might trigger the same problem and I'll fix it then. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From jsiwek at illinois.edu Fri Mar 30 10:27:20 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 30 Mar 2012 17:27:20 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: <6B10BFBC-AB4F-45B1-9C23-23ED6E15D8FE@icir.org> References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> <4D796578-DB02-4A1D-B11E-A21B60C1E54D@illinois.edu> <6B10BFBC-AB4F-45B1-9C23-23ED6E15D8FE@icir.org> Message-ID: On Mar 30, 2012, at 11:47 AM, Seth Hall wrote: > > On Mar 30, 2012, at 12:43 PM, Siwek, Jonathan Luke wrote: > >> In your branch, you can probably try just removing the IPAddr:: scoping. > > > I actually tried that already? > > /Users/seth/bro/bro.merging/src/IP.cc: In member function ?RecordVal* IPv6_Hdr::BuildRecordVal(VectorVal*) const?: > /Users/seth/bro/bro.merging/src/IP.cc:77: error: no matching function for call to ?AddrVal::AddrVal(const in6_addr&)? > /Users/seth/bro/bro.merging/src/IP.cc:78: error: no matching function for call to ?AddrVal::AddrVal(const in6_addr&)? > /Users/seth/bro/bro.merging/src/IP.cc: In member function ?void IPv6_Hdr_Chain::Init(const ip6_hdr*, bool, uint16)?: > /Users/seth/bro/bro.merging/src/IP.cc:317: error: no matching function for call to ?Reporter::Weird(IPAddr&, const in6_addr&, const char [17])? Those are because in topic/robin/log-threads, the appropriate IPAddr conversion ctors are marked as 'explicit', but they are not marked as such in master. I'd explicitly add the IPAddr() ctor at those lines to fix it since the implicit conversion might be harder to understand when someone's reading code. On Mar 30, 2012, at 11:52 AM, Bernhard Amann wrote: > The same thing happens when I merge master with my input branch. So - it does not seem to be branch-specific. Looks like your branch is probably downstream from Robin's, so it makes sense you get the same thing? On Mar 30, 2012, at 12:03 PM, Robin Sommer wrote: > Let me merge master into my logging branch, that might trigger the > same problem and I'll fix it then. I've done that and fixed it locally like I mentioned above. Let me know if it's easier for you if I just push it, else I'll let you fix it to avoid causing more chaos/confusion. +Jon From robin at icir.org Fri Mar 30 10:30:32 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 30 Mar 2012 10:30:32 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> <4D796578-DB02-4A1D-B11E-A21B60C1E54D@illinois.edu> <6B10BFBC-AB4F-45B1-9C23-23ED6E15D8FE@icir.org> Message-ID: <20120330173032.GT27503@icir.org> On Fri, Mar 30, 2012 at 17:27 +0000, you wrote: > Those are because in topic/robin/log-threads, the appropriate IPAddr > conversion ctors are marked as 'explicit', but they are not marked as > such in master. Yeah, I believe I was running into some ambiguity without the explicit. > I've done that and fixed it locally like I mentioned above. Let me > know if it's easier for you if I just push it, yes, please push. Thanks, Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Fri Mar 30 10:34:00 2012 From: seth at icir.org (Seth Hall) Date: Fri, 30 Mar 2012 13:34:00 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> <4D796578-DB02-4A1D-B11E-A21B60C1E54D@illinois.edu> <6B10BFBC-AB4F-45B1-9C23-23ED6E15D8FE@icir.org> Message-ID: On Mar 30, 2012, at 1:27 PM, Siwek, Jonathan Luke wrote: > I've done that and fixed it locally like I mentioned above. Let me know if it's easier for you if I just push it, else I'll let you fix it to avoid causing more chaos/confusion. I think Robin's fixing it in his branch now. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From jsiwek at illinois.edu Fri Mar 30 10:47:18 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 30 Mar 2012 17:47:18 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: <20120330173032.GT27503@icir.org> References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> <4D796578-DB02-4A1D-B11E-A21B60C1E54D@illinois.edu> <6B10BFBC-AB4F-45B1-9C23-23ED6E15D8FE@icir.org> <20120330173032.GT27503@icir.org> Message-ID: <3BE2A536-6FE8-4BA6-9F52-2968594166D6@illinois.edu> >> I've done that and fixed it locally like I mentioned above. Let me >> know if it's easier for you if I just push it, > > yes, please push. Thanks, Done, others can now try merging that in to their downstream branches. +Jon From seth at icir.org Fri Mar 30 11:00:16 2012 From: seth at icir.org (Seth Hall) Date: Fri, 30 Mar 2012 14:00:16 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/jsiwek/ipv6-ext-headers: Fix ipv6_ext_headers event and add routing0_data_to_addrs BIF. (5312a90) In-Reply-To: <3BE2A536-6FE8-4BA6-9F52-2968594166D6@illinois.edu> References: <201203141533.q2EFXlEk012119@bro-ids.icir.org> <63457315-5CBD-42DF-AA86-1FC2983BE862@icir.org> <4D796578-DB02-4A1D-B11E-A21B60C1E54D@illinois.edu> <6B10BFBC-AB4F-45B1-9C23-23ED6E15D8FE@icir.org> <20120330173032.GT27503@icir.org> <3BE2A536-6FE8-4BA6-9F52-2968594166D6@illinois.edu> Message-ID: <94E82978-7B27-40A8-A02E-DBE04330DE7A@icir.org> On Mar 30, 2012, at 1:47 PM, Siwek, Jonathan Luke wrote: > Done, others can now try merging that in to their downstream branches. Thanks Jon! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Fri Mar 30 12:04:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:04:38 -0000 Subject: [Bro-Dev] #434: Fix secondary path In-Reply-To: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> References: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> Message-ID: <062.973c4f13348b265230cc7709b560b7d6@tracker.bro-ids.org> #434: Fix secondary path ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by seth): I have *completely* forgotten why I wanted to keep the secondary path around. Are there any remaining wishes to keep? I think I was the last one wanting to keep it. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:06:08 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:06:08 -0000 Subject: [Bro-Dev] #434: Fix secondary path In-Reply-To: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> References: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> Message-ID: <062.9c47d538e31cdff55771335cb6dab079@tracker.bro-ids.org> #434: Fix secondary path ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by robin): On Fri, Mar 30, 2012 at 19:04 -0000, you wrote: > I have *completely* forgotten why I wanted to keep the secondary path > around. Are there any remaining wishes to keep? I think I was the last > one wanting to keep it. I believe Vern wnated to keep it too. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:06:59 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:06:59 -0000 Subject: [Bro-Dev] #780: file extraction trunctation. In-Reply-To: <048.841e7736f715e5a45f63422aac975ed4@tracker.bro-ids.org> References: <048.841e7736f715e5a45f63422aac975ed4@tracker.bro-ids.org> Message-ID: <063.762b27eabb1cb9d240a4aa55d9059cf0@tracker.bro-ids.org> #780: file extraction trunctation. ----------------------+--------------------------------------- Reporter: justin | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: file extraction truncated ----------------------+--------------------------------------- Changes (by seth): * priority: Normal => High Comment: I'm bumping priority on this because it pretty thoroughly breaks file extraction in many cases. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:10:22 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:10:22 -0000 Subject: [Bro-Dev] #699: Reorganizing layout of protocol analyzers In-Reply-To: <047.bdd8ceb4ce1b04ea31b0648ad0c67b84@tracker.bro-ids.org> References: <047.bdd8ceb4ce1b04ea31b0648ad0c67b84@tracker.bro-ids.org> Message-ID: <062.61d4b09f19b54491dd2559585f8156f6@tracker.bro-ids.org> #699: Reorganizing layout of protocol analyzers ---------------------+------------------------ Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by seth): * milestone: Bro2.1 => Bro2.2 Comment: I'm going to bump this back to 2.2 since Robin and I both agreed that it makes more sense to delay this until the Binpac++ integration. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:11:15 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:11:15 -0000 Subject: [Bro-Dev] #583: system function with feedback In-Reply-To: <046.d03023985c47130a6af60bec0359f9f5@tracker.bro-ids.org> References: <046.d03023985c47130a6af60bec0359f9f5@tracker.bro-ids.org> Message-ID: <061.ff10bbc083b7e1440dc43c4996c078bb@tracker.bro-ids.org> #583: system function with feedback ------------------------------+---------------------- Reporter: seth | Owner: Type: Feature Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: Rejected | Keywords: language ------------------------------+---------------------- Changes (by seth): * status: new => closed * resolution: => Rejected Comment: I don't like this proposal anymore so I'm closing this ticket. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:12:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:12:38 -0000 Subject: [Bro-Dev] #579: Syslog logging writer In-Reply-To: <046.4e6efad585e65a2ccb68427348524651@tracker.bro-ids.org> References: <046.4e6efad585e65a2ccb68427348524651@tracker.bro-ids.org> Message-ID: <061.3bff4fd7c93162d22abebc9c4a399ffd@tracker.bro-ids.org> #579: Syslog logging writer ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by seth): * priority: Normal => High Comment: Bumping priority because this could make integration with Martin Holste's ELSA project much easier. I need to talk to him more about how it should work to integrate best. I expect that the threaded logging framework should make the integration very easy though. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:15:33 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:15:33 -0000 Subject: [Bro-Dev] #353: Restore and improve IDMEF support In-Reply-To: <046.3a20146b674cb9679a0be4ea55b0c147@tracker.bro-ids.org> References: <046.3a20146b674cb9679a0be4ea55b0c147@tracker.bro-ids.org> Message-ID: <061.48d17f2a69c113f6af5c2cd4797efb31@tracker.bro-ids.org> #353: Restore and improve IDMEF support ---------------------+-------------------- Reporter: seth | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by seth): Also refer to ticket #802 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:18:15 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:18:15 -0000 Subject: [Bro-Dev] #267: Patch to fix internal log_encryption_key handling In-Reply-To: <046.f3b2f414141b2c225cf35ac25c8fd93f@tracker.bro-ids.org> References: <046.f3b2f414141b2c225cf35ac25c8fd93f@tracker.bro-ids.org> Message-ID: <061.dcf9b08d8c8db2bff53f4c3e2501a7f2@tracker.bro-ids.org> #267: Patch to fix internal log_encryption_key handling ----------------------+-------------------- Reporter: seth | Owner: Type: Patch | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: 1.5.2 Resolution: Invalid | Keywords: sprint ----------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Invalid Comment: I'm just going to close this ticket since it's related to the old logging style. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:18:59 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:18:59 -0000 Subject: [Bro-Dev] #772: Problem with $path_func in Log filters In-Reply-To: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> References: <046.95ffca20b458194651563c9048428086@tracker.bro-ids.org> Message-ID: <061.e2372591f3118ac1a5b7b98106a0363c@tracker.bro-ids.org> #772: Problem with $path_func in Log filters ----------------------+------------------------ Reporter: seth | Owner: seth Type: Problem | Status: assigned Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by seth): * owner: => seth * status: new => assigned -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:19:29 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:19:29 -0000 Subject: [Bro-Dev] #353: Restore and improve IDMEF support In-Reply-To: <046.3a20146b674cb9679a0be4ea55b0c147@tracker.bro-ids.org> References: <046.3a20146b674cb9679a0be4ea55b0c147@tracker.bro-ids.org> Message-ID: <061.4f082f18383d79a75b15b7d497fa63fc@tracker.bro-ids.org> #353: Restore and improve IDMEF support ---------------------+-------------------- Reporter: seth | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by robin): Alternative idea: if we had a Python logging backend, this should be become pretty straight-forward. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:38:09 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:38:09 -0000 Subject: [Bro-Dev] #309: Work with Endace to get their code back in In-Reply-To: <047.b48356ad4c0bc5e98558c47d09be301b@tracker.bro-ids.org> References: <047.b48356ad4c0bc5e98558c47d09be301b@tracker.bro-ids.org> Message-ID: <062.8c29f57a27b6048eb19a45bcc07dd8cc@tracker.bro-ids.org> #309: Work with Endace to get their code back in ---------------------+------------------------ Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by seth): * version: => git/master * milestone: Bro2.1 => Bro2.2 Comment: Unfortunately I don't think the endace support will make it back it yet. Especially since we're looking toward a larger restructuring of the packet input pipeline. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:40:13 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:40:13 -0000 Subject: [Bro-Dev] #496: bro -H undocumented In-Reply-To: <046.b7a7f615494ff19ec5f13223043fefcf@tracker.bro-ids.org> References: <046.b7a7f615494ff19ec5f13223043fefcf@tracker.bro-ids.org> Message-ID: <061.9fdc6146ac491c2ed26b61b304400d50@tracker.bro-ids.org> #496: bro -H undocumented -----------------------------+-------------------- Reporter: vern | Owner: Type: Problem | Status: closed Priority: Low | Milestone: Component: Bro | Version: Resolution: Solved/Applied | Keywords: -----------------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Solved/Applied Comment: It's gone in 2.0. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:41:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:41:56 -0000 Subject: [Bro-Dev] #698: HTTP vs MIME events In-Reply-To: <047.7f6f47529b886fee475b591da0309f6c@tracker.bro-ids.org> References: <047.7f6f47529b886fee475b591da0309f6c@tracker.bro-ids.org> Message-ID: <062.1378ccbb24b1ba025dd51a0565c3778d@tracker.bro-ids.org> #698: HTTP vs MIME events ----------------------+--------------------- Reporter: robin | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: cleanup ----------------------+--------------------- Changes (by seth): * milestone: => Bro2.1 Comment: I'd like to see at least some of this make it into 2.1 if we have time. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:44:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:44:51 -0000 Subject: [Bro-Dev] #353: Restore and improve IDMEF support In-Reply-To: <046.3a20146b674cb9679a0be4ea55b0c147@tracker.bro-ids.org> References: <046.3a20146b674cb9679a0be4ea55b0c147@tracker.bro-ids.org> Message-ID: <061.3e066668c81d15467e7b0689f7f60f8b@tracker.bro-ids.org> #353: Restore and improve IDMEF support ---------------------+-------------------- Reporter: seth | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by amannb): And I still want to do the python backend - but with all the other things that are going on at the moment that will probably take a while. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:50:36 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:50:36 -0000 Subject: [Bro-Dev] #551: Potential alternate signature loading method? In-Reply-To: <046.8fa46d7f399c279d6a4c4e9463558fac@tracker.bro-ids.org> References: <046.8fa46d7f399c279d6a4c4e9463558fac@tracker.bro-ids.org> Message-ID: <061.25f1033be7689c04e7f423e0fb8be580@tracker.bro-ids.org> #551: Potential alternate signature loading method? ------------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ------------------------------+-------------------- Changes (by seth): * milestone: => Bro2.1 Comment: This is so painful that it's missing. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 12:58:50 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 19:58:50 -0000 Subject: [Bro-Dev] #120: Need run time option to set recv socket buffer size or a bro policy to do so In-Reply-To: <063.5c42236525b6d888346929e8b1096da5@tracker.bro-ids.org> References: <063.5c42236525b6d888346929e8b1096da5@tracker.bro-ids.org> Message-ID: <078.3c8f6def2958c876da37e2989097b30d@tracker.bro-ids.org> #120: Need run time option to set recv socket buffer size or a bro policy to do so -----------------------+--------------------------------- Reporter: jones@? | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Component: Bro | Version: branches-robin-work Resolution: Rejected | Keywords: -----------------------+--------------------------------- Changes (by seth): * status: needs information => closed * resolution: => Rejected -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 13:02:32 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 20:02:32 -0000 Subject: [Bro-Dev] #762: Add eof line to logfiles In-Reply-To: <048.b69152fe8b8e0e80e5715b13977d82cb@tracker.bro-ids.org> References: <048.b69152fe8b8e0e80e5715b13977d82cb@tracker.bro-ids.org> Message-ID: <063.31d62575b42c1a9fc34503a8fb265209@tracker.bro-ids.org> #762: Add eof line to logfiles ------------------------------+------------------------ Reporter: amannb | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Changes (by seth): * milestone: => Bro2.1 Comment: Assigning this to a milestone to make sure we don't lose track of the ticket. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 13:03:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 20:03:21 -0000 Subject: [Bro-Dev] #781: Case sensitive (non-normalized) HTTP header names In-Reply-To: <048.1cb22f83281a8d29460fe152fe9d53ce@tracker.bro-ids.org> References: <048.1cb22f83281a8d29460fe152fe9d53ce@tracker.bro-ids.org> Message-ID: <063.2384e1cbc46862e1921b5d6454fdf046@tracker.bro-ids.org> #781: Case sensitive (non-normalized) HTTP header names ------------------------------+------------------------ Reporter: sconzo | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Changes (by seth): * milestone: => Bro2.1 Comment: Let's look into doing this along with the other http analyzer changes for 2.1 -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Mar 30 14:34:39 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 30 Mar 2012 21:34:39 -0000 Subject: [Bro-Dev] #434: Fix secondary path In-Reply-To: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> References: <047.5ba5da3b865ab8b2a3fd986f3ba5b573@tracker.bro-ids.org> Message-ID: <062.c72254aa02fa2dbd6e28d7917bf5d184@tracker.bro-ids.org> #434: Fix secondary path ---------------------+-------------------- Reporter: robin | Owner: Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ---------------------+-------------------- Comment (by vern): Right, I find benefit from large-conns.bro, which relies on it. -- Ticket URL: Bro Tracker Bro Issue Tracker