From bro at tracker.bro-ids.org Fri Jun 1 07:57:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 01 Jun 2012 14:57:38 -0000 Subject: [Bro-Dev] #824: Default the connection and alarm summaries to once per day In-Reply-To: <056.dadb50a21009b083c39e1a280f60c9e9@tracker.bro-ids.org> References: <056.dadb50a21009b083c39e1a280f60c9e9@tracker.bro-ids.org> Message-ID: <071.d62d3214215e2041ddba4340a3404d47@tracker.bro-ids.org> #824: Default the connection and alarm summaries to once per day ------------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ------------------------------+------------------------ Changes (by robin): * milestone: => Bro2.1 Comment: Change to the alarm summaries is in the works. The connection summaries however can't be detached from the log rotation/compression. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 1 12:11:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 01 Jun 2012 19:11:52 -0000 Subject: [Bro-Dev] #551: Potential alternate signature loading method? In-Reply-To: <046.8fa46d7f399c279d6a4c4e9463558fac@tracker.bro-ids.org> References: <046.8fa46d7f399c279d6a4c4e9463558fac@tracker.bro-ids.org> Message-ID: <061.8267e8bb148362c68ea1f44a424a0be4@tracker.bro-ids.org> #551: Potential alternate signature loading method? ------------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ------------------------------+-------------------- Comment (by jsiwek): In [dd4dd0ca6ea0adf94a9cbc87de6e322c34365508/bro]: {{{ #!CommitTicketReference repository="bro" revision="dd4dd0ca6ea0adf94a9cbc87de6e322c34365508" Add @load-sigs directive for loading signature files (addresses #551). }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 1 12:12:43 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 01 Jun 2012 19:12:43 -0000 Subject: [Bro-Dev] #551: Potential alternate signature loading method? In-Reply-To: <046.8fa46d7f399c279d6a4c4e9463558fac@tracker.bro-ids.org> References: <046.8fa46d7f399c279d6a4c4e9463558fac@tracker.bro-ids.org> Message-ID: <061.a8640d5f19f33839dcbbf2a59a549193@tracker.bro-ids.org> #551: Potential alternate signature loading method? ----------------------------+-------------------- Reporter: seth | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: | Keywords: ----------------------------+-------------------- Changes (by jsiwek): * type: Feature Request => Merge Request Comment: In `topic/jsiwek/load-sigs` -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 1 16:47:15 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 01 Jun 2012 23:47:15 -0000 Subject: [Bro-Dev] #825: topic/dnthayer/bif-tests Message-ID: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> #825: topic/dnthayer/bif-tests ---------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ On this branch I have added tests for many previously-untested BIFs. In the process I found and fixed some bugs: "join_string_vec" was broken and is now fixed, the data type of 2nd argument of "system_env" has been changed from "any" to "table_string_of_string" because that is the only type that seems to work, and "parse_dotted_addr" has been deprecated. I've also improved or fixed the documentation comments for several BIFs. There are still a few unresolved issues (marked with "TODO" comments in the test code): 1) the "order" BIF doesn't seem to work correctly (the results appear inconsistent and don't make sense to me), 2) the "type_name" BIF always returns "vector" for vector types (I would expect a more precise result, similar to the results it gives for sets and tables), and it gives perhaps an overly precise result ("file of string") for file types, 3) the documentation comments for "fmt" claim that you can specify a non-string as the first argument (in which case it's supposed to concatenate all the arguments, like "cat"), but it rejects non-string first arguments. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Jun 2 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 2 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206020700.q52702A5003623@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 551 [1] | seth | | Normal | Potential alternate signature loading method? Bro | 825 [2] | dnthayer | | Normal | topic/dnthayer/bif-tests [3] [1] #551: http://tracker.bro-ids.org/bro/ticket/551 [2] #825: http://tracker.bro-ids.org/bro/ticket/825 [3] bif-tests: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/bif-tests From noreply at bro-ids.org Sun Jun 3 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 3 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206030700.q537020E018754@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 551 [1] | seth | | Normal | Potential alternate signature loading method? Bro | 825 [2] | dnthayer | | Normal | topic/dnthayer/bif-tests [3] [1] #551: http://tracker.bro-ids.org/bro/ticket/551 [2] #825: http://tracker.bro-ids.org/bro/ticket/825 [3] bif-tests: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/bif-tests From noreply at bro-ids.org Mon Jun 4 00:00:01 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 4 Jun 2012 00:00:01 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206040700.q54701Ix016875@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 551 [1] | seth | | Normal | Potential alternate signature loading method? Bro | 825 [2] | dnthayer | | Normal | topic/dnthayer/bif-tests [3] [1] #551: http://tracker.bro-ids.org/bro/ticket/551 [2] #825: http://tracker.bro-ids.org/bro/ticket/825 [3] bif-tests: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/bif-tests From bro at tracker.bro-ids.org Mon Jun 4 07:09:42 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 04 Jun 2012 14:09:42 -0000 Subject: [Bro-Dev] #826: Brocontrol disk space management problems Message-ID: <046.4949ee5c7b0a6686aef2ebd307b2595f@tracker.bro-ids.org> #826: Brocontrol disk space management problems ------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Keywords: | ------------------------+------------------------ It looks like these two settings don't work in BroControl: {{{ # Expiration interval for log files in LogDir. Files older than this many days # will be deleted. LogExpireInterval = 2 # Lower threshold for space available on the disk that holds SpoolDir. If less # space is available, BroControl starts sending out warning emails. MinDiskSpace = 5 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 4 07:30:22 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 04 Jun 2012 14:30:22 -0000 Subject: [Bro-Dev] #826: Brocontrol disk space management problems In-Reply-To: <046.4949ee5c7b0a6686aef2ebd307b2595f@tracker.bro-ids.org> References: <046.4949ee5c7b0a6686aef2ebd307b2595f@tracker.bro-ids.org> Message-ID: <061.995c17b69378c46a0fe437aa067324d0@tracker.bro-ids.org> #826: Brocontrol disk space management problems -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: Rejected | Keywords: -------------------------+------------------------ Changes (by seth): * status: new => closed * resolution: => Rejected Comment: Arg, this was user error. These features seem to be working. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 4 07:47:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 04 Jun 2012 14:47:14 -0000 Subject: [Bro-Dev] #822: Segmentation fault ICMP Analizer In-Reply-To: <056.c05bcf7b45c705fd44326f7779a83361@tracker.bro-ids.org> References: <056.c05bcf7b45c705fd44326f7779a83361@tracker.bro-ids.org> Message-ID: <071.251bea9f137f79cfebd72cdd44687358@tracker.bro-ids.org> #822: Segmentation fault ICMP Analizer -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by jsiwek): * status: new => closed * resolution: => Solved/Applied Comment: The fix for this is in the master branch now. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 4 22:38:25 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 05 Jun 2012 05:38:25 -0000 Subject: [Bro-Dev] #827: MIME analyzer doesn't decode "encoded-word" encoding. Message-ID: <046.c53d86328568edbcd557ac89698567bb@tracker.bro-ids.org> #827: MIME analyzer doesn't decode "encoded-word" encoding. ---------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ It seems like the MIME analyzer should be doing this decoding for us directly. http://en.wikipedia.org/wiki/MIME#Encoded-Word -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Jun 5 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 5 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206050700.q557028j001233@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 551 [1] | seth | | Normal | Potential alternate signature loading method? Bro | 825 [2] | dnthayer | | Normal | topic/dnthayer/bif-tests [3] [1] #551: http://tracker.bro-ids.org/bro/ticket/551 [2] #825: http://tracker.bro-ids.org/bro/ticket/825 [3] bif-tests: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/bif-tests From robin at icir.org Tue Jun 5 08:42:40 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 5 Jun 2012 08:42:40 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Fixed a bug with the MIME analyzer not removing whitespace on wrapped headers. (89cb103) In-Reply-To: <201206051525.q55FPHxx022989@bro-ids.icir.org> References: <201206051525.q55FPHxx022989@bro-ids.icir.org> Message-ID: <20120605154240.GF61042@icir.org> On Tue, Jun 05, 2012 at 08:25 -0700, Seth Hall wrote: > - No test due to lack of tracefile with wrapped header. Were you able to test this in any form? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Tue Jun 5 09:32:28 2012 From: seth at icir.org (Seth Hall) Date: Tue, 5 Jun 2012 12:32:28 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Fixed a bug with the MIME analyzer not removing whitespace on wrapped headers. (89cb103) In-Reply-To: <20120605154240.GF61042@icir.org> References: <201206051525.q55FPHxx022989@bro-ids.icir.org> <20120605154240.GF61042@icir.org> Message-ID: On Jun 5, 2012, at 11:42 AM, Robin Sommer wrote: > On Tue, Jun 05, 2012 at 08:25 -0700, Seth Hall wrote: > >> - No test due to lack of tracefile with wrapped header. > > Were you able to test this in any form? Yes, I have a non-distributable tracefile that this fixes. It should be easy to find an example though, wrapped headers in SMTP is really common we could probably even find one in our test trace files somewhere, we just need to find it. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Tue Jun 5 09:39:58 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 05 Jun 2012 16:39:58 -0000 Subject: [Bro-Dev] #828: topic/dnthayer/cronenabled Message-ID: <050.90e73239e2a7f57e02f9cb47628022f3@tracker.bro-ids.org> #828: topic/dnthayer/cronenabled ---------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Keywords: | ---------------------------+------------------------ This branch fixes a bug with the broctl "cron" command. Initially, there is a config option "cronenabled=1". When a user does "cron disable", a dynamic state variable is set ("cronenabled=0") that does not affect the original config option (both values are visible from the broctl "config" command). At that point, doing a "cron ?" shows "cron enabled" because it grabs the config option instead of the dynamic state variable. Fixed by removing the "cronenabled" config option. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Jun 6 00:00:01 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 6 Jun 2012 00:00:01 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206060700.q56701BZ026528@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 551 [1] | seth | | Normal | Potential alternate signature loading method? Bro | 825 [2] | dnthayer | | Normal | topic/dnthayer/bif-tests [3] BroControl | 828 [4] | dnthayer | | Normal | topic/dnthayer/cronenabled [5] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 89cb103 | Seth Hall | 2012-06-05 | Fixed a bug with the MIME analyzer not removing whitespace on wrapped headers. [6] [1] #551: http://tracker.bro-ids.org/bro/ticket/551 [2] #825: http://tracker.bro-ids.org/bro/ticket/825 [3] bif-tests: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/bif-tests [4] #828: http://tracker.bro-ids.org/bro/ticket/828 [5] cronenabled: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbrocontrol&old=master&new_path=%2Fbrocontrol&new=topic/dnthayer/cronenabled [6] fastpath: http://tracker.bro-ids.org/bro/changeset/89cb103a2c07aede9969ee586225c4d7b0411a29/bro From bro at tracker.bro-ids.org Wed Jun 6 11:10:17 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 06 Jun 2012 18:10:17 -0000 Subject: [Bro-Dev] #825: topic/dnthayer/bif-tests In-Reply-To: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> References: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> Message-ID: <065.ea1dbec759979b6da45ab8d106e5648a@tracker.bro-ids.org> #825: topic/dnthayer/bif-tests ----------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Impressive, many thanks for the tests! -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jun 6 11:44:55 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 06 Jun 2012 18:44:55 -0000 Subject: [Bro-Dev] #825: topic/dnthayer/bif-tests In-Reply-To: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> References: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> Message-ID: <065.9b91f279c492101f9284a51ce3b69a3c@tracker.bro-ids.org> #825: topic/dnthayer/bif-tests ----------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): - type_name(): I've extended the vector description. For files, you're right, but I'm leaving that as is for now; the right fix would be to remove the file's type argument altogether (I don't think it's used for anything). - fmt(): I've removed documentation for that, seems like an odd use case anyways. Also remove the stringn check in the code; that's already taken care of elsewhere. - order(): the output seems to be correct to me; what doesn't make sense to you? -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Wed Jun 6 12:20:59 2012 From: robin at icir.org (Robin Sommer) Date: Wed, 6 Jun 2012 12:20:59 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Fixed a bug with the MIME analyzer not removing whitespace on wrapped headers. (89cb103) In-Reply-To: <20120605154240.GF61042@icir.org> References: <201206051525.q55FPHxx022989@bro-ids.icir.org> <20120605154240.GF61042@icir.org> Message-ID: <20120606192059.GA33440@icir.org> I had applied this, but I'm not sure it's actually correct. RFC 822 states: Unfolding is accomplished by regarding CRLF immediately followed by a LWSP-char as equivalent to the LWSP-char. Which seems to say the white-space should be left in except for the CRLF. Also, by removing it, the two parts now flow togehter. Here's an example I found with the test-suite: Old: ([xx.xx.xx.xx])\x09by xxx.xxx.gov New: ([xx.xx.xx.xx])by xxx.xxx.gov So I'm reverting this for now until we've decided what's right. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From bro at tracker.bro-ids.org Wed Jun 6 12:32:57 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 06 Jun 2012 19:32:57 -0000 Subject: [Bro-Dev] #551: Potential alternate signature loading method? In-Reply-To: <046.8fa46d7f399c279d6a4c4e9463558fac@tracker.bro-ids.org> References: <046.8fa46d7f399c279d6a4c4e9463558fac@tracker.bro-ids.org> Message-ID: <061.2f74c279f7deecafb40cd346746b337b@tracker.bro-ids.org> #551: Potential alternate signature loading method? ----------------------------+-------------------- Reporter: seth | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: Resolution: fixed | Keywords: ----------------------------+-------------------- Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [45f59005473a56a523ca44c1acd15ad7251e0656/bro]: {{{ #!CommitTicketReference repository="bro" revision="45f59005473a56a523ca44c1acd15ad7251e0656" Merge remote-tracking branch 'origin/topic/jsiwek/load-sigs' * origin/topic/jsiwek/load-sigs: Add @load-sigs directive for loading signature files (addresses #551). Closes #551. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jun 6 12:33:14 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 06 Jun 2012 19:33:14 -0000 Subject: [Bro-Dev] #828: topic/dnthayer/cronenabled In-Reply-To: <050.90e73239e2a7f57e02f9cb47628022f3@tracker.bro-ids.org> References: <050.90e73239e2a7f57e02f9cb47628022f3@tracker.bro-ids.org> Message-ID: <065.746d469b24852688a41ba4b7d23651bc@tracker.bro-ids.org> #828: topic/dnthayer/cronenabled ----------------------------+------------------------ Reporter: dnthayer | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: => robin * status: new => closed * resolution: => fixed Comment: In [589cb04c3d7e28a81aa07454e2b9b6b092f0e1af/broctl]: {{{ #!CommitTicketReference repository="broctl" revision="589cb04c3d7e28a81aa07454e2b9b6b092f0e1af" Merge remote-tracking branch 'origin/topic/dnthayer/cronenabled' * origin/topic/dnthayer/cronenabled: Fix the "cron disable" command Closes #828. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jun 6 13:05:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 06 Jun 2012 20:05:48 -0000 Subject: [Bro-Dev] #825: topic/dnthayer/bif-tests In-Reply-To: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> References: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> Message-ID: <065.3e61c3a8c48dc73aebc96c2688ad8a17@tracker.bro-ids.org> #825: topic/dnthayer/bif-tests ----------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by jsiwek): The baseline for the val_size() isn't going to be correct for some platforms where the sizeof some datatypes is different from wherever the baseline was created. Not sure how to alter the test, maybe just check a range of "reasonable" sizes? Also the baseline for the rand() test isn't matching what was expected on a couple different systems where I tried it (and both system's expectations were different from each other even). -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Jun 7 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 7 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206070700.q57702pP001581@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 825 [1] | dnthayer | | Normal | topic/dnthayer/bif-tests [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | d1e4e6e | Jon Siwek | 2012-06-06 | Include header for usleep(), caused compile failure on Archlinux. [3] [1] #825: http://tracker.bro-ids.org/bro/ticket/825 [2] bif-tests: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/bif-tests [3] fastpath: http://tracker.bro-ids.org/bro/changeset/d1e4e6e812a99274aa85a29c16093b87bebaa499/bro From bro at tracker.bro-ids.org Thu Jun 7 09:33:25 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 07 Jun 2012 16:33:25 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' Message-ID: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' ----------------------------+--------------------- Reporter: Tyler.Schoenke | Type: Problem Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: ----------------------------+--------------------- {{{ #0 0x00007fb04706aa75 in raise () from /lib/libc.so.6 #1 0x00007fb04706e5c0 in abort () from /lib/libc.so.6 #2 0x00007fb0479208c5 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/libstdc++.so.6 #3 0x00007fb04791ecf6 in ?? () from /usr/lib/libstdc++.so.6 #4 0x00007fb04791ed23 in std::terminate() () from /usr/lib/libstdc++.so.6 #5 0x00007fb04791ee1e in __cxa_throw () from /usr/lib/libstdc++.so.6 #6 0x00007fb0478ba987 in std::__throw_logic_error(char const*) () from /usr/lib/libstdc++.so.6 #7 0x00007fb0478fc911 in ?? () from /usr/lib/libstdc++.so.6 #8 0x00007fb0478fca33 in std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) () from /usr/lib/libstdc++.so.6 #9 0x00000000008020b8 in logging::Manager::ValToLogVal (this=0xc542a0, val=0x7fb0349103e0, ty=0x7fb0301ea1d0) at /root/src/bro-git-20120606/src/logging/Manager.cc:822 #10 0x0000000000802512 in logging::Manager::ValToLogVal (this=0xc542a0, val=0x7fb034928da0, ty=0x7fb0301e9ee0) at /root/src/bro-git-20120606/src/logging/Manager.cc:886 #11 0x00000000008027d9 in logging::Manager::RecordToFilterVals (this=0xc542a0, stream=0x1e81260, filter=0x1ea8940, columns=0x7fb0345b6350) at /root/src/bro-git-20120606/src/logging/Manager.cc:944 #12 0x0000000000801e14 in logging::Manager::Write (this=0xc542a0, id=0x1090450, columns=0x7fb0345b6350) at /root/src/bro-git-20120606/src/logging/Manager.cc:781 #13 0x00000000006ee67f in BifFunc::Log::bro___write (frame=0x7fb0345c2670, BiF_ARGS=0x7fb0346c7e30) at logging.bif:47 #14 0x00000000006d8dc4 in BuiltinFunc::Call (this=0xe8b130, args=0x7fb0346c7e30, parent=0x7fb0345c2670) at /root/src/bro-git-20120606/src/Func.cc:475 #15 0x00000000006c5db0 in CallExpr::Eval (this=0xe51120, f=0x7fb0345c2670) at /root/src/bro-git-20120606/src/Expr.cc:4519 #16 0x00000000007b942d in ReturnStmt::Exec (this=0xe511c0, f=0x7fb0345c2670, flow=@0x7fff510a1184) at /root/src/bro-git-20120606/src/Stmt.cc:1344 #17 0x00000000007b9b2b in StmtList::Exec (this=0xe50ba0, f=0x7fb0345c2670, flow=@0x7fff510a1184) at /root/src/bro-git-20120606/src/Stmt.cc:1404 #18 0x00000000006d802d in BroFunc::Call (this=0xe51290, args=0x7fb0345c26e0, parent=0x7fb0345c25e0) at /root/src/bro-git-20120606/src/Func.cc:332 #19 0x00000000006c5db0 in CallExpr::Eval (this=0x1090890, f=0x7fb0345c25e0) at /root/src/bro-git-20120606/src/Expr.cc:4519 #20 0x00000000007b413f in ExprStmt::Exec (this=0x1090930, f=0x7fb0345c25e0, flow=@0x7fff510a1494) at /root/src/bro-git-20120606/src/Stmt.cc:369 #21 0x00000000007b4bd5 in IfStmt::DoExec (this=0x1090a90, f=0x7fb0345c25e0, v=0x7fb0346c7fc0, flow=@0x7fff510a1494) at /root/src/bro-git-20120606/src/Stmt.cc:484 #22 0x00000000007b4173 in ExprStmt::Exec (this=0x1090a90, f=0x7fb0345c25e0, flow=@0x7fff510a1494) at /root/src/bro-git-20120606/src/Stmt.cc:373 #23 0x00000000007b9b2b in StmtList::Exec (this=0x108eda0, f=0x7fb0345c25e0, flow=@0x7fff510a1494) at /root/src/bro-git-20120606/src/Stmt.cc:1404 #24 0x00000000006d802d in BroFunc::Call (this=0x10646b0, args=0x7fb0346c7e50, parent=0x0) at /root/src/bro-git-20120606/src/Func.cc:332 #25 0x00000000006932f4 in EventHandler::Call (this=0x1064820, vl=0x7fb0346c7e50, no_remote=false) at /root/src/bro-git-20120606/src/EventHandler.cc:72 #26 0x00000000006247a4 in Event::Dispatch (this=0x7fb0346c81a0, no_remote=false) at /root/src/bro-git-20120606/src/Event.h:46 #27 0x0000000000692915 in EventMgr::Dispatch (this=0xb6a180) at /root/src/bro-git-20120606/src/Event.cc:105 #28 0x0000000000692990 in EventMgr::Drain (this=0xb6a180) at /root/src/bro-git-20120606/src/Event.cc:117 #29 0x000000000074235c in net_run () at /root/src/bro-git-20120606/src/Net.cc:502 #30 0x0000000000623e6e in main (argc=16, argv=0x7fff510a1cd8) at /root/src/bro-git-20120606/src/main.cc:1065 }}} core [New Thread 28841] [New Thread 29110] [New Thread 29116] [New Thread 29101] [New Thread 29097] [New Thread 29111] [New Thread 29105] [New Thread 29099] [New Thread 28842] [New Thread 29113] [New Thread 29112] [New Thread 29114] [New Thread 28845] [New Thread 29117] [New Thread 29115] [New Thread 29118] [New Thread 31021] [New Thread 28844] [New Thread 29093] [New Thread 29103] [New Thread 29104] [New Thread 29106] [New Thread 29107] [New Thread 29108] [New Thread 29109] [New Thread 31020] Core was generated by `/usr/local/bro-20120606/bin/bro -U .status -p broctl -p broctl-live -p local -p'. Program terminated with signal 6, Aborted. #0 0x00007fb04706aa75 in raise () from /lib/libc.so.6 ==== reporter.log 0.000000 Reporter::ERROR field value missing [Notice::n$src] /usr/local/bro-20120606/share/bro/site/tts/tts-email-per-orig.bro, line 21 0.000000 Reporter::ERROR field value missing [Notice::n$src] /usr/local/bro-20120606/share/bro/site/tts/tts-email-per-orig.bro, line 21 0.000000 Reporter::ERROR field value missing [Notice::n$src] /usr/local/bro-20120606/share/bro/site/tts/tts-email-per-orig.bro, line 21 0.000000 Reporter::ERROR field value missing [Notice::n$src] /usr/local/bro-20120606/share/bro/site/tts/tts-email-per-orig.bro, line 21 0.000000 Reporter::ERROR field value missing [Notice::n$src] /usr/local/bro-20120606/share/bro/site/tts/tts-email-per-orig.bro, line 21 0.000000 Reporter::ERROR field value missing [Notice::n$src] /usr/local/bro-20120606/share/bro/site/tts/tts-email-per-orig.bro, line 21 0.000000 Reporter::ERROR field value missing [Notice::n$src] /usr/local/bro-20120606/share/bro/site/tts/tts-email-per-orig.bro, line 21 0.000000 Reporter::ERROR field value missing [Notice::n$src] /usr/local/bro-20120606/share/bro/site/tts/tts-email-per-orig.bro, line 21 0.000000 Reporter::ERROR field value missing [Notice::n$src] /usr/local/bro-20120606/share/bro/site/tts/tts-email-per-orig.bro, line 21 0.000000 Reporter::ERROR field value missing [Notice::n$src] /usr/local/bro-20120606/share/bro/site/tts/tts-email-per-orig.bro, line 21 ==== stderr.log terminate called after throwing an instance of 'std::logic_error' what(): basic_string::_S_construct NULL not valid /usr/local/bro-20120606/share/broctl/scripts/run-bro: line 60: 28841 Aborted (core dumped) nohup $mybro $@ ==== stdout.log unlimited unlimited unlimited ==== .cmdline -U .status -p broctl -p broctl-live -p local -p manager local.bro broctl base/frameworks/cluster local-manager.bro broctl/auto ==== .env_vars PATH=/usr/local/bro-20120606/bin:/usr/local/bro-20120606/share/broctl/scripts:/usr/local/bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games BROPATH=/usr/local/bro-20120606/spool/installed-scripts-do-not- touch/site::/usr/local/bro-20120606/spool/installed-scripts-do-not- touch/auto:/usr/local/bro-20120606/share/bro:/usr/local/bro-20120606/share/bro/policy:/usr/local/bro-20120606/share/bro/site CLUSTER_NODE=manager ==== .status RUNNING [net_run] ==== No prof.log ==== packet_filter.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path packet_filter #fields ts node filter init success #types time string string bool bool ==== loaded_scripts.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path loaded_scripts #fields name #types string /usr/local/bro-20120606/share/bro/base/init-bare.bro /usr/local/bro-20120606/share/bro/base/const.bif.bro /usr/local/bro-20120606/share/bro/base/types.bif.bro /usr/local/bro-20120606/share/bro/base/strings.bif.bro /usr/local/bro-20120606/share/bro/base/bro.bif.bro /usr/local/bro-20120606/share/bro/base/reporter.bif.bro /usr/local/bro-20120606/share/bro/base/event.bif.bro /usr/local/bro-20120606/share/bro/base/frameworks/logging/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/logging/./main.bro /usr/local/bro-20120606/share/bro/base/logging.bif.bro /usr/local/bro-20120606/share/bro/base/frameworks/logging/./postprocessors/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/logging/./postprocessors/./scp.bro /usr/local/bro-20120606/share/bro/base/frameworks/logging/./postprocessors/./sftp.bro /usr/local/bro-20120606/share/bro/base/frameworks/logging/./writers/ascii.bro /usr/local/bro-20120606/share/bro/base/frameworks/logging/./writers/dataseries.bro /usr/local/bro-20120606/share/bro/base/frameworks/input/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/input/./main.bro /usr/local/bro-20120606/share/bro/base/input.bif.bro /usr/local/bro-20120606/share/bro/base/frameworks/input/./readers/ascii.bro /usr/local/bro-20120606/share/bro/base/frameworks/input/./readers/raw.bro /usr/local/bro-20120606/share/bro/base/frameworks/input/./readers/benchmark.bro /usr/local/bro-20120606/share/bro/base/init-default.bro /usr/local/bro-20120606/share/bro/base/utils/site.bro /usr/local/bro-20120606/share/bro/base/utils/./patterns.bro /usr/local/bro-20120606/share/bro/base/utils/addrs.bro /usr/local/bro-20120606/share/bro/base/utils/conn-ids.bro /usr/local/bro-20120606/share/bro/base/utils/directions-and-hosts.bro /usr/local/bro-20120606/share/bro/base/utils/files.bro /usr/local/bro-20120606/share/bro/base/utils/numbers.bro /usr/local/bro-20120606/share/bro/base/utils/paths.bro /usr/local/bro-20120606/share/bro/base/utils/strings.bro /usr/local/bro-20120606/share/bro/base/utils/thresholds.bro /usr/local/bro-20120606/share/bro/base/frameworks/notice/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/notice/./main.bro /usr/local/bro-20120606/share/bro/base/frameworks/notice/./weird.bro /usr/local/bro-20120606/share/bro/base/frameworks/notice/./actions/drop.bro /usr/local/bro-20120606/share/bro/base/frameworks/notice/./actions/email_admin.bro /usr/local/bro-20120606/share/bro/base/frameworks/notice/./actions/page.bro /usr/local/bro-20120606/share/bro/base/frameworks/notice/./actions/add- geodata.bro /usr/local/bro-20120606/share/bro/base/frameworks/notice/./extend- email/hostnames.bro /usr/local/bro-20120606/share/bro/base/frameworks/cluster/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/cluster/./main.bro /usr/local/bro-20120606/share/bro/base/frameworks/control/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/control/./main.bro /usr/local/bro-20120606/spool/installed-scripts-do-not-touch/auto/cluster- layout.bro /usr/local/bro-20120606/share/bro/base/frameworks/cluster/./setup- connections.bro /usr/local/bro-20120606/share/bro/base/frameworks/communication/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/communication/./main.bro /usr/local/bro-20120606/share/bro/base/frameworks/packet- filter/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/packet-filter/./main.bro /usr/local/bro-20120606/share/bro/base/frameworks/packet- filter/./netstats.bro /usr/local/bro-20120606/share/bro/policy/frameworks/communication/listen.bro /usr/local/bro-20120606/share/bro/base/frameworks/cluster/./nodes/manager.bro /usr/local/bro-20120606/share/bro/base/frameworks/notice/./cluster.bro /usr/local/bro-20120606/share/bro/base/frameworks/notice/./actions/pp- alarms.bro /usr/local/bro-20120606/share/bro/base/frameworks/dpd/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/dpd/./main.bro /usr/local/bro-20120606/share/bro/base/frameworks/signatures/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/signatures/./main.bro /usr/local/bro-20120606/share/bro/base/frameworks/software/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/software/./main.bro /usr/local/bro-20120606/share/bro/base/frameworks/metrics/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/metrics/./main.bro /usr/local/bro-20120606/share/bro/base/frameworks/metrics/./cluster.bro /usr/local/bro-20120606/share/bro/base/frameworks/intel/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/intel/./main.bro /usr/local/bro-20120606/share/bro/base/frameworks/reporter/__load__.bro /usr/local/bro-20120606/share/bro/base/frameworks/reporter/./main.bro /usr/local/bro-20120606/share/bro/base/protocols/conn/__load__.bro /usr/local/bro-20120606/share/bro/base/protocols/conn/./main.bro /usr/local/bro-20120606/share/bro/base/protocols/conn/./contents.bro /usr/local/bro-20120606/share/bro/base/protocols/conn/./inactivity.bro /usr/local/bro-20120606/share/bro/base/protocols/dns/__load__.bro /usr/local/bro-20120606/share/bro/base/protocols/dns/./consts.bro /usr/local/bro-20120606/share/bro/base/protocols/dns/./main.bro /usr/local/bro-20120606/share/bro/base/protocols/ftp/__load__.bro /usr/local/bro-20120606/share/bro/base/protocols/ftp/./utils-commands.bro /usr/local/bro-20120606/share/bro/base/protocols/ftp/./main.bro /usr/local/bro-20120606/share/bro/base/protocols/ftp/./file-extract.bro /usr/local/bro-20120606/share/bro/base/protocols/http/__load__.bro /usr/local/bro-20120606/share/bro/base/protocols/http/./main.bro /usr/local/bro-20120606/share/bro/base/protocols/http/./utils.bro /usr/local/bro-20120606/share/bro/base/protocols/http/./file-ident.bro /usr/local/bro-20120606/share/bro/base/protocols/http/./file-hash.bro /usr/local/bro-20120606/share/bro/base/protocols/http/./file-extract.bro /usr/local/bro-20120606/share/bro/base/protocols/irc/__load__.bro /usr/local/bro-20120606/share/bro/base/protocols/irc/./main.bro /usr/local/bro-20120606/share/bro/base/protocols/irc/./dcc-send.bro /usr/local/bro-20120606/share/bro/base/protocols/smtp/__load__.bro /usr/local/bro-20120606/share/bro/base/protocols/smtp/./main.bro /usr/local/bro-20120606/share/bro/base/protocols/smtp/./entities.bro /usr/local/bro-20120606/share/bro/base/protocols/smtp/./entities- excerpt.bro /usr/local/bro-20120606/share/bro/base/protocols/ssh/__load__.bro /usr/local/bro-20120606/share/bro/base/protocols/ssh/./main.bro /usr/local/bro-20120606/share/bro/base/protocols/ssl/__load__.bro /usr/local/bro-20120606/share/bro/base/protocols/ssl/./consts.bro /usr/local/bro-20120606/share/bro/base/protocols/ssl/./main.bro /usr/local/bro-20120606/share/bro/base/protocols/ssl/./mozilla-ca-list.bro /usr/local/bro-20120606/share/bro/base/protocols/syslog/__load__.bro /usr/local/bro-20120606/share/bro/base/protocols/syslog/./consts.bro /usr/local/bro-20120606/share/bro/base/protocols/syslog/./main.bro /usr/local/bro-20120606/spool/installed-scripts-do-not- touch/site/local.bro /usr/local/bro-20120606/share/bro/policy/misc/loaded-scripts.bro /usr/local/bro-20120606/share/bro/policy/tuning/defaults/__load__.bro /usr/local/bro-20120606/share/bro/policy/tuning/defaults/./packet- fragments.bro /usr/local/bro-20120606/share/bro/policy/tuning/defaults/./warnings.bro /usr/local/bro-20120606/share/bro/policy/frameworks/software/vulnerable.bro /usr/local/bro-20120606/share/bro/policy/frameworks/software/version- changes.bro /usr/local/bro-20120606/share/bro/site/bro-scripts/http-exe-bad- attributes.bro /usr/local/bro-20120606/share/bro/site/bro-scripts/scan.bro /usr/local/bro-20120606/share/bro/site/bro-scripts/scan.cluster.bro /usr/local/bro-20120606/share/bro/policy/misc/capture-loss.bro /usr/local/bro-20120606/share/bro/site/robin/alarm-summary-interval.bro /usr/local/bro-20120606/share/bro/site/tts/tts-bpf-packetfilter.bro /usr/local/bro-20120606/share/bro/site/tts/tts-sig-once.bro /usr/local/bro-20120606/share/bro/policy/protocols/ftp/software.bro /usr/local/bro-20120606/share/bro/policy/protocols/smtp/software.bro /usr/local/bro-20120606/share/bro/policy/protocols/ssh/software.bro /usr/local/bro-20120606/share/bro/policy/protocols/http/software.bro /usr/local/bro-20120606/share/bro/policy/protocols/dns/detect-external- names.bro /usr/local/bro-20120606/share/bro/policy/protocols/ftp/detect.bro /usr/local/bro-20120606/share/bro/policy/protocols/conn/known-hosts.bro /usr/local/bro-20120606/share/bro/policy/protocols/conn/known-services.bro /usr/local/bro-20120606/share/bro/policy/protocols/ssl/known-certs.bro /usr/local/bro-20120606/share/bro/policy/protocols/ssl/cert-hash.bro /usr/local/bro-20120606/share/bro/policy/protocols/ssl/validate-certs.bro /usr/local/bro-20120606/share/bro/site/tts/mozilla-ruby-cert.bro /usr/local/bro-20120606/share/bro/policy/protocols/ssh/geo-data.bro /usr/local/bro-20120606/share/bro/policy/protocols/ssh/detect- bruteforcing.bro /usr/local/bro-20120606/share/bro/policy/protocols/ssh/interesting- hostnames.bro /usr/local/bro-20120606/share/bro/policy/protocols/http/detect-MHR.bro /usr/local/bro-20120606/share/bro/policy/protocols/http/detect-sqli.bro /usr/local/bro-20120606/share/bro/broctl/__load__.bro /usr/local/bro-20120606/share/bro/broctl/./main.bro /usr/local/bro-20120606/share/bro/policy/frameworks/control/controllee.bro /usr/local/bro-20120606/spool/installed-scripts-do-not-touch/site/local- manager.bro /usr/local/bro-20120606/share/bro/site/tts/tts-http.bro /usr/local/bro-20120606/share/bro/site/tts/tts-email-per-orig.bro /usr/local/bro-20120606/share/bro/broctl/auto.bro /usr/local/bro-20120606/spool/installed-scripts-do-not-touch/auto/local- networks.bro /usr/local/bro-20120606/spool/installed-scripts-do-not-touch/auto/broctl- config.bro -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jun 7 14:07:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 07 Jun 2012 21:07:21 -0000 Subject: [Bro-Dev] #830: topic/tunnels Message-ID: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> #830: topic/tunnels --------------------+------------------------ Reporter: jsiwek | Owner: seth Type: Task | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | --------------------+------------------------ This branch is in `bro`, `cmake`, `bro-testing`, and `bro-testing-private` so far. It adds support for different forms of tunnel decapsulation: IPv{4,6}-in-IPv{4,6}, Teredo, and AYIYA. The usual packet processing will recurse on the encapsulated packets, and the presence of tunnels is conveyed in three major ways at the script layer: 1) base/frameworks/tunnels creates a new tunnel.log to log discovery of new tunnels 2) connection records have been extended with a *tunnel* field to indicate whether the connection exists within a tunnel 3) base/protocols/conn will log in conn.log the UIDs of all tunnels a given connection was seen encapsulated within, and that UID can be used to cross reference the tunnel UIDs in tunnel.log or other connection UIDs in conn.log A SOCKs v4 analyzer was also (re)added and currently any SOCKS requests register themselves at the scripting layer as a type of tunnel. Seth, can you look into the following and turn into a merge request after: * Does the representation of tunnels at the scripting-layer and/or format of tunnel.log need any tweaks? * Are the connections analyzed via SOCKS understandable or does something more have to be done in the logs to make it more clear that one endpoint is still the proxy and maybe not the real destination? (I think being able to cross reference tunnel.log by UID or lookup in the `Tunnel::active` table by `conn_id` can be helpful here) * Anything you want to cleanup with base/protocols/socks/main.bro (tons of commented out stuff in there) ? * SOCKS needs a test case. Robin, if you had time to start looking at the branch and want to make suggestions, that would help, too. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jun 7 15:12:25 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 07 Jun 2012 22:12:25 -0000 Subject: [Bro-Dev] #825: topic/dnthayer/bif-tests In-Reply-To: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> References: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> Message-ID: <065.9ac1a60e92cfe8cab384e47a4a673c3c@tracker.bro-ids.org> #825: topic/dnthayer/bif-tests ----------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): val_size(): Yeah, how about just checking for two numbers being printed, not worth doing much more. rand(): When testing, random values should be consistent, even across platform (through setting BRO_SEED_FILE). If that's not the case, that's worth fixing. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jun 7 15:30:31 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 07 Jun 2012 22:30:31 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.43afa5ce5db41c27d6fe14323d28b979@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Changes (by robin): * milestone: => Bro2.1 -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Jun 8 00:00:01 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 8 Jun 2012 00:00:01 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206080700.q58701PW027597@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 825 [1] | dnthayer | | Normal | topic/dnthayer/bif-tests [2] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ btest | 569d62b | Daniel Thayer | 2012-06-07 | Fix typos in README and btest help output [3] [1] #825: http://tracker.bro-ids.org/bro/ticket/825 [2] bif-tests: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/bif-tests [3] fastpath: http://tracker.bro-ids.org/bro/changeset/569d62b03a02214c3b5c169bd2ddc5282deaca84/btest From bro at tracker.bro-ids.org Fri Jun 8 08:19:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 08 Jun 2012 15:19:03 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.5a76c37c56847f38a788bbd0f01da39c@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by jsiwek): In [e9c18b51a31fb1a7e6aba802d8fd1cc16f4927f7/bro]: {{{ #!CommitTicketReference repository="bro" revision="e9c18b51a31fb1a7e6aba802d8fd1cc16f4927f7" Add more error handling code to logging of enum vals. (addresses #829) If lookup of enum name by value fails, an error is now sent through the reporter framework and the value logged will be an empty string (as opposed to trying to construct a string with null pointer which throws a logic_error and aborts Bro). }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 8 08:29:18 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 08 Jun 2012 15:29:18 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.d5729a932c4ac1e49372dfc596dc742e@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Changes (by jsiwek): * type: Problem => Merge Request Comment: > and the value logged will be an empty string Or maybe it should log the string representation of the internal integer value of the enum? There's probably another issue that was leading to this crash that could be caught at an earlier time, but I don't understand the conditions where an enum's type could fail to have a name mapping for it's value. The new errors that will be put in reporter.log from the patch in `topic/jsiwek /enum-log-error-handling` might help pinpoint how to reproduce it. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 8 10:49:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 08 Jun 2012 17:49:56 -0000 Subject: [Bro-Dev] #825: topic/dnthayer/bif-tests In-Reply-To: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> References: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> Message-ID: <065.6b0df4378c9cecd3f0bb807aa529d447@tracker.bro-ids.org> #825: topic/dnthayer/bif-tests ----------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by dnthayer): Replying to [comment:2 robin]: > > - type_name(): I've extended the vector description. For files, you're right, but I'm leaving that as is for now; the right fix would be to remove the file's type argument altogether (I don't think it's used for anything). > > - fmt(): I've removed documentation for that, seems like an odd use case anyways. Also remove the stringn check in the code; that's already taken care of elsewhere. OK, thanks. > - order(): the output seems to be correct to me; what doesn't make sense to you? I looked at it again, and I think I understand it now. The problem is the documentation is not clear (I will improve that). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 8 16:51:11 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 08 Jun 2012 23:51:11 -0000 Subject: [Bro-Dev] #831: Memory leak in print Message-ID: <048.ea67e86bfb2a6633bbc5ada34ca295e5@tracker.bro-ids.org> #831: Memory leak in print ------------------------+--------------------- Reporter: amannb | Type: Problem Status: new | Priority: Normal Milestone: Bro2.1 | Component: Bro Version: git/master | Keywords: ------------------------+--------------------- The following bro script apparently triggers a memory-leak in the print statement. {{{ event HTTP::log_http(rec: HTTP::Info) { print fmt("%s %s", rec$md5, rec); } }}} To reproduce run bro using 2009-M57-day11-18.trace. pprof output is attached. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Jun 9 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 9 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206090700.q59702oA027576@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 825 [1] | dnthayer | | Normal | topic/dnthayer/bif-tests [2] Bro | 829 [3] | Tyler.Schoenke | | Normal | terminate called after throwing an instance of 'std::logic_error' > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 191994a | Daniel Thayer | 2012-06-08 | Fix summary lines for BIF documentation [4] bro | 18e61fc | Daniel Thayer | 2012-06-08 | Fix val_size BIF tests and improve docs [5] btest | cab9c8b | Daniel Thayer | 2012-06-08 | Remove code to expand env. vars. on cmdline [6] btest | 2cbbefe | Daniel Thayer | 2012-06-08 | Add more explanation about expansion of env. vars. [7] btest | 569d62b | Daniel Thayer | 2012-06-07 | Fix typos in README and btest help output [8] [1] #825: http://tracker.bro-ids.org/bro/ticket/825 [2] bif-tests: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/bif-tests [3] #829: http://tracker.bro-ids.org/bro/ticket/829 [4] fastpath: http://tracker.bro-ids.org/bro/changeset/191994a60a8050bc04a900098da5bf2c66821a54/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/18e61fcdfcb7dca87fba5e07232bf52f21eb7814/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/cab9c8b48daa5216b57a4a14d1284d7cb5dd80e1/btest [7] fastpath: http://tracker.bro-ids.org/bro/changeset/2cbbefed7bfcccd5633f12fdbe4151337577e73c/btest [8] fastpath: http://tracker.bro-ids.org/bro/changeset/569d62b03a02214c3b5c169bd2ddc5282deaca84/btest From noreply at bro-ids.org Sun Jun 10 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 10 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206100700.q5A702nS025727@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 825 [1] | dnthayer | | Normal | topic/dnthayer/bif-tests [2] Bro | 829 [3] | Tyler.Schoenke | | Normal | terminate called after throwing an instance of 'std::logic_error' > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 191994a | Daniel Thayer | 2012-06-08 | Fix summary lines for BIF documentation [4] bro | 18e61fc | Daniel Thayer | 2012-06-08 | Fix val_size BIF tests and improve docs [5] btest | cab9c8b | Daniel Thayer | 2012-06-08 | Remove code to expand env. vars. on cmdline [6] btest | 2cbbefe | Daniel Thayer | 2012-06-08 | Add more explanation about expansion of env. vars. [7] btest | 569d62b | Daniel Thayer | 2012-06-07 | Fix typos in README and btest help output [8] [1] #825: http://tracker.bro-ids.org/bro/ticket/825 [2] bif-tests: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/bif-tests [3] #829: http://tracker.bro-ids.org/bro/ticket/829 [4] fastpath: http://tracker.bro-ids.org/bro/changeset/191994a60a8050bc04a900098da5bf2c66821a54/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/18e61fcdfcb7dca87fba5e07232bf52f21eb7814/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/cab9c8b48daa5216b57a4a14d1284d7cb5dd80e1/btest [7] fastpath: http://tracker.bro-ids.org/bro/changeset/2cbbefed7bfcccd5633f12fdbe4151337577e73c/btest [8] fastpath: http://tracker.bro-ids.org/bro/changeset/569d62b03a02214c3b5c169bd2ddc5282deaca84/btest From noreply at bro-ids.org Mon Jun 11 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 11 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206110700.q5B702mS029683@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 825 [1] | dnthayer | | Normal | topic/dnthayer/bif-tests [2] Bro | 829 [3] | Tyler.Schoenke | | Normal | terminate called after throwing an instance of 'std::logic_error' > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 191994a | Daniel Thayer | 2012-06-08 | Fix summary lines for BIF documentation [4] bro | 18e61fc | Daniel Thayer | 2012-06-08 | Fix val_size BIF tests and improve docs [5] btest | cab9c8b | Daniel Thayer | 2012-06-08 | Remove code to expand env. vars. on cmdline [6] btest | 2cbbefe | Daniel Thayer | 2012-06-08 | Add more explanation about expansion of env. vars. [7] btest | 569d62b | Daniel Thayer | 2012-06-07 | Fix typos in README and btest help output [8] [1] #825: http://tracker.bro-ids.org/bro/ticket/825 [2] bif-tests: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/dnthayer/bif-tests [3] #829: http://tracker.bro-ids.org/bro/ticket/829 [4] fastpath: http://tracker.bro-ids.org/bro/changeset/191994a60a8050bc04a900098da5bf2c66821a54/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/18e61fcdfcb7dca87fba5e07232bf52f21eb7814/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/cab9c8b48daa5216b57a4a14d1284d7cb5dd80e1/btest [7] fastpath: http://tracker.bro-ids.org/bro/changeset/2cbbefed7bfcccd5633f12fdbe4151337577e73c/btest [8] fastpath: http://tracker.bro-ids.org/bro/changeset/569d62b03a02214c3b5c169bd2ddc5282deaca84/btest From bro at tracker.bro-ids.org Mon Jun 11 08:21:02 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 11 Jun 2012 15:21:02 -0000 Subject: [Bro-Dev] #831: Memory leak in print In-Reply-To: <048.ea67e86bfb2a6633bbc5ada34ca295e5@tracker.bro-ids.org> References: <048.ea67e86bfb2a6633bbc5ada34ca295e5@tracker.bro-ids.org> Message-ID: <063.e6225eac32696fabd694840d2662a2e1@tracker.bro-ids.org> #831: Memory leak in print ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by robin): * priority: Normal => High Comment: This seems potentially significant. Anyone up for taking a look? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 11 08:24:16 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 11 Jun 2012 15:24:16 -0000 Subject: [Bro-Dev] #831: Memory leak in print In-Reply-To: <048.ea67e86bfb2a6633bbc5ada34ca295e5@tracker.bro-ids.org> References: <048.ea67e86bfb2a6633bbc5ada34ca295e5@tracker.bro-ids.org> Message-ID: <063.0efdfc9d45bf898751179cb360fee829@tracker.bro-ids.org> #831: Memory leak in print ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): Replying to [comment:1 robin]: > This seems potentially significant. Anyone up for taking a look? Yeah, I think I know what's going on, might have a patch shortly. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 11 13:00:35 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 11 Jun 2012 20:00:35 -0000 Subject: [Bro-Dev] #831: Memory leak in print In-Reply-To: <048.ea67e86bfb2a6633bbc5ada34ca295e5@tracker.bro-ids.org> References: <048.ea67e86bfb2a6633bbc5ada34ca295e5@tracker.bro-ids.org> Message-ID: <063.1ae075985e8656f394b9d3fe257b4c6a@tracker.bro-ids.org> #831: Memory leak in print ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): What's happening is the reference to `rec$md5` does not first check whether that optional field is initialized (e.g. `rec?$md5`), so when it isn't, an internal `InterpreterException` gets thrown and there's no cleanup of heap allocations during the stack unwind. I'm not sure that catching, cleaning up allocated resources, and rethrowing that exception every place it can occur is going to be the greatest way to fix this. E.g. currently that exception only gets thrown from a couple implementations of `UnaryExpr::Fold`, but that's called from a couple implementations of `Expr::Eval`, which maybe used in ~70 places, then still the callers of those places also potentially need to handle the exception to do cleanup, etc. Would it be better to wait and fix this through overhauling memory management to use smart pointers? That's planned, right? Any script that tries to reference a missing field should be corrected, anyway (these errors are logged in reporter.log). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 11 13:26:23 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 11 Jun 2012 20:26:23 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.e8bb7f7bb91da619ca738eda92c5bd36@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by Tyler.Schoenke): Hi Jon, I applied your code change, and now I get this error: {{{ #0 0x00007f6393816a75 in raise () from /lib/libc.so.6 #1 0x00007f639381a5c0 in abort () from /lib/libc.so.6 #2 0x000000000072e2c3 in Reporter::InternalError (this=0xc68da0, fmt=0x86f588 "unexpected IP proto in ICMP analyzer") at /root/src/bro-git-20120611/src/Reporter.cc:118 #3 0x0000000000713103 in ICMP_Analyzer::DeliverPacket (this=0x379b550, len=108, data=0x7f6392f7b068
, is_orig=true, seq=-1, ip=0x7fffade206d0, caplen=108) at /root/src/bro-git-20120611/src/ICMP.cc:67 #4 0x000000000062c595 in Analyzer::NextPacket (this=0x379b550, len=108, data=0x7f6392f7b068
, is_orig=true, seq=-1, ip=0x7fffade206d0, caplen=108) at /root/src/bro-git-20120611/src/Analyzer.cc:323 #5 0x00000000006577b1 in Connection::NextPacket (this=0x37a1300, t=1339443931.350776, is_orig=1, ip=0x7fffade206d0, len=108, caplen=108, data=@0x7fffade20508, record_packet=@0x7fffade20548, record_content=@0x7fffade20544, hdr=0x1cbffb0, pkt=0x7f6392f7b046
, hdr_size=14) at /root/src/bro-git-20120611/src/Conn.cc:225 #6 0x00000000007a2255 in NetSessions::DoNextPacket (this=0x1cc0e10, t=1339443931.350776, hdr=0x1cbffb0, ip_hdr=0x7fffade206d0, pkt=0x7f6392f7b046
, hdr_size=14) at /root/src/bro-git-20120611/src/Sessions.cc:663 #7 0x00000000007a0f30 in NetSessions::NextPacket (this=0x1cc0e10, t=1339443931.350776, hdr=0x1cbffb0, pkt=0x7f6392f7b046
, hdr_size=14, pkt_elem=0x0) at /root/src/bro-git-20120611/src/Sessions.cc:279 #8 0x00000000007a0cce in NetSessions::DispatchPacket (this=0x1cc0e10, t=1339443931.350776, hdr=0x1cbffb0, pkt=0x7f6392f7b046
, hdr_size=14, src_ps=0x1cbff70, pkt_elem=0x0) at /root/src/bro-git-20120611/src/Sessions.cc:227 #9 0x0000000000741e45 in net_packet_dispatch (t=1339443931.350776, hdr=0x1cbffb0, pkt=0x7f6392f7b046
, hdr_size=14, src_ps=0x1cbff70, pkt_elem=0x0) at /root/src/bro-git-20120611/src/Net.cc:353 #10 0x0000000000742096 in net_packet_arrival (t=1339443931.350776, hdr=0x1cbffb0, pkt=0x7f6392f7b046
, hdr_size=14, src_ps=0x1cbff70) at /root/src/bro- git-20120611/src/Net.cc:416 #11 0x000000000075647d in PktSrc::Process (this=0x1cbff70) at /root/src/bro-git-20120611/src/PktSrc.cc:273 #12 0x00000000007421ce in net_run () at /root/src/bro-git-20120611/src/Net.cc:446 #13 0x0000000000623e6e in main (argc=18, argv=0x7fffade20ed8) at /root/src/bro-git-20120611/src/main.cc:1065 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 11 13:48:26 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 11 Jun 2012 20:48:26 -0000 Subject: [Bro-Dev] #832: topic/jsiwek/interpreter-exception-fix Message-ID: <048.e93f60cdf7d95813fb6080ab714865ae@tracker.bro-ids.org> #832: topic/jsiwek/interpreter-exception-fix ---------------------------+------------------------ Reporter: jsiwek | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ This changes it so that an interpreter exception in one event handler body still allows other event handler bodies to run. (At least I think this was a bug and not a feature...) -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 11 14:49:08 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 11 Jun 2012 21:49:08 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.d974d6e31b5ba8172a37cab907ae49bc@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by jsiwek): Replying to [comment:4 Tyler.Schoenke]: > I applied your code change, and now I get this error: > > {{{ > #0 0x00007f6393816a75 in raise () from /lib/libc.so.6 > #1 0x00007f639381a5c0 in abort () from /lib/libc.so.6 > #2 0x000000000072e2c3 in Reporter::InternalError (this=0xc68da0, > fmt=0x86f588 "unexpected IP proto in ICMP analyzer") > at /root/src/bro-git-20120611/src/Reporter.cc:118 > }}} Looks like it's probably a separate problem. Do you have a pcap file that can reproduce this? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 11 15:57:16 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 11 Jun 2012 22:57:16 -0000 Subject: [Bro-Dev] #825: topic/dnthayer/bif-tests In-Reply-To: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> References: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> Message-ID: <065.89870dbed58405bca6afab44793b449d@tracker.bro-ids.org> #825: topic/dnthayer/bif-tests -----------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 11 16:51:57 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 11 Jun 2012 23:51:57 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.28b53f209828741bfa5ca5020113f99f@tracker.bro-ids.org> #830: topic/tunnels ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by robin): * owner: seth => robin * status: new => assigned * type: Task => Merge Request Comment: I've started to go through this. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 11 17:03:37 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 12 Jun 2012 00:03:37 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.f6cff96b40adbfca834f33e4f8bbf0fe@tracker.bro-ids.org> #830: topic/tunnels ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): Do we really want to remove encap_hdr_size and tunnel_port? In particular the former can be useful and I wouldn't be surprised if somebody is relying on it and won't be able to use Bro 2.1 otherwise. Or is there a replacement that I'm missing? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 11 17:31:44 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 12 Jun 2012 00:31:44 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.74b52c6fccb485a42a228904e9005303@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by robin): In [d1512ef462364e617751f4958fd9c4ae45fee175/bro]: {{{ #!CommitTicketReference repository="bro" revision="d1512ef462364e617751f4958fd9c4ae45fee175" Merge remote-tracking branch 'origin/topic/jsiwek/enum-log-error-handling' * origin/topic/jsiwek/enum-log-error-handling: Add more error handling code to logging of enum vals. (addresses #829) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 11 17:32:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 12 Jun 2012 00:32:34 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.c4a7bae2e73a4615def84ccd02c55c6f@tracker.bro-ids.org> #830: topic/tunnels ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): I've pushed a checkpoint of the merge to topic/robin/tunnel-merge. It's not done yet. Jon, I've marked a few locations with TODO-Jon. If you want, go ahead and look into them; or wait until I'm done, which has the advantage that at that point I may be understanding things better so that some TODOs may have solved itself. :-) -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Jun 12 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 12 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206120700.q5C702oH011483@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 829 [1] | Tyler.Schoenke | | Normal | terminate called after throwing an instance of 'std::logic_error' Bro | 830 [2] | jsiwek | robin | Normal | topic/tunnels [3] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ btest | cab9c8b | Daniel Thayer | 2012-06-08 | Remove code to expand env. vars. on cmdline [4] btest | 2cbbefe | Daniel Thayer | 2012-06-08 | Add more explanation about expansion of env. vars. [5] btest | 569d62b | Daniel Thayer | 2012-06-07 | Fix typos in README and btest help output [6] [1] #829: http://tracker.bro-ids.org/bro/ticket/829 [2] #830: http://tracker.bro-ids.org/bro/ticket/830 [3] tunnels: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/tunnels [4] fastpath: http://tracker.bro-ids.org/bro/changeset/cab9c8b48daa5216b57a4a14d1284d7cb5dd80e1/btest [5] fastpath: http://tracker.bro-ids.org/bro/changeset/2cbbefed7bfcccd5633f12fdbe4151337577e73c/btest [6] fastpath: http://tracker.bro-ids.org/bro/changeset/569d62b03a02214c3b5c169bd2ddc5282deaca84/btest From bro at tracker.bro-ids.org Tue Jun 12 08:05:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 12 Jun 2012 15:05:48 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.b758b5ff42d98237fc2b195f9def0afb@tracker.bro-ids.org> #830: topic/tunnels ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by jsiwek): Replying to [comment:2 robin]: > Do we really want to remove encap_hdr_size and tunnel_port? In particular the former can be useful and I wouldn't be surprised if somebody is relying on it and won't be able to use Bro 2.1 otherwise. Or is there a replacement that I'm missing? I think those were removed in Gregor's original work on tunnels because his implementation worked as a replacement (at least for the later), but that's no longer true. Adding back `encap_hdr_size` seems easy and might be good. Adding back `tunnel_port` and/or `BifConst::parse_udp_tunnels` needs some work to handle IPv6 and generally I don't like the approach they use to decapsulate packets because the presence of an outer packet is lost (not logged or analyzed). Also I think internally, it currently looks for a script identifier called `udp_tunnel_port`, not `tunnel_port` like it's currently named. So it already seems broken and may give an idea of how many people depend on that feature :). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jun 12 08:30:00 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 12 Jun 2012 15:30:00 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.d5c6b633da2127f65a6ab40c4c3e05d3@tracker.bro-ids.org> #830: topic/tunnels ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): On Tue, Jun 12, 2012 at 15:05 -0000, you wrote: > Adding back `encap_hdr_size` seems easy and might be good. Ok, let's add that back in. I know Seth doesn't like it, but sometimes it comes in handy. :) Agreed that tunnel_port isn't worth keeping. Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jun 12 14:30:12 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 12 Jun 2012 21:30:12 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.3bc33d509a5fcb7abbce8c7b6c251786@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by Tyler.Schoenke): Replying to [comment:5 jsiwek]: > > Looks like it's probably a separate problem. Do you have a pcap file that can reproduce this? I'm a bit mystified about this. If I run via the commandline directly on the interface, bro will crash. If I run a trace from the same time period as the crash, bro won't crash. Any ideas why that is happening? I noticed tcpdump didn't capture every packet received by filter: {{{ 1013576 packets captured 1013599 packets received by filter 0 packets dropped by kernel }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jun 12 14:51:35 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 12 Jun 2012 21:51:35 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.c326f2b25addc7b44746bb99b13ee098@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by Tyler.Schoenke): I tried a real-time debug, and saw that caplen equals len, which is thowing the error in src/ICMP.cc. I'm not sure what to make of it beyond that. {{{ (gdb) print caplen $2 = 40 (gdb) print len $3 = 40 (gdb) print icmp Attempt to use a type name as an expression (gdb) print icmpp $4 = (const icmp *) 0x7ffff58cd068 (gdb) print ip $5 = (const IP_Hdr *) 0x7fffffffe310 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jun 12 15:25:05 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 12 Jun 2012 22:25:05 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.514c4acb2079f40a20b201751ad722dc@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by jsiwek): Replying to [comment:7 Tyler.Schoenke]: > I'm a bit mystified about this. If I run via the commandline directly on the interface, bro will crash. If I run a trace from the same time period as the crash, bro won't crash. Any ideas why that is happening? Was the trace captured with the same `snaplen` setting that Bro was using when it was reading a live interface? Bro would indicate the `snaplen` like: {{{ $ bro -i en0 listening on en0, capture length 8192 bytes }}} And then you could replicate in `tcpdump` with the `-s` option: {{{ $ tcpdump -s 8192 -i en0 -w test.pcap }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jun 12 15:51:54 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 12 Jun 2012 22:51:54 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.58cd638d34cf3b870931048dbb02cb32@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by Tyler.Schoenke): Replying to [comment:9 jsiwek]: > Was the trace captured with the same `snaplen` setting that Bro was using when it was reading a live interface? > I just tried those commands, and get the same result. It crashes immediately when I run bro on the interface, and doesn't crash when run against the pcap. Does tcpdump discard bad ICMP packets? -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Jun 13 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 13 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206130700.q5D7027D004354@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 829 [1] | Tyler.Schoenke | | Normal | terminate called after throwing an instance of 'std::logic_error' Bro | 830 [2] | jsiwek | robin | Normal | topic/tunnels [3] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 1f60c3d | Daniel Thayer | 2012-06-12 | Fix many errors in the event documentation [4] btest | cab9c8b | Daniel Thayer | 2012-06-08 | Remove code to expand env. vars. on cmdline [5] btest | 2cbbefe | Daniel Thayer | 2012-06-08 | Add more explanation about expansion of env. vars. [6] btest | 569d62b | Daniel Thayer | 2012-06-07 | Fix typos in README and btest help output [7] [1] #829: http://tracker.bro-ids.org/bro/ticket/829 [2] #830: http://tracker.bro-ids.org/bro/ticket/830 [3] tunnels: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/tunnels [4] fastpath: http://tracker.bro-ids.org/bro/changeset/1f60c3db074d52dd71558d5fd8378b41d2f75375/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/cab9c8b48daa5216b57a4a14d1284d7cb5dd80e1/btest [6] fastpath: http://tracker.bro-ids.org/bro/changeset/2cbbefed7bfcccd5633f12fdbe4151337577e73c/btest [7] fastpath: http://tracker.bro-ids.org/bro/changeset/569d62b03a02214c3b5c169bd2ddc5282deaca84/btest From bro at tracker.bro-ids.org Wed Jun 13 08:33:41 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 13 Jun 2012 15:33:41 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.6310a9a927ce4884462c2a004a74864b@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by jsiwek): > I just tried those commands, and get the same result. It crashes immediately when I run bro on the interface, and doesn't crash when run against the pcap. Does tcpdump discard bad ICMP packets? I don't think so. Just curious, what was Bro's `snaplen` setting? (and to clarify, I meant that if it differed from the default 8192, to use that value with tcpdump). But maybe better is to just have Bro itself be dumping the packets as it sees them, e.g.: {{{ bro -w test.pcap -i eth0 record_all_packets=T }}} And then see if rerunning on that test.pcap after it crashes can reproduce it. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jun 13 09:02:35 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 13 Jun 2012 16:02:35 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.2ca97c50e7892b2273113168c1b2b824@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by Tyler.Schoenke): Replying to [comment:11 jsiwek]: > I don't think so. Just curious, what was Bro's `snaplen` setting? (and to clarify, I meant that if it differed from the default 8192, to use that value with tcpdump). Bro's snaplen was 8192. I guess it didn't matter. > But maybe better is to just have Bro itself be dumping the packets as it sees them, e.g.: > > {{{ > bro -w test.pcap -i eth0 record_all_packets=T > }}} > > And then see if rerunning on that test.pcap after it crashes can reproduce it. How is this for weird? {{{ root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339603109.086308 internal error in , line 1: unexpected IP proto in ICMP analyzer Aborted (core dumped) root at browrk3:~/test# ls -l total 179704 -rw-r--r-- 1 root root 3780 2012-06-13 09:58 conn.log -rw------- 1 root root 156737536 2012-06-13 09:58 core -rw-r--r-- 1 root root 0 2012-06-13 09:58 debug.log -rw-r--r-- 1 root root 89776 2012-06-13 09:58 dns.log -rw-r--r-- 1 root root 2742 2012-06-13 09:58 dpd.log -rw-r--r-- 1 root root 96979 2012-06-13 09:58 http.log -rw-r--r-- 1 root root 1176 2012-06-13 09:58 notice.log -rw-r--r-- 1 root root 1075 2012-06-13 09:58 notice_policy.log -rw-r--r-- 1 root root 198 2012-06-13 09:58 packet_filter.log -rw-r--r-- 1 root root 499 2012-06-13 09:58 smtp_entities.log -rw-r--r-- 1 root root 1033 2012-06-13 09:58 smtp.log -rw-r--r-- 1 root root 24981 2012-06-13 09:58 ssl.log -rw-r--r-- 1 root root 460 2012-06-13 09:58 syslog.log -rw-r--r-- 1 root root 128022520 2012-06-13 09:58 test.pcap -rw-r--r-- 1 root root 81166 2012-06-13 09:58 weird.log root at browrk3:~/test# bro -r test.pcap root at browrk3:~/test# echo $? 0 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Jun 13 15:27:07 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 13 Jun 2012 22:27:07 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.89fdd3ba920d0a4b7cfeb01e6523fdd8@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by jsiwek): If you `git checkout fastpath && git pull` and rebuild, I included in the error message the value of `ip->NextProto()` that it doesn't like, if you could tell me what that is and whether it's consistent across crashes, that might give me a hint. Also I'm curious about what kinds of weirds are in weird.log, `awk 'NR > 7' < weird.log | cut -f7 | sort | uniq` should give all unique ones, but mostly I'm wondering about any starting with "unknown_protocol_". -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Jun 14 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 14 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206140700.q5E702Vb017046@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 829 [1] | Tyler.Schoenke | | Normal | terminate called after throwing an instance of 'std::logic_error' Bro | 830 [2] | jsiwek | robin | Normal | topic/tunnels [3] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 22fb039 | Jon Siwek | 2012-06-13 | Improve an error message in ICMP analyzer. [4] bro | b66b022 | Daniel Thayer | 2012-06-13 | Fix a warning message [5] bro | 1f60c3d | Daniel Thayer | 2012-06-12 | Fix many errors in the event documentation [6] broccoli | 6834df8 | Daniel Thayer | 2012-06-13 | Fix a warning, and fix other typos in documentation [7] btest | cab9c8b | Daniel Thayer | 2012-06-08 | Remove code to expand env. vars. on cmdline [8] btest | 2cbbefe | Daniel Thayer | 2012-06-08 | Add more explanation about expansion of env. vars. [9] btest | 569d62b | Daniel Thayer | 2012-06-07 | Fix typos in README and btest help output [10] [1] #829: http://tracker.bro-ids.org/bro/ticket/829 [2] #830: http://tracker.bro-ids.org/bro/ticket/830 [3] tunnels: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/tunnels [4] fastpath: http://tracker.bro-ids.org/bro/changeset/22fb039e8384448b94091407ef08c2403f75cfb5/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/b66b022be0090b1327042953b49b3cbbdc5054f3/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/1f60c3db074d52dd71558d5fd8378b41d2f75375/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/6834df8d750c99900052925b09572b01f38fc89e/broccoli [8] fastpath: http://tracker.bro-ids.org/bro/changeset/cab9c8b48daa5216b57a4a14d1284d7cb5dd80e1/btest [9] fastpath: http://tracker.bro-ids.org/bro/changeset/2cbbefed7bfcccd5633f12fdbe4151337577e73c/btest [1] fastpath: http://tracker.bro-ids.org/bro/changeset/569d62b03a02214c3b5c169bd2ddc5282deaca84/btest From bro at tracker.bro-ids.org Thu Jun 14 15:10:39 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 14 Jun 2012 22:10:39 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.3837cbd8d2e1fa93c17ab8c213c908c6@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by Tyler.Schoenke): Replying to [comment:13 jsiwek]: > If you `git checkout fastpath && git pull` and rebuild, I included in the error message the value of `ip->NextProto()` that it doesn't like, if you could tell me what that is and whether it's consistent across crashes, that might give me a hint. > > Also I'm curious about what kinds of weirds are in weird.log, `awk 'NR > 7' < weird.log | cut -f7 | sort | uniq` should give all unique ones, but mostly I'm wondering about any starting with "unknown_protocol_". Here are the unknown protocols: {{{ unknown_protocol_103 unknown_protocol_47 unknown_protocol_50 unknown_protocol_97 }}} Below is the output of multiple back-to-back crashes. I also noticed it is crashing with two different error messages. {{{ root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710831.110451 internal error in , line 1: unexpected IP proto in ICMP analyzer: 17 Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710836.652673 internal error in , line 1: unexpected IP proto in ICMP analyzer: 6 Aborted (core dumped) root at browrk3:~/test# root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710843.062141 internal error in , line 1: unexpected next protocol in ICMP::DeliverPacket() Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710850.854367 internal error in , line 1: unexpected next protocol in ICMP::DeliverPacket() Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710855.403844 internal error in , line 1: unexpected next protocol in ICMP::DeliverPacket() Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710859.580805 internal error in , line 1: unexpected IP proto in ICMP analyzer: 6 Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710865.303795 internal error in , line 1: unexpected next protocol in ICMP::DeliverPacket() Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710867.725665 internal error in , line 1: unexpected IP proto in ICMP analyzer: 6 Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710872.459743 internal error in , line 1: unexpected IP proto in ICMP analyzer: 6 Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710875.625401 internal error in , line 1: unexpected IP proto in ICMP analyzer: 6 Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710878.200259 internal error in , line 1: unexpected IP proto in ICMP analyzer: 17 Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710882.961858 internal error in , line 1: unexpected IP proto in ICMP analyzer: 6 Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710885.948716 internal error in , line 1: unexpected next protocol in ICMP::DeliverPacket() Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710888.620009 internal error in , line 1: unexpected IP proto in ICMP analyzer: 6 Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710890.593817 internal error in , line 1: unexpected IP proto in ICMP analyzer: 6 Aborted (core dumped) root at browrk3:~/test# bro -w test.pcap -i eth1 record_all_packets=T , line 1: listening on eth1, capture length 8192 bytes 1339710894.223593 internal error in , line 1: unexpected IP proto in ICMP analyzer: 17 Aborted (core dumped) }}} From the debugger, I seem to be getting some garbage data in the packets. Neither IP addresses is in our address space. in /root/src/bro-git-20120611/src/ICMP.cc (gdb) print ip->ip4->ip_src $1 = {s_addr = 731220608} (43.149.138.128) (gdb) print ip->ip4->ip_dst $2 = {s_addr = 1882264748} (112.49.20.172) (gdb) print icmpp->icmp_type $3 = 1 '\001' (gdb) print icmpp->icmp_code $4 = 189 '\275' On a different run, I had this for ICMP type and code, which look out of range. (gdb) print icmpp->icmp_type $5 = 228 '\344' (gdb) print icmpp->icmp_code $6 = 115 's' (gdb) print ip->ip4->ip_src $18 = {s_addr = 1261144704} (75.43.138.128) (gdb) print ip->ip4->ip_dst $19 = {s_addr = 1828190024} (108.247.247.72) -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Jun 15 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 15 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206150700.q5F702pW024495@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 829 [1] | Tyler.Schoenke | | Normal | terminate called after throwing an instance of 'std::logic_error' Bro | 830 [2] | jsiwek | robin | Normal | topic/tunnels [3] > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 22fb039 | Jon Siwek | 2012-06-13 | Improve an error message in ICMP analyzer. [4] bro | b66b022 | Daniel Thayer | 2012-06-13 | Fix a warning message [5] bro | 1f60c3d | Daniel Thayer | 2012-06-12 | Fix many errors in the event documentation [6] broccoli | 6834df8 | Daniel Thayer | 2012-06-13 | Fix a warning, and fix other typos in documentation [7] btest | cab9c8b | Daniel Thayer | 2012-06-08 | Remove code to expand env. vars. on cmdline [8] btest | 2cbbefe | Daniel Thayer | 2012-06-08 | Add more explanation about expansion of env. vars. [9] btest | 569d62b | Daniel Thayer | 2012-06-07 | Fix typos in README and btest help output [10] [1] #829: http://tracker.bro-ids.org/bro/ticket/829 [2] #830: http://tracker.bro-ids.org/bro/ticket/830 [3] tunnels: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/tunnels [4] fastpath: http://tracker.bro-ids.org/bro/changeset/22fb039e8384448b94091407ef08c2403f75cfb5/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/b66b022be0090b1327042953b49b3cbbdc5054f3/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/1f60c3db074d52dd71558d5fd8378b41d2f75375/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/6834df8d750c99900052925b09572b01f38fc89e/broccoli [8] fastpath: http://tracker.bro-ids.org/bro/changeset/cab9c8b48daa5216b57a4a14d1284d7cb5dd80e1/btest [9] fastpath: http://tracker.bro-ids.org/bro/changeset/2cbbefed7bfcccd5633f12fdbe4151337577e73c/btest [1] fastpath: http://tracker.bro-ids.org/bro/changeset/569d62b03a02214c3b5c169bd2ddc5282deaca84/btest From bro at tracker.bro-ids.org Fri Jun 15 10:51:36 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Jun 2012 17:51:36 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.5bb0b6e5925d4d225cc6ae60e294fb7d@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by grigorescu): Replying to [comment:4 Tyler.Schoenke]: > {{{ > #2 0x000000000072e2c3 in Reporter::InternalError (this=0xc68da0, > fmt=0x86f588 "unexpected IP proto in ICMP analyzer") > at /root/src/bro-git-20120611/src/Reporter.cc:118 > }}} I removed the libpcap that came with Ubuntu 10.04,installed libpcap-1.2.1.tar.gz from http://www.tcpdump.org/#latest-release , and the problem went away for me. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 15 11:04:23 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Jun 2012 18:04:23 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.47958f54c952be5964c6539fdb9e3f45@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by jsiwek): Replying to [comment:14 Tyler.Schoenke]: > From the debugger, I seem to be getting some garbage data in the packets. Neither IP addresses is in our address space. They're stored in network-byte order. If you reverse the octets in the dotted-quads you gave, I think it looks reasonable. > On a different run, I had this for ICMP type and code, which look out of range. Yeah, that's probably because the packet isn't actually ICMP/ICMPv6, but rather TCP or UDP packets are getting in there somehow (protocol numbers 6 and 17). Replying to [comment:15 grigorescu]: > I removed the libpcap that came with Ubuntu 10.04,installed libpcap-1.2.1.tar.gz from http://www.tcpdump.org/#latest-release , and the problem went away for me. Thanks, I'll try to reproduce it on that OS and see if there's something more that needs to be done, but if that version of libpcap, when reading live, was replacing the packet buffer out from under us in between the time the packet is first seen and when it gets delivered to the ICMP analyzer, that could explain what's going on. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 15 13:43:46 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Jun 2012 20:43:46 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.3d76ba172a5cfdaede235a99632ab6a3@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by Tyler.Schoenke): Replying to [comment:15 grigorescu]: > I removed the libpcap that came with Ubuntu 10.04,installed libpcap-1.2.1.tar.gz from http://www.tcpdump.org/#latest-release , and the problem went away for me. Thanks Vlad! That worked for me as well. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 15 14:36:07 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Jun 2012 21:36:07 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.3cf52eaef18051429b0ab44795320f4b@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: -----------------------------+------------------------ Comment (by Tyler.Schoenke): Replying to [comment:16 jsiwek]: > Replying to [comment:14 Tyler.Schoenke]: > They're stored in network-byte order. If you reverse the octets in the dotted-quads you gave, I think it looks reasonable. That makes sense. I typically don't dig into the code this much, so I suspected that I was wrong with my analysis. > > On a different run, I had this for ICMP type and code, which look out of range. > > Yeah, that's probably because the packet isn't actually ICMP/ICMPv6, but rather TCP or UDP packets are getting in there somehow (protocol numbers 6 and 17). I thought it was a bit strange tcp and upd protocols were showing up in the icmp analyzer, but didn't understand enough about the code to know if that was normal or not. > Replying to [comment:15 grigorescu]: > Thanks, I'll try to reproduce it on that OS and see if there's something more that needs to be done, but if that version of libpcap, when reading live, was replacing the packet buffer out from under us in between the time the packet is first seen and when it gets delivered to the ICMP analyzer, that could explain what's going on. I think I was using libpcap 0.8. As I mentioned, the upgrade to 1.2.1 got rid of the error and crashing. Thanks, Tyler -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 15 15:23:06 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Jun 2012 22:23:06 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.45bfedc3723182d635f76a30243714e4@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied Comment: So this seems solved and the code is already merged, so closing ticket. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 15 16:05:01 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Jun 2012 23:05:01 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.ef3b899d3d119a0d17db53687c6cc2b3@tracker.bro-ids.org> #830: topic/tunnels ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): I'm adding a SOCKS test base, however I'm not sure the output is right, can somebody look at conn.log and output? (will come with an upcoming push) Also, I'm removing the socks.bro code that's commented out for now. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 15 16:16:35 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Jun 2012 23:16:35 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.f251b823925c8d2b2344c0b6ade6b655@tracker.bro-ids.org> #830: topic/tunnels ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Comment (by robin): I was about to merge this into master and hence are moving the todos from the code to here (these are mostly cosmetically tasks that could be done after the merge as well). However, I did *not* merge because of a large problem, see next commit I'll add to the ticket. Current code in topic/robin/tunnels-merge. - The process of creating an EncapsulatingConn and feeding packet would benefit from a bit more refactoring. Code like this appears a few times (whereever a tunnel is found): {{{ Encapsulation* outer = new Encapsulation(e); EncapsulatingConn ec(c, BifEnum::Tunnel::AYIYA); outer->Add(ec); sessions->DoNextInnerPacket(network_time(), 0, inner, outer); delete inner; delete outer; }}} Can we factor that out somehow? Perhaps move it all into the DoNextInnerPacketMethod() and rename that one? - event.bif: tunnel_changed(): I initially didn't understand the last sentence about the tunnel field. I now get it after reading the code. Not sure how to improve. - Sessions.cc: This is a bigger one.The ip_tunnels map needs to do state management to automatically discard old entries, like those not used for X hours. Can be done with a timer per index. - NetSessions::DoNextInnerPacket: Would it make sense use network_time in the non-hdr case? - TunnelEncapsulation: I find the class name Encapsulation misleading as it's really set chain of encapsulations. Rename to EncapsulationChain or EncapsulationStack? - Encapsulation::Encapsulation(const Encapsulation* other) I don't like the ptr constructor. When reading code using that, I can't tell what it does with the pointer (i.e., that it actually deep-copies the object). Can we use just the reference version? (I realize that means more "if ( not null )" at the caller end). - tunnel/main.bro: I suggest to make the expiration interval configurable. - tunnel/main.bro: Does the active tabel really need to be &synchronized? - tunnel/main.bro: tunnel_changed() event: there's something here I don't understand. Shouldn't c$tunnel already be registered? And what if a layer goes away, does that need to be removed here? Or is that done separately? Also, conn/main.bro has a tunnel_changed handler at the same priority that *sets* c$tunnel. That's seems undefined behaviour. - conn.log: "parents" lacks context, it's hard to say what it means if you doesn't already have tunnels in mind. Rename to "tunnel_parents" or "tunneled_in" or just "tunnels"? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 15 16:22:30 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Jun 2012 23:22:30 -0000 Subject: [Bro-Dev] #823: Remaining input framework todos In-Reply-To: <047.5e447178368a3a40b99fd343d1c743c6@tracker.bro-ids.org> References: <047.5e447178368a3a40b99fd343d1c743c6@tracker.bro-ids.org> Message-ID: <062.736291484e04af150008e22c4fa15b02@tracker.bro-ids.org> #823: Remaining input framework todos ----------------------+------------------------ Reporter: robin | Owner: amannb Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by robin): * owner: bernhard => amannb * status: new => assigned Comment: Are these all done now? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 15 16:28:16 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Jun 2012 23:28:16 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.c5d1a057e766c90b3d398c916ab21ee4@tracker.bro-ids.org> #830: topic/tunnels ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by robin): * owner: robin => jsiwek * type: Merge Request => Task Comment: Looks like we have a major problem in this branch: execution time roughly doubles. I tried two traces and both showed that effect. Here is one: With robin/topic/tunnels-merge: {{{ > zcat ../testing/external/bro- testing/Traces/2009-M57-day11-18.trace.gz | time bro -r - 29.68user 3.02system 0:30.56elapsed 107%CPU (0avgtext+0avgdata 32272maxresident)k }}} With current master: {{{ > zcat ../testing/external/bro- testing/Traces/2009-M57-day11-18.trace.gz | time bro -r - 14.64user 2.75system 0:18.66elapsed 93%CPU (0avgtext+0avgdata 31920maxresident)k }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 15 16:36:11 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Jun 2012 23:36:11 -0000 Subject: [Bro-Dev] #823: Remaining input framework todos In-Reply-To: <047.5e447178368a3a40b99fd343d1c743c6@tracker.bro-ids.org> References: <047.5e447178368a3a40b99fd343d1c743c6@tracker.bro-ids.org> Message-ID: <062.d933d2422ebcf561b3a8ea940eba5bf5@tracker.bro-ids.org> #823: Remaining input framework todos ----------------------+------------------------ Reporter: robin | Owner: amannb Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by amannb): Everything but the memory leaks is done. I am still waiting on feedback on those, but I hope that they are also fixed. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Jun 16 00:00:05 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 16 Jun 2012 00:00:05 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206160700.q5G705Xk024481@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ broccoli | 6834df8 | Daniel Thayer | 2012-06-13 | Fix a warning, and fix other typos in documentation [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/6834df8d750c99900052925b09572b01f38fc89e/broccoli From bro at tracker.bro-ids.org Sat Jun 16 16:25:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 16 Jun 2012 23:25:19 -0000 Subject: [Bro-Dev] #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events Message-ID: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events ------------------------+-------------------- Reporter: aashish | Type: Patch Status: new | Priority: Normal Milestone: Bro2.1 | Component: Bro Version: git/master | Keywords: ------------------------+-------------------- Patch to provide access to the ICMPv6 payload in the scripting layer for both neighbor advertisements and neighbor solicitation messages. Payload is needed for extracting mac-addresses which is useful for: 1) to get bindings of mac with v6 IP addresses. This is needed to be able to dhcp jail IPv6 hosts. 2) Alert on fake router advertisements 3) Build Neighbor Caches and flag of spoofing etc. I believe instead of payload we can send just the mac-address to the scripting layer (extract it in ICMP.cc - instead of in the script events). I am open to that thought. Not sure if there is anything else, apart from the mac-address that we eventually might need from the payload in these events ( icmp_neighbor_solicitation and event icmp_neighbor_advertisement) Also, attaching a skeleton policy file to log IPv6 and mac-addr bindings. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sun Jun 17 00:00:02 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 17 Jun 2012 00:00:02 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206170700.q5H7028J019469@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ broccoli | 6834df8 | Daniel Thayer | 2012-06-13 | Fix a warning, and fix other typos in documentation [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/6834df8d750c99900052925b09572b01f38fc89e/broccoli From bro at tracker.bro-ids.org Sun Jun 17 18:53:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 01:53:48 -0000 Subject: [Bro-Dev] #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events In-Reply-To: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> References: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> Message-ID: <064.5ef1e465b7e5f5f0cc5ac3a8e6d0f2c9@tracker.bro-ids.org> #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events ----------------------+------------------------ Reporter: aashish | Owner: Type: Patch | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by robin): * priority: Normal => High Comment: This would be good to add in. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Mon Jun 18 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 18 Jun 2012 00:00:04 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206180700.q5I704Nl008134@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ broccoli | 6834df8 | Daniel Thayer | 2012-06-13 | Fix a warning, and fix other typos in documentation [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/6834df8d750c99900052925b09572b01f38fc89e/broccoli From bro at tracker.bro-ids.org Mon Jun 18 07:53:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 14:53:19 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.4f481cc6c8b7df2e2a2fdd7ba9e20f4d@tracker.bro-ids.org> #830: topic/tunnels ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by jsiwek): Replying to [comment:10 robin]: > Looks like we have a major problem in this branch: execution time roughly doubles. I see similar execution times between master and the original topic/tunnels on 64-bit Linux. What OS are you on? > With robin/topic/tunnels-merge: I don't see this branch (or topic/robin/tunnels-merge), did it get pushed? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 07:59:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 14:59:10 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.6f04c276c8eb9acfe786798d8d3b66d2@tracker.bro-ids.org> #830: topic/tunnels ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by robin): On Mon, Jun 18, 2012 at 14:53 -0000, you wrote: > I see similar execution times between master and the original > topic/tunnels on 64-bit Linux. What OS are you on? Same (a FC15). You're saying you don't see a difference between master and topic/tunnels? Then I may have broken something. Let me know how topic/robin/tunnels-merge looks for you. > I don't see this branch (or topic/robin/tunnels-merge), did it get pushed? Oops, it didn't. Pushed now (t/r/s of course) Robin -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 08:03:29 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 15:03:29 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.42b5e2575cc87de546dfcb24d473fbfb@tracker.bro-ids.org> #830: topic/tunnels ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by jsiwek): > Same (a FC15). You're saying you don't see a difference between master > and topic/tunnels? Right. > Then I may have broken something. Let me know how > topic/robin/tunnels-merge looks for you. Yeah, topic/robin/tunnels-merge is doing poorer than topic/tunnels and master for me, too. I can check it out as I look into your suggestions, thanks. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 08:08:45 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 15:08:45 -0000 Subject: [Bro-Dev] #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events In-Reply-To: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> References: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> Message-ID: <064.187c9d65ea97c1e373bb338d144f2113@tracker.bro-ids.org> #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events ----------------------+------------------------ Reporter: aashish | Owner: Type: Patch | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): Let's try to add in least the *.cc part for the beta. However, rather than passing the whole packet payload to script-land, we should parse out the fields (like MAC address) and just pass just them in the events. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 08:09:03 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 15:09:03 -0000 Subject: [Bro-Dev] #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events In-Reply-To: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> References: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> Message-ID: <064.eb9e00cb15295bbcf95dc5b6c6db25cb@tracker.bro-ids.org> #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events ----------------------+------------------------ Reporter: aashish | Owner: Type: Patch | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by robin): Seth, can you take a look at the Bro scripts? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 08:49:21 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 15:49:21 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.12811bfa19706f6f027f669513a9f367@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Comment (by slagell): I think this is very odd as the Security Onion uses the Ubuntu 1004 LTS and I have not heard of problems there. In fact I am running it now with no problems. See the versions below: dpkg -l *pcap* Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig- pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Description +++-=======================-=======================-============================================================== ii libnet-pcap-perl 0.16-2 Perl binding to the LBL pcap packet capture library ii libpcap-dev 1.0.0-6 development library for libpcap (transitional package) un libpcap0.7-dev (no description available) ii libpcap0.8 1.0.0-6 system interface for user-level packet capture ii libpcap0.8-dev 1.0.0-6 development library and header files for libpcap0.8 ii pcapcat 0.21 no description given ii securityonion-pcap-agen 20120224 no description given Maybe 32bit PAE version is different and the error is only for 64-bit ubuntu? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 09:16:17 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 16:16:17 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.1cbbe5ac16dbfb09ba78b2d191f7d307@tracker.bro-ids.org> #830: topic/tunnels ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by jsiwek): In [146cb47d6ae76c1478569bca3dfb60cc44fcb700/bro]: {{{ #!CommitTicketReference repository="bro" revision="146cb47d6ae76c1478569bca3dfb60cc44fcb700" Fix performance problem checking packet encapsulation. (addresses #830) Connections were creating a new encapsulation object for nearly every packet even if no tunnels were ever involved with the Connection. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 09:25:48 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 16:25:48 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.6ee211c9a2b2d8e39531bb79a8cff051@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Comment (by seth): > I think this is very odd as the Security Onion uses the Ubuntu 1004 LTS > and I have not heard of problems there. In fact I am running it now with > no problems. See the versions below: Security Onion is 32-bit only. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 09:56:50 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 16:56:50 -0000 Subject: [Bro-Dev] #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events In-Reply-To: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> References: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> Message-ID: <064.ff407cb721cf930dd2275707406ca748@tracker.bro-ids.org> #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events ----------------------+------------------------ Reporter: aashish | Owner: Type: Patch | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by seth): > Seth, can you take a look at the Bro scripts? Functionally these scripts need restructured quite a bit, but probably won't take much time. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 10:37:23 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 17:37:23 -0000 Subject: [Bro-Dev] #829: terminate called after throwing an instance of 'std::logic_error' In-Reply-To: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> References: <056.197838d5842b96219f9edd16cfb35b6a@tracker.bro-ids.org> Message-ID: <071.59ff1ea9a71e36863cd04891a9936282@tracker.bro-ids.org> #829: terminate called after throwing an instance of 'std::logic_error' -----------------------------+------------------------ Reporter: Tyler.Schoenke | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Comment (by slagell): Right. But I saw nothing in this bug report saying it was for 64 bit only. Hence my speculation and question about their platform. I can see the original poster had 64-bit addresses in the error logs, but it would be reassuring to know that all the people that had this issue were on 64 bit platforms. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 11:58:43 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 18:58:43 -0000 Subject: [Bro-Dev] #831: Memory leak in print In-Reply-To: <048.ea67e86bfb2a6633bbc5ada34ca295e5@tracker.bro-ids.org> References: <048.ea67e86bfb2a6633bbc5ada34ca295e5@tracker.bro-ids.org> Message-ID: <063.5fd081d580529ab85d56d5d5dae35c3d@tracker.bro-ids.org> #831: Memory leak in print ----------------------+------------------------ Reporter: amannb | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Changes (by robin): * priority: High => Normal * milestone: Bro2.1 => Comment: Hmmm ? That indeed sounds like a bigger problem. The exception is a relatively recent addition, before that Bro would just abort with an internal error for many runtime errors. So not much we can do right now. Regarding smart pointers, yes, in principle, but that's a *big* task, with potential to introduce very subtle problems. So while it's on the roadmap, I'm actually reluctant to tackle it soon. Also, perhaps this can be taken core of by eventually compiling Bro scripts, which will remove lots of the relevant code anyway long-term. But that's all something for a longer discussion. I'm leaving this ticket open, but I'm removing the milestone. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 12:22:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 19:22:51 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.1bb46fdb55eff9aa4ed5dd06f3f11ba8@tracker.bro-ids.org> #830: topic/tunnels ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by robin): That indeed fixes it. Thanks! I'll wait for the other changes, and Seth going over the tunnel logs, before doing the final merge. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 15:34:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 22:34:19 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.fe0edd75c1d8b2ad1caaf7af314bf32b@tracker.bro-ids.org> #830: topic/tunnels ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by jsiwek): I addressed all your suggestions in [comment:9 comment 9] in `topic/jsiwek /tunnels-merge` that you can take a look at. Probably the conn.log baselines in `bro-testing` and `bro-testing-private` need updating again since I changed the "parents" field to be named "tunnel_parents". This one I thought might need more explanation than just looking at the code diffs: > - tunnel/main.bro: tunnel_changed() event: there's something here I don't > understand. Shouldn't c$tunnel already be registered? Yes, c$tunnel should have already been registered, so that was redundant to have in the tunnel/main.bro handler. > what if a layer goes away, does that need to be removed > here? Or is that done separately? The handler in conn/main.bro keeps track of the current encapsulation stack by setting c$tunnel. Doing that works for all cases where tunnel_changed happens: a layer gets added, a layer gets removed, or a layer changed. > Also, conn/main.bro has a > tunnel_changed handler at the same priority that *sets* > c$tunnel. That's seems undefined behaviour. I think that concern goes away now that the handler in tunnel/main.bro doesn't inspect c$tunnel at all anymore. Let me know if not. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 15:59:10 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 22:59:10 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.96a49e2de46aac04a17d00dbc160e141@tracker.bro-ids.org> #830: topic/tunnels ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by robin): In [2f6a76c2ae4c737c43fd0373fcfe519ff45098ed/bro]: {{{ #!CommitTicketReference repository="bro" revision="2f6a76c2ae4c737c43fd0373fcfe519ff45098ed" Merge remote-tracking branch 'origin/topic/jsiwek/tunnels-merge' into topic/robin/tunnels-merge * origin/topic/jsiwek/tunnels-merge: Fix performance problem checking packet encapsulation. (addresses #830) }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Jun 18 16:22:32 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 18 Jun 2012 23:22:32 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.df8bb3a4b35e8c282f3825bf5dd89fa4@tracker.bro-ids.org> #830: topic/tunnels ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by robin): Looks good, merged into topic/robin/tunnels-merge. Remaining things: - Seth to look at scripts/logs. - Check the SOCKS output. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Jun 19 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 19 Jun 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206190700.q5J703TK017469@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ broccoli | 6834df8 | Daniel Thayer | 2012-06-13 | Fix a warning, and fix other typos in documentation [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/6834df8d750c99900052925b09572b01f38fc89e/broccoli From robin at icir.org Tue Jun 19 15:22:53 2012 From: robin at icir.org (Robin Sommer) Date: Tue, 19 Jun 2012 15:22:53 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/rotate-info: Log writer API change: providing writers with rotation interval and base time when rotating a log. (bc33cb5) In-Reply-To: <201206192220.q5JMKrlZ012091@bro-ids.icir.org> References: <201206192220.q5JMKrlZ012091@bro-ids.icir.org> Message-ID: <20120619222253.GK38637@icir.org> On Tue, Jun 19, 2012 at 15:20 -0700, I wrote: > On branch : topic/robin/rotate-info Seth, does this do what you were looking for? it passes the tests, but I haven't checked that the new values being passed are actually right ... Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From seth at icir.org Tue Jun 19 17:44:19 2012 From: seth at icir.org (Seth Hall) Date: Tue, 19 Jun 2012 20:44:19 -0400 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/rotate-info: Log writer API change: providing writers with rotation interval and base time when rotating a log. (bc33cb5) In-Reply-To: <20120619222253.GK38637@icir.org> References: <201206192220.q5JMKrlZ012091@bro-ids.icir.org> <20120619222253.GK38637@icir.org> Message-ID: <796B156B-DAF9-406B-806D-9B83235D2BF8@icir.org> On Jun 19, 2012, at 6:22 PM, Robin Sommer wrote: > Seth, does this do what you were looking for? I think so, thanks. > it passes the tests, but I haven't checked that the new values being > passed are actually right ... I'll keep an eye out for inconsistencies. Thanks! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From noreply at bro-ids.org Wed Jun 20 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 20 Jun 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206200700.q5K703Wx025526@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ broccoli | 6834df8 | Daniel Thayer | 2012-06-13 | Fix a warning, and fix other typos in documentation [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/6834df8d750c99900052925b09572b01f38fc89e/broccoli From seth at icir.org Wed Jun 20 11:21:45 2012 From: seth at icir.org (Seth Hall) Date: Wed, 20 Jun 2012 14:21:45 -0400 Subject: [Bro-Dev] Tunnels updates done Message-ID: Robin, I'm done with my tunnel and SOCKS updates. My topic/seth/tunnels-merge is ready to merged into your tunnels-merge branch. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Wed Jun 20 11:45:29 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 20 Jun 2012 18:45:29 -0000 Subject: [Bro-Dev] #834: Compile warning on RHEL 6.2 Message-ID: <046.fd396e56746c3541b2691ce86ee8cf2b@tracker.bro-ids.org> #834: Compile warning on RHEL 6.2 ---------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ This would be good to get fixed before the beta. {{{ src/IPAddr.h: In member function ?bool SocketComm::AcceptConnection(int)?: src/IPAddr.h:61: warning: dereferencing pointer ?client.1094? does break strict-aliasing rules src/RemoteSerializer.cc:4232: note: initialized from here src/RemoteSerializer.cc:4235: warning: dereferencing pointer ?client.1093? does break strict-aliasing rules src/RemoteSerializer.cc:4235: note: initialized from here src/RemoteSerializer.cc:4236: warning: dereferencing pointer ?client.1094? does break strict-aliasing rules src/RemoteSerializer.cc:4236: note: initialized from here }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Jun 21 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 21 Jun 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206210700.q5L703fE031044@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ broccoli | 6834df8 | Daniel Thayer | 2012-06-13 | Fix a warning, and fix other typos in documentation [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/6834df8d750c99900052925b09572b01f38fc89e/broccoli From bro at tracker.bro-ids.org Thu Jun 21 07:57:56 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 21 Jun 2012 14:57:56 -0000 Subject: [Bro-Dev] #832: topic/jsiwek/interpreter-exception-fix In-Reply-To: <048.e93f60cdf7d95813fb6080ab714865ae@tracker.bro-ids.org> References: <048.e93f60cdf7d95813fb6080ab714865ae@tracker.bro-ids.org> Message-ID: <063.0a5095ef3e3281de7864a70a96ba4873@tracker.bro-ids.org> #832: topic/jsiwek/interpreter-exception-fix ----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: assigned Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by seth): * owner: => robin * priority: Normal => High * status: new => assigned * milestone: => Bro2.1 Comment: Hm, this needs merged into master but it's not going out in the nightly merge status reports. Someone reported a problem today that's actually being caused by this bug. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jun 21 10:33:05 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 21 Jun 2012 17:33:05 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.ef88d1fdcbddb7b4d86560bcbc71e053@tracker.bro-ids.org> #830: topic/tunnels ---------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Comment (by robin): In [b096168318775427f97f88fa2453e2a470f2d17f/bro]: {{{ #!CommitTicketReference repository="bro" revision="b096168318775427f97f88fa2453e2a470f2d17f" Merge branch 'topic/robin/tunnels-merge' * topic/robin/tunnels-merge: (51 commits) Updating baselines and NEWS. Remove &synchronized from Tunnel::active table. Refactor of interal tunnel analysis code. Add state management of NetSessions's IP tunnel map. Add "encap_hdr_size" option back in. Script-layer tunnel interface cleanup. Fix performance problem checking packet encapsulation. (addresses #830) Adding a SOCKS test case. Updating DataSeries baselines. Moving my todos over to the tracker ticket. Extend weird names that occur in core packet processing during decapsulation. Add Teredo analysis option to reduce false positive decapsulation. Just some cleanup/documentation of new tunnel-handling code. Memory leak fixes Add a config.h definition for IPPROTO_IPV4. Add AYIYA tunnel decapsulation unit test. Add Teredo-specific events. Refactor some of the NetSessions routines that recurse on IP packets. Add independent options to toggle the different decapsulation methods Add more sanity checks before recursing on encapsulated IP packets. ... Conflicts: src/event.bif }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Jun 22 00:00:04 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 22 Jun 2012 00:00:04 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206220700.q5M704Ri004997@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 832 [1] | jsiwek | robin | High | topic/jsiwek/interpreter-exception-fix [2] [1] #832: http://tracker.bro-ids.org/bro/ticket/832 [2] interpreter-exception-fix: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/jsiwek/interpreter-exception-fix From bro at tracker.bro-ids.org Fri Jun 22 07:52:19 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 22 Jun 2012 14:52:19 -0000 Subject: [Bro-Dev] #832: topic/jsiwek/interpreter-exception-fix In-Reply-To: <048.e93f60cdf7d95813fb6080ab714865ae@tracker.bro-ids.org> References: <048.e93f60cdf7d95813fb6080ab714865ae@tracker.bro-ids.org> Message-ID: <063.1447c766822f9692321a85e4dcecd4ef@tracker.bro-ids.org> #832: topic/jsiwek/interpreter-exception-fix -----------------------------+------------------------ Reporter: jsiwek | Owner: robin Type: Merge Request | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: assigned => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From robin at icir.org Fri Jun 22 13:50:12 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 22 Jun 2012 13:50:12 -0700 Subject: [Bro-Dev] Seeing crashes with master Message-ID: <20120622205012.GI46519@icir.org> I'm seeing crashes with current master in a cluster setup. From a quick look, it might be related to the logging code, but I can't immediately tell what's triggering it. Anybody else seeing problems with current master? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From robin at icir.org Fri Jun 22 14:17:30 2012 From: robin at icir.org (Robin Sommer) Date: Fri, 22 Jun 2012 14:17:30 -0700 Subject: [Bro-Dev] Seeing crashes with master In-Reply-To: <20120622205012.GI46519@icir.org> References: <20120622205012.GI46519@icir.org> Message-ID: <20120622211730.GA54112@icir.org> On Fri, Jun 22, 2012 at 13:50 -0700, I wrote: > I'm seeing crashes with current master in a cluster setup. From a > quick look, it might be related to the logging code, but I can't > immediately tell what's triggering it. Hmmm... Seems to be gone now, it may have been a mixup in my installation with an older Bro version. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From slagell at illinois.edu Fri Jun 22 14:19:16 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Fri, 22 Jun 2012 21:19:16 +0000 Subject: [Bro-Dev] Seeing crashes with master In-Reply-To: <20120622211730.GA54112@icir.org> References: <20120622205012.GI46519@icir.org>, <20120622211730.GA54112@icir.org> Message-ID: <3628C31A-D798-4BAB-B9B4-BA369ED6D735@illinois.edu> Sent from my mobile On Jun 22, 2012, at 4:17 PM, "Robin Sommer" wrote: > > On Fri, Jun 22, 2012 at 13:50 -0700, I wrote: > >> I'm seeing crashes with current master in a cluster setup. From a >> quick look, it might be related to the logging code, but I can't >> immediately tell what's triggering it. > > Hmmm... Seems to be gone now, it may have been a mixup in my > installation with an older Bro version I am pretty sure we are running master just fine. From seth at icir.org Sun Jun 24 17:33:15 2012 From: seth at icir.org (Seth Hall) Date: Sun, 24 Jun 2012 20:33:15 -0400 Subject: [Bro-Dev] Seeing crashes with master In-Reply-To: <3628C31A-D798-4BAB-B9B4-BA369ED6D735@illinois.edu> References: <20120622205012.GI46519@icir.org>, <20120622211730.GA54112@icir.org> <3628C31A-D798-4BAB-B9B4-BA369ED6D735@illinois.edu> Message-ID: <1219F2DF-F2FA-4121-8344-1807161038E7@icir.org> On Jun 22, 2012, at 5:19 PM, Slagell, Adam J wrote: > I am pretty sure we are running master just fine. Yep, the dev cluster is running master with no crashes. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From bro at tracker.bro-ids.org Tue Jun 26 15:10:17 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 26 Jun 2012 22:10:17 -0000 Subject: [Bro-Dev] #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events In-Reply-To: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> References: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> Message-ID: <064.c94e2d68333a46b20482ae7a46dc6b29@tracker.bro-ids.org> #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events ----------------------+------------------------ Reporter: aashish | Owner: Type: Patch | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [9ae9b2aa4dca3c7fe1c4cb310dac8563caa36700/bro]: {{{ #!CommitTicketReference repository="bro" revision="9ae9b2aa4dca3c7fe1c4cb310dac8563caa36700" Extract ICMPv6 NDP options and include in ICMP events (addresses #833). This adds a new parameter of type "icmp6_nd_options" to the ICMPv6 neighbor discovery events (icmp_redirect, icmp_router_solicitation, icmp_router_advertisement, icmp_neighbor_solicitation, icmp_neighbor_advertisement) which includes data extracted from all neighbor discovery options (RFC 4861) that are present in the ICMPv6 message. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Jun 26 15:15:01 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 26 Jun 2012 22:15:01 -0000 Subject: [Bro-Dev] #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events In-Reply-To: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> References: <049.9def06a546029afffcfdaeecc3ae85a0@tracker.bro-ids.org> Message-ID: <064.8e4378358bedabc86206d8ace34335d7@tracker.bro-ids.org> #833: ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events ----------------------------+------------------------ Reporter: aashish | Owner: Type: Merge Request | Status: new Priority: High | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * type: Patch => Merge Request Comment: `topic/jsiwek/icmp6-ndp-options` has new code to parse out ICMPv6 NDP message options and include them in all the relevant events. (This branch doesn't address adopting the example policy script given in the ticket). -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Jun 27 00:00:06 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 27 Jun 2012 00:00:06 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206270700.q5R7068p018975@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 833 [1] | aashish | | High | ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 94f0bf2 | Daniel Thayer | 2012-06-26 | Fix typos in event documentation [2] bro | 5ab2545 | Daniel Thayer | 2012-06-26 | Fix typos in NEWS for Bro 2.1 beta [3] [1] #833: http://tracker.bro-ids.org/bro/ticket/833 [2] fastpath: http://tracker.bro-ids.org/bro/changeset/94f0bf215783b7b529a7960da6bb463e4fe8c0cf/bro [3] fastpath: http://tracker.bro-ids.org/bro/changeset/5ab2545ff3da7b210e368b81fff87c12614d6ab8/bro From bro at tracker.bro-ids.org Wed Jun 27 09:36:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 27 Jun 2012 16:36:34 -0000 Subject: [Bro-Dev] #834: Compile warning on RHEL 6.2 In-Reply-To: <046.fd396e56746c3541b2691ce86ee8cf2b@tracker.bro-ids.org> References: <046.fd396e56746c3541b2691ce86ee8cf2b@tracker.bro-ids.org> Message-ID: <061.b1c6671a7e23699c30ea78be78185829@tracker.bro-ids.org> #834: Compile warning on RHEL 6.2 ----------------------+------------------------ Reporter: seth | Owner: jsiwek Type: Problem | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------+------------------------ Changes (by jsiwek): * owner: => jsiwek * status: new => closed * resolution: => fixed Comment: In [a651185ff9f93fedb3a82575e5107dd7460475de/bro]: {{{ #!CommitTicketReference repository="bro" revision="a651185ff9f93fedb3a82575e5107dd7460475de" Fix strict-aliasing warning in RemoteSerializer.cc (fixes #834). }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Jun 28 00:00:03 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 28 Jun 2012 00:00:03 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206280700.q5S703XL014422@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 833 [1] | aashish | | High | ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | a651185 | Jon Siwek | 2012-06-27 | Fix strict-aliasing warning in RemoteSerializer.cc (fixes #834). [2] bro | 94f0bf2 | Daniel Thayer | 2012-06-26 | Fix typos in event documentation [3] bro | 5ab2545 | Daniel Thayer | 2012-06-26 | Fix typos in NEWS for Bro 2.1 beta [4] [1] #833: http://tracker.bro-ids.org/bro/ticket/833 [2] fastpath: http://tracker.bro-ids.org/bro/changeset/a651185ff9f93fedb3a82575e5107dd7460475de/bro [3] fastpath: http://tracker.bro-ids.org/bro/changeset/94f0bf215783b7b529a7960da6bb463e4fe8c0cf/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/5ab2545ff3da7b210e368b81fff87c12614d6ab8/bro From bro at tracker.bro-ids.org Thu Jun 28 08:12:38 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 28 Jun 2012 15:12:38 -0000 Subject: [Bro-Dev] #835: Porting Drop and Catch-n-release to 2.0 Message-ID: <049.54449caa3016b34f217c36247f756cfa@tracker.bro-ids.org> #835: Porting Drop and Catch-n-release to 2.0 ------------------------+----------------------------- Reporter: aashish | Type: Feature Request Status: new | Priority: Normal Milestone: | Component: Bro Version: git/master | Keywords: ------------------------+----------------------------- The following patch ports the drop.bro to bro-2.0+ (along with catch-n-release functionality) [policies originally written for bro-1.5.3 and prior versions by Jim Mellander and Robin Sommer] Also attaching scan.bro (which is ported to 2.0) scan.bro and drop.bro files need to go into policy/protocols/conn/ folder. Also adding test-drop-connectivity and test-restore-connectivity scripts which should go into aux/broctl/bin/ This patch and policies have been operational at LBNL for a few months now with bro-2.0. (sorry haven't created my own branch to commit these) - please let me know if this need to be otherwise. -- Ticket URL: Bro Tracker Bro Issue Tracker From jsiwek at illinois.edu Thu Jun 28 11:43:41 2012 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Thu, 28 Jun 2012 18:43:41 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] fastpath: Drain events before terminating log/thread managers. (21a0e74) In-Reply-To: <201206281753.q5SHravS006796@bro-ids.icir.org> References: <201206281753.q5SHravS006796@bro-ids.icir.org> Message-ID: On RHEL 6.3 (gcc 4.4.6) and Ubuntu 12.04 (gcc 4.6.3), when not using --enable-debug, this fixes many tests that were failing, but I think there could still be problems that I don't know what to do about. In main.cc's terminate_bro() there is: log_mgr->Terminate(); thread_mgr->Terminate(); mgr.Drain(); Is it possible anywhere in those Terminate() calls for an event to be raised for which a handler exists that tries to use the logging framework (i.e. cause an attempt to write after logging/threading terminates) ? If so, threading::Manager::Terminate() seems like it's `delete`ing backend threads without the WriterFrontend being aware, which then tries to access it later during those final log writes. Do events need to also be drained periodically during/inside those Terminate() calls? Jon On Jun 28, 2012, at 12:53 PM, Jonathan Siwek wrote: > Repository : ssh://git at bro-ids.icir.org/bro > > On branch : fastpath > Link : http://tracker.bro-ids.org/bro/changeset/21a0e74d682f0584288c6e631496bb4083e5d33f/bro > >> --------------------------------------------------------------- > > commit 21a0e74d682f0584288c6e631496bb4083e5d33f > Author: Jon Siwek > Date: Thu Jun 28 12:42:32 2012 -0500 > > Drain events before terminating log/thread managers. > > Using the default scripts, the events from RemoteSerializer::LogStats() > were attempting to use the logging framework after logging/threading > had been terminated which never worked right and sometimes caused > crashes with "fatal error: cannot lock mutex". > > Also made communication log baseline test pass more reliably. > > >> --------------------------------------------------------------- > > 21a0e74d682f0584288c6e631496bb4083e5d33f > src/main.cc | 2 + > .../send.log | 29 ++++++++++--------- > .../communication/communication_log_baseline.bro | 9 ++++-- > 3 files changed, 23 insertions(+), 17 deletions(-) > > diff --git a/src/main.cc b/src/main.cc > index b1d0a4d..d94a32d 100644 > --- a/src/main.cc > +++ b/src/main.cc > @@ -313,6 +313,8 @@ void terminate_bro() > if ( remote_serializer ) > remote_serializer->LogStats(); > > + mgr.Drain(); > + > log_mgr->Terminate(); > thread_mgr->Terminate(); > > diff --git a/testing/btest/Baseline/scripts.base.frameworks.communication.communication_log_baseline/send.log b/testing/btest/Baseline/scripts.base.frameworks.communication.communication_log_baseline/send.log > index d3c14c8..94e0403 100644 > --- a/testing/btest/Baseline/scripts.base.frameworks.communication.communication_log_baseline/send.log > +++ b/testing/btest/Baseline/scripts.base.frameworks.communication.communication_log_baseline/send.log > @@ -5,17 +5,18 @@ > #path communication > #fields ts peer src_name connected_peer_desc connected_peer_addr connected_peer_port level message > #types time string string string addr port string string > -1326492291.485390 bro parent - - - info [#1/127.0.0.1:47757] added peer > -1326492291.491731 bro child - - - info [#1/127.0.0.1:47757] connected > -1326492291.492024 bro parent - - - info [#1/127.0.0.1:47757] peer connected > -1326492291.492024 bro parent - - - info [#1/127.0.0.1:47757] phase: version > -1326492291.492740 bro script - - - info connection established > -1326492291.492740 bro script - - - info requesting events matching /^?(NOTHING)$?/ > -1326492291.492740 bro script - - - info accepting state > -1326492291.493800 bro parent - - - info [#1/127.0.0.1:47757] phase: handshake > -1326492291.493800 bro parent - - - info warning: no events to request > -1326492291.494161 bro parent - - - info [#1/127.0.0.1:47757] peer_description is bro > -1326492291.494404 bro parent - - - info [#1/127.0.0.1:47757] peer supports keep-in-cache; using that > -1326492291.494404 bro parent - - - info [#1/127.0.0.1:47757] phase: running > -1326492291.494404 bro parent - - - info terminating... > -1326492291.494404 bro parent - - - info [#1/127.0.0.1:47757] closing connection > +1340904724.781527 bro parent - - - info [#1/127.0.0.1:47757] added peer > +1340904724.784954 bro child - - - info [#1/127.0.0.1:47757] connected > +1340904724.786168 bro parent - - - info [#1/127.0.0.1:47757] peer connected > +1340904724.786168 bro parent - - - info [#1/127.0.0.1:47757] phase: version > +1340904724.786168 bro script - - - info connection established > +1340904724.786168 bro script - - - info requesting events matching /^?(NOTHING)$?/ > +1340904724.786168 bro script - - - info accepting state > +1340904724.787645 bro parent - - - info [#1/127.0.0.1:47757] phase: handshake > +1340904724.787645 bro parent - - - info warning: no events to request > +1340904724.788857 bro parent - - - info [#1/127.0.0.1:47757] peer_description is bro > +1340904724.829480 bro parent - - - info [#1/127.0.0.1:47757] peer supports keep-in-cache; using that > +1340904724.829480 bro parent - - - info [#1/127.0.0.1:47757] phase: running > +1340904724.829480 bro parent - - - info terminating... > +1340904724.832952 bro child - - - info terminating > +1340904724.834082 bro parent - - - info [#1/127.0.0.1:47757] closing connection > diff --git a/testing/btest/scripts/base/frameworks/communication/communication_log_baseline.bro b/testing/btest/scripts/base/frameworks/communication/communication_log_baseline.bro > index 3d80ef7..4a2ed73 100644 > --- a/testing/btest/scripts/base/frameworks/communication/communication_log_baseline.bro > +++ b/testing/btest/scripts/base/frameworks/communication/communication_log_baseline.bro > @@ -5,7 +5,7 @@ > # @TEST-EXEC: btest-bg-wait -k 10 > # > # Don't diff the receiver log just because port is always going to change > -# @TEST-EXEC: egrep -v 'pid|socket buffer size' sender/communication.log >send.log > +# @TEST-EXEC: egrep -v 'CPU|bytes|pid|socket buffer size' sender/communication.log >send.log > # @TEST-EXEC: btest-diff send.log > > @TEST-START-FILE sender.bro > @@ -19,6 +19,10 @@ redef Communication::nodes += { > event remote_connection_handshake_done(p: event_peer) > { > terminate_communication(); > + } > + > +event remote_connection_closed(p: event_peer) > + { > terminate(); > } > > @@ -30,9 +34,8 @@ event remote_connection_handshake_done(p: event_peer) > > @load frameworks/communication/listen > > -event remote_connection_handshake_done(p: event_peer) > +event remote_connection_closed(p: event_peer) > { > - terminate_communication(); > terminate(); > } > > > _______________________________________________ > bro-commits mailing list > bro-commits at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits > From bro at tracker.bro-ids.org Thu Jun 28 12:45:57 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 28 Jun 2012 19:45:57 -0000 Subject: [Bro-Dev] #836: Make reporter.log errors go to stderr when run from command-line Message-ID: <048.f55ab9255b4a29b56b8218bba951e661@tracker.bro-ids.org> #836: Make reporter.log errors go to stderr when run from command-line -----------------------------+------------------------ Reporter: amannb | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Component: Bro | Version: git/master Keywords: | -----------------------------+------------------------ After talking with Seth for a bit about this I think we should consider outputting error messages that go to reporter.log (and perhaps warnings) to stderr when Bro is running for the command line. broctl could disable those outputs (with a command-line or script-level switch). I think this would make important errors in scripts much more visible to the user. It is easy to forget to check reporter.log when experimenting with stuff, when reporter.log might show the solution at once (happened to me a couple of times). And I am apparently not alone in that - especially new users might not know that fatal error messages will end up in reporter.log (see for example todays input-framework question on the mailing list). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jun 28 13:54:34 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 28 Jun 2012 20:54:34 -0000 Subject: [Bro-Dev] #837: broctl load order incorrect Message-ID: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> #837: broctl load order incorrect ------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Keywords: | ------------------------+------------------------ Right now broctl script loading looks like this... {{{ -U .status -p broctl -p broctl-live -p local -p manager local.bro broctl base/frameworks/cluster local-manager.bro broctl/auto }}} It needs to load the local.bro script last so that any setting can be changed by settings in local.bro. I think the load order should look like this... {{{ -U .status -p broctl -p broctl-live -p local -p manager broctl base/frameworks/cluster broctl/auto local.bro local-manager.bro }}} It would be interesting to find out if the "base/frameworks/cluster" load needs to be in there too. It may make more sense to just load that in the broctl module that is loaded first if the cluster is enabled. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jun 28 13:59:52 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 28 Jun 2012 20:59:52 -0000 Subject: [Bro-Dev] #837: broctl load order incorrect In-Reply-To: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> References: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> Message-ID: <061.9bd818d2eb8c29fc2b2cc692a521c7c0@tracker.bro-ids.org> #837: broctl load order incorrect -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by seth): Hm, this is a little more complicated than I first remembered. The problem is that this is an option that is potentially set by broctl. logrotationinterval Of course, if I set something directly in a bro script I would like for broctl to not overwrite that setting. I vote for changing the order and keeping more advanced users happy that might be setting this stuff directly. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Jun 28 15:45:28 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 28 Jun 2012 22:45:28 -0000 Subject: [Bro-Dev] #837: broctl load order incorrect In-Reply-To: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> References: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> Message-ID: <061.630c25e4affe86f202a75348453cd268@tracker.bro-ids.org> #837: broctl load order incorrect -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by dnthayer): Replying to [comment:1 seth]: > Hm, this is a little more complicated than I first remembered. The problem is that this is an option that is potentially set by broctl. > > logrotationinterval > > Of course, if I set something directly in a bro script I would like for broctl to not overwrite that setting. I vote for changing the order and keeping more advanced users happy that might be setting this stuff directly. I would think that the broctl setting should take priority, because I want to be able to trust the output of "broctl config". Otherwise, if a bro script could override it, then I'd have to check the broctl source code to find the name of the script variable (for broctl option "logrotationinterval", the bro script variable name is "default_rotation_interval"). Then, if I do a "grep -R default_rotation_interval *" in the install tree, I see it assigned 4 different values in 8 different places. Alternatively, how about we just remove those broctl options that clash with bro script variables? I guess as long as we have multiple ways of setting a single parameter it will inevitably lead to confusion. Looking at the broctl README, I see some factual errors in the "Site-specific Customization" section. I will try to clarify the instructions there. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Jun 29 00:00:05 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 29 Jun 2012 00:00:05 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206290700.q5T705SA011443@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 833 [1] | aashish | | High | ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 41f1544 | Jon Siwek | 2012-06-28 | Add front-end name to InitMessage from WriterFrontend to Backend. [2] bro | 1bbd639 | Jon Siwek | 2012-06-28 | Small tweak to make test complete quicker. [3] bro | 21a0e74 | Jon Siwek | 2012-06-28 | Drain events before terminating log/thread managers. [4] bro | a651185 | Jon Siwek | 2012-06-27 | Fix strict-aliasing warning in RemoteSerializer.cc (fixes #834). [5] bro | 94f0bf2 | Daniel Thayer | 2012-06-26 | Fix typos in event documentation [6] bro | 5ab2545 | Daniel Thayer | 2012-06-26 | Fix typos in NEWS for Bro 2.1 beta [7] pysubnettree | 00cc7fa | Daniel Thayer | 2012-06-28 | Fix indentation of an "else" statement [8] pysubnettree | d9c2160 | Jon Siwek | 2012-06-28 | Fix compile warnings and dependencies of swig-generated files. [9] [1] #833: http://tracker.bro-ids.org/bro/ticket/833 [2] fastpath: http://tracker.bro-ids.org/bro/changeset/41f1544332cddfa9a636c05f41371698a891de63/bro [3] fastpath: http://tracker.bro-ids.org/bro/changeset/1bbd63970a9fe5529cc9c6898c510d47ea5472af/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/21a0e74d682f0584288c6e631496bb4083e5d33f/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/a651185ff9f93fedb3a82575e5107dd7460475de/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/94f0bf215783b7b529a7960da6bb463e4fe8c0cf/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/5ab2545ff3da7b210e368b81fff87c12614d6ab8/bro [8] fastpath: http://tracker.bro-ids.org/bro/changeset/00cc7fa9a53410cff1369501eb3d4ae28ca4bc9d/pysubnettree [9] fastpath: http://tracker.bro-ids.org/bro/changeset/d9c2160980c319db5df0c3c6d958270f71743622/pysubnettree From bro at tracker.bro-ids.org Fri Jun 29 07:21:51 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 29 Jun 2012 14:21:51 -0000 Subject: [Bro-Dev] #837: broctl load order incorrect In-Reply-To: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> References: <046.1d3cbfcc6103575de82e3e2b73e67985@tracker.bro-ids.org> Message-ID: <061.fa5f3811e2c832256250174cf50b81c8@tracker.bro-ids.org> #837: broctl load order incorrect -------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: new Priority: Normal | Milestone: Bro2.1 Component: BroControl | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by seth): > Alternatively, how about we just remove those broctl options > that clash with bro script variables? I guess as long as we > have multiple ways of setting a single parameter it will > inevitably lead to confusion. I'm starting to lean this way as well. This confusion between where a value is set is pretty frustrating and hard to debug sometimes. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 29 09:32:24 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 29 Jun 2012 16:32:24 -0000 Subject: [Bro-Dev] #301: Switch to binary logging In-Reply-To: <046.26b2c9b3fb5b3c5022b521115d956856@tracker.bro-ids.org> References: <046.26b2c9b3fb5b3c5022b521115d956856@tracker.bro-ids.org> Message-ID: <061.46d70901512724b19add498ad36f0385@tracker.bro-ids.org> #301: Switch to binary logging ------------------------------+-------------------- Reporter: seth | Owner: Type: Feature Request | Status: closed Priority: High | Milestone: Bro2.1 Component: Bro | Version: Resolution: Solved/Applied | Keywords: ------------------------------+-------------------- Changes (by seth): * status: new => closed * resolution: => Solved/Applied Comment: I'm not sure this is going to happen. The binary logs are merged in, let's just leave it at that for now. -- Ticket URL: Bro Tracker Bro Issue Tracker From slagell at illinois.edu Fri Jun 29 09:35:03 2012 From: slagell at illinois.edu (Slagell, Adam J) Date: Fri, 29 Jun 2012 16:35:03 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/seth/elasticsearch-merge-tmp: ElasticSearch plugin fixes. (49b19f2) In-Reply-To: <201206291609.q5TG9fIc012953@bro-ids.icir.org> References: <201206291609.q5TG9fIc012953@bro-ids.icir.org> Message-ID: <42C56F3F-9DF4-4362-8EE4-24C4396508A2@illinois.edu> I'd like to learn more about Elastic Search sometime Seth. Maybe a blog entry or a talk at the Bro Exchange. :-) On Jun 29, 2012, at 11:09 AM, Seth Hall wrote: > Repository : ssh://git at bro-ids.icir.org/bro > > On branch : topic/seth/elasticsearch-merge-tmp > Link : http://tracker.bro-ids.org/bro/changeset/49b19f244fc8284f75469f05d96c4ac16acd2cfe/bro > >> --------------------------------------------------------------- > > commit 49b19f244fc8284f75469f05d96c4ac16acd2cfe > Author: Seth Hall > Date: Fri Jun 29 12:09:34 2012 -0400 > > ElasticSearch plugin fixes. > > - Fixed memory leak. > > - Now using Fmt instead of the thread unsafe fmt. > > >> --------------------------------------------------------------- > > 49b19f244fc8284f75469f05d96c4ac16acd2cfe > src/logging/writers/ElasticSearch.cc | 33 +++++++++++++++++---------------- > src/logging/writers/ElasticSearch.h | 3 +++ > 2 files changed, 20 insertions(+), 16 deletions(-) > > diff --git a/src/logging/writers/ElasticSearch.cc b/src/logging/writers/ElasticSearch.cc > index 0e388df..19cd296 100644 > --- a/src/logging/writers/ElasticSearch.cc > +++ b/src/logging/writers/ElasticSearch.cc > @@ -32,9 +32,11 @@ ElasticSearch::ElasticSearch(WriterFrontend* frontend) : WriterBackend(frontend) > > index_prefix = string((const char*) BifConst::LogElasticSearch::index_prefix->Bytes(), BifConst::LogElasticSearch::index_prefix->Len()); > > - es_server = string(fmt("http://%s:%d/", BifConst::LogElasticSearch::server_host->Bytes(), > - (int) BifConst::LogElasticSearch::server_port)); > + es_server = string(Fmt("http://%s:%d", BifConst::LogElasticSearch::server_host->Bytes(), > + (int) BifConst::LogElasticSearch::server_port)); > + bulk_url = string(Fmt("%s/_bulk", es_server.c_str())); > > + http_headers = curl_slist_append(NULL, "Content-Type: text/json; charset=utf-8"); > buffer.Clear(); > counter = 0; > current_index = string(); > @@ -56,13 +58,14 @@ bool ElasticSearch::DoInit(const WriterInfo& info, int num_fields, const threadi > > bool ElasticSearch::DoFlush() > { > - // Do something here? > + BatchIndex(); > return true; > } > > bool ElasticSearch::DoFinish() > { > BatchIndex(); > + curl_slist_free_all(http_headers); > curl_easy_cleanup(curl_handle); > return WriterBackend::DoFinish(); > } > @@ -70,8 +73,7 @@ bool ElasticSearch::DoFinish() > bool ElasticSearch::BatchIndex() > { > curl_easy_reset(curl_handle); > - string url = es_server + "_bulk"; > - curl_easy_setopt(curl_handle, CURLOPT_URL, url.c_str()); > + curl_easy_setopt(curl_handle, CURLOPT_URL, bulk_url.c_str()); > curl_easy_setopt(curl_handle, CURLOPT_POST, 1); > curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, (curl_off_t)buffer.Len()); > curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDS, buffer.Bytes()); > @@ -264,7 +266,7 @@ bool ElasticSearch::UpdateIndex(double now, double rinterval, double rbase) > current_index = index_prefix + "-" + buf; > } > > - //printf("%s - prev:%s current:%s\n", Info().path.c_str(), prev_index.c_str(), current_index.c_str()); > + printf("%s - prev:%s current:%s\n", Info().path.c_str(), prev_index.c_str(), current_index.c_str()); > return true; > } > > @@ -281,7 +283,7 @@ bool ElasticSearch::DoRotate(string rotated_path, const RotateInfo& info, bool t > > // Compress the previous index > //curl_easy_reset(curl_handle); > - //curl_easy_setopt(curl_handle, CURLOPT_URL, fmt("%s%s/_settings", es_server.c_str(), prev_index.c_str())); > + //curl_easy_setopt(curl_handle, CURLOPT_URL, Fmt("%s/%s/_settings", es_server.c_str(), prev_index.c_str())); > //curl_easy_setopt(curl_handle, CURLOPT_CUSTOMREQUEST, "PUT"); > //curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDS, "{\"index\":{\"store.compress.stored\":\"true\"}}"); > //curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, (curl_off_t) 42); > @@ -290,14 +292,14 @@ bool ElasticSearch::DoRotate(string rotated_path, const RotateInfo& info, bool t > // Optimize the previous index. > // TODO: make this into variables. > //curl_easy_reset(curl_handle); > - //curl_easy_setopt(curl_handle, CURLOPT_URL, fmt("%s%s/_optimize?max_num_segments=1&wait_for_merge=false", es_server.c_str(), prev_index.c_str())); > + //curl_easy_setopt(curl_handle, CURLOPT_URL, Fmt("%s/%s/_optimize?max_num_segments=1&wait_for_merge=false", es_server.c_str(), prev_index.c_str())); > //HTTPSend(curl_handle); > } > > - if ( ! FinishedRotation(current_index, prev_index, info, terminating) ) > - { > - Error(Fmt("error rotating %s to %s", prev_index.c_str(), current_index.c_str())); > - } > + //if ( ! FinishedRotation(current_index, prev_index, info, terminating) ) > + // { > + // Error(Fmt("error rotating %s to %s", prev_index.c_str(), current_index.c_str())); > + // } > > return true; > } > @@ -325,7 +327,7 @@ CURL* ElasticSearch::HTTPSetup() > CURL* handle = curl_easy_init(); > if ( ! handle ) > { > - Error(fmt("cURL did not initialize correctly.")); > + Error("cURL did not initialize correctly."); > return 0; > } > > @@ -340,8 +342,7 @@ bool ElasticSearch::HTTPReceive(void* ptr, int size, int nmemb, void* userdata) > > bool ElasticSearch::HTTPSend(CURL *handle) > { > - struct curl_slist *headers = curl_slist_append(NULL, "Content-Type: text/json; charset=utf-8"); > - curl_easy_setopt(handle, CURLOPT_HTTPHEADER, headers); > + curl_easy_setopt(handle, CURLOPT_HTTPHEADER, http_headers); > curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, &logging::writer::ElasticSearch::HTTPReceive); // This gets called with the result. > // HTTP 1.1 likes to use chunked encoded transfers, which aren't good for speed. > // The best (only?) way to disable that is to just use HTTP 1.0 > @@ -361,7 +362,7 @@ bool ElasticSearch::HTTPSend(CURL *handle) > uint http_code = 0; > curl_easy_getinfo(curl_handle, CURLINFO_RESPONSE_CODE, &http_code); > if ( http_code != 200 ) > - Error(fmt("Received a non-successful status code back from ElasticSearch server.")); > + Error(Fmt("Received a non-successful status code back from ElasticSearch server.")); > > return true; > } > diff --git a/src/logging/writers/ElasticSearch.h b/src/logging/writers/ElasticSearch.h > index 71238a9..aa56dba 100644 > --- a/src/logging/writers/ElasticSearch.h > +++ b/src/logging/writers/ElasticSearch.h > @@ -59,6 +59,9 @@ private: > int cluster_name_len; > > string es_server; > + string bulk_url; > + > + struct curl_slist *http_headers; > > string path; > string index_prefix; > > _______________________________________________ > bro-commits mailing list > bro-commits at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits > ------ Adam J. Slagell, CISO, CISSP Chief Information Security Officer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info 217.244.8965 "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From vladg at cmu.edu Fri Jun 29 09:47:47 2012 From: vladg at cmu.edu (Vlad Grigorescu) Date: Fri, 29 Jun 2012 16:47:47 +0000 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/seth/elasticsearch-merge-tmp: ElasticSearch plugin fixes. (49b19f2) In-Reply-To: <21433_1340987713_q5TGZBSi028626_42C56F3F-9DF4-4362-8EE4-24C4396508A2@illinois.edu> Message-ID: I'll be presenting at the Bro Exchange on ElasticSearch, and specifically a web interface I've been working on for accessing Bro logs in ElasticSearch. If people are interested in playing around and learning more about ElasticSearch until then, just let me know - I'd be happy to help. Personally, I think it's really cool (or as their website describes it, "bonsai cool.") --Vlad On 6/29/12 12:35 PM, "Slagell, Adam J" wrote: I'd like to learn more about Elastic Search sometime Seth. Maybe a blog entry or a talk at the Bro Exchange. :-) On Jun 29, 2012, at 11:09 AM, Seth Hall wrote: > Repository : ssh://git at bro-ids.icir.org/bro > > On branch : topic/seth/elasticsearch-merge-tmp > Link : >http://tracker.bro-ids.org/bro/changeset/49b19f244fc8284f75469f05d96c4ac16 >acd2cfe/bro > >> --------------------------------------------------------------- > > commit 49b19f244fc8284f75469f05d96c4ac16acd2cfe > Author: Seth Hall > Date: Fri Jun 29 12:09:34 2012 -0400 > > ElasticSearch plugin fixes. > > - Fixed memory leak. > > - Now using Fmt instead of the thread unsafe fmt. > > >> --------------------------------------------------------------- > > 49b19f244fc8284f75469f05d96c4ac16acd2cfe > src/logging/writers/ElasticSearch.cc | 33 >+++++++++++++++++---------------- > src/logging/writers/ElasticSearch.h | 3 +++ > 2 files changed, 20 insertions(+), 16 deletions(-) > > diff --git a/src/logging/writers/ElasticSearch.cc >b/src/logging/writers/ElasticSearch.cc > index 0e388df..19cd296 100644 > --- a/src/logging/writers/ElasticSearch.cc > +++ b/src/logging/writers/ElasticSearch.cc > @@ -32,9 +32,11 @@ ElasticSearch::ElasticSearch(WriterFrontend* >frontend) : WriterBackend(frontend) > > index_prefix = string((const char*) >BifConst::LogElasticSearch::index_prefix->Bytes(), >BifConst::LogElasticSearch::index_prefix->Len()); > > - es_server = string(fmt("http://%s:%d/", >BifConst::LogElasticSearch::server_host->Bytes(), > - (int) >BifConst::LogElasticSearch::server_port)); > + es_server = string(Fmt("http://%s:%d", >BifConst::LogElasticSearch::server_host->Bytes(), > + (int) >BifConst::LogElasticSearch::server_port)); > + bulk_url = string(Fmt("%s/_bulk", es_server.c_str())); > > + http_headers = curl_slist_append(NULL, "Content-Type: text/json; >charset=utf-8"); > buffer.Clear(); > counter = 0; > current_index = string(); > @@ -56,13 +58,14 @@ bool ElasticSearch::DoInit(const WriterInfo& info, >int num_fields, const threadi > > bool ElasticSearch::DoFlush() > { > - // Do something here? > + BatchIndex(); > return true; > } > > bool ElasticSearch::DoFinish() > { > BatchIndex(); > + curl_slist_free_all(http_headers); > curl_easy_cleanup(curl_handle); > return WriterBackend::DoFinish(); > } > @@ -70,8 +73,7 @@ bool ElasticSearch::DoFinish() > bool ElasticSearch::BatchIndex() > { > curl_easy_reset(curl_handle); > - string url = es_server + "_bulk"; > - curl_easy_setopt(curl_handle, CURLOPT_URL, url.c_str()); > + curl_easy_setopt(curl_handle, CURLOPT_URL, bulk_url.c_str()); > curl_easy_setopt(curl_handle, CURLOPT_POST, 1); > curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, >(curl_off_t)buffer.Len()); > curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDS, buffer.Bytes()); > @@ -264,7 +266,7 @@ bool ElasticSearch::UpdateIndex(double now, double >rinterval, double rbase) > current_index = index_prefix + "-" + buf; > } > > - //printf("%s - prev:%s current:%s\n", Info().path.c_str(), >prev_index.c_str(), current_index.c_str()); > + printf("%s - prev:%s current:%s\n", Info().path.c_str(), >prev_index.c_str(), current_index.c_str()); > return true; > } > > @@ -281,7 +283,7 @@ bool ElasticSearch::DoRotate(string rotated_path, >const RotateInfo& info, bool t > > // Compress the previous index > //curl_easy_reset(curl_handle); > - //curl_easy_setopt(curl_handle, CURLOPT_URL, fmt("%s%s/_settings", >es_server.c_str(), prev_index.c_str())); > + //curl_easy_setopt(curl_handle, CURLOPT_URL, Fmt("%s/%s/_settings", >es_server.c_str(), prev_index.c_str())); > //curl_easy_setopt(curl_handle, CURLOPT_CUSTOMREQUEST, "PUT"); > //curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDS, >"{\"index\":{\"store.compress.stored\":\"true\"}}"); > //curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDSIZE_LARGE, >(curl_off_t) 42); > @@ -290,14 +292,14 @@ bool ElasticSearch::DoRotate(string rotated_path, >const RotateInfo& info, bool t > // Optimize the previous index. > // TODO: make this into variables. > //curl_easy_reset(curl_handle); > - //curl_easy_setopt(curl_handle, CURLOPT_URL, >fmt("%s%s/_optimize?max_num_segments=1&wait_for_merge=false", >es_server.c_str(), prev_index.c_str())); > + //curl_easy_setopt(curl_handle, CURLOPT_URL, >Fmt("%s/%s/_optimize?max_num_segments=1&wait_for_merge=false", >es_server.c_str(), prev_index.c_str())); > //HTTPSend(curl_handle); > } > > - if ( ! FinishedRotation(current_index, prev_index, info, terminating) ) > - { > - Error(Fmt("error rotating %s to %s", prev_index.c_str(), >current_index.c_str())); > - } > + //if ( ! FinishedRotation(current_index, prev_index, info, >terminating) ) > + // { > + // Error(Fmt("error rotating %s to %s", prev_index.c_str(), >current_index.c_str())); > + // } > > return true; > } > @@ -325,7 +327,7 @@ CURL* ElasticSearch::HTTPSetup() > CURL* handle = curl_easy_init(); > if ( ! handle ) > { > - Error(fmt("cURL did not initialize correctly.")); > + Error("cURL did not initialize correctly."); > return 0; > } > > @@ -340,8 +342,7 @@ bool ElasticSearch::HTTPReceive(void* ptr, int size, >int nmemb, void* userdata) > > bool ElasticSearch::HTTPSend(CURL *handle) > { > - struct curl_slist *headers = curl_slist_append(NULL, "Content-Type: >text/json; charset=utf-8"); > - curl_easy_setopt(handle, CURLOPT_HTTPHEADER, headers); > + curl_easy_setopt(handle, CURLOPT_HTTPHEADER, http_headers); > curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, >&logging::writer::ElasticSearch::HTTPReceive); // This gets called with >the result. > // HTTP 1.1 likes to use chunked encoded transfers, which aren't good >for speed. > // The best (only?) way to disable that is to just use HTTP 1.0 > @@ -361,7 +362,7 @@ bool ElasticSearch::HTTPSend(CURL *handle) > uint http_code = 0; > curl_easy_getinfo(curl_handle, CURLINFO_RESPONSE_CODE, &http_code); > if ( http_code != 200 ) > - Error(fmt("Received a non-successful status code back from >ElasticSearch server.")); > + Error(Fmt("Received a non-successful status code back from >ElasticSearch server.")); > > return true; > } > diff --git a/src/logging/writers/ElasticSearch.h >b/src/logging/writers/ElasticSearch.h > index 71238a9..aa56dba 100644 > --- a/src/logging/writers/ElasticSearch.h > +++ b/src/logging/writers/ElasticSearch.h > @@ -59,6 +59,9 @@ private: > int cluster_name_len; > > string es_server; > + string bulk_url; > + > + struct curl_slist *http_headers; > > string path; > string index_prefix; > > _______________________________________________ > bro-commits mailing list > bro-commits at bro-ids.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits > ------ Adam J. Slagell, CISO, CISSP Chief Information Security Officer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info 217.244.8965 "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." _______________________________________________ bro-dev mailing list bro-dev at bro-ids.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From bro at tracker.bro-ids.org Fri Jun 29 10:12:23 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 29 Jun 2012 17:12:23 -0000 Subject: [Bro-Dev] #830: topic/tunnels In-Reply-To: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> References: <048.8768fa661ed9e0dbb57d68c4a581df5c@tracker.bro-ids.org> Message-ID: <063.fa05ede8ad4ec2d780de9018183911fb@tracker.bro-ids.org> #830: topic/tunnels -----------------------------+------------------------ Reporter: jsiwek | Owner: jsiwek Type: Task | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by seth): * status: assigned => closed * resolution: => Solved/Applied Comment: This is complete. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Fri Jun 29 14:38:13 2012 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 29 Jun 2012 21:38:13 -0000 Subject: [Bro-Dev] #825: topic/dnthayer/bif-tests In-Reply-To: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> References: <050.2e0e3700c053a8f4983c7d1c137a6256@tracker.bro-ids.org> Message-ID: <065.2afabd68f7e116c603e5a9183ae3845a@tracker.bro-ids.org> #825: topic/dnthayer/bif-tests -----------------------------+------------------------ Reporter: dnthayer | Owner: Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.1 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Comment (by jsiwek): In [34ead91f992cbc40dcb81053343e2ef60a3aff61/bro]: {{{ #!CommitTicketReference repository="bro" revision="34ead91f992cbc40dcb81053343e2ef60a3aff61" Fix inconsistencies in random number generation. The srand()/rand() interface was being intermixed with the srandom()/random() one. The later is now used throughout. Changed the srand() and rand() BIFs to work deterministically if Bro was given a seed file (addresses #825). They also now wrap the system's srandom() and random() instead of srand() and rand() as per the above. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Jun 30 00:00:06 2012 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 30 Jun 2012 00:00:06 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201206300700.q5U706WF026285@bro-ids.icir.org> > Open Merge Requests for Bro2.1 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 833 [1] | aashish | | High | ICMPv6:Patch to add payload as a parameter to neighbor advertisements and neighbor solicitation events > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 34ead91 | Jon Siwek | 2012-06-29 | Fix inconsistencies in random number generation. [2] bro | 0e48fda | Jon Siwek | 2012-06-29 | Updating input framework unit tests. [3] bro | 41f1544 | Jon Siwek | 2012-06-28 | Add front-end name to InitMessage from WriterFrontend to Backend. [4] bro | 1bbd639 | Jon Siwek | 2012-06-28 | Small tweak to make test complete quicker. [5] bro | 21a0e74 | Jon Siwek | 2012-06-28 | Drain events before terminating log/thread managers. [6] bro | a651185 | Jon Siwek | 2012-06-27 | Fix strict-aliasing warning in RemoteSerializer.cc (fixes #834). [7] bro | 94f0bf2 | Daniel Thayer | 2012-06-26 | Fix typos in event documentation [8] bro | 5ab2545 | Daniel Thayer | 2012-06-26 | Fix typos in NEWS for Bro 2.1 beta [9] pysubnettree | 00cc7fa | Daniel Thayer | 2012-06-28 | Fix indentation of an "else" statement [10] pysubnettree | d9c2160 | Jon Siwek | 2012-06-28 | Fix compile warnings and dependencies of swig-generated files. [11] [1] #833: http://tracker.bro-ids.org/bro/ticket/833 [2] fastpath: http://tracker.bro-ids.org/bro/changeset/34ead91f992cbc40dcb81053343e2ef60a3aff61/bro [3] fastpath: http://tracker.bro-ids.org/bro/changeset/0e48fda6ffa0be4cec2d763305a1394e19b32778/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/41f1544332cddfa9a636c05f41371698a891de63/bro [5] fastpath: http://tracker.bro-ids.org/bro/changeset/1bbd63970a9fe5529cc9c6898c510d47ea5472af/bro [6] fastpath: http://tracker.bro-ids.org/bro/changeset/21a0e74d682f0584288c6e631496bb4083e5d33f/bro [7] fastpath: http://tracker.bro-ids.org/bro/changeset/a651185ff9f93fedb3a82575e5107dd7460475de/bro [8] fastpath: http://tracker.bro-ids.org/bro/changeset/94f0bf215783b7b529a7960da6bb463e4fe8c0cf/bro [9] fastpath: http://tracker.bro-ids.org/bro/changeset/5ab2545ff3da7b210e368b81fff87c12614d6ab8/bro [1] fastpath: http://tracker.bro-ids.org/bro/changeset/00cc7fa9a53410cff1369501eb3d4ae28ca4bc9d/pysubnettree [1] fastpath: http://tracker.bro-ids.org/bro/changeset/d9c2160980c319db5df0c3c6d958270f71743622/pysubnettree