From noreply at bro-ids.org Fri Feb 1 00:00:02 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 1 Feb 2013 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302010800.r11802b5002438@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 928 [1] | matthias | seth | Normal | Incorporate ICSI certificate notary into SSL logging Broccoli | 937 [2] | seth | robin | Normal | topic/seth/sendpackets: A test program for sending packets through Broccoli. [3] [1] #928: http://tracker.bro-ids.org/bro/ticket/928 [2] #937: http://tracker.bro-ids.org/bro/ticket/937 [3] sendpackets:: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbroccoli&old=master&new_path=%2Fbroccoli&new=topic/seth/sendpackets: From bro at tracker.bro-ids.org Fri Feb 1 15:49:46 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 01 Feb 2013 23:49:46 -0000 Subject: [Bro-Dev] #941: Merge topic/bernhard/input-tests-exit-after-terminate Message-ID: <048.abfbd9407b5706e16223bb647db37331@tracker.bro-ids.org> #941: Merge topic/bernhard/input-tests-exit-after-terminate ---------------------------+------------------------ Reporter: amannb | Owner: Type: Merge Request | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------------+------------------------ Branch changes the input tests to run parallel using topic/robin/exit- after-terminate -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Feb 2 00:00:04 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 2 Feb 2013 00:00:04 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302020800.r128040H028246@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 928 [1] | matthias | seth | Normal | Incorporate ICSI certificate notary into SSL logging Bro | 941 [2] | amannb | | Low | Merge topic/bernhard/input-tests-exit-after-terminate [3] Broccoli | 937 [4] | seth | robin | Normal | topic/seth/sendpackets: A test program for sending packets through Broccoli. [5] [1] #928: http://tracker.bro-ids.org/bro/ticket/928 [2] #941: http://tracker.bro-ids.org/bro/ticket/941 [3] input-tests-exit-after-terminate: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/bernhard/input-tests-exit-after-terminate [4] #937: http://tracker.bro-ids.org/bro/ticket/937 [5] sendpackets:: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbroccoli&old=master&new_path=%2Fbroccoli&new=topic/seth/sendpackets: From noreply at bro-ids.org Sun Feb 3 00:00:02 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 3 Feb 2013 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302030800.r138024p016800@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 928 [1] | matthias | seth | Normal | Incorporate ICSI certificate notary into SSL logging Bro | 941 [2] | amannb | | Low | Merge topic/bernhard/input-tests-exit-after-terminate [3] Broccoli | 937 [4] | seth | robin | Normal | topic/seth/sendpackets: A test program for sending packets through Broccoli. [5] [1] #928: http://tracker.bro-ids.org/bro/ticket/928 [2] #941: http://tracker.bro-ids.org/bro/ticket/941 [3] input-tests-exit-after-terminate: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/bernhard/input-tests-exit-after-terminate [4] #937: http://tracker.bro-ids.org/bro/ticket/937 [5] sendpackets:: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbroccoli&old=master&new_path=%2Fbroccoli&new=topic/seth/sendpackets: From noreply at bro-ids.org Mon Feb 4 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 4 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302040800.r14803iH028037@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 928 [1] | matthias | seth | Normal | Incorporate ICSI certificate notary into SSL logging Bro | 941 [2] | amannb | | Low | Merge topic/bernhard/input-tests-exit-after-terminate [3] Broccoli | 937 [4] | seth | robin | Normal | topic/seth/sendpackets: A test program for sending packets through Broccoli. [5] [1] #928: http://tracker.bro-ids.org/bro/ticket/928 [2] #941: http://tracker.bro-ids.org/bro/ticket/941 [3] input-tests-exit-after-terminate: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/bernhard/input-tests-exit-after-terminate [4] #937: http://tracker.bro-ids.org/bro/ticket/937 [5] sendpackets:: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbroccoli&old=master&new_path=%2Fbroccoli&new=topic/seth/sendpackets: From bro at tracker.bro-ids.org Mon Feb 4 09:42:14 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 04 Feb 2013 17:42:14 -0000 Subject: [Bro-Dev] #937: topic/seth/sendpackets: A test program for sending packets through Broccoli. In-Reply-To: <046.f92285419706b2cf191af3753376acc9@tracker.bro-ids.org> References: <046.f92285419706b2cf191af3753376acc9@tracker.bro-ids.org> Message-ID: <061.bd93bbe453eea50dfd60d339a2b901a5@tracker.bro-ids.org> #937: topic/seth/sendpackets: A test program for sending packets through Broccoli. -----------------------------+------------------------ Reporter: seth | Owner: robin Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Broccoli | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 4 13:23:12 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 04 Feb 2013 21:23:12 -0000 Subject: [Bro-Dev] #942: Generic log delaying mechanism for logging framework Message-ID: <046.eb19defcf4e96fac77bff6d8a6732743@tracker.bro-ids.org> #942: Generic log delaying mechanism for logging framework -----------------------------+------------------------ Reporter: seth | Owner: seth Type: Feature Request | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | -----------------------------+------------------------ We need to add a mechanism for delaying log writes within the logging framework for the case where some asynchronous lookup needs to happen in a non-base script. There are a few requirements: - The mechanism needs to copy the log record so that future modifications of the record aren't impacted unless deliberately modifying the delayed record. - Three functions in Log:: namespace to register and unregister delays for logs and one to get access to the delayed log by it's delay token. - Additional configuration option in logging framework to configure a default logging delay. It's possible that we should set the default stream delay in the stream configuration record. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 4 23:55:45 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 05 Feb 2013 07:55:45 -0000 Subject: [Bro-Dev] #928: Incorporate ICSI certificate notary into SSL logging In-Reply-To: <050.670adea7e8363a03ddb2651421087ccb@tracker.bro-ids.org> References: <050.670adea7e8363a03ddb2651421087ccb@tracker.bro-ids.org> Message-ID: <065.a4effaa2634bd6eaf40983394073db06@tracker.bro-ids.org> #928: Incorporate ICSI certificate notary into SSL logging ----------------------------+------------------------ Reporter: matthias | Owner: seth Type: Merge Request | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: fixed | Keywords: ----------------------------+------------------------ Changes (by seth): * status: assigned => closed * resolution: => fixed Comment: In [changeset:a2556642e6dea9f61ee1a1093d4378615f152e48/bro]: {{{ #!CommitTicketReference repository="bro" revision="a2556642e6dea9f61ee1a1093d4378615f152e48" Merge remote-tracking branch 'origin/topic/matthias/notary' * origin/topic/matthias/notary: Small cosmetic changes. Give log buffer the correct name. Simplify delayed logging of SSL records. Implement delay-token style SSL logging. More style tweaks: replace spaces with tabs. Factor notary code into separte file. Adhere to Bro coding style guidelines. Enhance ssl.log with information from notary. Closes #928 }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Feb 5 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 5 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302050800.r15803GW001346@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 941 [1] | amannb | | Low | Merge topic/bernhard/input-tests-exit-after-terminate [2] [1] #941: http://tracker.bro-ids.org/bro/ticket/941 [2] input-tests-exit-after-terminate: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/bernhard/input-tests-exit-after-terminate From bro at tracker.bro-ids.org Tue Feb 5 07:50:19 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 05 Feb 2013 15:50:19 -0000 Subject: [Bro-Dev] #941: Merge topic/bernhard/input-tests-exit-after-terminate In-Reply-To: <048.abfbd9407b5706e16223bb647db37331@tracker.bro-ids.org> References: <048.abfbd9407b5706e16223bb647db37331@tracker.bro-ids.org> Message-ID: <063.df4decd4ec7ec9f34c6d960d1d2d781f@tracker.bro-ids.org> #941: Merge topic/bernhard/input-tests-exit-after-terminate -----------------------------+------------------------ Reporter: amannb | Owner: Type: Merge Request | Status: closed Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: Solved/Applied | Keywords: -----------------------------+------------------------ Changes (by robin): * status: new => closed * resolution: => Solved/Applied -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Feb 5 08:57:35 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 05 Feb 2013 16:57:35 -0000 Subject: [Bro-Dev] #394: all.bro loads non-existent policy and segfaults In-Reply-To: <048.54639988ae284713b2b5e1414267fed3@tracker.bro-ids.org> References: <048.54639988ae284713b2b5e1414267fed3@tracker.bro-ids.org> Message-ID: <063.674f661a3f76ee57ce0532917b176345@tracker.bro-ids.org> #394: all.bro loads non-existent policy and segfaults -----------------------+------------------------ Reporter: gregor | Owner: seth Type: Problem | Status: closed Priority: Normal | Milestone: Component: Bro | Version: git/master Resolution: Rejected | Keywords: maketest -----------------------+------------------------ Changes (by seth): * status: accepted => closed * resolution: => Rejected Comment: I'm not sure if this problem is still in the core or not, but I have no clue how to express it at the scriptland. I'm going to close this for now and we'll see if we bump into it again. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Feb 5 08:58:35 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 05 Feb 2013 16:58:35 -0000 Subject: [Bro-Dev] #938: topic/seth/software-version-updates Updates to vulnerable software checking. to robin Owner set to seth (was: robin) In-Reply-To: <047.618e6bbadcf6aef8dacc8730c2c4b08a@tracker.bro-ids.org> References: <047.618e6bbadcf6aef8dacc8730c2c4b08a@tracker.bro-ids.org> Message-ID: <062.f400a41a3f867de76d1942e64db4425e@tracker.bro-ids.org> #938: topic/seth/software-version-updates Updates to vulnerable software checking. to robin Owner set to seth ----------------------+------------------------ Reporter: robin | Owner: seth Type: Problem | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: robin ----------------------+------------------------ -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 6 11:06:30 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 06 Feb 2013 19:06:30 -0000 Subject: [Bro-Dev] #943: PF_Ring plugin to support load balancing while sniffing multiple interfaces Message-ID: <046.13f2a01000dce0d7f5e3936f93cfb271@tracker.bro-ids.org> #943: PF_Ring plugin to support load balancing while sniffing multiple interfaces ------------------------+------------------------ Reporter: seth | Owner: dnthayer Type: Problem | Status: new Priority: Medium | Milestone: Bro2.2 Component: BroControl | Version: git/master Keywords: | ------------------------+------------------------ As reported by Jordi Ros-Giralt, if you want to sniff two interfaces on the same host and load balance each interface across several workers there are problems where all of the traffic is not monitored because it's all put into the same pf_ring cluster id. I suspect we need to do two things: - Make cluster_id settable in the worker config as an override for the global option. - Make the plugin watch for multiple load balancing rings on the same host (is this possible with the current plugin architecture?) and adapt by making the second load balancing ring use the global value +1. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 6 12:20:58 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Wed, 06 Feb 2013 20:20:58 -0000 Subject: [Bro-Dev] #938: topic/seth/software-version-updates2 Updates to vulnerable software checking. (was: topic/seth/software-version-updates Updates to vulnerable software checking. to robin Owner set to seth) In-Reply-To: <047.618e6bbadcf6aef8dacc8730c2c4b08a@tracker.bro-ids.org> References: <047.618e6bbadcf6aef8dacc8730c2c4b08a@tracker.bro-ids.org> Message-ID: <062.0b69b11f7d4505db6b4562bda0ba0ac6@tracker.bro-ids.org> #938: topic/seth/software-version-updates2 Updates to vulnerable software checking. ----------------------------+------------------------ Reporter: robin | Owner: robin Type: Merge Request | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by seth): * keywords: robin => * owner: seth => robin * type: Problem => Merge Request Comment: I must have accidentally merged another branch in at some point. I cherry picked the commits into a new branch and I'll delete the old one. The new branch to merge is now: topic/seth/software-version-updates2 -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Feb 7 00:00:02 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 7 Feb 2013 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302070800.r178029d018630@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From seth at icir.org Thu Feb 7 07:00:21 2013 From: seth at icir.org (Seth Hall) Date: Thu, 7 Feb 2013 10:00:21 -0500 Subject: =?iso-8859-1?q?_=5BBro-Dev=5D_=08Rules_for_base/?= Message-ID: I took a few notes of the rules of thumb that I try to use for the base/ scripts directory. Let me know if there are questions or thoughts about other rules for these scripts. http://bro-ids.org/development/script-conventions.html#rules-for-base .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130207/a44db384/attachment.bin From bro at tracker.bro-ids.org Thu Feb 7 07:52:58 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 07 Feb 2013 15:52:58 -0000 Subject: [Bro-Dev] #944: @bro-meta index in ES writer Message-ID: <046.759215c78861b625799e92f75485d382@tracker.bro-ids.org> #944: @bro-meta index in ES writer ---------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ The elasticsearch writer isn't creating/modifying the required (for Brownian) @bro-meta index when using the ReLog script to import old logs because rotation is disabled when importing logs. For now the right answer is to probably just leave off out the start and end fields and write to the index in the UpdateIndex method if rotation is disabled. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 7 13:05:03 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 07 Feb 2013 21:05:03 -0000 Subject: [Bro-Dev] #934: GPRS Tunneling Protocol (GTP) Analyzer In-Reply-To: <053.e35358da3dd6bfe2ce077946ca6f01bd@tracker.bro-ids.org> References: <053.e35358da3dd6bfe2ce077946ca6f01bd@tracker.bro-ids.org> Message-ID: <068.56a814295cce0cf2f70ef0aaff385146@tracker.bro-ids.org> #934: GPRS Tunneling Protocol (GTP) Analyzer ------------------------------+----------------------------------------- Reporter: liamrandall | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: GTP GPRS Tunneling Protocol ------------------------------+----------------------------------------- Comment (by jsiwek): In [changeset:26bf99c5a334c8bd23dd436aa3dcfad9c0e80535/bro]: {{{ #!CommitTicketReference repository="bro" revision="26bf99c5a334c8bd23dd436aa3dcfad9c0e80535" Add parsing for GTPv1 extension headers and control messages. Added a generic gtpv1_message event generated for any GTP message type. Added specific events for the create/update/delete PDP context request/response messages. Addresses #934. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 7 13:22:02 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 07 Feb 2013 21:22:02 -0000 Subject: [Bro-Dev] #934: GPRS Tunneling Protocol (GTP) Analyzer In-Reply-To: <053.e35358da3dd6bfe2ce077946ca6f01bd@tracker.bro-ids.org> References: <053.e35358da3dd6bfe2ce077946ca6f01bd@tracker.bro-ids.org> Message-ID: <068.28b3495d4e58f86d4e0b9ca9976d94c8@tracker.bro-ids.org> #934: GPRS Tunneling Protocol (GTP) Analyzer ------------------------------+----------------------------------------- Reporter: liamrandall | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: GTP GPRS Tunneling Protocol ------------------------------+----------------------------------------- Comment (by jsiwek): Liam, are you able to test the `topic/jsiwek/gtp-enhancements` branch to see if it fixes your issue with GTP-U decapsulation? If not, an example pcap would be helpful. I wasn't seeing anything wrong with ones you attached or other public ones I found. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 7 13:36:14 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 07 Feb 2013 21:36:14 -0000 Subject: [Bro-Dev] #934: GPRS Tunneling Protocol (GTP) Analyzer In-Reply-To: <053.e35358da3dd6bfe2ce077946ca6f01bd@tracker.bro-ids.org> References: <053.e35358da3dd6bfe2ce077946ca6f01bd@tracker.bro-ids.org> Message-ID: <068.19efdc6ba1b88a22c79151ed8504dafd@tracker.bro-ids.org> #934: GPRS Tunneling Protocol (GTP) Analyzer ------------------------------+----------------------------------------- Reporter: liamrandall | Owner: Type: Feature Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: GTP GPRS Tunneling Protocol ------------------------------+----------------------------------------- Comment (by liamrandall): Replying to [comment:5 jsiwek]: > Liam, are you able to test the `topic/jsiwek/gtp-enhancements` branch to see if it fixes your issue with GTP-U decapsulation? Yes, I will try it out tonight. If not, an example pcap would be helpful. I wasn't seeing anything wrong with ones you attached or other public ones I found. I have some pcaps that we can experiment with privately; if this doesn't fix them I will let you know. -- Ticket URL: Bro Tracker Bro Issue Tracker From jsiwek at illinois.edu Thu Feb 7 13:44:02 2013 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 7 Feb 2013 15:44:02 -0600 Subject: =?iso-8859-1?q?Re=3A_=5BBro-Dev=5D_=08Rules_for_base/?= In-Reply-To: References: Message-ID: <51142022.2020501@illinois.edu> On 2/7/2013 9:00 AM, Seth Hall wrote: > > * All consts in export sections must be tagged with &redef. > Is that actually supposed to be the thing about making options that alter script behavior be consts with &redefs as opposed to globals to prevent runtime modification? Because there's places where it makes sense for something to be const without &redef. E.g.: module Math; export { const PI = 3.14; } Scripts in base/protocols (dns/syslog/ssl/modbus/socks) have consts of that style. Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130207/559941d0/attachment.html From seth at icir.org Thu Feb 7 13:49:55 2013 From: seth at icir.org (Seth Hall) Date: Thu, 7 Feb 2013 16:49:55 -0500 Subject: =?iso-8859-1?q?Re=3A_=5BBro-Dev=5D_=08Rules_for_base/?= In-Reply-To: <51142022.2020501@illinois.edu> References: <51142022.2020501@illinois.edu> Message-ID: <01102A1B-3FBA-410F-8CCB-1EF9B8BE241D@icir.org> On Feb 7, 2013, at 4:44 PM, "Siwek, Jon" wrote: > Because there's places where it makes sense for something to be const without &redef. E.g.: > > module Math; export { const PI = 3.14; } Damn, you're right. I'll reword that a bit tonight or remove it. Thanks for looking over the list. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From noreply at bro-ids.org Fri Feb 8 00:00:02 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 8 Feb 2013 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302080800.r18802PW019681@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From vladg at cmu.edu Fri Feb 8 11:50:02 2013 From: vladg at cmu.edu (Vlad Grigorescu) Date: Fri, 8 Feb 2013 19:50:02 +0000 Subject: [Bro-Dev] Duplicate Notice Actions Regression? Message-ID: <1202BE242E080642B0CD0AD0A03E855297E6AA@PGH-MSGMB-03.andrew.ad.cmu.edu> Recently, I've been seeing Bro perform duplicate notice actions. I think this commit might have introduced a regression: > # The notice policy is completely handled by the manager and shouldn't be > # done by workers or proxies to save time for packet processing. > -event bro_init() &priority=11 > - { > - Notice::policy = table(); > - } > +redef Notice::policy = table(); Specifically, reading from the commit that fixed the duplicate notice action issue (): > The problem was that Notice::policy is used to populate the internal Notice::ordered_policy vector in a priority 10 bro_init handler (in scripts/base/frameworks/notice/main.bro) and then that is what is used when applying policy to notices. In order for scripts/base/frameworks/notice/cluster.bro to prevent Notice::policy from being used on non-manager nodes, it needs to clear it in a bro_init hander of higher priority than 10. Am I on the right track here? If not, does anyone have any other ideas of what might be causing this? Thanks, --Vlad -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130208/c3c83c38/attachment.bin From jsiwek at illinois.edu Fri Feb 8 12:29:40 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 8 Feb 2013 20:29:40 +0000 Subject: [Bro-Dev] Duplicate Notice Actions Regression? In-Reply-To: <1202BE242E080642B0CD0AD0A03E855297E6AA@PGH-MSGMB-03.andrew.ad.cmu.edu> References: <1202BE242E080642B0CD0AD0A03E855297E6AA@PGH-MSGMB-03.andrew.ad.cmu.edu> Message-ID: On Feb 8, 2013, at 1:50 PM, Vlad Grigorescu wrote: > Recently, I've been seeing Bro perform duplicate notice actions. I think this commit might have introduced a regression: > >> # The notice policy is completely handled by the manager and shouldn't be >> # done by workers or proxies to save time for packet processing. >> -event bro_init() &priority=11 >> - { >> - Notice::policy = table(); >> - } >> +redef Notice::policy = table(); I also thought that could have broken the notice de-duplication/suppression, but it seemed to work in my testing. A simple check is to do `broctl print Notice::ordered_policy`. If it's empty on all the worker nodes, but populated for the manager node, then it's still working like I expected and probably something else is wrong. > Am I on the right track here? If not, does anyone have any other ideas of what might be causing this? Are you getting 2 of the same exact email as if from both the worker and manager, or is it just that you get many emails within the suppression interval for the same "logical" notice $identifier? And is it for all notice types or just certain ones? If it's certain custom ones you're creating, can you post examples of how you call NOTICE() to generate them? Have you changed any of the "suppression_interval" settings? Jon From vladg at cmu.edu Fri Feb 8 12:40:07 2013 From: vladg at cmu.edu (Vlad Grigorescu) Date: Fri, 8 Feb 2013 20:40:07 +0000 Subject: [Bro-Dev] Duplicate Notice Actions Regression? In-Reply-To: <19737_1360355386_r18KTjnI003351_B3E27C90B7D2DD4CAEF09215A29EB0D116F1EEEA@CITESMBX2.ad.uillinois.edu> References: <1202BE242E080642B0CD0AD0A03E855297E6AA@PGH-MSGMB-03.andrew.ad.cmu.edu> <19737_1360355386_r18KTjnI003351_B3E27C90B7D2DD4CAEF09215A29EB0D116F1EEEA@CITESMBX2.ad.uillinois.edu> Message-ID: <1202BE242E080642B0CD0AD0A03E855297E8AB@PGH-MSGMB-03.andrew.ad.cmu.edu> On Feb 8, 2013, at 3:29 PM, "Siwek, Jonathan Luke" wrote: > I also thought that could have broken the notice de-duplication/suppression, but it seemed to work in my testing. A simple check is to do `broctl print Notice::ordered_policy`. If it's empty on all the worker nodes, but populated for the manager node, then it's still working like I expected and probably something else is wrong. It's populated on all the nodes. I'm not redefing Notice::emailed_types, which is what the original commit says causes this, but I am redefing Notice::mail_dest. > Are you getting 2 of the same exact email as if from both the worker and manager, or is it just that you get many emails within the suppression interval for the same "logical" notice $identifier? Same exact e-mail. > And is it for all notice types or just certain ones? If it's certain custom ones you're creating, can you post examples of how you call NOTICE() to generate them? Hmm. I believe only custom ones. I don't think I'm doing anything with the default ones, except for ACTION_LOG, which isn't duplicated. I used to use sync_functions to generate them (example here: https://gist.github.com/grigorescu/2925e938f1bcc13a1964), but I've changed to just using the notice event to see if that fixes this, e.g.: > event notice(n: Notice: Info) &priority=-5 > { > if ( ACTION_EMAIL_ISO_IR in n$actions ) > email_notice_to(n, "iso-ir at cmu.edu", T); > } > Have you changed any of the "suppression_interval" settings? Some of my notices have a non-default suppress_for interval, but I haven't change the interval globally. Thanks, --Vlad From jsiwek at illinois.edu Fri Feb 8 14:22:42 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 8 Feb 2013 22:22:42 +0000 Subject: [Bro-Dev] Duplicate Notice Actions Regression? In-Reply-To: <1202BE242E080642B0CD0AD0A03E855297E8AB@PGH-MSGMB-03.andrew.ad.cmu.edu> References: <1202BE242E080642B0CD0AD0A03E855297E6AA@PGH-MSGMB-03.andrew.ad.cmu.edu> <19737_1360355386_r18KTjnI003351_B3E27C90B7D2DD4CAEF09215A29EB0D116F1EEEA@CITESMBX2.ad.uillinois.edu> <1202BE242E080642B0CD0AD0A03E855297E8AB@PGH-MSGMB-03.andrew.ad.cmu.edu> Message-ID: >> I also thought that could have broken the notice de-duplication/suppression, but it seemed to work in my testing. A simple check is to do `broctl print Notice::ordered_policy` > > It's populated on all the nodes. Then I think that change to using redef to clear Notice::policy isn't working, probably because it depends on @load ordering -- it would always have to be the last script to be loaded that redefs Notice::policy. Seth said he'll make some changes so that the application of notice policy is better delegated to the manager in a cluster environment. Can you create a bug ticket for him? Jon From hlin33 at illinois.edu Fri Feb 8 14:37:47 2013 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Fri, 8 Feb 2013 16:37:47 -0600 Subject: [Bro-Dev] binpac usage of &length = -1 Message-ID: Hi, When I check the binpac source code related to FlowBuffer, I find this comment: // A negative frame_length represents a frame till EOF void NewFrame(int frame_length, bool chunked_); My guess that this frame_length is record attribute &length. So I played with an a record type req = record{ header : uint8; data : bytestring &restofdata; } &length = -1 ; I defined the event handler to print out header values, but I found that in .cc file, the binpac NewData is called, but nothing is printed out. So any one has idea how this &length = -1 is used? -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130208/d40b4c2e/attachment.html From robin at icir.org Fri Feb 8 15:19:22 2013 From: robin at icir.org (Robin Sommer) Date: Fri, 8 Feb 2013 15:19:22 -0800 Subject: [Bro-Dev] binpac usage of &length = -1 In-Reply-To: References: Message-ID: <20130208231922.GE74757@icir.org> On Fri, Feb 08, 2013 at 16:37 -0600, you wrote: > type req = record{ > header : uint8; > data : bytestring &restofdata; > } > &length = -1 ; So one thing is that I'm not sure the &length attribute transfers over to the NewFrame() call, don't know the internals there. Second, the &restofdata should already tell BinPAC++ that you want all the remaining data; do you find that a &length is needed in addition? (As usual, I'm arguing from the perspective here how I believe that binpac *should* work; that doesn't always match what it actually does/requires ..). Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From noreply at bro-ids.org Sat Feb 9 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 9 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302090800.r19803k8011091@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From bro at tracker.bro-ids.org Sat Feb 9 14:44:53 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sat, 09 Feb 2013 22:44:53 -0000 Subject: [Bro-Dev] #945: Fix Cluster Notice::Policy Delegation Message-ID: <052.7e2fd385c61aa2312ab00e9358e9abfe@tracker.bro-ids.org> #945: Fix Cluster Notice::Policy Delegation ------------------------+--------------------- Reporter: grigorescu | Type: Problem Status: new | Priority: Medium Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: ------------------------+--------------------- The Notice::Policy isn't clearing correctly on workers, which causes duplicate notice actions. The current system is dependent on @load ordering, and is rather fragile. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sun Feb 10 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 10 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302100800.r1A803Qf026717@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From hlin33 at illinois.edu Sun Feb 10 13:48:41 2013 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Sun, 10 Feb 2013 15:48:41 -0600 Subject: [Bro-Dev] binpac usage of &length = -1 In-Reply-To: <3b708151a4a449029a4fcd5f0478fec7@CHIHT3.ad.uillinois.edu> References: <3b708151a4a449029a4fcd5f0478fec7@CHIHT3.ad.uillinois.edu> Message-ID: On Fri, Feb 8, 2013 at 5:19 PM, Robin Sommer wrote: > > > On Fri, Feb 08, 2013 at 16:37 -0600, you wrote: > > > type req = record{ > > header : uint8; > > data : bytestring &restofdata; > > } > > &length = -1 ; > > So one thing is that I'm not sure the &length attribute transfers over > to the NewFrame() call, don't know the internals there. Second, the > &restofdata should already tell BinPAC++ that you want all the > remaining data; do you find that a &length is needed in addition? (As > usual, I'm arguing from the perspective here how I believe that binpac > *should* work; that doesn't always match what it actually > does/requires ..). > Unfortunately, yes, you needed the &length field. Otherwise: /home/****/bro/src/dnp3-protocol.pac:42: error : cannot handle incremental input Then the interesting question is when binpac can handle incremental input? > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130210/9e2257f5/attachment.html From noreply at bro-ids.org Mon Feb 11 00:00:02 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 11 Feb 2013 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302110800.r1B802YA012279@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From robin at icir.org Mon Feb 11 08:00:07 2013 From: robin at icir.org (Robin Sommer) Date: Mon, 11 Feb 2013 08:00:07 -0800 Subject: [Bro-Dev] binpac usage of &length = -1 In-Reply-To: References: <3b708151a4a449029a4fcd5f0478fec7@CHIHT3.ad.uillinois.edu> Message-ID: <20130211160007.GV2699@icir.org> On Sun, Feb 10, 2013 at 15:48 -0600, you wrote: > Unfortunately, yes, you needed the &length field. Otherwise: > > /home/****/bro/src/dnp3-protocol.pac:42: error : cannot handle incremental > input Too bad, that sounds like a case it should be able to support rather easily. > Then the interesting question is when binpac can handle incremental input? I have no idea ... It might actually be worth looking at the code and see where it stumbles. And/or try the dce_rpc approach as we discussed. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org From hugolin615 at gmail.com Mon Feb 11 19:21:38 2013 From: hugolin615 at gmail.com (Hugo) Date: Mon, 11 Feb 2013 21:21:38 -0600 Subject: [Bro-Dev] binpac usage of &length = -1 In-Reply-To: <12da22aa7b2d4613b7d9344a157a6b46@CHIHT1.ad.uillinois.edu> References: <3b708151a4a449029a4fcd5f0478fec7@CHIHT3.ad.uillinois.edu> <12da22aa7b2d4613b7d9344a157a6b46@CHIHT1.ad.uillinois.edu> Message-ID: On Mon, Feb 11, 2013 at 10:00 AM, Robin Sommer wrote: > > On Sun, Feb 10, 2013 at 15:48 -0600, you wrote: > > > Unfortunately, yes, you needed the &length field. Otherwise: > > > > /home/****/bro/src/dnp3-protocol.pac:42: error : cannot handle > incremental > > input > > Too bad, that sounds like a case it should be able to support rather > easily. > > > Then the interesting question is when binpac can handle incremental > input? > > I have no idea ... It might actually be worth looking at the code and > see where it stumbles. And/or try the dce_rpc approach as we > discussed. > Reaching a stuck point. I included some of my studies here and also my question. ********************************************************************************************** In dce_rpc, it is actually easy for his purpose. type DCE_RPC_PDU = record { # Set header's byteorder to little-endian (or big-endian) to # avoid cyclic dependency. header : DCE_RPC_Header; frag : bytestring &length = body_length; auth : DCE_RPC_Auth(header); } &let { body_length: int = header.frag_length - sizeof(header) - header.auth_length; frag_reassembled: bool = $context.flow.reassemble_fragment(frag, header.lastfrag); body: DCE_RPC_Body(header) withinput $context.flow.reassembled_body() &if frag_reassembled; } &byteorder = header.byteorder, &length = header.frag_length; # length of the PDU frag already receive byte string, so it uses a separate flow buffer to reorder or reassemble the byte string in frag. "withinput" can be understood as take the following bytes as byte string and give it to "body" to parse. ************************************************************************************************** I need to play flow_buffer_. The flow buffer object that is directly used by binpac. Let's say I have two network packets of 28 bytes. 1 2 3 ....28 29 30 ... 56 If I define a record like following to parse network packets: type req = record { header: Header; #(8 bytes) payload : uint16 (2bytes) } &length = 10 &let{ incBuff : bool = $context.flow.increaseBuffer(18); } See that I fixed the length of req record with correct length. In increaseBuffer function, I directly call GrowFrame function. This can increase flow buffer from 10 to 18. Then I can see binpac do incremental parsing. Binpac does not care at all where those bytes spanned, it just parse like 1 2 3 ....18 19 ... 36 37 ... 54. 55, 56 (probably discarded) However, the problem is that I can not read bytestring stored in the flow buffer if it is spanned in two network packets. I try to use flow_buffer_->begin and ->end to read the starting and ending point of the flow buffer. But that only works for my manually increase flow buffer within same network packet (because bytes are all ready). When flow buffer is spanned into two network packet (like, 19...36), the flow buffer still fetch data, but I can not use begin and end function, as the bytes are not ready (assert is generated). Also those bytes are not associated with any record member. So I could not catch those bytes. These bytes are what I need for parsing. So any idea to solve this? Two more study experiments are under go....To be continued. > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130211/06575bb1/attachment.html From noreply at bro-ids.org Tue Feb 12 00:00:02 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 12 Feb 2013 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302120800.r1C802aZ025476@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From noreply at bro-ids.org Wed Feb 13 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 13 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302130800.r1D803BW005882@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From noreply at bro-ids.org Thu Feb 14 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 14 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302140800.r1E803c2004751@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From bro at tracker.bro-ids.org Thu Feb 14 20:10:08 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Feb 2013 04:10:08 -0000 Subject: [Bro-Dev] #946: Async scriptland functions stack explosion Message-ID: <046.fb6c1eb0faa749f204ff3782fa29fb77@tracker.bro-ids.org> #946: Async scriptland functions stack explosion ---------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Medium | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ The example directly from the CHANGES file results in a stack explosion. I was doing some work with callbacks and realized that I needed a continuation but this is the only mechanism currently available in scriptland for doing continuations (just to point out that this is actually blocking me right now). {{{ global X: table[string] of count; function a() : count { # This delays until condition becomes true. return when ( "a" in X ) { return X["a"]; } timeout 5 min { return 0; } } event bro_init() { # Installs a trigger which fires if a() returns 42. when ( a() == 42 ) { print "Yippie!"; } X["a"] = 42; } }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Fri Feb 15 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 15 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302150800.r1F803rw007184@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From bro at tracker.bro-ids.org Fri Feb 15 10:26:33 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 15 Feb 2013 18:26:33 -0000 Subject: [Bro-Dev] #946: Async scriptland functions stack explosion In-Reply-To: <046.fb6c1eb0faa749f204ff3782fa29fb77@tracker.bro-ids.org> References: <046.fb6c1eb0faa749f204ff3782fa29fb77@tracker.bro-ids.org> Message-ID: <061.df99903ef7656209dd128a75b96e09e7@tracker.bro-ids.org> #946: Async scriptland functions stack explosion ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Medium | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): This patch may work around the issue: {{{ diff --git a/scripts/base/frameworks/notice/actions/drop.bro b/scripts/base/fram index 0116dd4..0a6ee7e 100644 --- a/scripts/base/frameworks/notice/actions/drop.bro +++ b/scripts/base/frameworks/notice/actions/drop.bro @@ -17,20 +17,19 @@ export { }; } +function drop_func(n: Notice::Info) + { + if ( ACTION_DROP !in n$actions ) return; + + #local drop = React::drop_address(n$src, ""); + #local addl = drop?$sub ? fmt(" %s", drop$sub) : ""; + #n$dropped = drop$note != Drop::AddressDropIgnored; + #n$msg += fmt(" [%s%s]", drop$note, addl); + } + # This is a little awkward because we want to inject drop along with the # synchronous functions. event bro_init() { - local drop_func = function(n: Notice::Info) - { - if ( ACTION_DROP in n$actions ) - { - #local drop = React::drop_address(n$src, ""); - #local addl = drop?$sub ? fmt(" %s", drop$sub) : ""; - #n$dropped = drop$note != Drop::AddressDropIgnored; - #n$msg += fmt(" [%s%s]", drop$note, addl); - } - }; - add Notice::sync_functions[drop_func]; } }}} I don't know the intention of the original code, but I think that refactor is equivalent in that it still does nothing... As for the reason why that fixes the unrelated example code... it goes something like: 1) Event (or hook) handlers share a frame. 2) The bro_init handler in the example script has a when block which causes its frame to be cloned. 3) The cloning of the bro_init handler's frame starts serializing the value of the local drop_func variable which is an anonymous-function 4) That serialization never ends, looks like maybe an infinite loop of "serialize anonymous-function ID" ->"serialize function value bound to ID" -> "serialize ID of function". Haven't played with fixes for that last step yet, but maybe the workaround is enough to unblock you. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Feb 16 00:00:01 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 16 Feb 2013 00:00:01 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302160800.r1G801Sa031103@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From noreply at bro-ids.org Sun Feb 17 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 17 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302170800.r1H803oB002174@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From abenson at gmail.com Fri Feb 15 10:49:26 2013 From: abenson at gmail.com (Andrew Benson) Date: Fri, 15 Feb 2013 10:49:26 -0800 Subject: [Bro-Dev] Error with Multiple_Sig_Responders Message-ID: We noticed that one of our hosts was causing this error to be thrown 1360776820.430495 Reporter::ERROR non-optional field "ts" missing in initialization ([note=Signatures::Multiple_Sig_Responders, src_addr=, sig_id=, event_msg=, host_count=5, sub_msg= has triggered signature on 5 hosts]) I looked into it, and it looks to me like Multiple_Sig_Responders is in fact missing that field. --- a/scripts/base/frameworks/signatures/main.bro +++ b/scripts/base/frameworks/signatures/main.bro @@ -270,7 +270,7 @@ orig, sig_id, hcount); Log::write(Signatures::LOG, - [$note=Multiple_Sig_Responders, + [$ts=network_time(), $note=Multiple_Sig_Responders, $src_addr=orig, $sig_id=sig_id, $event_msg=msg, $host_count=hcount, $sub_msg=horz_scan_msg]); -- AndrewB Knowing is Half the Battle. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130215/8d28a63d/attachment.html From noreply at bro-ids.org Mon Feb 18 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 18 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302180800.r1I803Qp025690@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From noreply at bro-ids.org Tue Feb 19 00:00:05 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 19 Feb 2013 00:00:05 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302190800.r1J805BF023743@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 938 [1] | robin | robin | Normal | topic/seth/software-version-updates2 Updates to vulnerable software checking. [2] [1] #938: http://tracker.bro-ids.org/bro/ticket/938 [2] software-version-updates2: http://tracker.bro-ids.org/bro/changeset?old_path=%2Fbro&old=master&new_path=%2Fbro&new=topic/seth/software-version-updates2 From bro at tracker.bro-ids.org Tue Feb 19 07:22:40 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 19 Feb 2013 15:22:40 -0000 Subject: [Bro-Dev] #938: topic/seth/software-version-updates2 Updates to vulnerable software checking. In-Reply-To: <047.618e6bbadcf6aef8dacc8730c2c4b08a@tracker.bro-ids.org> References: <047.618e6bbadcf6aef8dacc8730c2c4b08a@tracker.bro-ids.org> Message-ID: <062.9ae7f85dd6d7e1ffde6c57b047f65fb5@tracker.bro-ids.org> #938: topic/seth/software-version-updates2 Updates to vulnerable software checking. ---------------------+------------------------ Reporter: robin | Owner: seth Type: Task | Status: assigned Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ---------------------+------------------------ Changes (by robin): * owner: robin => seth * type: Merge Request => Task Comment: External tests don't pass yet, removing merge request for the time being. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Feb 19 10:13:40 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 19 Feb 2013 18:13:40 -0000 Subject: [Bro-Dev] #946: Async scriptland functions stack explosion In-Reply-To: <046.fb6c1eb0faa749f204ff3782fa29fb77@tracker.bro-ids.org> References: <046.fb6c1eb0faa749f204ff3782fa29fb77@tracker.bro-ids.org> Message-ID: <061.7306b312f92398230642f6ebf0f2d519@tracker.bro-ids.org> #946: Async scriptland functions stack explosion ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Medium | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [changeset:7e5115460c2da630825b7ef994591f95aef4f4dc/bro]: {{{ #!CommitTicketReference repository="bro" revision="7e5115460c2da630825b7ef994591f95aef4f4dc" Fix three bugs with 'when' and 'return when' statements. Addresses #946 - 'when' statements were problematic when used in a function/event/hook that had local variables with an assigned function value. This was because 'when' blocks operate on a clone of the frame and the cloning process serializes locals and the serialization of functions had an infinite cycle in it (ID -> BroFunc -> ID -> BroFunc ...). The ID was only used for the function name and type information, so refactoring Func and subclasses to depend on those two things instead fixes the issue. - 'return when' blocks, specifically, didn't work whenever execution of the containing function's body does another function call before reaching the 'return when' block, because of an assertion. This was was due to logic in CallExpr::Eval always clearing the CallExpr associated with the Frame after doing the call, instead of restoring any previous CallExpr, which the code in Trigger::Eval expected to have available. - An assert could be reached when the condition of a 'when' statement depended on checking the value of global state variables. The assert in Trigger::QueueTrigger that checks that the Trigger isn't disabled would get hit because Trigger::Eval/Timeout disable themselves after running, but don't unregister themselves from the NotifierRegistry, which keeps calling QueueTrigger for every state access of the global. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Feb 19 14:19:38 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 19 Feb 2013 22:19:38 -0000 Subject: [Bro-Dev] #946: Async scriptland functions stack explosion In-Reply-To: <046.fb6c1eb0faa749f204ff3782fa29fb77@tracker.bro-ids.org> References: <046.fb6c1eb0faa749f204ff3782fa29fb77@tracker.bro-ids.org> Message-ID: <061.d3e89976e444c2c52987c5eef76cc652@tracker.bro-ids.org> #946: Async scriptland functions stack explosion ----------------------+------------------------ Reporter: seth | Owner: Type: Problem | Status: new Priority: Medium | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------+------------------------ Comment (by jsiwek): In [changeset:d158c7ffdfed8ac58e107efaba660a1d541cb7d0/bro]: {{{ #!CommitTicketReference repository="bro" revision="d158c7ffdfed8ac58e107efaba660a1d541cb7d0" Fix memory leaks resulting from 'when' and 'return when' statements. Addresses #946. }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Feb 19 14:23:10 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 19 Feb 2013 22:23:10 -0000 Subject: [Bro-Dev] #946: Async scriptland functions stack explosion In-Reply-To: <046.fb6c1eb0faa749f204ff3782fa29fb77@tracker.bro-ids.org> References: <046.fb6c1eb0faa749f204ff3782fa29fb77@tracker.bro-ids.org> Message-ID: <061.e1d896db2139bd2feb1a20cf63a90d9d@tracker.bro-ids.org> #946: Async scriptland functions stack explosion ----------------------------+------------------------ Reporter: seth | Owner: Type: Merge Request | Status: new Priority: Medium | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: ----------------------------+------------------------ Changes (by jsiwek): * type: Problem => Merge Request Comment: Fixes for the issue in the ticket description as well as others are in `topic/jsiwek/ticket946`. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Feb 19 14:39:21 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 19 Feb 2013 22:39:21 -0000 Subject: [Bro-Dev] #947: Incorrect size calculation for SSH failed/successful heuristic Message-ID: <052.ac2c8b74ac05bb424b2a165a70b9355e@tracker.bro-ids.org> #947: Incorrect size calculation for SSH failed/successful heuristic ------------------------+--------------------- Reporter: grigorescu | Type: Problem Status: new | Priority: Low Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: ------------------------+--------------------- We're getting a lot of false positives for successful SSH logins from a source that we recently blackholed. I suspect what's happening is that the retransmissions keep bumping up the size of the connection, until it crosses the threshold for a "successful" connection. With the changes from #730: Find and fix tcp sequence counting bugs, is it possible to improve the accuracy of the reported size? -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Feb 20 00:00:02 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 20 Feb 2013 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302200800.r1K802Ab004888@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 946 [1] | seth | | Medium | Async scriptland functions stack explosion [1] #946: http://tracker.bro-ids.org/bro/ticket/946 From bro at tracker.bro-ids.org Wed Feb 20 19:59:48 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 21 Feb 2013 03:59:48 -0000 Subject: [Bro-Dev] #947: Incorrect size calculation for SSH failed/successful heuristic In-Reply-To: <052.ac2c8b74ac05bb424b2a165a70b9355e@tracker.bro-ids.org> References: <052.ac2c8b74ac05bb424b2a165a70b9355e@tracker.bro-ids.org> Message-ID: <067.fafee3e5b4486f3886f3f411e015383e@tracker.bro-ids.org> #947: Incorrect size calculation for SSH failed/successful heuristic -------------------------+------------------------ Reporter: grigorescu | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by vern): Retransmissions shouldn't affect connection size, which is computed in terms of the TCP bytestream rather than anything from packet volume. It would be great to have a trace llustrating the problem. With luck this should be pretty much non-sensitive since the logins themselves will all be part of the encrypted payload. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Wed Feb 20 20:05:07 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 21 Feb 2013 04:05:07 -0000 Subject: [Bro-Dev] #947: Incorrect size calculation for SSH failed/successful heuristic In-Reply-To: <052.ac2c8b74ac05bb424b2a165a70b9355e@tracker.bro-ids.org> References: <052.ac2c8b74ac05bb424b2a165a70b9355e@tracker.bro-ids.org> Message-ID: <067.8d281b072120f60bbd1570c7623f4d7b@tracker.bro-ids.org> #947: Incorrect size calculation for SSH failed/successful heuristic -------------------------+------------------------ Reporter: grigorescu | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by seth): There is a trace attached to the fixed ticket that Vlad referenced. Bro was picking the wrong TCP sequence ID to follow in some cases. For the trace attached to the ticket, there is a middle box that sends a RST to kill the connection. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Thu Feb 21 00:00:02 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 21 Feb 2013 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302210800.r1L802RS022168@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 946 [1] | seth | | Medium | Async scriptland functions stack explosion [1] #946: http://tracker.bro-ids.org/bro/ticket/946 From luoshoufu at gmail.com Thu Feb 21 10:22:14 2013 From: luoshoufu at gmail.com (Sean Shoufu Luo) Date: Thu, 21 Feb 2013 13:22:14 -0500 Subject: [Bro-Dev] Compile Bro in OpenBSD 5.2 Message-ID: Hi all, I'm trying to compile Bro in OpenBSD 5.2. Unfortunately, several errors stop me. The following are logs. Anyone can give some quick advise? I appreciate your comments. Thanks, Shoufu -------------- /usr/include/sys/socket.h:162: error: expected specifier-qualifier-list before 'u_int8_t' /usr/include/sys/socket.h:462: error: expected declaration specifiers or '...' before 'socklen_t' /usr/include/sys/socket.h:470: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'recv' /usr/include/sys/socket.h:477: error: expected declaration specifiers or '...' before 'socklen_t' In file included from /home/sluo/bro/bro/src/bro_inet_ntop.c:19: {home}/bro/bro/src/bro_inet_ntop.h:10: error: expected declaration specifiers or '...' before 'socklen_t' {home}/bro/bro/src/bro_inet_ntop.c:52: error: conflicting types for 'bro_inet_ntop' {home}/bro/bro/src/bro_inet_ntop.h:10: error: previous declaration of 'bro_inet_ntop' was here *** Error code 1 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130221/f5a1776d/attachment.html From bro at tracker.bro-ids.org Thu Feb 21 10:52:32 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 21 Feb 2013 18:52:32 -0000 Subject: [Bro-Dev] #947: Incorrect size calculation for SSH failed/successful heuristic In-Reply-To: <052.ac2c8b74ac05bb424b2a165a70b9355e@tracker.bro-ids.org> References: <052.ac2c8b74ac05bb424b2a165a70b9355e@tracker.bro-ids.org> Message-ID: <067.6d5160c9f8792fbb09a767549538bc68@tracker.bro-ids.org> #947: Incorrect size calculation for SSH failed/successful heuristic -------------------------+------------------------ Reporter: grigorescu | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by jsiwek): Replying to [comment:2 seth]: > There is a trace attached to the fixed ticket that Vlad referenced. Bro was picking the wrong TCP sequence ID to follow in some cases. For the trace attached to the ticket, there is a middle box that sends a RST to kill the connection. But that doesn't demonstrate Vlad's current issue, so an example pcap would still help. The heuristic for SSH login looks like it primarily uses the packet-wise size calculations from conn_size_analyzer, but only falls back on the TCP stream size based on sequence numbers if there looked like there was something wonky with the packet-wise size (for which I see a TODO comment about that being fragile in some cases like IPv6). So making the choice of heuristic more flexible/user-controllable might help. E.g. maybe refactoring it to use ConnPolling stuff would work, and you could provide some standard/default callbacks that demonstrate checking by packet-wise size versus TCP stream-wise size. (And make the default be a stream-wise size check if #730 was the only reason why it wasn't in the first place). -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 21 11:10:06 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 21 Feb 2013 19:10:06 -0000 Subject: [Bro-Dev] #947: Incorrect size calculation for SSH failed/successful heuristic In-Reply-To: <052.ac2c8b74ac05bb424b2a165a70b9355e@tracker.bro-ids.org> References: <052.ac2c8b74ac05bb424b2a165a70b9355e@tracker.bro-ids.org> Message-ID: <067.909bd1760762404b29c1dcb0841db6b1@tracker.bro-ids.org> #947: Incorrect size calculation for SSH failed/successful heuristic -------------------------+------------------------ Reporter: grigorescu | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: -------------------------+------------------------ Comment (by seth): > The heuristic for SSH login looks like it primarily uses the packet-wise > size calculations from conn_size_analyzer, but only falls back on the TCP > stream size based on sequence numbers if there looked like there was > something wonky with the packet-wise size (for which I see a TODO comment > about that being fragile in some cases like IPv6). Ah, the problem is the conn_size_analyzer counts retransmissions and includes the size of the ip and tcp headers. I only want the payload size. The current implementation is horribly brain-dead. > So making the choice of heuristic more flexible/user-controllable might > help. E.g. maybe refactoring it to use ConnPolling stuff would work, and > you could provide some standard/default callbacks that demonstrate > checking by packet-wise size versus TCP stream-wise size. (And make the > default be a stream-wise size check if #730 was the only reason why it > wasn't in the first place). You got it right on all accounts. -- Ticket URL: Bro Tracker Bro Issue Tracker From jsiwek at illinois.edu Thu Feb 21 11:31:02 2013 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 21 Feb 2013 13:31:02 -0600 Subject: [Bro-Dev] Compile Bro in OpenBSD 5.2 In-Reply-To: References: Message-ID: <512675F6.8020001@illinois.edu> On 2/21/2013 12:22 PM, Sean Shoufu Luo wrote: > > I'm trying to compile Bro in OpenBSD 5.2. Unfortunately, several > errors stop me. The following are logs. Anyone can give some quick > advise? I appreciate your comments. is supposed to be included before , and I think OpenBSD is being sensitive about places that don't do that in Bro's code. Could you try the patch below and let me know if it helps? diff --git a/src/bro_inet_ntop.h b/src/bro_inet_ntop.h index 00326b0..c018403 100644 --- a/src/bro_inet_ntop.h +++ b/src/bro_inet_ntop.h @@ -5,6 +5,7 @@ extern "C" { #endif +#include #include const char * diff --git a/src/threading/SerialTypes.h b/src/threading/SerialTypes.h index 60aee24..f4f0bc0 100644 --- a/src/threading/SerialTypes.h +++ b/src/threading/SerialTypes.h @@ -2,6 +2,7 @@ #ifndef THREADING_SERIALIZATIONTYPES_H #define THREADING_SERIALIZATIONTYPES_H +#include #include #include #include From luoshoufu at gmail.com Thu Feb 21 12:20:29 2013 From: luoshoufu at gmail.com (Sean Shoufu Luo) Date: Thu, 21 Feb 2013 15:20:29 -0500 Subject: [Bro-Dev] Compile Bro in OpenBSD 5.2 In-Reply-To: <512675F6.8020001@illinois.edu> References: <512675F6.8020001@illinois.edu> Message-ID: Yes, it works! Thank you, Jon! Thanks, Shoufu On Thu, Feb 21, 2013 at 2:31 PM, Siwek, Jon wrote: > On 2/21/2013 12:22 PM, Sean Shoufu Luo wrote: > >> >> I'm trying to compile Bro in OpenBSD 5.2. Unfortunately, several errors >> stop me. The following are logs. Anyone can give some quick advise? I >> appreciate your comments. >> > is supposed to be included before , and I > think OpenBSD is being sensitive about places that don't do that in Bro's > code. Could you try the patch below and let me know if it helps? > > diff --git a/src/bro_inet_ntop.h b/src/bro_inet_ntop.h > index 00326b0..c018403 100644 > --- a/src/bro_inet_ntop.h > +++ b/src/bro_inet_ntop.h > @@ -5,6 +5,7 @@ > extern "C" { > #endif > > +#include > #include > > const char * > diff --git a/src/threading/SerialTypes.h b/src/threading/SerialTypes.h > index 60aee24..f4f0bc0 100644 > --- a/src/threading/SerialTypes.h > +++ b/src/threading/SerialTypes.h > @@ -2,6 +2,7 @@ > #ifndef THREADING_SERIALIZATIONTYPES_H > #define THREADING_SERIALIZATIONTYPES_H > > +#include > #include > #include > #include > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20130221/1a425787/attachment.html From bro at tracker.bro-ids.org Thu Feb 21 13:18:24 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 21 Feb 2013 21:18:24 -0000 Subject: [Bro-Dev] #934: GPRS Tunneling Protocol (GTP) Analyzer In-Reply-To: <053.e35358da3dd6bfe2ce077946ca6f01bd@tracker.bro-ids.org> References: <053.e35358da3dd6bfe2ce077946ca6f01bd@tracker.bro-ids.org> Message-ID: <068.ec47625eaf77e98d749852b4398200ef@tracker.bro-ids.org> #934: GPRS Tunneling Protocol (GTP) Analyzer ----------------------------+----------------------------------------- Reporter: liamrandall | Owner: Type: Merge Request | Status: new Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: GTP GPRS Tunneling Protocol ----------------------------+----------------------------------------- Changes (by jsiwek): * type: Feature Request => Merge Request Comment: `topic/jsiwek/gtp-enhancements` is still the latest additions that seem ok to merge. -- Ticket URL: Bro Tracker Bro Issue Tracker From seth at icir.org Thu Feb 21 23:32:14 2013 From: seth at icir.org (Seth Hall) Date: Fri, 22 Feb 2013 02:32:14 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/seth/smb-smb2-work's head updated: Fixed some small issues to adapt to Bro updates after merge. (886ed46) In-Reply-To: <201302220728.r1M7SJiI013879@bro-ids.icir.org> References: <201302220728.r1M7SJiI013879@bro-ids.icir.org> Message-ID: On Feb 22, 2013, at 2:28 AM, Seth Hall wrote: > Branch 'topic/seth/smb-smb2-work' now includes: If anyone's interested in playing around, this branch works now. It can do some form of logging (take a look and let me know what you think!) and it supports file identification and extraction for the ANDX read and write commands (configurable with options in the base/protocols/smb/file-extract.bro script). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From seth at icir.org Thu Feb 21 23:33:37 2013 From: seth at icir.org (Seth Hall) Date: Fri, 22 Feb 2013 02:33:37 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/seth/smb-smb2-work's head updated: Fixed some small issues to adapt to Bro updates after merge. (886ed46) In-Reply-To: References: <201302220728.r1M7SJiI013879@bro-ids.icir.org> Message-ID: <4320E1E3-3C3C-4C26-B5DF-46874892732D@icir.org> On Feb 22, 2013, at 2:32 AM, Seth Hall wrote: > and it supports file identification and extraction Oh, and MD5 hashing. It looks like it's hashing all files by default right now. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro-ids.org/ From noreply at bro-ids.org Fri Feb 22 00:00:02 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Fri, 22 Feb 2013 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302220800.r1M802Ad000625@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 934 [1] | liamrandall | | Normal | GPRS Tunneling Protocol (GTP) Analyzer Bro | 946 [2] | seth | | Medium | Async scriptland functions stack explosion [1] #934: http://tracker.bro-ids.org/bro/ticket/934 [2] #946: http://tracker.bro-ids.org/bro/ticket/946 From bro at tracker.bro-ids.org Fri Feb 22 00:05:22 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 22 Feb 2013 08:05:22 -0000 Subject: [Bro-Dev] #935: Enhance logging framework with a delay mechanism In-Reply-To: <050.6cfff68c7c77ec0180294b572ee6681c@tracker.bro-ids.org> References: <050.6cfff68c7c77ec0180294b572ee6681c@tracker.bro-ids.org> Message-ID: <065.2253053f4065739244b0fa6d8ce089f4@tracker.bro-ids.org> #935: Enhance logging framework with a delay mechanism -----------------------+------------------------ Reporter: matthias | Owner: seth Type: Task | Status: closed Priority: Normal | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: Invalid | Keywords: -----------------------+------------------------ Changes (by seth): * status: new => closed * resolution: => Invalid Comment: I accidentally refiled this iticket as #942 but it has more information so I'm closing this one. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Sat Feb 23 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Sat, 23 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302230800.r1N8037M003197@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 934 [1] | liamrandall | | Normal | GPRS Tunneling Protocol (GTP) Analyzer Bro | 946 [2] | seth | | Medium | Async scriptland functions stack explosion > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | dd9f361 | Jon Siwek | 2013-02-22 | Fix build on OpenBSD 5.2. [3] [1] #934: http://tracker.bro-ids.org/bro/ticket/934 [2] #946: http://tracker.bro-ids.org/bro/ticket/946 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/dd9f361bc739f5aa4bc11f70569499e9115d0d50/bro From noreply at bro-ids.org Sun Feb 24 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Sun, 24 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302240800.r1O803RN009577@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 934 [1] | liamrandall | | Normal | GPRS Tunneling Protocol (GTP) Analyzer Bro | 946 [2] | seth | | Medium | Async scriptland functions stack explosion > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | dd9f361 | Jon Siwek | 2013-02-22 | Fix build on OpenBSD 5.2. [3] [1] #934: http://tracker.bro-ids.org/bro/ticket/934 [2] #946: http://tracker.bro-ids.org/bro/ticket/946 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/dd9f361bc739f5aa4bc11f70569499e9115d0d50/bro From bro at tracker.bro-ids.org Sun Feb 24 15:52:31 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Sun, 24 Feb 2013 23:52:31 -0000 Subject: [Bro-Dev] #948: add bif for URI -> binary decoding Message-ID: <051.4d28e54091640b9950b95d2bc7babbc1@tracker.bro-ids.org> #948: add bif for URI -> binary decoding ------------------------+----------------------------- Reporter: scampbell | Type: Feature Request Status: new | Priority: Low Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: ------------------------+----------------------------- The current URI_decode() bif returns non-ascii data in a x\nn format which is safe, but not useful in all situations (such as when you need the literal binary data). thanks! scott -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Mon Feb 25 00:00:05 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Mon, 25 Feb 2013 00:00:05 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302250800.r1P805Qf007062@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 934 [1] | liamrandall | | Normal | GPRS Tunneling Protocol (GTP) Analyzer Bro | 946 [2] | seth | | Medium | Async scriptland functions stack explosion > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | dd9f361 | Jon Siwek | 2013-02-22 | Fix build on OpenBSD 5.2. [3] [1] #934: http://tracker.bro-ids.org/bro/ticket/934 [2] #946: http://tracker.bro-ids.org/bro/ticket/946 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/dd9f361bc739f5aa4bc11f70569499e9115d0d50/bro From bro at tracker.bro-ids.org Mon Feb 25 07:01:28 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 25 Feb 2013 15:01:28 -0000 Subject: [Bro-Dev] #949: High CPU from polling loop on low traffic links Message-ID: <053.0bdaeaf204bc1a075b5e23a5d9c4cc09@tracker.bro-ids.org> #949: High CPU from polling loop on low traffic links -------------------------+--------------------- Reporter: liamrandall | Type: Problem Status: new | Priority: Low Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: -------------------------+--------------------- In low or no traffic links Bro consumes a large amount of the CPU. Bro has a core processing loop that it needs to do regularly which involves checking for packets from interfaces, checking for communication traffic (in the case of a cluster of Bro processes), processing scheduled events, etc. It's this processing loop that is causing the CPU utilization when there is no traffic. Moving Bro to standalone mode will reduce the CPU load on sensors with out traffic. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Mon Feb 25 07:25:37 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Mon, 25 Feb 2013 15:25:37 -0000 Subject: [Bro-Dev] #949: High CPU from polling loop on low traffic links In-Reply-To: <053.0bdaeaf204bc1a075b5e23a5d9c4cc09@tracker.bro-ids.org> References: <053.0bdaeaf204bc1a075b5e23a5d9c4cc09@tracker.bro-ids.org> Message-ID: <068.f9a7bc135d1369b28084502a0a43cc1c@tracker.bro-ids.org> #949: High CPU from polling loop on low traffic links --------------------------+------------------------ Reporter: liamrandall | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: --------------------------+------------------------ Comment (by liamrandall): To change Bro to "Standalone Mode" on SecurityOnion /opt/bro/etc/node.cfg Backup and replace with: {{{ [bro] type=standalone host=localhost interface=eth0 }}} Where eth0 is interface to monitor, then: {{{ sudo broctl check sudo broctl install sudo broctl restart }}} -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Tue Feb 26 00:00:03 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Tue, 26 Feb 2013 00:00:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302260800.r1Q803fj006647@bro-ids.icir.org> > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | dd9f361 | Jon Siwek | 2013-02-22 | Fix build on OpenBSD 5.2. [1] [1] fastpath: http://tracker.bro-ids.org/bro/changeset/dd9f361bc739f5aa4bc11f70569499e9115d0d50/bro From bro at tracker.bro-ids.org Tue Feb 26 09:57:10 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 26 Feb 2013 17:57:10 -0000 Subject: [Bro-Dev] #949: High CPU from polling loop on low traffic links In-Reply-To: <053.0bdaeaf204bc1a075b5e23a5d9c4cc09@tracker.bro-ids.org> References: <053.0bdaeaf204bc1a075b5e23a5d9c4cc09@tracker.bro-ids.org> Message-ID: <068.bc352b1b55a9af3e68e9cdd3081f60ed@tracker.bro-ids.org> #949: High CPU from polling loop on low traffic links --------------------------+------------------------ Reporter: liamrandall | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: --------------------------+------------------------ Comment (by james.lay): Ironically this is how I have mine set up already, carbon copy of the above. Just tested and still maxed out CPU. I've tested with downloading a file as well over the link, still 100% CPU. What next? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Tue Feb 26 15:18:19 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Tue, 26 Feb 2013 23:18:19 -0000 Subject: [Bro-Dev] #950: Add client/server random to SSL hello events Message-ID: <047.04f49e4ee2be1fa6348edecfe5ba9c89@tracker.bro-ids.org> #950: Add client/server random to SSL hello events ------------------------+------------------- Reporter: ewust | Type: Patch Status: new | Priority: Low Milestone: Bro2.2 | Component: Bro Version: git/master | Keywords: ------------------------+------------------- ssl_client_hello and ssl_server_hello should provide applications with the nonces (client/server random) in the SSL hello messages. This can be used for steganographic applications, or can be used to detect entropy problems. -- Ticket URL: Bro Tracker Bro Issue Tracker From noreply at bro-ids.org Wed Feb 27 00:00:10 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Wed, 27 Feb 2013 00:00:10 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302270800.r1R80Ame031480@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 934 [1] | liamrandall | | Normal | GPRS Tunneling Protocol (GTP) Analyzer Bro | 946 [2] | seth | | Medium | Async scriptland functions stack explosion > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | dd9f361 | Jon Siwek | 2013-02-22 | Fix build on OpenBSD 5.2. [3] [1] #934: http://tracker.bro-ids.org/bro/ticket/934 [2] #946: http://tracker.bro-ids.org/bro/ticket/946 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/dd9f361bc739f5aa4bc11f70569499e9115d0d50/bro From noreply at bro-ids.org Thu Feb 28 00:00:02 2013 From: noreply at bro-ids.org (Merge Tracker) Date: Thu, 28 Feb 2013 00:00:02 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201302280800.r1S802JS027472@bro-ids.icir.org> > Open Merge Requests for Bro2.2 > ============================== Component | Id | Reporter | Owner | Prio | Summary ------------------------------------------------------------------------------------------------------------------ Bro | 934 [1] | liamrandall | | Normal | GPRS Tunneling Protocol (GTP) Analyzer Bro | 946 [2] | seth | | Medium | Async scriptland functions stack explosion > Unmerged Fastpath Commits > ========================= Component | Revision | Committer | Date | Summary ------------------------------------------------------------------------------------------------------------------ bro | 2481f9f | Jon Siwek | 2013-02-27 | Fix possible null pointer dereference in identify_data BIF. [3] bro | dd9f361 | Jon Siwek | 2013-02-22 | Fix build on OpenBSD 5.2. [4] [1] #934: http://tracker.bro-ids.org/bro/ticket/934 [2] #946: http://tracker.bro-ids.org/bro/ticket/946 [3] fastpath: http://tracker.bro-ids.org/bro/changeset/2481f9f83772a4e934f72c1bf9ac35fd0ea7c096/bro [4] fastpath: http://tracker.bro-ids.org/bro/changeset/dd9f361bc739f5aa4bc11f70569499e9115d0d50/bro From bro at tracker.bro-ids.org Thu Feb 28 07:26:07 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 28 Feb 2013 15:26:07 -0000 Subject: [Bro-Dev] #949: High CPU from polling loop on low traffic links In-Reply-To: <053.0bdaeaf204bc1a075b5e23a5d9c4cc09@tracker.bro-ids.org> References: <053.0bdaeaf204bc1a075b5e23a5d9c4cc09@tracker.bro-ids.org> Message-ID: <068.efb64c6f6a4a66d814138ec65dcf7cb6@tracker.bro-ids.org> #949: High CPU from polling loop on low traffic links --------------------------+------------------------ Reporter: liamrandall | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: --------------------------+------------------------ Comment (by james.lay): Any movement on this? I'm unable to proceed forward until this issue is addressed. -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 28 08:04:24 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Thu, 28 Feb 2013 16:04:24 -0000 Subject: [Bro-Dev] #949: High CPU from polling loop on low traffic links In-Reply-To: <053.0bdaeaf204bc1a075b5e23a5d9c4cc09@tracker.bro-ids.org> References: <053.0bdaeaf204bc1a075b5e23a5d9c4cc09@tracker.bro-ids.org> Message-ID: <068.fa82efa789e6b405b7ff7bbd4000380d@tracker.bro-ids.org> #949: High CPU from polling loop on low traffic links --------------------------+------------------------ Reporter: liamrandall | Owner: Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Resolution: | Keywords: --------------------------+------------------------ Comment (by seth): > Any movement on this? I'm unable to proceed forward until this issue is > addressed. No, this is not something we are addressing at the moment with our current resources. Why is this a blocking issue for what you are trying to do? -- Ticket URL: Bro Tracker Bro Issue Tracker From bro at tracker.bro-ids.org Thu Feb 28 18:38:36 2013 From: bro at tracker.bro-ids.org (Bro Tracker) Date: Fri, 01 Mar 2013 02:38:36 -0000 Subject: [Bro-Dev] #951: Error message from SSL delay logging Message-ID: <046.97667d3d68f5ffdd31e369b5847affe0@tracker.bro-ids.org> #951: Error message from SSL delay logging ---------------------+------------------------ Reporter: seth | Owner: seth Type: Problem | Status: new Priority: Low | Milestone: Bro2.2 Component: Bro | Version: git/master Keywords: | ---------------------+------------------------ This is mostly a reminder for me.. {{{ 1362104865.826919 ./scripts/base/protocols/ssl/./main.bro, lines 193-194: SSL delay tokens not released in time ({ notary }) }}} I got that message in my console when I ran Bro with the local.bro script. I think that we should just go ahead and log the line without waiting if Bro is terminating. I'd rather get the log line even without any extended information and just skip the error message. -- Ticket URL: Bro Tracker Bro Issue Tracker