From noreply at bro.org Sat Feb 1 00:00:12 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 1 Feb 2014 00:00:12 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402010800.s1180Cxd013898@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- --------------------------------- BIT-1122 [1] Bro Jon Siwek Seth Hall 2014-01-30 2.3 Normal topic/jsiwek/dns-improvements [2] BIT-1119 [3] Bro Jon Siwek - 2014-01-31 2.3 Normal topic/jsiwek/tcp-improvements [4] [1] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [2] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements [3] BIT-1119 https://bro-tracker.atlassian.net/browse/BIT-1119 [4] tcp-improvements https://github.com/bro/bro/tree/topic/jsiwek/tcp-improvements From noreply at bro.org Sun Feb 2 00:00:15 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 2 Feb 2014 00:00:15 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402020800.s1280FH4030051@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- --------------------------------- BIT-1122 [1] Bro Jon Siwek Seth Hall 2014-01-30 2.3 Normal topic/jsiwek/dns-improvements [2] BIT-1119 [3] Bro Jon Siwek - 2014-01-31 2.3 Normal topic/jsiwek/tcp-improvements [4] [1] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [2] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements [3] BIT-1119 https://bro-tracker.atlassian.net/browse/BIT-1119 [4] tcp-improvements https://github.com/bro/bro/tree/topic/jsiwek/tcp-improvements From noreply at bro.org Mon Feb 3 00:00:14 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 3 Feb 2014 00:00:14 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402030800.s1380Ev2005629@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- --------------------------------- BIT-1122 [1] Bro Jon Siwek Seth Hall 2014-01-30 2.3 Normal topic/jsiwek/dns-improvements [2] BIT-1119 [3] Bro Jon Siwek - 2014-01-31 2.3 Normal topic/jsiwek/tcp-improvements [4] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- -------------------------------- 688df8a [5] broctl Daniel Thayer 2014-02-02 Fix a few sporadic test failures [1] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [2] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements [3] BIT-1119 https://bro-tracker.atlassian.net/browse/BIT-1119 [4] tcp-improvements https://github.com/bro/bro/tree/topic/jsiwek/tcp-improvements [5] 688df8a https://github.com/bro/broctl/commit/688df8a84e3e096d0f4aa5f6a3cfaa769b6fff28 From noreply at bro.org Tue Feb 4 00:00:18 2014 From: noreply at bro.org (Merge Tracker) Date: Tue, 4 Feb 2014 00:00:18 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402040800.s1480IDP022390@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- --------------------------------- BIT-1122 [1] Bro Jon Siwek Seth Hall 2014-01-30 2.3 Normal topic/jsiwek/dns-improvements [2] BIT-1119 [3] Bro Jon Siwek - 2014-01-31 2.3 Normal topic/jsiwek/tcp-improvements [4] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ---------------------------------------- ab45084 [5] bro Jon Siwek 2014-02-03 Minor unified2 script documentation fix. 688df8a [6] broctl Daniel Thayer 2014-02-02 Fix a few sporadic test failures [1] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [2] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements [3] BIT-1119 https://bro-tracker.atlassian.net/browse/BIT-1119 [4] tcp-improvements https://github.com/bro/bro/tree/topic/jsiwek/tcp-improvements [5] ab45084 https://github.com/bro/bro/commit/ab4508486e2337ba9eccf123343f6133530d6813 [6] 688df8a https://github.com/bro/broctl/commit/688df8a84e3e096d0f4aa5f6a3cfaa769b6fff28 From gc355804 at ohio.edu Tue Feb 4 10:28:50 2014 From: gc355804 at ohio.edu (Gilbert Clark) Date: Tue, 04 Feb 2014 13:28:50 -0500 Subject: [Bro-Dev] Test timing measurements (Re: [Bro-Commits] [git/btest] topic/robin/timing: Adding a timing mode that records test execution times per host. (808fd76)) In-Reply-To: <20140131192428.GP3813@icir.org> References: <201401311701.s0VH1eU7005355@bro-ids.icir.org> <1D657054C5E2994C98FE68E14DD10A2B3894345B04@EXMAIL1.ohio.edu> <20140131192428.GP3813@icir.org> Message-ID: <52F13162.7000606@ohio.edu> On 1/31/14 2:24 PM, Robin Sommer wrote: > (Moving from bro-commits to bro-dev). > > On Fri, Jan 31, 2014 at 12:51 -0500, you wrote: > >> Instruction counts are probably going to have a strong dependency on >> the compiler version / options used to generate the code. I believe >> these counts could additionally be influenced by e.g. library >> upgrades, even when restricted to a single host and using a specific >> compiler / options. > True, but I'm not sure that's necessarily a bad thing. I don't think it's a bad thing either, exactly. When I wrote this, I was thinking that it'd make it a little easier to interpret the results if code changes could be tested independently of system changes ... but you're right that the system *is* going to have an effect on how bro runs and should therefore be included in the benchmark. I do think it's important to be able to isolate system-level vs. code-level changes, but just storing the commit should be enough to do that. I think this is already happening, so ... looks good to me. > What I'm mostly wondering about is if it's worth commiting data that's > very specific to a single user/machine to the repos? From an academic standpoint, think it could be useful to review these counts just to see how much they actually vary from platform to platform. Also, if the instruction counts *are* pretty consistent cross-platform, then it might e.g. be an indication that we / the compiler aren't taking advantage of some more complex instructions that exist on a particular platform. Additionally, given the number of people working on the project, this kind of data probably wouldn't take much space. It also seems like it'd be pretty easy to purge this data in the event it didn't end up being useful. --Gilbert From jira at bro-tracker.atlassian.net Tue Feb 4 14:12:39 2014 From: jira at bro-tracker.atlassian.net (Jeannette Dopheide (JIRA)) Date: Tue, 4 Feb 2014 16:12:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1127) topic/jdopheid/bro_documentation In-Reply-To: References: Message-ID: Jeannette Dopheide created BIT-1127: --------------------------------------- Summary: topic/jdopheid/bro_documentation Key: BIT-1127 URL: https://bro-tracker.atlassian.net/browse/BIT-1127 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Jeannette Dopheide Fix For: 2.3 Updates to bro manual that I worked on last week. I wanted to get these merged before I forget about them. -- This message was sent by Atlassian JIRA (v6.2-OD-08-034#6251) From jira at bro-tracker.atlassian.net Tue Feb 4 14:12:39 2014 From: jira at bro-tracker.atlassian.net (Jeannette Dopheide (JIRA)) Date: Tue, 4 Feb 2014 16:12:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1127) topic/jdopheid/bro_documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeannette Dopheide updated BIT-1127: ------------------------------------ Status: Merge Request (was: Open) > topic/jdopheid/bro_documentation > -------------------------------- > > Key: BIT-1127 > URL: https://bro-tracker.atlassian.net/browse/BIT-1127 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jeannette Dopheide > Fix For: 2.3 > > > Updates to bro manual that I worked on last week. I wanted to get these merged before I forget about them. -- This message was sent by Atlassian JIRA (v6.2-OD-08-034#6251) From jira at bro-tracker.atlassian.net Tue Feb 4 22:35:39 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 5 Feb 2014 00:35:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1119) topic/jsiwek/tcp-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1119: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/jsiwek/tcp-improvements > ----------------------------- > > Key: BIT-1119 > URL: https://bro-tracker.atlassian.net/browse/BIT-1119 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Fix For: 2.3 > > Attachments: signature.asc > > > This branch is in the bro, bro-testing, and bro-testing-private repos and has a few changes to improve reporting of TCP connection sizes and gaps (commit messages explain in more detail). > The baseline changes in the external repos all seemed reasonable/explainable (or actually fix a problem). There's too much changed to go through case-by-case and actually check things, but I did do closer examinations of unique differences as I came across them (e.g. try to corroborate Bro results via wireshark). Then for those that seem to follow the same trend as something I already inspected, I wouldn't manually check. -- This message was sent by Atlassian JIRA (v6.2-OD-08-034#6251) From noreply at bro.org Wed Feb 5 00:00:12 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 5 Feb 2014 00:00:12 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402050800.s1580Can022067@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------------ ---------- ---------- ------------- ---------- ------------------------------------ BIT-1127 [1] Bro Jeannette Dopheide - 2014-02-04 2.3 Normal topic/jdopheid/bro_documentation [2] BIT-1122 [3] Bro Jon Siwek Seth Hall 2014-01-30 2.3 Normal topic/jsiwek/dns-improvements [4] [1] BIT-1127 https://bro-tracker.atlassian.net/browse/BIT-1127 [2] bro_documentation https://github.com/bro/bro/tree/topic/jdopheid/bro_documentation [3] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [4] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements From noreply at bro.org Thu Feb 6 00:00:14 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 6 Feb 2014 00:00:14 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402060800.s1680EJH015485@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------------ ---------- ---------- ------------- ---------- ------------------------------------ BIT-1127 [1] Bro Jeannette Dopheide - 2014-02-04 2.3 Normal topic/jdopheid/bro_documentation [2] BIT-1122 [3] Bro Jon Siwek Seth Hall 2014-01-30 2.3 Normal topic/jsiwek/dns-improvements [4] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ------------------------------------------ 4b63b30 [5] bro Bernhard Amann 2014-02-05 Fix x509-extension test sometimes failing. [1] BIT-1127 https://bro-tracker.atlassian.net/browse/BIT-1127 [2] bro_documentation https://github.com/bro/bro/tree/topic/jdopheid/bro_documentation [3] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [4] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements [5] 4b63b30 https://github.com/bro/bro/commit/4b63b3090165b59863356a9e4ee228c2f1ac5809 From jira at bro-tracker.atlassian.net Thu Feb 6 13:47:39 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 6 Feb 2014 15:47:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1126) Logs disappearing after bro termination In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1126: ------------------------------- Status: Merge Request (was: Open) > Logs disappearing after bro termination > --------------------------------------- > > Key: BIT-1126 > URL: https://bro-tracker.atlassian.net/browse/BIT-1126 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Environment: freebsd > Reporter: Aashish Sharma > Priority: High > Labels: BroControl, log > > I have noticed several times that in the event of bro termination after expiration of StopTimeout, bro logs disappear. > This is generally seen when log sizes are much bigger (for example after overnight) > This issue was present in bro-2.1 and continue to be present in bro-2.2 > I see (kill from control.py - kick in often when stopping or restarting bro) because catch-n-release is still trying to flush its tables (which takes long time). Then there is no logs from overnight! > I can provide more information if desired (or even a test case). > Thanks, > Aashish > -- This message was sent by Atlassian JIRA (v6.2-OD-08-034#6251) From jira at bro-tracker.atlassian.net Thu Feb 6 13:47:39 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 6 Feb 2014 15:47:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1126) Logs disappearing after bro termination In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400#comment-15400 ] Daniel Thayer commented on BIT-1126: ------------------------------------ Branch topic/dnthayer/ticket1126 has a fix for this. > Logs disappearing after bro termination > --------------------------------------- > > Key: BIT-1126 > URL: https://bro-tracker.atlassian.net/browse/BIT-1126 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Environment: freebsd > Reporter: Aashish Sharma > Priority: High > Labels: BroControl, log > > I have noticed several times that in the event of bro termination after expiration of StopTimeout, bro logs disappear. > This is generally seen when log sizes are much bigger (for example after overnight) > This issue was present in bro-2.1 and continue to be present in bro-2.2 > I see (kill from control.py - kick in often when stopping or restarting bro) because catch-n-release is still trying to flush its tables (which takes long time). Then there is no logs from overnight! > I can provide more information if desired (or even a test case). > Thanks, > Aashish > -- This message was sent by Atlassian JIRA (v6.2-OD-08-034#6251) From jira at bro-tracker.atlassian.net Thu Feb 6 20:55:39 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 6 Feb 2014 22:55:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1126) Logs disappearing after bro termination In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1126: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Logs disappearing after bro termination > --------------------------------------- > > Key: BIT-1126 > URL: https://bro-tracker.atlassian.net/browse/BIT-1126 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.2 > Environment: freebsd > Reporter: Aashish Sharma > Priority: High > Labels: BroControl, log > > I have noticed several times that in the event of bro termination after expiration of StopTimeout, bro logs disappear. > This is generally seen when log sizes are much bigger (for example after overnight) > This issue was present in bro-2.1 and continue to be present in bro-2.2 > I see (kill from control.py - kick in often when stopping or restarting bro) because catch-n-release is still trying to flush its tables (which takes long time). Then there is no logs from overnight! > I can provide more information if desired (or even a test case). > Thanks, > Aashish > -- This message was sent by Atlassian JIRA (v6.2-OD-08-034#6251) From noreply at bro.org Fri Feb 7 00:00:17 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 7 Feb 2014 00:00:17 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402070800.s1780HuD029374@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------------ ---------- ---------- ------------- ---------- ------------------------------------ BIT-1127 [1] Bro Jeannette Dopheide - 2014-02-04 2.3 Normal topic/jdopheid/bro_documentation [2] BIT-1122 [3] Bro Jon Siwek Seth Hall 2014-01-30 2.3 Normal topic/jsiwek/dns-improvements [4] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ------------------------------------------ 4b63b30 [5] bro Bernhard Amann 2014-02-05 Fix x509-extension test sometimes failing. [1] BIT-1127 https://bro-tracker.atlassian.net/browse/BIT-1127 [2] bro_documentation https://github.com/bro/bro/tree/topic/jdopheid/bro_documentation [3] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [4] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements [5] 4b63b30 https://github.com/bro/bro/commit/4b63b3090165b59863356a9e4ee228c2f1ac5809 From robin at icir.org Fri Feb 7 08:09:05 2014 From: robin at icir.org (Robin Sommer) Date: Fri, 7 Feb 2014 08:09:05 -0800 Subject: [Bro-Dev] Test timing measurements (Re: [Bro-Commits] [git/btest] topic/robin/timing: Adding a timing mode that records test execution times per host. (808fd76)) In-Reply-To: <52F13162.7000606@ohio.edu> References: <201401311701.s0VH1eU7005355@bro-ids.icir.org> <1D657054C5E2994C98FE68E14DD10A2B3894345B04@EXMAIL1.ohio.edu> <20140131192428.GP3813@icir.org> <52F13162.7000606@ohio.edu> Message-ID: <20140207160905.GQ19640@icir.org> On Tue, Feb 04, 2014 at 13:28 -0500, you wrote: > From an academic standpoint, think it could be useful to review these > counts just to see how much they actually vary from platform to > platform. Yeah, I think you're right, I've commited my baselines for the external tests now. Thinking of this, maybe btest should also record a short descripting of the platform along with that. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jira at bro-tracker.atlassian.net Fri Feb 7 10:29:39 2014 From: jira at bro-tracker.atlassian.net (Marek Balint (JIRA)) Date: Fri, 7 Feb 2014 12:29:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (TM-16) Index not working when traffic encapsulated in 802.1q trunk In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/TM-16?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marek Balint updated TM-16: --------------------------- Attachment: tm-16.patch Attaching patch which should solve this problem. > Index not working when traffic encapsulated in 802.1q trunk > ----------------------------------------------------------- > > Key: TM-16 > URL: https://bro-tracker.atlassian.net/browse/TM-16 > Project: Time Machine > Issue Type: Problem > Affects Versions: git/master > Environment: Ubuntu 10.04 , pf_ring > Reporter: tyler.schoenke > Labels: 802.1Q, indexes > Attachments: tm-16.patch > > > Hi All, > When I query the time machine index, I am not receiving any results. > I just restarted time machine, and checked one of the recent class files to see there is traffic for a particular IP address. > tcpdump -e -v -n -r class_all_1385406639.023206 "vlan and host 128.138.44.198" > It shows some traffic, example: > 128.138.44.198.54014 > 74.125.225.209.443: Flags [.], cksum 0x8d2c (correct), seq 1283940799:1283940800, ack 615539104, win 16311, length 1 > 19:11:00.571731632 10:8c:cf:57:46:00 > 00:1d:09:6a:d9:a9, ethertype 802.1Q (0x8100), length 70: vlan 987, p 0, ethertype IPv4, (tos 0x0, ttl 56, id 17482, offset 0, flags [none], proto TCP (6), length 52) > When I telnet localhost 42042 and run the following command, I don't receive any results. > query to_file "128.138.44.198.pcap" index ip "128.138.44.198" > In the above tcpdump, you can see my traffic is 802.1Q trunked. I have to use the "vlan" BPF to extract it with tcpdump, and am wondering if the trunking is causing problems with indexing? > I tested the same version of time machine on non-trunked traffic, and the index works fine. > Let me know if you need any other configuration info. > Tyler -- This message was sent by Atlassian JIRA (v6.2-OD-08-034#6251) From jira at bro-tracker.atlassian.net Fri Feb 7 10:29:39 2014 From: jira at bro-tracker.atlassian.net (Marek Balint (JIRA)) Date: Fri, 7 Feb 2014 12:29:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (TM-16) Index not working when traffic encapsulated in 802.1q trunk In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/TM-16?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marek Balint updated TM-16: --------------------------- Status: In Progress (was: Open) > Index not working when traffic encapsulated in 802.1q trunk > ----------------------------------------------------------- > > Key: TM-16 > URL: https://bro-tracker.atlassian.net/browse/TM-16 > Project: Time Machine > Issue Type: Problem > Affects Versions: git/master > Environment: Ubuntu 10.04 , pf_ring > Reporter: tyler.schoenke > Labels: 802.1Q, indexes > Attachments: tm-16.patch > > > Hi All, > When I query the time machine index, I am not receiving any results. > I just restarted time machine, and checked one of the recent class files to see there is traffic for a particular IP address. > tcpdump -e -v -n -r class_all_1385406639.023206 "vlan and host 128.138.44.198" > It shows some traffic, example: > 128.138.44.198.54014 > 74.125.225.209.443: Flags [.], cksum 0x8d2c (correct), seq 1283940799:1283940800, ack 615539104, win 16311, length 1 > 19:11:00.571731632 10:8c:cf:57:46:00 > 00:1d:09:6a:d9:a9, ethertype 802.1Q (0x8100), length 70: vlan 987, p 0, ethertype IPv4, (tos 0x0, ttl 56, id 17482, offset 0, flags [none], proto TCP (6), length 52) > When I telnet localhost 42042 and run the following command, I don't receive any results. > query to_file "128.138.44.198.pcap" index ip "128.138.44.198" > In the above tcpdump, you can see my traffic is 802.1Q trunked. I have to use the "vlan" BPF to extract it with tcpdump, and am wondering if the trunking is causing problems with indexing? > I tested the same version of time machine on non-trunked traffic, and the index works fine. > Let me know if you need any other configuration info. > Tyler -- This message was sent by Atlassian JIRA (v6.2-OD-08-034#6251) From jira at bro-tracker.atlassian.net Fri Feb 7 10:29:39 2014 From: jira at bro-tracker.atlassian.net (Marek Balint (JIRA)) Date: Fri, 7 Feb 2014 12:29:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (TM-16) Index not working when traffic encapsulated in 802.1q trunk In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/TM-16?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401#comment-15401 ] Marek Balint edited comment on TM-16 at 2/7/14 12:29 PM: --------------------------------------------------------- Attaching patch which should solve this problem (tm-16.patch). was (Author: mareq): Attaching patch which should solve this problem. > Index not working when traffic encapsulated in 802.1q trunk > ----------------------------------------------------------- > > Key: TM-16 > URL: https://bro-tracker.atlassian.net/browse/TM-16 > Project: Time Machine > Issue Type: Problem > Affects Versions: git/master > Environment: Ubuntu 10.04 , pf_ring > Reporter: tyler.schoenke > Labels: 802.1Q, indexes > Attachments: tm-16.patch > > > Hi All, > When I query the time machine index, I am not receiving any results. > I just restarted time machine, and checked one of the recent class files to see there is traffic for a particular IP address. > tcpdump -e -v -n -r class_all_1385406639.023206 "vlan and host 128.138.44.198" > It shows some traffic, example: > 128.138.44.198.54014 > 74.125.225.209.443: Flags [.], cksum 0x8d2c (correct), seq 1283940799:1283940800, ack 615539104, win 16311, length 1 > 19:11:00.571731632 10:8c:cf:57:46:00 > 00:1d:09:6a:d9:a9, ethertype 802.1Q (0x8100), length 70: vlan 987, p 0, ethertype IPv4, (tos 0x0, ttl 56, id 17482, offset 0, flags [none], proto TCP (6), length 52) > When I telnet localhost 42042 and run the following command, I don't receive any results. > query to_file "128.138.44.198.pcap" index ip "128.138.44.198" > In the above tcpdump, you can see my traffic is 802.1Q trunked. I have to use the "vlan" BPF to extract it with tcpdump, and am wondering if the trunking is causing problems with indexing? > I tested the same version of time machine on non-trunked traffic, and the index works fine. > Let me know if you need any other configuration info. > Tyler -- This message was sent by Atlassian JIRA (v6.2-OD-08-034#6251) From jira at bro-tracker.atlassian.net Fri Feb 7 10:37:39 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 7 Feb 2014 12:37:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1127) topic/jdopheid/bro_documentation In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1127: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/jdopheid/bro_documentation > -------------------------------- > > Key: BIT-1127 > URL: https://bro-tracker.atlassian.net/browse/BIT-1127 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jeannette Dopheide > Fix For: 2.3 > > > Updates to bro manual that I worked on last week. I wanted to get these merged before I forget about them. -- This message was sent by Atlassian JIRA (v6.2-OD-08-034#6251) From noreply at bro.org Sat Feb 8 00:00:13 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 8 Feb 2014 00:00:13 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402080800.s1880DDi023332@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- --------------------------------- BIT-1122 [1] Bro Jon Siwek Seth Hall 2014-01-30 2.3 Normal topic/jsiwek/dns-improvements [2] [1] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [2] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements From noreply at bro.org Sun Feb 9 00:00:13 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 9 Feb 2014 00:00:13 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402090800.s1980DWG031493@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- --------------------------------- BIT-1122 [1] Bro Jon Siwek Seth Hall 2014-01-30 2.3 Normal topic/jsiwek/dns-improvements [2] [1] BIT-1122 https://bro-tracker.atlassian.net/browse/BIT-1122 [2] dns-improvements https://github.com/bro/bro/tree/topic/jsiwek/dns-improvements From jira at bro-tracker.atlassian.net Sun Feb 9 20:58:38 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 22:58:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1122) topic/jsiwek/dns-improvements In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1122: --------------------------- Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) Done > topic/jsiwek/dns-improvements > ----------------------------- > > Key: BIT-1122 > URL: https://bro-tracker.atlassian.net/browse/BIT-1122 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Seth Hall > Fix For: 2.3 > > > This branch is in bro, bro-testing, and bro-testing-private repos. > - Fixes incorrect parsing of DNS message format for messages with empty question sections. > - Changes dns.log to only include standard queries (opcode == 1). > - Adds "dns_unknown_reply" event for RR types that Bro doesn't know how to parse, which improves accuracy of request-reply pair matching performed by the default DNS scripts. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:00:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:00:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-521) Bro's secondary path does not handle IPv6) (only v4 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-521: -------------------------- Resolution: Fixed Status: Closed (was: Open) I'm just going to close this since the secondary path code is already gone in the plugins branch. > Bro's secondary path does not handle IPv6) (only v4 > --------------------------------------------------- > > Key: BIT-521 > URL: https://bro-tracker.atlassian.net/browse/BIT-521 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: gregor > Labels: IPv6 > Fix For: 2.3 > > > Bro's secondary path only handles IPv4 packets > See NetSessions::NextPacketSecondary -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:02:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:02:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-348) Reassembler integer overflow issues. Data not delivered after 2GB In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15502#comment-15502 ] Seth Hall commented on BIT-348: ------------------------------- Does anyone have an estimate of how much effort this would be to fix? I suspect it's not too bad, but I don't trust my judgement quite that much. I'd like to see this addressed, it feels like a time bomb waiting to go off. > Reassembler integer overflow issues. Data not delivered after 2GB > ----------------------------------------------------------------- > > Key: BIT-348 > URL: https://bro-tracker.atlassian.net/browse/BIT-348 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: gregor > Priority: High > Labels: inttypes > Fix For: 2.3 > > > {noformat} > #!rst > The TCP Reassembler does not deliver any data to analyzers after the first 2GB due to signed integer overflow (Actually it will deliver again between 4--6GB, etc.) This happens silently, i.e., without content_gap events or Undelivered calls. > This report superseded BIT-315, BIT-137 > The TCP Reassembler (and Reassem) base class use ``int`` to keep track of sequence numbers and ``seq_delta`` to check for differences. If a connection exceeds 2GB, the relative sequence numbers (int) used by the Reassembler become negative. While many parts of the Reassembler still work (because seq_delta still reports the correct difference) some parts do not. In particular ``seq_to_skip`` is broken (and fails silently). There might well be other parts of the Reassembler that fail > silently as well, that I haven't found yet. > See Comments in TCP_Reassembler.cc for more details. > The Reassembler should use int64. However this will require deep changes to the Reassembler and the TCP Analyzer and TCP_Endpoint classes (since we also store sequence numbers there). Also, the analyzer framework will need tweaks as well (e.g., Undelivered uses ``int`` for sequence numbers, also has to go to 64 bit) > As a hotfix that seems to work I disabled the ``seq_to_skip`` features. It wasn't used by any analyzer or policy script (Note, that seq_to_skip is different from skip_deliveries). Hotfix is in > topic/gregor/reassembler-hotfix > {noformat} -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:04:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:04:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-781) Case sensitive (non-normalized) HTTP header names In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-781: -------------------------- Resolution: Fixed Status: Closed (was: Open) I'm just going to close this ticket since it will just be part of the binpac++ HTTP analyzer work. This ticket is kind of redundant. > Case sensitive (non-normalized) HTTP header names > ------------------------------------------------- > > Key: BIT-781 > URL: https://bro-tracker.atlassian.net/browse/BIT-781 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: sconzo > Labels: analyzer > Fix For: 2.3 > > > There are a lot of usecases to have access to the normalized HTTP header names, I'm asking for an extension that will allow access to the non-normalized version. This can be used to find non-standard/suspicious HTTP connections. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:08:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:08:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1102) Logs use tab delimiter yet do not remove tabs from logged data In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1102: --------------------------- Resolution: Feedback Missing Status: Closed (was: In Progress) Could not reproduce the problem. > Logs use tab delimiter yet do not remove tabs from logged data > -------------------------------------------------------------- > > Key: BIT-1102 > URL: https://bro-tracker.atlassian.net/browse/BIT-1102 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: ubuntu 12.04LTS server > Reporter: April Lorenzen > Assignee: Seth Hall > Labels: http.log > Attachments: 4872c1c6d6983c4b6cf4e0f83baf5ec5.pcap > > > The attached pcap processed with bro -r pcapname will demonstrate the issue. View the http.log after processing and note that the field delimiter appears within a field. > I tried searching the official github mirror but don't know bro well enough to find the spot to look. I'd be happy to work on fixing this if you point me to the problem area (that's the hard part, right?) -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:08:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:08:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1111) will not start In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1111: --------------------------- Resolution: Fixed Status: Closed (was: Open) This issue was fixed with the recent merge of libmagic into master. > will not start > -------------- > > Key: BIT-1111 > URL: https://bro-tracker.atlassian.net/browse/BIT-1111 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Mac OSX 10.9 > Reporter: Chas DiFatta > Priority: High > Labels: broctl > > there is a problem when starting as i get the following error > internal error: can't load magic file : no magic files loaded > /opt/bro/share/broctl/scripts/run-bro: line 82: 1175 Abort trap: 6 (core dumped) nohup $mybro "$@" > have a new mac running 10.9. downloaded 2.2 for OSX, it installed fine. followed the recommended changes to the config files. ran the following command as root. > sh-3.2# ./broctl > Welcome to BroControl 1.2 > Type "help" for help. > [BroControl] > install > removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/site ... done. > removing old policies in /opt/bro/spool/installed-scripts-do-not-touch/auto ... done. > creating policy directories ... done. > installing site policies ... done. > generating standalone-layout.bro ... done. > generating local-networks.bro ... done. > generating broctl-config.bro ... done. > updating nodes ... done. > [BroControl] > start > starting bro ... > . > bro terminated immediately after starting; check output with "diag" > [BroControl] > diag > [bro] > Bro 2.2 > Darwin 13.0.0 > No gdb installed. > ==== No reporter.log > ==== stderr.log > internal error: can't load magic file : no magic files loaded > /opt/bro/share/broctl/scripts/run-bro: line 82: 1175 Abort trap: 6 (core dumped) nohup $mybro "$@" > ==== stdout.log > unlimited > unlimited > unlimited > ==== .cmdline > -i eh0 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto > ==== .env_vars > PATH=/opt/bro/bin:/opt/bro/share/broctl/scripts:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin > BROPATH=/opt/bro/spool/installed-scripts-do-not-touch/site::/opt/bro/spool/installed-scripts-do-not-touch/auto:/opt/bro/share/bro:/opt/bro/share/bro/policy:/opt/bro/share/bro/site > CLUSTER_NODE= > ==== .status > TERMINATED [internal_error] > ==== No prof.log > ==== No packet_filter.log > ==== No loaded_scripts.log -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:10:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:10:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-276) broctl status commands should run regardless of lock In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-276: -------------------------- Resolution: Rejected Status: Closed (was: Open) I'm going to close this ticket due to the impending rework coming to broctl which will invalidate this ticket. > broctl status commands should run regardless of lock > ---------------------------------------------------- > > Key: BIT-276 > URL: https://bro-tracker.atlassian.net/browse/BIT-276 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Affects Versions: 1.5.2 > Reporter: solomon > Assignee: Robin Sommer > Priority: High > > broctl should allow the informational commands (top, capstats, netstats, diag, peerstatus, status, et al.) to run without interference from a concurrently locked broctl process. If there is 'broctl cron' or 'broctl start' command runnning, say, from the system crontab, use of broctl to check on the status is often delayed or prevented. Allowing the informational commands to run unimpeded would allow monitoring suites, such as Nagios, to more easily watch the status of the cluster. With the frequent 'cannot get lock' errors, Nagios cannot work with bro & broctl. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:10:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:10:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-724) Changing semantics of ConnSizeAnalyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-724: -------------------------- Resolution: Rejected Status: Closed (was: Open) There was some contention over this and we've left it for a few releases so I think it's stuck at this point. :) > Changing semantics of ConnSizeAnalyzer > -------------------------------------- > > Key: BIT-724 > URL: https://bro-tracker.atlassian.net/browse/BIT-724 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Seth Hall > Priority: High > > I think we should change what the conn size analyzer is measuring. It currently measures the size of the connection from the IP header down (or up, depending on how you look at it). From my perspective that data is rarely (if ever?) useful. What is more useful is a counted value for the connection size. c$(orig|resp)$size takes it's measurement from sequence counting and can get confused in some cases (chinese firewall sending RST packets for instance). > This is the patch I'm proposing: > {noformat} > diff --git a/scripts/base/init-bare.bro b/scripts/base/init-bare.bro > index 859a69f..21a9b60 100644 > --- a/scripts/base/init-bare.bro > +++ b/scripts/base/init-bare.bro > @@ -66,10 +66,10 @@ type endpoint: record { > > ## Number of packets on the wire > ## Set if :bro:id:`use_conn_size_analyzer` is true. > - num_pkts: count &optional; > - ## Number of IP-level bytes on the wire > + counted_pkts: count &optional; > + ## Number of content bytes on the wire > ## Set if :bro:id:`use_conn_size_analyzer` is true. > - num_bytes_ip: count &optional; > + counted_bytes: count &optional; > }; > > type endpoint_stats: record { > diff --git a/src/ConnSizeAnalyzer.cc b/src/ConnSizeAnalyzer.cc > index a1b892f..5d0efcd 100644 > --- a/src/ConnSizeAnalyzer.cc > +++ b/src/ConnSizeAnalyzer.cc > @@ -39,12 +39,12 @@ void ConnSize_Analyzer::DeliverPacket(int len, const u_char* data, bool is_orig, > > if ( is_orig ) > { > - orig_bytes += ip->TotalLen(); > + orig_bytes += len; > orig_pkts ++; > } > else > { > - resp_bytes += ip->TotalLen(); > + resp_bytes += len; > resp_pkts ++; > } > } > {noformat} > If no one has a problem with this, I'd like to make the change for the 2.0 release because I'm having trouble currently with counting bytes for the SSH analyzer and we're getting more false positives than we should be seeing. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:14:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:14:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1110) BRO_DISABLE_BROXYGEN env variable not working In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1110: --------------------------- Assignee: Jon Siwek > BRO_DISABLE_BROXYGEN env variable not working > --------------------------------------------- > > Key: BIT-1110 > URL: https://bro-tracker.atlassian.net/browse/BIT-1110 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.2 > Environment: Scientific linux v6.2 > Reporter: scampbell > Assignee: Jon Siwek > Priority: High > Labels: broxygen > Attachments: Manager.cc.diff > > > When running bro from the command line - for example: > [scottc at sigma-n SSHD_BRO]$ bin/bro test > you get the error: > internal error: Broxygen can't get mtime of bro binary : No such file or directory > Aborted > even when the bro binary is in the runtime path. > Setting: > [scottc at sigma-n SSHD_BRO]$ env | grep BRO > BRO_DISABLE_BROXYGEN=T > does not help - see patch below. As well, running with -X or --broxygen did not change anything. A quick hack "fixed" the problem, but the problem should probably be looked over more closely as I stopped with the bandaid. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:18:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:18:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1099) ./build/bro-git-20131129/lib/broctl/plugins/lb_myricom.py In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1099: --------------------------- Resolution: Fixed Status: Closed (was: Open) I believe this ticket is taken care of. Please reopen if there is still something to address. > ./build/bro-git-20131129/lib/broctl/plugins/lb_myricom.py > --------------------------------------------------------- > > Key: BIT-1099 > URL: https://bro-tracker.atlassian.net/browse/BIT-1099 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: git/master > Environment: RHEL6 > Myricom 10G > Reporter: tyler.schoenke > Labels: Myricom, RHEL6 > > Hi All, > First, unrelated, thanks for fixing whatever was wrong with PktSrc.cc:222. I was core dumping on that with the Myricom 10G until I upgraded to the latest git version of Bro. > I had been running with my Myricom environment variables configured in .bash_profile, but decided to move them to etc/node.cfg because that feature is now available in Bro 2.2. > When I tried to start the workers, they failed and I got the message to run diag. When I ran diag, I noticed it showed the following. > 28366 snf.0.-1 P (environ) SNF_DATARING_SIZE = 2147483648 (0x80000000) (2048.0 MiB) > 28366 snf.0.-1 P (default) SNF_DESCRING_SIZE = 536870912 (0x20000000) (512.0 MiB) > 28366 snf.0.-1 P (userset) SNF_FLAGS = 257 (0x101) > 28366 snf.0.-1 P (environ) SNF_DEBUG_MASK = 3 (0x3) > 28366 snf.0.-1 P (default) SNF_DEBUG_FILENAME = stderr > Everything looks great except for SNF_FLAGS. Myricom defaults it to 0x1, and that was working when I ran bro from the command line. > I found it was getting set in ./build/bro-git-20131129/lib/broctl/plugins/lb_myricom.py > According to /opt/snf/include/snf.h, the only valid values for SNF_FLAGS are 0x1, 0x2, and 0x300. No possible combination of those will add up to 0x101. I'm guessing that is a typo. If not, please explain why it is defaulted to that, otherwise you may want to default to 0x1. > Thanks, > Tyler -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:20:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:20:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1017) MPLS in VLAN not considered/stripped In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1017: --------------------------- Assignee: Robin Sommer > MPLS in VLAN not considered/stripped > ------------------------------------ > > Key: BIT-1017 > URL: https://bro-tracker.atlassian.net/browse/BIT-1017 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: ckanich > Assignee: Robin Sommer > Fix For: 2.3 > > Attachments: mpls-in-vlan.patch, newfile.pcap > > > PktSrc.cc doesn't seem to consider mpls packets in vlan. Here's a quick patch. I couldn't figure a straightforward way to incorporate L2 stripping so there's some code duplication. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:20:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:20:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1017) MPLS in VLAN not considered/stripped In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509#comment-15509 ] Seth Hall commented on BIT-1017: -------------------------------- Looks like this might be one to deal with in the branch where you are doing the pktsrc plugin work. > MPLS in VLAN not considered/stripped > ------------------------------------ > > Key: BIT-1017 > URL: https://bro-tracker.atlassian.net/browse/BIT-1017 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: ckanich > Assignee: Robin Sommer > Fix For: 2.3 > > Attachments: mpls-in-vlan.patch, newfile.pcap > > > PktSrc.cc doesn't seem to consider mpls packets in vlan. Here's a quick patch. I couldn't figure a straightforward way to incorporate L2 stripping so there's some code duplication. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:34:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:34:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-735) Clean up and merge the TCPStats analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510#comment-15510 ] Seth Hall commented on BIT-735: ------------------------------- Is this dead in the water or is there still some hope for it? > Clean up and merge the TCPStats analyzer > ---------------------------------------- > > Key: BIT-735 > URL: https://bro-tracker.atlassian.net/browse/BIT-735 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro > Reporter: Seth Hall > Fix For: 2.3 > > > Katrina wants to get her TCPStats analyzer merged. Let's aim for getting it cleaned up and ready for the 2.1 release. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:34:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:34:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-862) btest path length limitations In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-862: -------------------------- Assignee: Robin Sommer > btest path length limitations > ----------------------------- > > Key: BIT-862 > URL: https://bro-tracker.atlassian.net/browse/BIT-862 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BTest > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.3 > > > btest looks like it fails to create a unix socket when running in paths that are particularly long: > {noformat} > jenkins at ubuntu12-04:btest$ pwd > /home/jenkins/workspace/BuildAll/label/Ubuntu_12.04_x86_64/bro/testing/btest > jenkins at ubuntu12-04:btest$ ../../aux/btest/btest core/ > Process TestManager-1: > Traceback (most recent call last): > File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap > self.run() > File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run > self._target(*self._args, **self._kwargs) > File "/usr/lib/python2.7/multiprocessing/managers.py", line 550, in _run_server > server = cls._Server(registry, address, authkey, serializer) > File "/usr/lib/python2.7/multiprocessing/managers.py", line 162, in __init__ > self.listener = Listener(address=address, backlog=16) > File "/usr/lib/python2.7/multiprocessing/connection.py", line 132, in __init__ > self._listener = SocketListener(address, family, backlog) > File "/usr/lib/python2.7/multiprocessing/connection.py", line 254, in __init__ > self._socket.bind(address) > File "/usr/lib/python2.7/socket.py", line 224, in meth > return getattr(self._sock,name)(*args) > error: AF_UNIX path too long > Traceback (most recent call last): > File "../../aux/btest/btest", line 1162, in > (succeeded, failed, skipped) = TestManager().run(copy.deepcopy(tests), output_handler) > File "../../aux/btest/btest", line 136, in run > self.start() > File "/usr/lib/python2.7/multiprocessing/managers.py", line 528, in start > self._address = reader.recv() > EOFError > {noformat} > Doing this change fixes it: > {noformat} > iff --git a/btest b/btest > index fedaa63..dee3247 100755 > --- a/btest > +++ b/btest > @@ -129,8 +129,8 @@ ConfigParser.ConfigParser._interpolate = cpExpandBackticks > # Main class distributing the work across threads. > class TestManager(multiprocessing.managers.SyncManager): > - def __init__(self): > - super(TestManager, self).__init__() > + def __init__(self, *args, **kwargs): > + super(TestManager, self).__init__(*args, **kwargs) > def run(self, tests, output_handler): > self.start() > @@ -1158,7 +1158,7 @@ mkdir(BaselineDir) > mkdir(TmpDir) > try: > - (succeeded, failed, skipped) = TestManager().run(copy.deepcopy(tests), output_handler) > + (succeeded, failed, skipped) = TestManager(address="/tmp/blah").run(copy.deepcopy(tests), output_handler) > total = succeeded + failed + skipped > except KeyboardInterrupt: > print >>sys.stderr, "Aborted." > {noformat} > But obviously the path of the socket should be determined more dynamically. I didn't know if it always makes sense to try to use something short in /tmp or should it only be changed through a command line switch? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:36:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:36:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15511#comment-15511 ] Seth Hall commented on BIT-700: ------------------------------- Robin, I think you're going to have to make the call on this one. > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.3 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:36:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:36:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-700: -------------------------- Assignee: Robin Sommer (was: Jon Siwek) > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.3 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:38:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:38:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-434) Fix secondary path In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-434: -------------------------- Resolution: Invalid Status: Closed (was: Open) Hah, another secondary path ticket to close! > Fix secondary path > ------------------ > > Key: BIT-434 > URL: https://bro-tracker.atlassian.net/browse/BIT-434 > Project: Bro Issue Tracker > Issue Type: Task > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Fix For: 2.3 > > > We decided that want to keep the secondary path, but it seems there are some pieces whereit is not well integrated with the rest. Non-standard packet sources is one area (like the Endace support); there may be more. > See its paper for more information on the secondary path. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:40:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:40:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-418) Add analysis support for SCTP In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-418: -------------------------- Resolution: Rejected Status: Closed (was: Open) This is a really pointless ticket. > Add analysis support for SCTP > ----------------------------- > > Key: BIT-418 > URL: https://bro-tracker.atlassian.net/browse/BIT-418 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Reporter: Seth Hall > > Like the summary says. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:48:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:48:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-180) field value missing in secondary filter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-180: -------------------------- Resolution: Invalid Status: Closed (was: Open) Wow! Another secondary path ticket! > field value missing in secondary filter > --------------------------------------- > > Key: BIT-180 > URL: https://bro-tracker.atlassian.net/browse/BIT-180 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 1.5.2 > Reporter: Robin Sommer > > This is with trunk as of r6926, with a plain ./configure. > bro \-r ~/data/traces/surfing all > Unknown destination > Unknown destination > ((vlan) and ((not (port 25 or port 587)) and (not (port 80 or port 8000 or port 8080)))) and (((((((((((((((((((((((((((((((((((((((((((tcp port 514) or (udp)) or (tcp port 989)) or (tcp port 994)) or (tcp dst port 80 or tcp dst port 8080 or tcp dst port 8000)) or (port 110)) or (port 6667)) or (tcp port 113)) or (tcp)) or (tcp port 585)) or (tcp port 614)) or (tcp port 990)) or (port 2049)) or (tcp port 524)) or (port 445)) or (tcp port 992)) or (port ftp)) or (port telnet or tcp port 513)) or ((ip[6:2] & 0x3fff \!= 0) and tcp)) or (arp)) or (tcp[13] & 7 \!= 0)) or (port 135)) or (port 53)) or (udp port 137)) or (udp port 69)) or (udp port 2002 and src net 134.96)) or (icmp)) or (tcp port 443)) or (tcp port 563)) or (tcp port 995)) or (port finger)) or ((ip[6:2] & 0x3fff \!= 0) and udp)) or (port 6346 or port 8436)) or (tcp port smtp or tcp port 587)) or (tcp port 22)) or (port 6666)) or (port 111)) or (tcp port 22)) or (tcp port 993)) or (tcp)) or (tcp src port 80 or tcp src port 8080 or tcp src port 8000)) or (tcp port 636)) or (udp port 123)) > a_udp_event() > 1144876537.668448 ../policy/secondary-filter.bro, line 15 (pkt$udp): internal error: field value missing -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:50:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:50:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-168) Patch to enable/disable nonblocking dns In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-168: -------------------------- Resolution: Fixed Status: Closed (was: Open) Somehow this ticket managed to stay open from the 1.4 days? > Patch to enable/disable nonblocking dns > --------------------------------------- > > Key: BIT-168 > URL: https://bro-tracker.atlassian.net/browse/BIT-168 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 1.5.2 > Reporter: mccreary > Assignee: mccreary > Attachments: bro_nbdns_patch, configure.patch > > > Here is a patch to support disabling DNS with a configure switch. Note that I also needed to add a missing #ifdef block around some new code in DNS_Mgr.cc, as DNS_Mgr.h removes the method definitions if HAVE_NB_DNS is not defined. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:52:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:52:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-67) Grammar ambiguity with "local" for introducing "when" variables In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-67?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-67: ------------------------- Assignee: Robin Sommer > Grammar ambiguity with "local" for introducing "when" variables > --------------------------------------------------------------- > > Key: BIT-67 > URL: https://bro-tracker.atlassian.net/browse/BIT-67 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Vern Paxson > Assignee: Robin Sommer > > Currently there are two ways in the grammar to parse "local", one as a variable declaration (including a type, initialization, and attributes), and the other as an expression (no type or attributes, just essentially an initialization). I believe we added this second to support introducing variables to hold the delayed values resulting from "when" expressions. However, this leads to parsing ambiguities and in particular the inability to associate attributes with "local" declarations. (One such attribute we need is &raw_output.) So we instead need to use a different keyword for "when" values. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Sun Feb 9 21:52:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Sun, 9 Feb 2014 23:52:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-67) Grammar ambiguity with "local" for introducing "when" variables In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-67?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15516#comment-15516 ] Seth Hall commented on BIT-67: ------------------------------ This one seems to have slipped through the cracks. Should it be merged? > Grammar ambiguity with "local" for introducing "when" variables > --------------------------------------------------------------- > > Key: BIT-67 > URL: https://bro-tracker.atlassian.net/browse/BIT-67 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Vern Paxson > > Currently there are two ways in the grammar to parse "local", one as a variable declaration (including a type, initialization, and attributes), and the other as an expression (no type or attributes, just essentially an initialization). I believe we added this second to support introducing variables to hold the delayed values resulting from "when" expressions. However, this leads to parsing ambiguities and in particular the inability to associate attributes with "local" declarations. (One such attribute we need is &raw_output.) So we instead need to use a different keyword for "when" values. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Mon Feb 10 05:48:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Mon, 10 Feb 2014 07:48:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-25) TRW should be more flexible in determining what connections to skip In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-25: ------------------------- Resolution: Invalid Status: Closed (was: Open) There is no TRW implementation in Bro at the moment and since this is an implementation specific request I'm going to close it. > TRW should be more flexible in determining what connections to skip > ------------------------------------------------------------------- > > Key: BIT-25 > URL: https://bro-tracker.atlassian.net/browse/BIT-25 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Reporter: Vern Paxson > > [Eric Thomas, Sandia|From] > Instead of using a set lookup (the honeypot global) to determine whether > a connection is related to a honeypot, introduce a function variable that > gets set to a function which takes a connection record as input and returns > a boolean. The return value specifies T/F whether the connection is > associated with a honeypot. This function is called in check_TRW_scan > (trw-impl.bro) instead of the set lookup in honeypot. > > The default function would do the simple set lookup, as is done now. But it > allows others to create a function that performs more complex operations. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Mon Feb 10 05:50:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Mon, 10 Feb 2014 07:50:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-73) Document time sources in Bro In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-73: ------------------------- Resolution: Rejected Status: Closed (was: Open) There is no real action to take from this ticket so I'm going to close it. > Document time sources in Bro > ---------------------------- > > Key: BIT-73 > URL: https://bro-tracker.atlassian.net/browse/BIT-73 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 1.5.2 > Reporter: gregor > Labels: current_time,, network_time,, source, time > > Document the different time sources in Bro (network_time, current_timestamp, current_time, etc.) and when to use them. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Mon Feb 10 07:26:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 10 Feb 2014 09:26:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1110) BRO_DISABLE_BROXYGEN env variable not working In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1110: --------------------------- Resolution: Fixed Fix Version/s: 2.3 Status: Closed (was: Open) A fix for this was merge to git/master last month. > BRO_DISABLE_BROXYGEN env variable not working > --------------------------------------------- > > Key: BIT-1110 > URL: https://bro-tracker.atlassian.net/browse/BIT-1110 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.2 > Environment: Scientific linux v6.2 > Reporter: scampbell > Assignee: Jon Siwek > Priority: High > Labels: broxygen > Fix For: 2.3 > > Attachments: Manager.cc.diff > > > When running bro from the command line - for example: > [scottc at sigma-n SSHD_BRO]$ bin/bro test > you get the error: > internal error: Broxygen can't get mtime of bro binary : No such file or directory > Aborted > even when the bro binary is in the runtime path. > Setting: > [scottc at sigma-n SSHD_BRO]$ env | grep BRO > BRO_DISABLE_BROXYGEN=T > does not help - see patch below. As well, running with -X or --broxygen did not change anything. A quick hack "fixed" the problem, but the problem should probably be looked over more closely as I stopped with the bandaid. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Mon Feb 10 09:04:38 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 10 Feb 2014 11:04:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-67) Grammar ambiguity with "local" for introducing "when" variables In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-67?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15520#comment-15520 ] Jon Siwek commented on BIT-67: ------------------------------ I don't think the change in the commit fully resolves the grammar ambiguity, but just makes a workaround of declaring the local w/ an attribute outside the when stmt actually work for more cases. Generally, I consider my work on that branch more of a sandbox to play around w/ fixing the various problems with attributes. It fixes many common usages to be more intuitive, but makes some other things a bit weird. There's a lot of discussion scattered over the tickets referenced in that commit, but no one's come up w/ a definitive solution for all the attribute weirdness in Bro. So no: don't merge that in order to fix the problem in this ticket. > Grammar ambiguity with "local" for introducing "when" variables > --------------------------------------------------------------- > > Key: BIT-67 > URL: https://bro-tracker.atlassian.net/browse/BIT-67 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Vern Paxson > Assignee: Robin Sommer > > Currently there are two ways in the grammar to parse "local", one as a variable declaration (including a type, initialization, and attributes), and the other as an expression (no type or attributes, just essentially an initialization). I believe we added this second to support introducing variables to hold the delayed values resulting from "when" expressions. However, this leads to parsing ambiguities and in particular the inability to associate attributes with "local" declarations. (One such attribute we need is &raw_output.) So we instead need to use a different keyword for "when" values. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 13:19:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 15:19:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1128) Add configure options for linking against jemalloc In-Reply-To: References: Message-ID: Robin Sommer created BIT-1128: --------------------------------- Summary: Add configure options for linking against jemalloc Key: BIT-1128 URL: https://bro-tracker.atlassian.net/browse/BIT-1128 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: git/master Reporter: Robin Sommer To gather experiences with using jemalloc, add a configure options --with-jemalloc= that links Bro against it if found. Default should be off. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 13:45:37 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Tue, 11 Feb 2014 15:45:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1129) RADIUS Protocol Analyzer In-Reply-To: References: Message-ID: grigorescu created BIT-1129: ------------------------------- Summary: RADIUS Protocol Analyzer Key: BIT-1129 URL: https://bro-tracker.atlassian.net/browse/BIT-1129 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: git/master Reporter: grigorescu topic/vladg/radius is ready to be merged. It's been running at CMU for a few months with no issues. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 13:45:37 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Tue, 11 Feb 2014 15:45:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1129) RADIUS Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] grigorescu updated BIT-1129: ---------------------------- Status: Merge Request (was: Open) > RADIUS Protocol Analyzer > ------------------------ > > Key: BIT-1129 > URL: https://bro-tracker.atlassian.net/browse/BIT-1129 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > > topic/vladg/radius is ready to be merged. It's been running at CMU for a few months with no issues. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 14:01:37 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Tue, 11 Feb 2014 16:01:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1130) Fix some misidentification of SOCKS traffic In-Reply-To: References: Message-ID: grigorescu created BIT-1130: ------------------------------- Summary: Fix some misidentification of SOCKS traffic Key: BIT-1130 URL: https://bro-tracker.atlassian.net/browse/BIT-1130 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Reporter: grigorescu Added a constraint that the SOCKS command be 1, 2 or 3, which are the defined commands. This helps to address some traffic that was being misidentified as SOCKS. Has been running at CMU without issues. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 14:01:37 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Tue, 11 Feb 2014 16:01:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1130) Fix some misidentification of SOCKS traffic In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] grigorescu updated BIT-1130: ---------------------------- Status: Merge Request (was: Open) > Fix some misidentification of SOCKS traffic > ------------------------------------------- > > Key: BIT-1130 > URL: https://bro-tracker.atlassian.net/browse/BIT-1130 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: grigorescu > > Added a constraint that the SOCKS command be 1, 2 or 3, which are the defined commands. This helps to address some traffic that was being misidentified as SOCKS. Has been running at CMU without issues. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:01:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:01:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-17) MemoryAllocation() core dumps at termination In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-17?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15521#comment-15521 ] Robin Sommer commented on BIT-17: --------------------------------- Looks like nobody has been seen this in a long time and the config for reproducing doesn't exist anymore. > MemoryAllocation() core dumps at termination > -------------------------------------------- > > Key: BIT-17 > URL: https://bro-tracker.atlassian.net/browse/BIT-17 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Assignee: Robin Sommer > > In rare situations, the final {{profiling_logger->Log();}} in {{terminate_bro}} core-dumps in some object's {{MemoryAllocation()}}, liklely because that object has already been deleted. > Not clear whether this is still a problem after the recent round of ref-counting fixes but I've disabled the final {{Log()}} for now. Need to recheck. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:03:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:03:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-127) Potential memory leak in async DNS lookups In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-127: ----------------------------- Resolution: Fixed Status: Closed (was: Open) I believe we've fixed a set of DNS problems along these lines so I'm going to assume this is fixed as well. > Potential memory leak in async DNS lookups > ------------------------------------------ > > Key: BIT-127 > URL: https://bro-tracker.atlassian.net/browse/BIT-127 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Assignee: Robin Sommer > Labels: leak > > perftools says: > (pprof) top > Total: 1131 objects > 337 29.8% 29.8% 958 84.7% LookupHostCallback::Resolved ??:0 > 329 29.1% 58.9% 638 56.4% StringVal::StringVal ??:0 > 309 27.3% 86.2% 309 27.3% BroString::Set ??:0 > 146 12.9% 99.1% 146 12.9% DNS_Mgr::IssueAsyncRequests ??:0 > 10 0.9% 100.0% 27 2.4% LookupHostCallback::Timeout ??:0 > 0 0.0% 100.0% 309 27.3% BroString::BroString ??:0 > 0 0.0% 100.0% 760 67.2% AssignExpr::Eval ??:0 > 0 0.0% 100.0% 1131 100.0% main ??:0 > 0 0.0% 100.0% 760 67.2% CallExpr::Eval ??:0 > 0 0.0% 100.0% 760 67.2% EventMgr::Dispatch ??:0 > 0 0.0% 100.0% 760 67.2% Trigger::Trigger ??:0 -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:03:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:03:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-17) MemoryAllocation() core dumps at termination In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-17?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-17: ---------------------------- Resolution: Cannot Reproduce Status: Closed (was: Open) > MemoryAllocation() core dumps at termination > -------------------------------------------- > > Key: BIT-17 > URL: https://bro-tracker.atlassian.net/browse/BIT-17 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Assignee: Robin Sommer > > In rare situations, the final {{profiling_logger->Log();}} in {{terminate_bro}} core-dumps in some object's {{MemoryAllocation()}}, liklely because that object has already been deleted. > Not clear whether this is still a problem after the recent round of ref-counting fixes but I've disabled the final {{Log()}} for now. Need to recheck. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:11:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:11:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-123) expire-logs doesn't expire stats/* In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-123: -------------------------------- Assignee: Daniel Thayer (was: Robin Sommer) > expire-logs doesn't expire stats/* > ---------------------------------- > > Key: BIT-123 > URL: https://bro-tracker.atlassian.net/browse/BIT-123 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Assignee: Daniel Thayer > Priority: Low > > It should however have a separate expiration period, as we might want to keep these for a different period. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:11:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:11:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-123) expire-logs doesn't expire stats/* In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15523#comment-15523 ] Robin Sommer commented on BIT-123: ---------------------------------- It's not very important but something we should probably still add. Daniel, mind taking a look? I think all we need here is an additional piece in expire-logs that removes from stats/stats.log all entries older than X, with X large by default like maybe year. Btw, I've also always wanted to visualize the data in stats/, it's already in parseable form. :) > expire-logs doesn't expire stats/* > ---------------------------------- > > Key: BIT-123 > URL: https://bro-tracker.atlassian.net/browse/BIT-123 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Assignee: Robin Sommer > Priority: Low > > It should however have a separate expiration period, as we might want to keep these for a different period. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:13:39 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:13:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-130) Potential memory leak in serialization cache In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-130: ----------------------------- Resolution: Cannot Reproduce Status: Closed (was: Open) Closing because unclear how to reproduce and whether this is still there. > Potential memory leak in serialization cache > -------------------------------------------- > > Key: BIT-130 > URL: https://bro-tracker.atlassian.net/browse/BIT-130 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Assignee: Robin Sommer > Labels: leak > > ==12541== 612 (48 direct, 564 indirect) bytes in 1 blocks are definitely lost in loss record 2,237 of 5,387 > ==12541== 612 (48 direct, 564 indirect) bytes in 1 blocks are definitely lost in loss record 2,237 of 5,387 > ==12541== at 0x400674E: operator new(unsigned int) (vg_replace_malloc.c:224) > ==12541== by 0x81A622D: SerializationCache::Register(SerialObj const*, unsigned long long, bool) (Serializer.cc:581) > ==12541== by 0x81A18EF: SerialObj::Serialize(SerialInfo*) const (SerialObj.cc:105) > ==12541== by 0x81FA2A7: Val::Serialize(SerialInfo*) const (Val.cc:98) > ==12541== by 0x81362C5: ID::DoSerialize(SerialInfo*) const (ID.cc:474) > ==12541== by 0x81A1A0E: SerialObj::Serialize(SerialInfo*) const (SerialObj.cc:123) > ==12541== by 0x81367B5: ID::Serialize(SerialInfo*) const (ID.cc:287) > ==12541== by 0x81A8555: Serializer::Serialize(SerialInfo*, ID const&) (Serializer.cc:113) > ==12541== by 0x817F853: RemoteSerializer::SendID(SerialInfo*, RemoteSerializer::Peer*, ID const&) (RemoteSerializer.cc:974) > ==12541== by 0x818152F: RemoteSerializer::ProcessConnected() (RemoteSerializer.cc:1831) > ==12541== by 0x818548F: RemoteSerializer::DoMessage() (RemoteSerializer.cc:1503) > ==12541== by 0x8185A4A: RemoteSerializer::Poll(bool) (RemoteSerializer.cc:1442) > ==12541== by 0x81858F5: RemoteSerializer::Poll(bool) (RemoteSerializer.cc:1419) > ==12541== by 0x8185EC6: RemoteSerializer::NextTimestamp(double*) (RemoteSerializer.cc:1251) > ==12541== by 0x813ABB2: IOSourceRegistry::FindSoonest(double*) (IOSource.cc:61) > ==12541== by 0x8158914: net_run() (Net.cc:507) > ==12541== by 0x804FB0F: main (main.cc:999) -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:13:39 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:13:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-131) Potential leak in remote serializer's logging In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-131: ----------------------------- Resolution: Cannot Reproduce Status: Closed (was: Open) Closing because unclear how to reproduce and whether this is still there. > Potential leak in remote serializer's logging > --------------------------------------------- > > Key: BIT-131 > URL: https://bro-tracker.atlassian.net/browse/BIT-131 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Assignee: Robin Sommer > Labels: leak > > ==12541== 44 bytes in 1 blocks are definitely lost in loss record 3,910 of 5,387 > ==12541== 44 bytes in 1 blocks are definitely lost in loss record 3,910 of 5,387 > ==12541== at 0x400674E: operator new(unsigned int) (vg_replace_malloc.c:224) > ==12541== by 0x817CE6A: RemoteSerializer::Log(RemoteSerializer::LogLevel, char const*, RemoteSerializer::Peer*, RemoteSerializer::LogSrc) (RemoteSerializer.cc:2354) > ==12541== by 0x817D0E3: RemoteSerializer::Log(RemoteSerializer::LogLevel, char const*) (RemoteSerializer.cc:2334) > ==12541== by 0x817D2EF: RemoteSerializer::LogStats() (RemoteSerializer.cc:2396) > ==12541== by 0x804D3F5: terminate_bro() (main.cc:271) > ==12541== by 0x804D813: termination_signal() (main.cc:297) > ==12541== by 0x8158BB3: net_run() (Net.cc:591) > ==12541== by 0x804FB0F: main (main.cc:999) > ==12541== 44 bytes in 1 blocks are definitely lost in loss record 3,929 of 5,387 > ==12541== 44 bytes in 1 blocks are definitely lost in loss record 3,929 of 5,387 > ==12541== at 0x400674E: operator new(unsigned int) (vg_replace_malloc.c:224) > ==12541== by 0x817CDE4: RemoteSerializer::Log(RemoteSerializer::LogLevel, char const*, RemoteSerializer::Peer*, RemoteSerializer::LogSrc) (RemoteSerializer.cc:2353) > ==12541== by 0x817D0E3: RemoteSerializer::Log(RemoteSerializer::LogLevel, char const*) (RemoteSerializer.cc:2334) > ==12541== by 0x817D2EF: RemoteSerializer::LogStats() (RemoteSerializer.cc:2396) > ==12541== by 0x804D3F5: terminate_bro() (main.cc:271) > ==12541== by 0x804D813: termination_signal() (main.cc:297) > ==12541== by 0x8158BB3: net_run() (Net.cc:591) > ==12541== by 0x804FB0F: main (main.cc:999) > ==12541== 198 (52 direct, 146 indirect) bytes in 1 blocks are definitely lost in loss record 4,635 of 5,387 > ==12541== 198 (52 direct, 146 indirect) bytes in 1 blocks are definitely lost in loss record 4,635 of 5,387 > ==12541== at 0x400674E: operator new(unsigned int) (vg_replace_malloc.c:224) > ==12541== by 0x817CEF0: RemoteSerializer::Log(RemoteSerializer::LogLevel, char const*, RemoteSerializer::Peer*, RemoteSerializer::LogSrc) (RemoteSerializer.cc:2355) > ==12541== by 0x817D0E3: RemoteSerializer::Log(RemoteSerializer::LogLevel, char const*) (RemoteSerializer.cc:2334) > ==12541== by 0x817D2EF: RemoteSerializer::LogStats() (RemoteSerializer.cc:2396) > ==12541== by 0x804D3F5: terminate_bro() (main.cc:271) > ==12541== by 0x804D813: termination_signal() (main.cc:297) > ==12541== by 0x8158BB3: net_run() (Net.cc:591) > ==12541== by 0x804FB0F: main (main.cc:999) -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:41:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:41:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-174) policy/xquery directory and files did not make it to trunk from Robin's branch In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-174: ----------------------------- Resolution: Invalid Status: Closed (was: Open) no longer applies > policy/xquery directory and files did not make it to trunk from Robin's branch > ------------------------------------------------------------------------------ > > Key: BIT-174 > URL: https://bro-tracker.atlassian.net/browse/BIT-174 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 1.5.2 > Reporter: Seth Hall > Assignee: Robin Sommer > Priority: Low > Labels: devmode > > When doing the install process from BroControl in trunk, it complains about that directory not existing. > With devmode enabled, here's my output... > {noformat} > [BroControl] > install > waiting for lock ....... ok > removing old policies in /bro/share/bro/.site ... done. > removing old policies in /bro/share/bro/broctl ... done. > creating policy directories ... done. > creating installation directories ... done. > installing files ...warning: file does not exist: /usr/home/seth/bro/policy/xquery/*.xq > done. > installing site policies ... done. > generating broctl-layout.bro ... done. > generating analysis-policy.bro ... done. > generating local-networks.bro ... done. > updating nodes ... [BroControl] > > {noformat} > It looks like a newline is missing after outputting all of the status messages too. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:43:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:43:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-132) Potential leak in remote serializer's AddPeer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-132: ----------------------------- Resolution: Cannot Reproduce Status: Closed (was: Open) No clear if it still exists and how to reproduce. > Potential leak in remote serializer's AddPeer > --------------------------------------------- > > Key: BIT-132 > URL: https://bro-tracker.atlassian.net/browse/BIT-132 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Assignee: Robin Sommer > Labels: leak > > ==12541== 10,645 (1,380 direct, 9,265 indirect) bytes in 15 blocks are definitely lost in loss record 3,808 of 5,387 > ==12541== 10,645 (1,380 direct, 9,265 indirect) bytes in 15 blocks are definitely lost in loss record 3,808 of 5,387 > ==12541== at 0x400674E: operator new(unsigned int) (vg_replace_malloc.c:224) > ==12541== by 0x817E4F0: RemoteSerializer::AddPeer(unsigned int, unsigned short, unsigned int) (RemoteSerializer.cc:1655) > ==12541== by 0x8181452: RemoteSerializer::ProcessConnected() (RemoteSerializer.cc:1812) > ==12541== by 0x818548F: RemoteSerializer::DoMessage() (RemoteSerializer.cc:1503) > ==12541== by 0x8185A4A: RemoteSerializer::Poll(bool) (RemoteSerializer.cc:1442) > ==12541== by 0x81858F5: RemoteSerializer::Poll(bool) (RemoteSerializer.cc:1419) > ==12541== by 0x8185EC6: RemoteSerializer::NextTimestamp(double*) (RemoteSerializer.cc:1251) > ==12541== by 0x813ABB2: IOSourceRegistry::FindSoonest(double*) (IOSource.cc:61) > ==12541== by 0x8158914: net_run() (Net.cc:507) > ==12541== by 0x804FB0F: main (main.cc:999) -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:45:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:45:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-79) DNS analyzer does not generate events on most NXDOMAIN In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-79?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-79: ------------------------------- Assignee: (was: Robin Sommer) > DNS analyzer does not generate events on most NXDOMAIN > ------------------------------------------------------ > > Key: BIT-79 > URL: https://bro-tracker.atlassian.net/browse/BIT-79 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 1.5.2 > Reporter: gregor > > With default settings the "old" DNS-Analyzer doesn't generate events on (most) NXDOMAINs because: most NXDOMAIN replies have additional (or authority) sections, dns_skip_all* is T by default, the dns_rejected event is only generated when all sections have zero count ==> no dns_rejeced event (because there's an additional) but the additional is not printed ==> no event is generated for this DNS reply. > Maybe solution: > a) generate a dns_rejected whenever RCode \!= 0 or > b) generate a dns_rejected when RCode\!=0 && ancount==0 > What do you think is the best semantic? > FYI: here's the code snippet from DNS.cc > {noformat} > else if ( msg->QR == 1 && > msg->ancount == 0 && msg->nscount == 0 && msg->arcount == 0 ) > // Service rejected in some fashion, and it won't be reported > // via a returned RR because there aren't any. > dns_event = dns_rejected; > {noformat} -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:45:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:45:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-178) BroControl's check process should check for ability to set "ulimit -d" In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-178: -------------------------------- Assignee: Daniel Thayer (was: Robin Sommer) > BroControl's check process should check for ability to set "ulimit -d" > ---------------------------------------------------------------------- > > Key: BIT-178 > URL: https://bro-tracker.atlassian.net/browse/BIT-178 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > Priority: Low > Labels: warning > > If the bro process(es) are run as a non-root user, the "ulimit \-d" call done during the run-bro script will fail (on freebsd at least) and cause obtuse failures when the Bro processes grow beyond the default 512M data segment size (on freebsd again). The check process could verify that setting can be set and possibly give recommendations for linux and freebsd on how to increase that setting globally. > For documentation purposes, to set it globally to the value set by the run-bro script put the following in the /boot/loader.conf file and reboot: > {noformat} > kern.maxdsiz=1610612736 > {noformat} -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:45:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:45:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-287) BroControl should warn if changes aren't "install"ed In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-287: -------------------------------- Assignee: (was: Robin Sommer) This sounds like something the new broctld could eventually do. > BroControl should warn if changes aren't "install"ed > ---------------------------------------------------- > > Key: BIT-287 > URL: https://bro-tracker.atlassian.net/browse/BIT-287 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Seth Hall > > At times it can be difficult to remember to "install" after making changes in the site directory before restarting with BroControl. BroControl should check to see if there are any files that are updated in the site directory and put up a warning to the user if they are restarting or starting without installing. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:47:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:47:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-288) Requesting BroControl feature to force flush log files. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-288: -------------------------------- Assignee: (was: Robin Sommer) > Requesting BroControl feature to force flush log files. > ------------------------------------------------------- > > Key: BIT-288 > URL: https://bro-tracker.atlassian.net/browse/BIT-288 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Seth Hall > > If would be handy for BroControl to have the ability to force flush log files as a command. In the default case of running "flush-logs" (or whatever it would be named) with a node name, it could flush only the manager (or standalone) since the other nodes wouldn't make much sense to flush and it would make the process take longer. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:47:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:47:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-721) Broccoli python doesn't support all possible data types In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-721: ----------------------------- Resolution: Won't Fix Status: Closed (was: Open) There's been some work on this but I don't see us fixing this with the new comm library in planning. > Broccoli python doesn't support all possible data types > ------------------------------------------------------- > > Key: BIT-721 > URL: https://bro-tracker.atlassian.net/browse/BIT-721 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: broccoli-python > Reporter: Seth Hall > Assignee: Robin Sommer > > It has no support built in for sending or receiving tables/sets/vectors. It's support for enums appears to be incomplete as well. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:49:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:49:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-12) Capture from multiple interfaces In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-12?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-12: ------------------------------- Assignee: (was: Robin Sommer) Is this something that's still desired? We should either bump the ticket to higher priority or close it if we punt. > Capture from multiple interfaces > -------------------------------- > > Key: BIT-12 > URL: https://bro-tracker.atlassian.net/browse/BIT-12 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Priority: Low > > Allow multiple interfaces to be specified for a node in {{node.cfg}}. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:53:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:53:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-244) assertion failure - trailing_CRLF In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-244: ----------------------------- Resolution: Fixed Status: Closed (was: Open) Both traces work fine for me with current git. > assertion failure - trailing_CRLF > --------------------------------- > > Key: BIT-244 > URL: https://bro-tracker.atlassian.net/browse/BIT-244 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 1.5.1 > Reporter: Vern Paxson > Assignee: Robin Sommer > Attachments: http-no-trailing-CRLF.2.trace, http-no-trailing-CRLF.trace > > > The appended trace when processed using http-reply http-body produces: > bro: HTTP.cc:88: virtual void HTTP_Entity::Deliver(int, const char*, int): Assertion `trailing_CRLF' failed. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 15:55:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 17:55:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1017) MPLS in VLAN not considered/stripped In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1017: ------------------------------ Status: Merge Request (was: Open) > MPLS in VLAN not considered/stripped > ------------------------------------ > > Key: BIT-1017 > URL: https://bro-tracker.atlassian.net/browse/BIT-1017 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: ckanich > Assignee: Robin Sommer > Fix For: 2.3 > > Attachments: mpls-in-vlan.patch, newfile.pcap > > > PktSrc.cc doesn't seem to consider mpls packets in vlan. Here's a quick patch. I couldn't figure a straightforward way to incorporate L2 stripping so there's some code duplication. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 16:13:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 18:13:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-862) btest path length limitations In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15532#comment-15532 ] Robin Sommer commented on BIT-862: ---------------------------------- Not quite sure either but what I'll do for now is take a short path inside TMPDIR and seed it with the current PID. That should be good enough. > btest path length limitations > ----------------------------- > > Key: BIT-862 > URL: https://bro-tracker.atlassian.net/browse/BIT-862 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BTest > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.3 > > > btest looks like it fails to create a unix socket when running in paths that are particularly long: > {noformat} > jenkins at ubuntu12-04:btest$ pwd > /home/jenkins/workspace/BuildAll/label/Ubuntu_12.04_x86_64/bro/testing/btest > jenkins at ubuntu12-04:btest$ ../../aux/btest/btest core/ > Process TestManager-1: > Traceback (most recent call last): > File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap > self.run() > File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run > self._target(*self._args, **self._kwargs) > File "/usr/lib/python2.7/multiprocessing/managers.py", line 550, in _run_server > server = cls._Server(registry, address, authkey, serializer) > File "/usr/lib/python2.7/multiprocessing/managers.py", line 162, in __init__ > self.listener = Listener(address=address, backlog=16) > File "/usr/lib/python2.7/multiprocessing/connection.py", line 132, in __init__ > self._listener = SocketListener(address, family, backlog) > File "/usr/lib/python2.7/multiprocessing/connection.py", line 254, in __init__ > self._socket.bind(address) > File "/usr/lib/python2.7/socket.py", line 224, in meth > return getattr(self._sock,name)(*args) > error: AF_UNIX path too long > Traceback (most recent call last): > File "../../aux/btest/btest", line 1162, in > (succeeded, failed, skipped) = TestManager().run(copy.deepcopy(tests), output_handler) > File "../../aux/btest/btest", line 136, in run > self.start() > File "/usr/lib/python2.7/multiprocessing/managers.py", line 528, in start > self._address = reader.recv() > EOFError > {noformat} > Doing this change fixes it: > {noformat} > iff --git a/btest b/btest > index fedaa63..dee3247 100755 > --- a/btest > +++ b/btest > @@ -129,8 +129,8 @@ ConfigParser.ConfigParser._interpolate = cpExpandBackticks > # Main class distributing the work across threads. > class TestManager(multiprocessing.managers.SyncManager): > - def __init__(self): > - super(TestManager, self).__init__() > + def __init__(self, *args, **kwargs): > + super(TestManager, self).__init__(*args, **kwargs) > def run(self, tests, output_handler): > self.start() > @@ -1158,7 +1158,7 @@ mkdir(BaselineDir) > mkdir(TmpDir) > try: > - (succeeded, failed, skipped) = TestManager().run(copy.deepcopy(tests), output_handler) > + (succeeded, failed, skipped) = TestManager(address="/tmp/blah").run(copy.deepcopy(tests), output_handler) > total = succeeded + failed + skipped > except KeyboardInterrupt: > print >>sys.stderr, "Aborted." > {noformat} > But obviously the path of the socket should be determined more dynamically. I didn't know if it always makes sense to try to use something short in /tmp or should it only be changed through a command line switch? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 16:15:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 18:15:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15533#comment-15533 ] Robin Sommer commented on BIT-700: ---------------------------------- I propose to remove the packet sorter. Yet another piece adding complexity but nobody (?) actually using it. Objections? > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.3 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 16:19:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 18:19:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-161) In standalone mode, broctl attempts to connect to wrong port. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-161: -------------------------------- Assignee: Daniel Thayer (was: Robin Sommer) I'm not quite sure how to test for this, I guess we'd have to load the config and print out the value. Is that worth it? > In standalone mode, broctl attempts to connect to wrong port. > ------------------------------------------------------------- > > Key: BIT-161 > URL: https://bro-tracker.atlassian.net/browse/BIT-161 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > Priority: Low > Labels: warning > > I have a standalone instance setup and the bro process is holding open port 47758/tcp, but the broctl interface is attempting to connect to port 47760/tcp when it tries to do anything with broccoli. > {noformat} > [BroControl] > netstats > bro: > {noformat} > {noformat} > seth at Blake3:~$ sudo lsof -i | grep LISTEN > bro 74156 root 0u IPv4 0xb3ad270 0t0 TCP *:47758 (LISTEN) > {noformat} -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 16:21:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 18:21:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-67) Grammar ambiguity with "local" for introducing "when" variables In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-67?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-67: ---------------------------- Labels: language (was: ) > Grammar ambiguity with "local" for introducing "when" variables > --------------------------------------------------------------- > > Key: BIT-67 > URL: https://bro-tracker.atlassian.net/browse/BIT-67 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Vern Paxson > Labels: language > > Currently there are two ways in the grammar to parse "local", one as a variable declaration (including a type, initialization, and attributes), and the other as an expression (no type or attributes, just essentially an initialization). I believe we added this second to support introducing variables to hold the delayed values resulting from "when" expressions. However, this leads to parsing ambiguities and in particular the inability to associate attributes with "local" declarations. (One such attribute we need is &raw_output.) So we instead need to use a different keyword for "when" values. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 16:21:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 18:21:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-862) btest path length limitations In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-862: ----------------------------- Resolution: Fixed Status: Closed (was: Open) > btest path length limitations > ----------------------------- > > Key: BIT-862 > URL: https://bro-tracker.atlassian.net/browse/BIT-862 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BTest > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Robin Sommer > Fix For: 2.3 > > > btest looks like it fails to create a unix socket when running in paths that are particularly long: > {noformat} > jenkins at ubuntu12-04:btest$ pwd > /home/jenkins/workspace/BuildAll/label/Ubuntu_12.04_x86_64/bro/testing/btest > jenkins at ubuntu12-04:btest$ ../../aux/btest/btest core/ > Process TestManager-1: > Traceback (most recent call last): > File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap > self.run() > File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run > self._target(*self._args, **self._kwargs) > File "/usr/lib/python2.7/multiprocessing/managers.py", line 550, in _run_server > server = cls._Server(registry, address, authkey, serializer) > File "/usr/lib/python2.7/multiprocessing/managers.py", line 162, in __init__ > self.listener = Listener(address=address, backlog=16) > File "/usr/lib/python2.7/multiprocessing/connection.py", line 132, in __init__ > self._listener = SocketListener(address, family, backlog) > File "/usr/lib/python2.7/multiprocessing/connection.py", line 254, in __init__ > self._socket.bind(address) > File "/usr/lib/python2.7/socket.py", line 224, in meth > return getattr(self._sock,name)(*args) > error: AF_UNIX path too long > Traceback (most recent call last): > File "../../aux/btest/btest", line 1162, in > (succeeded, failed, skipped) = TestManager().run(copy.deepcopy(tests), output_handler) > File "../../aux/btest/btest", line 136, in run > self.start() > File "/usr/lib/python2.7/multiprocessing/managers.py", line 528, in start > self._address = reader.recv() > EOFError > {noformat} > Doing this change fixes it: > {noformat} > iff --git a/btest b/btest > index fedaa63..dee3247 100755 > --- a/btest > +++ b/btest > @@ -129,8 +129,8 @@ ConfigParser.ConfigParser._interpolate = cpExpandBackticks > # Main class distributing the work across threads. > class TestManager(multiprocessing.managers.SyncManager): > - def __init__(self): > - super(TestManager, self).__init__() > + def __init__(self, *args, **kwargs): > + super(TestManager, self).__init__(*args, **kwargs) > def run(self, tests, output_handler): > self.start() > @@ -1158,7 +1158,7 @@ mkdir(BaselineDir) > mkdir(TmpDir) > try: > - (succeeded, failed, skipped) = TestManager().run(copy.deepcopy(tests), output_handler) > + (succeeded, failed, skipped) = TestManager(address="/tmp/blah").run(copy.deepcopy(tests), output_handler) > total = succeeded + failed + skipped > except KeyboardInterrupt: > print >>sys.stderr, "Aborted." > {noformat} > But obviously the path of the socket should be determined more dynamically. I didn't know if it always makes sense to try to use something short in /tmp or should it only be changed through a command line switch? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 16:21:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 18:21:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1017) MPLS in VLAN not considered/stripped In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1017: --------------------------------- Assignee: (was: Robin Sommer) > MPLS in VLAN not considered/stripped > ------------------------------------ > > Key: BIT-1017 > URL: https://bro-tracker.atlassian.net/browse/BIT-1017 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: ckanich > Fix For: 2.3 > > Attachments: mpls-in-vlan.patch, newfile.pcap > > > PktSrc.cc doesn't seem to consider mpls packets in vlan. Here's a quick patch. I couldn't figure a straightforward way to incorporate L2 stripping so there's some code duplication. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 16:21:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Feb 2014 18:21:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-67) Grammar ambiguity with "local" for introducing "when" variables In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-67?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-67: ------------------------------- Assignee: (was: Robin Sommer) > Grammar ambiguity with "local" for introducing "when" variables > --------------------------------------------------------------- > > Key: BIT-67 > URL: https://bro-tracker.atlassian.net/browse/BIT-67 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Vern Paxson > Labels: language > > Currently there are two ways in the grammar to parse "local", one as a variable declaration (including a type, initialization, and attributes), and the other as an expression (no type or attributes, just essentially an initialization). I believe we added this second to support introducing variables to hold the delayed values resulting from "when" expressions. However, this leads to parsing ambiguities and in particular the inability to associate attributes with "local" declarations. (One such attribute we need is &raw_output.) So we instead need to use a different keyword for "when" values. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 17:17:37 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Tue, 11 Feb 2014 19:17:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535#comment-15535 ] grigorescu commented on BIT-700: -------------------------------- I've actually been looking at PacketSorter recently, and I think it would be useful for at least CMU and another EDU. The way CMU taps the network is to aggregate SPANs from different routers, and to ensure no duplicate traffic is received, the upflow is received from one router, while the downflow is received from another. This can obviously lead to problems when the packets arrive out of order (which seems to happen). An option I've been considering is to use our Arista switch as the master timesource that all routers synchronize to, and then use the PacketSorter to sort the packets based on timestamp. I think as we see more people tapping their networks internally, we'll see more of this distributed tapping that will need to be sorted. Does that seem like a reasonable use case? Is there a different, cleaner, way to achieve that? > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.3 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 18:03:37 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 11 Feb 2014 20:03:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-161) In standalone mode, broctl attempts to connect to wrong port. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536#comment-15536 ] Daniel Thayer commented on BIT-161: ----------------------------------- Why are there three different default values for Communication::listen_port in Bro 2.2? (47757 in base/frameworks/communication/main.bro, 47758 in various broccoli test scripts such as broping, and broctl sets a value of 47760) > In standalone mode, broctl attempts to connect to wrong port. > ------------------------------------------------------------- > > Key: BIT-161 > URL: https://bro-tracker.atlassian.net/browse/BIT-161 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > Priority: Low > Labels: warning > > I have a standalone instance setup and the bro process is holding open port 47758/tcp, but the broctl interface is attempting to connect to port 47760/tcp when it tries to do anything with broccoli. > {noformat} > [BroControl] > netstats > bro: > {noformat} > {noformat} > seth at Blake3:~$ sudo lsof -i | grep LISTEN > bro 74156 root 0u IPv4 0xb3ad270 0t0 TCP *:47758 (LISTEN) > {noformat} -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 20:05:37 2014 From: jira at bro-tracker.atlassian.net (AK (JIRA)) Date: Tue, 11 Feb 2014 22:05:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1131) Global Variable Containing Trace Filename In-Reply-To: References: Message-ID: AK created BIT-1131: ----------------------- Summary: Global Variable Containing Trace Filename Key: BIT-1131 URL: https://bro-tracker.atlassian.net/browse/BIT-1131 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: 2.2 Environment: All. This is a feature for scriptland and is environment independent. It only benefits environments using Bro in post processing situations. Reporter: AK It would be nice to have a @PKTSOURCE variable similar to the @FILENAME and @DIR variables. Somehow exposing the filename of the pcap being processed is the end goal. One use case could be dynamically loading scripts with @if statements or altering control flow within scripts depending on the name of the pcap file. Consider if tcpdump is used to record (and rotate) daily packet captures and Bro is used in a post processing manner. Assuming the packet capture is named according to the day it was recorded on, it would be rather handy for scriptland to behave differently depending on the pcap name. Additionally, it would be handy to be able to include the name of the pcap file in log file names or log records. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 11 21:47:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Tue, 11 Feb 2014 23:47:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-12) Capture from multiple interfaces In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-12?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-12: ------------------------- Resolution: Won't Fix Status: Closed (was: Open) Let's close it. There is a lot of rework that is coming to how packets are handled and a lot of stuff that I would like to see happen around lower level tuning for packet capture that would intrinsically change how all of this works. > Capture from multiple interfaces > -------------------------------- > > Key: BIT-12 > URL: https://bro-tracker.atlassian.net/browse/BIT-12 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Priority: Low > > Allow multiple interfaces to be specified for a node in {{node.cfg}}. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Wed Feb 12 00:00:11 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 12 Feb 2014 00:00:11 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402120800.s1C80Beo022674@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------------------------- BIT-1130 [1] Bro grigorescu - 2014-02-11 - Normal Fix some misidentification of SOCKS traffic BIT-1129 [2] Bro grigorescu - 2014-02-11 - Normal RADIUS Protocol Analyzer BIT-1017 [3] Bro ckanich - 2014-02-11 2.3 Normal MPLS in VLAN not considered/stripped Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ------------------------------------ adfe3a0 [4] bro Bernhard Amann 2014-02-10 add channel_id tls extension number. [1] BIT-1130 https://bro-tracker.atlassian.net/browse/BIT-1130 [2] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [3] BIT-1017 https://bro-tracker.atlassian.net/browse/BIT-1017 [4] adfe3a0 https://github.com/bro/bro/commit/adfe3a0754d936de1dbe702848b5bf36fe8c19ed From jira at bro-tracker.atlassian.net Wed Feb 12 08:15:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 12 Feb 2014 10:15:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15538#comment-15538 ] Robin Sommer commented on BIT-700: ---------------------------------- How will you get the external timestamps into Bro? I've kind of always felt that this functionality doesn't really belong into Bro. Is this something where Click would be the better place for now, once we get the envisioned Click layer between NICs and Bro in place (netmap is going to have a large piece of the functionality required for that soon). So Click would do the reading from multiple interfaces, joining, and timestamp sorting; and then just pass on a single packet stream to Bro. > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.3 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 12 08:33:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 12 Feb 2014 10:33:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1131) Global Variable Containing Trace Filename In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15539#comment-15539 ] Robin Sommer commented on BIT-1131: ----------------------------------- What would happen if Bro reads from multiple sources (interfaces/files). The easiest way I can think of would be providing a built-in function that just provides a list of all of them. Also, we should probably wait with this until we've settled on how to handle packet sources in the future. There are a number ideas floating around, in particular allowing script-land to add/remove sources on the fly. > Global Variable Containing Trace Filename > ----------------------------------------- > > Key: BIT-1131 > URL: https://bro-tracker.atlassian.net/browse/BIT-1131 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.2 > Environment: All. This is a feature for scriptland and is environment independent. It only benefits environments using Bro in post processing situations. > Reporter: AK > Labels: language > > It would be nice to have a @PKTSOURCE variable similar to the @FILENAME and @DIR variables. Somehow exposing the filename of the pcap being processed is the end goal. > One use case could be dynamically loading scripts with @if statements or altering control flow within scripts depending on the name of the pcap file. Consider if tcpdump is used to record (and rotate) daily packet captures and Bro is used in a post processing manner. Assuming the packet capture is named according to the day it was recorded on, it would be rather handy for scriptland to behave differently depending on the pcap name. Additionally, it would be handy to be able to include the name of the pcap file in log file names or log records. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 12 09:11:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 12 Feb 2014 11:11:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1131) Global Variable Containing Trace Filename In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15540#comment-15540 ] Seth Hall commented on BIT-1131: -------------------------------- Yep, I really think this should wait a bit. Once a few more things are nailed down we should be able to approach. Ultimately I would not like this request implemented as it is because I would like the ability to load pcap files programmatically within scripts and this would be implementing an API that is born deprecated. > Global Variable Containing Trace Filename > ----------------------------------------- > > Key: BIT-1131 > URL: https://bro-tracker.atlassian.net/browse/BIT-1131 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.2 > Environment: All. This is a feature for scriptland and is environment independent. It only benefits environments using Bro in post processing situations. > Reporter: AK > Labels: language > > It would be nice to have a @PKTSOURCE variable similar to the @FILENAME and @DIR variables. Somehow exposing the filename of the pcap being processed is the end goal. > One use case could be dynamically loading scripts with @if statements or altering control flow within scripts depending on the name of the pcap file. Consider if tcpdump is used to record (and rotate) daily packet captures and Bro is used in a post processing manner. Assuming the packet capture is named according to the day it was recorded on, it would be rather handy for scriptland to behave differently depending on the pcap name. Additionally, it would be handy to be able to include the name of the pcap file in log file names or log records. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 12 09:17:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 12 Feb 2014 11:17:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-161) In standalone mode, broctl attempts to connect to wrong port. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15541#comment-15541 ] Robin Sommer commented on BIT-161: ---------------------------------- Using different ports allows to use them all in parallel. Say you're running a cluster but also want to do some independent Broccoli work/testing in parallel, or maybe start up a standalone Bro. Robin > In standalone mode, broctl attempts to connect to wrong port. > ------------------------------------------------------------- > > Key: BIT-161 > URL: https://bro-tracker.atlassian.net/browse/BIT-161 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Seth Hall > Assignee: Daniel Thayer > Priority: Low > Labels: warning > > I have a standalone instance setup and the bro process is holding open port 47758/tcp, but the broctl interface is attempting to connect to port 47760/tcp when it tries to do anything with broccoli. > {noformat} > [BroControl] > netstats > bro: > {noformat} > {noformat} > seth at Blake3:~$ sudo lsof -i | grep LISTEN > bro 74156 root 0u IPv4 0xb3ad270 0t0 TCP *:47758 (LISTEN) > {noformat} -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 12 11:15:37 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Wed, 12 Feb 2014 13:15:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542#comment-15542 ] grigorescu commented on BIT-700: -------------------------------- What I was hoping to do was to use the timestamp IP option to sort the packets, but I'm still trying to figure out if that's reliable with Cisco routers. Alternatively, for TCP the sorting by sequence number would also be useful. As for whether or not this belongs in Bro - I could see an argument for both sides. I think the fundamental question is how commonly does this occur? Is this an artifact of some issue with the tapping infrastructure, or are out of order packets expected traffic on most networks? If the current PacketSorter implementation needs reworking, perhaps the TCP reassembler should be extended to deal with out of order packets? I'm not trying to advocate for one particular approach, as long as I have some mechanism of dealing with this. :-) > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.3 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 12 19:41:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 12 Feb 2014 21:41:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1132) topic/seth/http-connect In-Reply-To: References: Message-ID: Seth Hall created BIT-1132: ------------------------------ Summary: topic/seth/http-connect Key: BIT-1132 URL: https://bro-tracker.atlassian.net/browse/BIT-1132 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: 2.3 Reporter: Seth Hall Assignee: Robin Sommer New support for the HTTP analyzer to correct support and decapsulate HTTP CONNECT proxying. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 12 19:41:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 12 Feb 2014 21:41:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1132) topic/seth/http-connect In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1132: --------------------------- Status: In Progress (was: Open) > topic/seth/http-connect > ----------------------- > > Key: BIT-1132 > URL: https://bro-tracker.atlassian.net/browse/BIT-1132 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.3 > Reporter: Seth Hall > Assignee: Robin Sommer > > New support for the HTTP analyzer to correct support and decapsulate HTTP CONNECT proxying. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 12 19:43:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 12 Feb 2014 21:43:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1132) topic/seth/http-connect In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1132: --------------------------- Status: Merge Request (was: In Progress) > topic/seth/http-connect > ----------------------- > > Key: BIT-1132 > URL: https://bro-tracker.atlassian.net/browse/BIT-1132 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.3 > Reporter: Seth Hall > Assignee: Robin Sommer > > New support for the HTTP analyzer to correct support and decapsulate HTTP CONNECT proxying. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 12 19:43:38 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 12 Feb 2014 21:43:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1132) topic/seth/http-connect In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543#comment-15543 ] Seth Hall commented on BIT-1132: -------------------------------- There is a same-name branch in the private test suite. > topic/seth/http-connect > ----------------------- > > Key: BIT-1132 > URL: https://bro-tracker.atlassian.net/browse/BIT-1132 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.3 > Reporter: Seth Hall > Assignee: Robin Sommer > > New support for the HTTP analyzer to correct support and decapsulate HTTP CONNECT proxying. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Thu Feb 13 00:00:11 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 13 Feb 2014 00:00:11 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402130800.s1D80BGZ022433@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------- BIT-1132 [1] Bro Seth Hall Robin Sommer 2014-02-12 - Normal topic/seth/http-connect [2] BIT-1130 [3] Bro grigorescu - 2014-02-11 - Normal Fix some misidentification of SOCKS traffic BIT-1129 [4] Bro grigorescu - 2014-02-11 - Normal RADIUS Protocol Analyzer BIT-1017 [5] Bro ckanich - 2014-02-11 2.3 Normal MPLS in VLAN not considered/stripped Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ------------------------------------- e844727 [6] bro Jon Siwek 2014-02-12 Increase timeouts of some unit tests. 6563b54 [7] bro Jon Siwek 2014-02-12 Fix memory leak in modbus analyzer. adfe3a0 [8] bro Bernhard Amann 2014-02-10 add channel_id tls extension number. [1] BIT-1132 https://bro-tracker.atlassian.net/browse/BIT-1132 [2] http-connect https://github.com/bro/bro/tree/topic/seth/http-connect [3] BIT-1130 https://bro-tracker.atlassian.net/browse/BIT-1130 [4] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [5] BIT-1017 https://bro-tracker.atlassian.net/browse/BIT-1017 [6] e844727 https://github.com/bro/bro/commit/e844727e7339a95054e05cdf8634d6fd47044e74 [7] 6563b54 https://github.com/bro/bro/commit/6563b544d8b5e532006682acc3313c5989ce0fe5 [8] adfe3a0 https://github.com/bro/bro/commit/adfe3a0754d936de1dbe702848b5bf36fe8c19ed From jira at bro-tracker.atlassian.net Thu Feb 13 08:13:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 13 Feb 2014 10:13:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15544#comment-15544 ] Robin Sommer commented on BIT-700: ---------------------------------- For using IP timestamps the packet sorter will need work, it's just looking at pcap timestamps right now. TCP is not a problem though: the TCP reassembler does already put things in order according to sequence numbers (having lots of reordering would impact performance though). I think there are two different things going in here: (1) normal packet reordering as it happens on the network: that's not a problem for Bro, the TCP reassembler takes care of that for TCP and in any case Bro is seeing the packets in the same order an the end systems, so its interpretation will match theirs as well, which is all we need. (2) tapping introducing additional reordering that the client systems do not see; that's a problem, and the most typical case is indeed having two interfaces with uni-directional streams being merged together. So if we focus on (2) here I argue that it's a problem better solved outside of Bro, namely where the packets are captured/passed on. That's where my suggestion of using Click is coming from. In addition, that would have the advantage of helping not only Bro but everybody. > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.3 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 13 13:19:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 13 Feb 2014 15:19:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: Jon Siwek created BIT-1133: ------------------------------ Summary: topic/jsiwek/dns-perf Key: BIT-1133 URL: https://bro-tracker.atlassian.net/browse/BIT-1133 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Affects Versions: git/master Reporter: Jon Siwek Fix For: 2.3 Refactored DNS script's state management to improve performance (due to reduced timer volume). Branch is in bro, bro-testing, bro-testing-private. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 13 13:19:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 13 Feb 2014 15:19:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1133: --------------------------- Status: Merge Request (was: Open) > topic/jsiwek/dns-perf > --------------------- > > Key: BIT-1133 > URL: https://bro-tracker.atlassian.net/browse/BIT-1133 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Labels: dns, performance > Fix For: 2.3 > > > Refactored DNS script's state management to improve performance (due to reduced timer volume). > Branch is in bro, bro-testing, bro-testing-private. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 13 15:55:37 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Thu, 13 Feb 2014 17:55:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1134) DNS_Mgr::LookupAddr does not respect DNS_FAKE In-Reply-To: References: Message-ID: Justin Azoff created BIT-1134: --------------------------------- Summary: DNS_Mgr::LookupAddr does not respect DNS_FAKE Key: BIT-1134 URL: https://bro-tracker.atlassian.net/browse/BIT-1134 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.2 Reporter: Justin Azoff Priority: Low -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 13 15:57:37 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Thu, 13 Feb 2014 17:57:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1134) DNS_Mgr::LookupAddr does not respect DNS_FAKE In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545#comment-15545 ] Justin Azoff commented on BIT-1134: ----------------------------------- DNS_Mgr::LookupHost checks for DNS_FAKE, but DNS_Mgr::LookupAddr which is used by the MHR policy, does not. > DNS_Mgr::LookupAddr does not respect DNS_FAKE > --------------------------------------------- > > Key: BIT-1134 > URL: https://bro-tracker.atlassian.net/browse/BIT-1134 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > Priority: Low > -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 13 17:25:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 13 Feb 2014 19:25:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546#comment-15546 ] Robin Sommer commented on BIT-1133: ----------------------------------- Merged, with the expected result: time goes down. Two "but" though: (1) I still see a 2% increase in instructions for tests.m57-short and 1.5% for tests.m57-long, is there something else in DNS that's still quite a bit more expensive now? (2) I'm wondering if max_pending_query_ids is the right model: a global threshold has the disadvantage that it doesn't scale with the traffic; it's the same for tiny networks as well as with 100Gbps links. That's ok if, independent of network size, it either rarely kicks or it really doesn't matter much at all when it kicks in; but I can't tell if that's the case here (I haven't looked too closely at the rest of the script's logic yet). So if that were a problem, is there a way to do it per flow instead (or is that what max_pending_msgs is basically already doing?) > topic/jsiwek/dns-perf > --------------------- > > Key: BIT-1133 > URL: https://bro-tracker.atlassian.net/browse/BIT-1133 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Labels: dns, performance > Fix For: 2.3 > > > Refactored DNS script's state management to improve performance (due to reduced timer volume). > Branch is in bro, bro-testing, bro-testing-private. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 13 17:27:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 13 Feb 2014 19:27:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546#comment-15546 ] Robin Sommer edited comment on BIT-1133 at 2/13/14 7:26 PM: ------------------------------------------------------------ Merged, with the expected result: time goes down. One "but" though: I still see a 2% increase in instructions for tests.m57-short and 1.5% for tests.m57-long, is there something else in DNS that's still quite a bit more expensive now? was (Author: robin): Merged, with the expected result: time goes down. Two "but" though: (1) I still see a 2% increase in instructions for tests.m57-short and 1.5% for tests.m57-long, is there something else in DNS that's still quite a bit more expensive now? (2) I'm wondering if max_pending_query_ids is the right model: a global threshold has the disadvantage that it doesn't scale with the traffic; it's the same for tiny networks as well as with 100Gbps links. That's ok if, independent of network size, it either rarely kicks or it really doesn't matter much at all when it kicks in; but I can't tell if that's the case here (I haven't looked too closely at the rest of the script's logic yet). So if that were a problem, is there a way to do it per flow instead (or is that what max_pending_msgs is basically already doing?) > topic/jsiwek/dns-perf > --------------------- > > Key: BIT-1133 > URL: https://bro-tracker.atlassian.net/browse/BIT-1133 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Labels: dns, performance > Fix For: 2.3 > > > Refactored DNS script's state management to improve performance (due to reduced timer volume). > Branch is in bro, bro-testing, bro-testing-private. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From robin at icir.org Thu Feb 13 17:27:52 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 13 Feb 2014 17:27:52 -0800 Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: <20140214012752.GF7138@icir.org> never mind my 2nd point, I didn't look right. Removed from the ticket. From jira at bro-tracker.atlassian.net Thu Feb 13 17:29:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 13 Feb 2014 19:29:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547#comment-15547 ] Robin Sommer commented on BIT-1133: ----------------------------------- never mind my 2nd point, I didn't look right. Removed from the ticket. > topic/jsiwek/dns-perf > --------------------- > > Key: BIT-1133 > URL: https://bro-tracker.atlassian.net/browse/BIT-1133 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Labels: dns, performance > Fix For: 2.3 > > > Refactored DNS script's state management to improve performance (due to reduced timer volume). > Branch is in bro, bro-testing, bro-testing-private. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 13 18:19:37 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Thu, 13 Feb 2014 20:19:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1134) DNS_Mgr::LookupAddr does not respect DNS_FAKE In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548#comment-15548 ] Justin Azoff commented on BIT-1134: ----------------------------------- Acually, it might be more that BRO_DNS_FAKE just isn't working at all. {code} # cat d.bro event bro_init() { when ( local result = lookup_hostname_txt("bro.wp.dg.cx") ) { print(result); } when ( local result2 = lookup_hostname("www.google.com") ) { print(result2); } } # BRO_DNS_FAKE=1 bro -h 2>&1|grep -i dns_fa $BRO_DNS_FAKE | disable DNS lookups (on) # BRO_DNS_FAKE=1 bro -i lo0 d.bro listening on lo0, capture length 8192 bytes Bro may refer to: Places in Sweden (Bro means bridge in Swedish): Bro, Sweden, a place in Upplands-Bro Municipality, Stockholm County, Bro or Broo, former town name of the city of Kristinehamn in Sweden http://en.wikipedia.org/wiki/Bro { 74.125.228.17, 74.125.228.16, 74.125.228.20, 74.125.228.18, 74.125.228.19, 2607:f8b0:4002:c06::67 } ^Creceived termination signal 0 packets received on interface lo0, 0 dropped {code} > DNS_Mgr::LookupAddr does not respect DNS_FAKE > --------------------------------------------- > > Key: BIT-1134 > URL: https://bro-tracker.atlassian.net/browse/BIT-1134 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > Priority: Low > -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Fri Feb 14 00:00:13 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 14 Feb 2014 00:00:13 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402140800.s1E80DKl025306@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------- BIT-1133 [1] Bro Jon Siwek - 2014-02-13 2.3 Normal topic/jsiwek/dns-perf [2] BIT-1132 [3] Bro Seth Hall Robin Sommer 2014-02-12 - Normal topic/seth/http-connect [4] BIT-1130 [5] Bro grigorescu - 2014-02-11 - Normal Fix some misidentification of SOCKS traffic BIT-1129 [6] Bro grigorescu - 2014-02-11 - Normal RADIUS Protocol Analyzer BIT-1017 [7] Bro ckanich - 2014-02-11 2.3 Normal MPLS in VLAN not considered/stripped [1] BIT-1133 https://bro-tracker.atlassian.net/browse/BIT-1133 [2] dns-perf https://github.com/bro/bro/tree/topic/jsiwek/dns-perf [3] BIT-1132 https://bro-tracker.atlassian.net/browse/BIT-1132 [4] http-connect https://github.com/bro/bro/tree/topic/seth/http-connect [5] BIT-1130 https://bro-tracker.atlassian.net/browse/BIT-1130 [6] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [7] BIT-1017 https://bro-tracker.atlassian.net/browse/BIT-1017 From jira at bro-tracker.atlassian.net Fri Feb 14 07:43:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 14 Feb 2014 09:43:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549#comment-15549 ] Jon Siwek commented on BIT-1133: -------------------------------- {quote} Merged, with the expected result: time goes down. One "but" though: I still see a 2% increase in instructions for tests.m57-short and 1.5% for tests.m57-long, is there something else in DNS that's still quite a bit more expensive now? {quote} I didn't measure instructions, but I thought I didn't see any significant *timing* difference on m57-long. There's extra conditional checks in some event handlers, there's extra events generated for unknown RR types, and there's extra weird events generated for unmatched messages, which could explain your observations. Also, are you measuring -O2 or -O0; differences in the optimized version probably warrant more emphasis/interest, right? But then that's also probably subject to more variance across arch/compiler (i.e. differing environments may make two people see differing perf changes) ? > topic/jsiwek/dns-perf > --------------------- > > Key: BIT-1133 > URL: https://bro-tracker.atlassian.net/browse/BIT-1133 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Labels: dns, performance > Fix For: 2.3 > > > Refactored DNS script's state management to improve performance (due to reduced timer volume). > Branch is in bro, bro-testing, bro-testing-private. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From robin at icir.org Fri Feb 14 07:50:53 2014 From: robin at icir.org (Robin Sommer) Date: Fri, 14 Feb 2014 07:50:53 -0800 Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: <20140214155053.GC15405@icir.org> On Fri, Feb 14, 2014 at 09:43 -0600, you wrote: > Also, are you measuring -O2 or -O0; differences in the optimized > version probably warrant more emphasis/interest, right? -O0. I'm using instructions as a proxy for "work"; so far the instruction number has turned out to be nicely stable across run on the same machine. So I'm not saying that there's a precise 2% impact but only that the DNS changes lead to now more work being done, which I think warrants understanding given that DNS is just a small of piece of everything. If we understand what's going on and deem it fine and worth the cost, no problem. I'll do some more measurements and see how actual timing looks like for me. From jira at bro-tracker.atlassian.net Fri Feb 14 07:51:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 14 Feb 2014 09:51:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15550#comment-15550 ] Robin Sommer commented on BIT-1133: ----------------------------------- -O0. I'm using instructions as a proxy for "work"; so far the instruction number has turned out to be nicely stable across run on the same machine. So I'm not saying that there's a precise 2% impact but only that the DNS changes lead to now more work being done, which I think warrants understanding given that DNS is just a small of piece of everything. If we understand what's going on and deem it fine and worth the cost, no problem. I'll do some more measurements and see how actual timing looks like for me. > topic/jsiwek/dns-perf > --------------------- > > Key: BIT-1133 > URL: https://bro-tracker.atlassian.net/browse/BIT-1133 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Labels: dns, performance > Fix For: 2.3 > > > Refactored DNS script's state management to improve performance (due to reduced timer volume). > Branch is in bro, bro-testing, bro-testing-private. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From robin at icir.org Fri Feb 14 08:00:40 2014 From: robin at icir.org (Robin Sommer) Date: Fri, 14 Feb 2014 08:00:40 -0800 Subject: [Bro-Dev] [JIRA] (BIT-1134) DNS_Mgr::LookupAddr does not respect DNS_FAKE In-Reply-To: References: Message-ID: <20140214160040.GD15405@icir.org> On Thu, Feb 13, 2014 at 20:19 -0600, you wrote: > Acually, it might be more that BRO_DNS_FAKE just isn't working at all. BRO_DNS_FAKE was historically for controlling how/if DNS constantns in scripts get resolved (i.e., set[add] = { www.google.com }). The lookup_* functions came later and might fail to take that into account. Also, I'm wondering if our test suite should set BRO_DNS_FAKE generally. It's not great because it changes quite a bit what's happening; on the other hand doing all the lookups every time the suite runs isn't ideal either. From jira at bro-tracker.atlassian.net Fri Feb 14 08:01:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 14 Feb 2014 10:01:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1134) DNS_Mgr::LookupAddr does not respect DNS_FAKE In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15551#comment-15551 ] Robin Sommer commented on BIT-1134: ----------------------------------- BRO_DNS_FAKE was historically for controlling how/if DNS constantns in scripts get resolved (i.e., set[add] = { www.google.com }). The lookup_* functions came later and might fail to take that into account. Also, I'm wondering if our test suite should set BRO_DNS_FAKE generally. It's not great because it changes quite a bit what's happening; on the other hand doing all the lookups every time the suite runs isn't ideal either. > DNS_Mgr::LookupAddr does not respect DNS_FAKE > --------------------------------------------- > > Key: BIT-1134 > URL: https://bro-tracker.atlassian.net/browse/BIT-1134 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > Priority: Low > -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 14 08:49:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 14 Feb 2014 10:49:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15552#comment-15552 ] Robin Sommer commented on BIT-1133: ----------------------------------- So with optimizations enabled, I compared user time between 741ae7a368f and master. Averaged over 3 runs each, I do see a rather consistent difference of 3-4%. The total numbers are small though, it's about one minute a run total on 2009-M57-day11-18. So dunno. If it's just necessary to get the dns.log right, there's not much we can do. And given that you aren't seeing a difference, it may also be system-specific effect. So should we close the ticket and move on? > topic/jsiwek/dns-perf > --------------------- > > Key: BIT-1133 > URL: https://bro-tracker.atlassian.net/browse/BIT-1133 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Labels: dns, performance > Fix For: 2.3 > > > Refactored DNS script's state management to improve performance (due to reduced timer volume). > Branch is in bro, bro-testing, bro-testing-private. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From seth at icir.org Fri Feb 14 09:49:33 2014 From: seth at icir.org (Seth Hall) Date: Fri, 14 Feb 2014 12:49:33 -0500 Subject: [Bro-Dev] [JIRA] (BIT-1134) DNS_Mgr::LookupAddr does not respect DNS_FAKE In-Reply-To: <20140214160040.GD15405@icir.org> References: <20140214160040.GD15405@icir.org> Message-ID: <6542C811-5A4A-486A-9226-98545C0B2921@icir.org> On Feb 14, 2014, at 11:00 AM, Robin Sommer wrote: > Also, I'm wondering if our test suite should set BRO_DNS_FAKE > generally. It's not great because it changes quite a bit what's > happening; on the other hand doing all the lookups every time the > suite runs isn't ideal either. I have a slightly different angle on that. I think it's reasonable to set BRO_DNS_FAKE (assuming it works more completely than it currently does) for the tests because eventually I don't think that the scripts in policy will be part of the base Bro distribution forever and I think it's reasonable to have them tested separately and with all the caveats that accompany testing those scripts. For the moment though, I still think it's reasonable though since our test suite shouldn't be relying on external DNS servers returning the same information forever. Perhaps we should come up with more general tests for testing that stuff to make sure that external services are still working and working like we expect them to. For example, if Team Cymru changed the output of their MHR service it would be nice to have a test that catches that so we can update appropriately but is not normally run for the performance testing. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20140214/c52a2993/attachment.bin From jira at bro-tracker.atlassian.net Fri Feb 14 09:51:38 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Fri, 14 Feb 2014 11:51:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1134) DNS_Mgr::LookupAddr does not respect DNS_FAKE In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1134: --------------------------- Attachment: signature.asc I have a slightly different angle on that. I think it's reasonable to set BRO_DNS_FAKE (assuming it works more completely than it currently does) for the tests because eventually I don't think that the scripts in policy will be part of the base Bro distribution forever and I think it's reasonable to have them tested separately and with all the caveats that accompany testing those scripts. For the moment though, I still think it's reasonable though since our test suite shouldn't be relying on external DNS servers returning the same information forever. Perhaps we should come up with more general tests for testing that stuff to make sure that external services are still working and working like we expect them to. For example, if Team Cymru changed the output of their MHR service it would be nice to have a test that catches that so we can update appropriately but is not normally run for the performance testing. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ > DNS_Mgr::LookupAddr does not respect DNS_FAKE > --------------------------------------------- > > Key: BIT-1134 > URL: https://bro-tracker.atlassian.net/browse/BIT-1134 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > Priority: Low > Attachments: signature.asc > > -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 14 10:13:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 14 Feb 2014 12:13:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15554#comment-15554 ] Jon Siwek commented on BIT-1133: -------------------------------- {quote} -O0. I'm using instructions as a proxy for "work"; so far the instruction number has turned out to be nicely stable across run on the same machine. So I'm not saying that there's a precise 2% impact but only that the DNS changes lead to now more work being done, {quote} Makes sense. {quote} which I think warrants understanding given that DNS is just a small of piece of everything {quote} Small piece relative to code or traffic? Not saying it's actually the case here (I agree, would be nice to fully understand), just that generally small changes might inevitably have observable perf. difference due traffic patterns/volume. In that case, it might be hard to fully optimize away the perf. hit without doing unrelated/low-level changes. {quote} So with optimizations enabled, I compared user time between 741ae7a368f and master. Averaged over 3 runs each, I do see a rather consistent difference of 3-4%. {quote} Looks like either system-specific difference or our methods differ. On brotestbed: {noformat} $ cc --version cc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4) $ cat /etc/issue Red Hat Enterprise Linux Server release 6.5 (Santiago) $ git checkout 741ae7a368f && git fix-submodules && ./configure --builddir=build-opt --disable-perftools && cd build-opt && make -j8 && . bro-path-dev.sh $ for i in {1..3}; do time bro -r ~/2009-M57-day11-18.trace local "Site::local_nets+={192.168.0.0/16}"; done real 0m38.847s user 0m41.886s sys 0m3.086s real 0m39.148s user 0m41.990s sys 0m3.579s real 0m38.963s user 0m42.336s sys 0m3.016s $ git checkout 7d0fbcd7b7551e9 && git fix-submodules && ./configure --builddir=build-opt --disable-perftools && cd build-opt && make -j8 && . bro-path-dev.sh $ for i in {1..3}; do time bro -r ~/2009-M57-day11-18.trace local "Site::local_nets+={192.168.0.0/16}"; done real 0m39.239s user 0m42.414s sys 0m3.033s real 0m39.829s user 0m42.580s sys 0m3.424s real 0m39.376s user 0m42.353s sys 0m3.274s {noformat} So I get avg. 42.07s user time before and avg. 42.449 user time after, about 1% diff. {quote} So dunno. If it's just necessary to get the dns.log right, there's not much we can do. And given that you aren't seeing a difference, it may also be system-specific effect. So should we close the ticket and move on? {quote} At least the results I got seem a reasonable tradeoff for pushing things towards correctness. There's probably bigger performance wins if someone were to do a more generalized profiling/optimization pass over Bro in its entirety than trying to eke out more cycles from the new DNS script logic in particular? > topic/jsiwek/dns-perf > --------------------- > > Key: BIT-1133 > URL: https://bro-tracker.atlassian.net/browse/BIT-1133 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Labels: dns, performance > Fix For: 2.3 > > > Refactored DNS script's state management to improve performance (due to reduced timer volume). > Branch is in bro, bro-testing, bro-testing-private. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 14 10:44:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 14 Feb 2014 12:44:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1133: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/jsiwek/dns-perf > --------------------- > > Key: BIT-1133 > URL: https://bro-tracker.atlassian.net/browse/BIT-1133 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Labels: dns, performance > Fix For: 2.3 > > > Refactored DNS script's state management to improve performance (due to reduced timer volume). > Branch is in bro, bro-testing, bro-testing-private. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 14 10:44:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 14 Feb 2014 12:44:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1133) topic/jsiwek/dns-perf In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15555#comment-15555 ] Robin Sommer commented on BIT-1133: ----------------------------------- I wasn't using local.bro. Rerunning with your command lines yields maybe a slightly smaller difference than before but not much. Anyways, I'm fine, closing ticket. Generally, I'm just trying to catch regressions. I agree that there's a lot more potential for optimization elsewhere. > topic/jsiwek/dns-perf > --------------------- > > Key: BIT-1133 > URL: https://bro-tracker.atlassian.net/browse/BIT-1133 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Labels: dns, performance > Fix For: 2.3 > > > Refactored DNS script's state management to improve performance (due to reduced timer volume). > Branch is in bro, bro-testing, bro-testing-private. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 14 11:26:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 14 Feb 2014 13:26:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1129) RADIUS Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15556#comment-15556 ] Robin Sommer commented on BIT-1129: ----------------------------------- Nice! Do you have a test trace? Two questions for {{scripts/base/protocols/radius/main.bro}}: - I'm not sure I understand the expiration logic: is the assumption that even after expire() has expired an entry, there'll be a further message coming in for that ID and then it will be logged? In other words, I would have expected expire() to log the entry itself. - the attribute list is a vector but no other elements than the first are used? > RADIUS Protocol Analyzer > ------------------------ > > Key: BIT-1129 > URL: https://bro-tracker.atlassian.net/browse/BIT-1129 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > > topic/vladg/radius is ready to be merged. It's been running at CMU for a few months with no issues. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 14 12:28:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 14 Feb 2014 14:28:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1017) MPLS in VLAN not considered/stripped In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1017: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > MPLS in VLAN not considered/stripped > ------------------------------------ > > Key: BIT-1017 > URL: https://bro-tracker.atlassian.net/browse/BIT-1017 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: ckanich > Fix For: 2.3 > > Attachments: mpls-in-vlan.patch, newfile.pcap > > > PktSrc.cc doesn't seem to consider mpls packets in vlan. Here's a quick patch. I couldn't figure a straightforward way to incorporate L2 stripping so there's some code duplication. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 14 12:28:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 14 Feb 2014 14:28:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1130) Fix some misidentification of SOCKS traffic In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1130: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Fix some misidentification of SOCKS traffic > ------------------------------------------- > > Key: BIT-1130 > URL: https://bro-tracker.atlassian.net/browse/BIT-1130 > Project: Bro Issue Tracker > Issue Type: Improvement > Components: Bro > Reporter: grigorescu > > Added a constraint that the SOCKS command be 1, 2 or 3, which are the defined commands. This helps to address some traffic that was being misidentified as SOCKS. Has been running at CMU without issues. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 14 23:12:37 2014 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sat, 15 Feb 2014 01:12:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15557#comment-15557 ] Vern Paxson commented on BIT-700: --------------------------------- I agree that this is better solved at the point of (multi-interface) packet capture. We added the sorter, however, so as to not presume that users can indeed change their kernels. It's certainly valid to consider that it might be more complexity than the benefits merit. > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.3 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Sat Feb 15 00:00:16 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 15 Feb 2014 00:00:16 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402150800.s1F80GIl015867@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- --------------------------- BIT-1132 [1] Bro Seth Hall Robin Sommer 2014-02-12 - Normal topic/seth/http-connect [2] BIT-1129 [3] Bro grigorescu - 2014-02-14 - Normal RADIUS Protocol Analyzer [1] BIT-1132 https://bro-tracker.atlassian.net/browse/BIT-1132 [2] http-connect https://github.com/bro/bro/tree/topic/seth/http-connect [3] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 From noreply at bro.org Sun Feb 16 00:00:12 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 16 Feb 2014 00:00:12 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402160800.s1G80CBG012617@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- --------------------------- BIT-1132 [1] Bro Seth Hall Robin Sommer 2014-02-12 - Normal topic/seth/http-connect [2] BIT-1129 [3] Bro grigorescu - 2014-02-14 - Normal RADIUS Protocol Analyzer [1] BIT-1132 https://bro-tracker.atlassian.net/browse/BIT-1132 [2] http-connect https://github.com/bro/bro/tree/topic/seth/http-connect [3] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 From noreply at bro.org Mon Feb 17 00:00:14 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 17 Feb 2014 00:00:14 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402170800.s1H80ECj019838@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- --------------------------- BIT-1132 [1] Bro Seth Hall Robin Sommer 2014-02-12 - Normal topic/seth/http-connect [2] BIT-1129 [3] Bro grigorescu - 2014-02-14 - Normal RADIUS Protocol Analyzer [1] BIT-1132 https://bro-tracker.atlassian.net/browse/BIT-1132 [2] http-connect https://github.com/bro/bro/tree/topic/seth/http-connect [3] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 From noreply at bro.org Tue Feb 18 00:00:14 2014 From: noreply at bro.org (Merge Tracker) Date: Tue, 18 Feb 2014 00:00:14 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402180800.s1I80Ext029556@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- --------------------------- BIT-1132 [1] Bro Seth Hall Robin Sommer 2014-02-12 - Normal topic/seth/http-connect [2] BIT-1129 [3] Bro grigorescu - 2014-02-14 - Normal RADIUS Protocol Analyzer [1] BIT-1132 https://bro-tracker.atlassian.net/browse/BIT-1132 [2] http-connect https://github.com/bro/bro/tree/topic/seth/http-connect [3] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 From jira at bro-tracker.atlassian.net Tue Feb 18 12:51:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 14:51:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15558#comment-15558 ] Robin Sommer commented on BIT-700: ---------------------------------- Moving this to Click should solve that as well, it can run in user-land there, though not as efficiently as if in kernel space (but probably not worse than inside Bro). I think this actually fits very nicely with the envisioned Click-layer giving us more control over packet land stuff, so I keep up my proposal of removing the code from Bro. > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.3 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 12:53:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 14:53:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1128) Add configure options for linking against jemalloc In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1128: ------------------------------ Fix Version/s: 2.3 > Add configure options for linking against jemalloc > -------------------------------------------------- > > Key: BIT-1128 > URL: https://bro-tracker.atlassian.net/browse/BIT-1128 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Fix For: 2.3 > > > To gather experiences with using jemalloc, add a configure options --with-jemalloc= that links Bro against it if found. Default should be off. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:13:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:13:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1128) Add configure options for linking against jemalloc In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1128: ------------------------------ Fix Version/s: (was: 2.4) 2.3 > Add configure options for linking against jemalloc > -------------------------------------------------- > > Key: BIT-1128 > URL: https://bro-tracker.atlassian.net/browse/BIT-1128 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Robin Sommer > Fix For: 2.3 > > > To gather experiences with using jemalloc, add a configure options --with-jemalloc= that links Bro against it if found. Default should be off. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:17:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:17:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1135) BRO_DNS_FAKE not respected everywhere In-Reply-To: References: Message-ID: Robin Sommer created BIT-1135: --------------------------------- Summary: BRO_DNS_FAKE not respected everywhere Key: BIT-1135 URL: https://bro-tracker.atlassian.net/browse/BIT-1135 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Robin Sommer Fix For: 2.3 Dynamic DNS lookups don't respect BRO_DNS_FAKE. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:17:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:17:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1136) Apply lib magic fix In-Reply-To: References: Message-ID: Robin Sommer created BIT-1136: --------------------------------- Summary: Apply lib magic fix Key: BIT-1136 URL: https://bro-tracker.atlassian.net/browse/BIT-1136 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Robin Sommer Assignee: Jon Siwek Fix For: 2.3 There's a fix for libmagic that we should apply to your version. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:23:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:23:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1137) Investigate sumstats / scan detector performance In-Reply-To: References: Message-ID: Robin Sommer created BIT-1137: --------------------------------- Summary: Investigate sumstats / scan detector performance Key: BIT-1137 URL: https://bro-tracker.atlassian.net/browse/BIT-1137 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Robin Sommer Fix For: 2.3 It's not clear if sumstats is causing more CPU and/or memory load than expected. There's also some indication that it may perform less well in standalone mode than cluster mode. Need to understand and potential improve. A part of this is also understanding how the new scan detector performs in terms of CPU/memory when compared against the 1.x version. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:25:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:25:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1138) UDP scan detection generates a large number of triggers In-Reply-To: References: Message-ID: Robin Sommer created BIT-1138: --------------------------------- Summary: UDP scan detection generates a large number of triggers Key: BIT-1138 URL: https://bro-tracker.atlassian.net/browse/BIT-1138 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Robin Sommer Fix For: 2.3 These triggers then cause high CPU load. We had a fix already but I'm not sure if it has been confirmed that it solved the problem? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:27:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:27:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1139) MHR lookups can cause significant CPU overhead in tests In-Reply-To: References: Message-ID: Robin Sommer created BIT-1139: --------------------------------- Summary: MHR lookups can cause significant CPU overhead in tests Key: BIT-1139 URL: https://bro-tracker.atlassian.net/browse/BIT-1139 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Robin Sommer Fix For: 2.3 Live operation seems fine, need to understand what's going on. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:29:39 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:29:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1140) Bloomfilter hashing problem In-Reply-To: References: Message-ID: Robin Sommer created BIT-1140: --------------------------------- Summary: Bloomfilter hashing problem Key: BIT-1140 URL: https://bro-tracker.atlassian.net/browse/BIT-1140 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Robin Sommer Assignee: Matthias Vallentin Fix For: 2.3 It seems bloomfilter hashing isn't working correctly. Has that been confirmed? Is there a fix? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:29:39 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:29:39 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1141) Investigate further improvements to file analysis performance In-Reply-To: References: Message-ID: Robin Sommer created BIT-1141: --------------------------------- Summary: Investigate further improvements to file analysis performance Key: BIT-1141 URL: https://bro-tracker.atlassian.net/browse/BIT-1141 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Robin Sommer Assignee: Jon Siwek Fix For: 2.3 Some further ideas for measuring and improving the performance of maintaining the handles were floating around. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:31:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:31:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1132) topic/seth/http-connect In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559#comment-15559 ] Robin Sommer commented on BIT-1132: ----------------------------------- It's not working yet, I'll take a look. > topic/seth/http-connect > ----------------------- > > Key: BIT-1132 > URL: https://bro-tracker.atlassian.net/browse/BIT-1132 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.3 > Reporter: Seth Hall > Assignee: Robin Sommer > Fix For: 2.3 > > > New support for the HTTP analyzer to correct support and decapsulate HTTP CONNECT proxying. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:31:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:31:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1132) topic/seth/http-connect In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1132: ------------------------------ Status: Open (was: Merge Request) > topic/seth/http-connect > ----------------------- > > Key: BIT-1132 > URL: https://bro-tracker.atlassian.net/browse/BIT-1132 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.3 > Reporter: Seth Hall > Assignee: Robin Sommer > Fix For: 2.3 > > > New support for the HTTP analyzer to correct support and decapsulate HTTP CONNECT proxying. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:31:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:31:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1132) topic/seth/http-connect In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1132: ------------------------------ Fix Version/s: 2.3 > topic/seth/http-connect > ----------------------- > > Key: BIT-1132 > URL: https://bro-tracker.atlassian.net/browse/BIT-1132 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.3 > Reporter: Seth Hall > Assignee: Robin Sommer > Fix For: 2.3 > > > New support for the HTTP analyzer to correct support and decapsulate HTTP CONNECT proxying. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:31:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:31:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1129) RADIUS Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1129: ------------------------------ Fix Version/s: 2.3 > RADIUS Protocol Analyzer > ------------------------ > > Key: BIT-1129 > URL: https://bro-tracker.atlassian.net/browse/BIT-1129 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Fix For: 2.3 > > > topic/vladg/radius is ready to be merged. It's been running at CMU for a few months with no issues. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:33:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:33:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1137) Investigate sumstats / scan detector performance In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1137: --------------------------------- Assignee: Gilbert Clark > Investigate sumstats / scan detector performance > ------------------------------------------------- > > Key: BIT-1137 > URL: https://bro-tracker.atlassian.net/browse/BIT-1137 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Assignee: Gilbert Clark > Fix For: 2.3 > > > It's not clear if sumstats is causing more CPU and/or memory load than expected. There's also some indication that it may perform less well in standalone mode than cluster mode. Need to understand and potential improve. > A part of this is also understanding how the new scan detector performs in terms of CPU/memory when compared against the 1.x version. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 13:39:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 15:39:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1139) MHR lookups can cause significant CPU overhead in tests In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1139: --------------------------------- Assignee: Gilbert Clark > MHR lookups can cause significant CPU overhead in tests > ------------------------------------------------------- > > Key: BIT-1139 > URL: https://bro-tracker.atlassian.net/browse/BIT-1139 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Assignee: Gilbert Clark > Fix For: 2.3 > > > Live operation seems fine, need to understand what's going on. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 14:07:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 18 Feb 2014 16:07:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1134) DNS_Mgr::LookupAddr does not respect DNS_FAKE In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1134: --------------------------- Fix Version/s: 2.3 > DNS_Mgr::LookupAddr does not respect DNS_FAKE > --------------------------------------------- > > Key: BIT-1134 > URL: https://bro-tracker.atlassian.net/browse/BIT-1134 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Reporter: Justin Azoff > Priority: Low > Fix For: 2.3 > > Attachments: signature.asc > > -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 14:11:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 18 Feb 2014 16:11:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1135) BRO_DNS_FAKE not respected everywhere In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560#comment-15560 ] Jon Siwek commented on BIT-1135: -------------------------------- Is this a duplicate of BIT-1134 or is there a separate part to it? > BRO_DNS_FAKE not respected everywhere > ------------------------------------- > > Key: BIT-1135 > URL: https://bro-tracker.atlassian.net/browse/BIT-1135 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Fix For: 2.3 > > > Dynamic DNS lookups don't respect BRO_DNS_FAKE. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 14:15:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 16:15:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1135) BRO_DNS_FAKE not respected everywhere In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1135: ------------------------------ Resolution: Duplicate Status: Closed (was: Open) > BRO_DNS_FAKE not respected everywhere > ------------------------------------- > > Key: BIT-1135 > URL: https://bro-tracker.atlassian.net/browse/BIT-1135 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Fix For: 2.3 > > > Dynamic DNS lookups don't respect BRO_DNS_FAKE. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 14:19:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 18 Feb 2014 16:19:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1138) UDP scan detection generates a large number of triggers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561#comment-15561 ] Jon Siwek commented on BIT-1138: -------------------------------- This was from a custom script that Aashish was running, not something distributed w/ Bro? But yeah, I don't recall if we found out if the suggestions helped. > UDP scan detection generates a large number of triggers > ------------------------------------------------------- > > Key: BIT-1138 > URL: https://bro-tracker.atlassian.net/browse/BIT-1138 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Fix For: 2.3 > > > These triggers then cause high CPU load. We had a fix already but I'm not sure if it has been confirmed that it solved the problem? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 14:41:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Feb 2014 16:41:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1138) UDP scan detection generates a large number of triggers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15562#comment-15562 ] Robin Sommer commented on BIT-1138: ----------------------------------- Yeah, I keep forgetting that we don't ship that script. But still, I would like to make sure we understand what was going on there. > UDP scan detection generates a large number of triggers > ------------------------------------------------------- > > Key: BIT-1138 > URL: https://bro-tracker.atlassian.net/browse/BIT-1138 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Fix For: 2.3 > > > These triggers then cause high CPU load. We had a fix already but I'm not sure if it has been confirmed that it solved the problem? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 18 14:43:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 18 Feb 2014 16:43:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1142) SNMP Analysis In-Reply-To: References: Message-ID: Jon Siwek created BIT-1142: ------------------------------ Summary: SNMP Analysis Key: BIT-1142 URL: https://bro-tracker.atlassian.net/browse/BIT-1142 Project: Bro Issue Tracker Issue Type: New Feature Components: BinPAC, Bro Affects Versions: git/master Reporter: Jon Siwek Assignee: Seth Hall Fix For: 2.3 /topic/jsiwek/snmp in bro, binpac, and bro-testing-private adds support for parsing SNMP datagrams. It's only absent a snmp.log. Seth, do you mind taking a look at what might make sense for a default snmp.log? I'm guessing it might look similar in concept to dns.log. A difference is I'm not sure how meaningful raw OID to value mappings will be. The code is in a merge-able state as it is in the branch/repos I mentioned, and IMO, has value even without a default snmp.log. So if you just want to flip to a merge request and postpone thinking up an snmp.log for later, I think that's fine, too. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From asharma at lbl.gov Tue Feb 18 15:38:20 2014 From: asharma at lbl.gov (Aashish Sharma) Date: Tue, 18 Feb 2014 15:38:20 -0800 Subject: [Bro-Dev] [JIRA] (BIT-1138) UDP scan detection generates a large number of triggers In-Reply-To: References: Message-ID: I haven't got chance to measure if the fix is effective or not yet. I have start measuring the CPU spikes in this week after putting in the fix for scan_udp.bro. I should have some results in a couple of days. Aashish On Tue, Feb 18, 2014 at 2:19 PM, Jon Siwek (JIRA) < jira at bro-tracker.atlassian.net> wrote: > > [ > https://bro-tracker.atlassian.net/browse/BIT-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561#comment-15561] > > Jon Siwek commented on BIT-1138: > -------------------------------- > > This was from a custom script that Aashish was running, not something > distributed w/ Bro? > > But yeah, I don't recall if we found out if the suggestions helped. > > > UDP scan detection generates a large number of triggers > > ------------------------------------------------------- > > > > Key: BIT-1138 > > URL: https://bro-tracker.atlassian.net/browse/BIT-1138 > > Project: Bro Issue Tracker > > Issue Type: Problem > > Components: Bro > > Reporter: Robin Sommer > > Fix For: 2.3 > > > > > > These triggers then cause high CPU load. We had a fix already but I'm > not sure if it has been confirmed that it solved the problem? > > > > -- > This message was sent by Atlassian JIRA > (v6.2-OD-09-036#6252) > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20140218/96120bee/attachment.html From jira at bro-tracker.atlassian.net Tue Feb 18 15:39:38 2014 From: jira at bro-tracker.atlassian.net (aashish (JIRA)) Date: Tue, 18 Feb 2014 17:39:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1138) UDP scan detection generates a large number of triggers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15563#comment-15563 ] aashish commented on BIT-1138: ------------------------------ I haven't got chance to measure if the fix is effective or not yet. I have start measuring the CPU spikes in this week after putting in the fix for scan_udp.bro. I should have some results in a couple of days. Aashish On Tue, Feb 18, 2014 at 2:19 PM, Jon Siwek (JIRA) < > UDP scan detection generates a large number of triggers > ------------------------------------------------------- > > Key: BIT-1138 > URL: https://bro-tracker.atlassian.net/browse/BIT-1138 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Fix For: 2.3 > > > These triggers then cause high CPU load. We had a fix already but I'm not sure if it has been confirmed that it solved the problem? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Wed Feb 19 00:00:15 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 19 Feb 2014 00:00:15 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402190800.s1J80FFl010008@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------ BIT-1129 [1] Bro grigorescu - 2014-02-18 2.3 Normal RADIUS Protocol Analyzer Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ---------------------------------------------- b712d64 [2] bro Bernhard Amann 2014-02-18 update 3rdparty submodule (new SQLite version) [1] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [2] b712d64 https://github.com/bro/bro/commit/b712d6436cfbfe48c0afcb0a2fd30bfd07a4687b From jira at bro-tracker.atlassian.net Wed Feb 19 05:17:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 19 Feb 2014 07:17:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1142) SNMP Analysis In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15564#comment-15564 ] Seth Hall commented on BIT-1142: -------------------------------- I'll take a look today, but you're right and I may end up falling back to just setting this to a merge request. Thanks! > SNMP Analysis > ------------- > > Key: BIT-1142 > URL: https://bro-tracker.atlassian.net/browse/BIT-1142 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BinPAC, Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Seth Hall > Fix For: 2.3 > > > /topic/jsiwek/snmp in bro, binpac, and bro-testing-private adds support for parsing SNMP datagrams. It's only absent a snmp.log. > Seth, do you mind taking a look at what might make sense for a default snmp.log? I'm guessing it might look similar in concept to dns.log. A difference is I'm not sure how meaningful raw OID to value mappings will be. > The code is in a merge-able state as it is in the branch/repos I mentioned, and IMO, has value even without a default snmp.log. So if you just want to flip to a merge request and postpone thinking up an snmp.log for later, I think that's fine, too. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 19 07:03:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 19 Feb 2014 09:03:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: Jon Siwek created BIT-1143: ------------------------------ Summary: Investigate replacing libmagic w/ signatures for file identificaiton Key: BIT-1143 URL: https://bro-tracker.atlassian.net/browse/BIT-1143 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: git/master Reporter: Jon Siwek Assignee: Jon Siwek Fix For: 2.3 I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 19 08:37:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 19 Feb 2014 10:37:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1136) Apply lib magic fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565#comment-15565 ] Jon Siwek commented on BIT-1136: -------------------------------- topic/jsiwek/new-libmagic for bro and 3rdparty repos. still plan on doing BIT-1143 to replace libmagic, but nice to have this in the meantime > Apply lib magic fix > ------------------- > > Key: BIT-1136 > URL: https://bro-tracker.atlassian.net/browse/BIT-1136 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Assignee: Jon Siwek > Fix For: 2.3 > > > There's a fix for libmagic that we should apply to your version. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 19 08:39:42 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 19 Feb 2014 10:39:42 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1136) Apply lib magic fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1136: --------------------------- Status: Merge Request (was: Open) > Apply lib magic fix > ------------------- > > Key: BIT-1136 > URL: https://bro-tracker.atlassian.net/browse/BIT-1136 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Assignee: Jon Siwek > Fix For: 2.3 > > > There's a fix for libmagic that we should apply to your version. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 19 09:51:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 19 Feb 2014 11:51:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566#comment-15566 ] Jon Siwek commented on BIT-1143: -------------------------------- A question about requirements. Bro currently uses libmagic for two types of file information -- simple mime type identification and also more verbose descriptions. E.g. "image/png" versus "PNG image data, 1435 x 170, 8-bit/color RGB, non-interlaced". Both types are exposed to users via the {{identify_data}} BIF. The generic file-over-tcp analyzer also can raise a {{file_transferred}} event that contains both types of info. Finally, the files framework only relies on the mime type. How necessary is it to keep the verbose file description functionality in absence of libmagic? The way to support it seems like it would be for the file signature regexes to include capture groups to extract all the variable info, but is that possible with Bro's regular expressions? > Investigate replacing libmagic w/ signatures for file identificaiton > -------------------------------------------------------------------- > > Key: BIT-1143 > URL: https://bro-tracker.atlassian.net/browse/BIT-1143 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.3 > > > I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 19 10:59:37 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 19 Feb 2014 12:59:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15567#comment-15567 ] Seth Hall commented on BIT-1143: -------------------------------- I actually don't like that the verbose descriptions are in there and we don't really use them anyway. The fact that it's handwritten bytestream parsing code written in C that isn't code we even wrote bothers me a quite a bit actually. I'm fine getting rid of it from the file analyzer and I don't think anyone would even notice. > Investigate replacing libmagic w/ signatures for file identificaiton > -------------------------------------------------------------------- > > Key: BIT-1143 > URL: https://bro-tracker.atlassian.net/browse/BIT-1143 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.3 > > > I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Thu Feb 20 00:00:16 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 20 Feb 2014 00:00:16 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402200800.s1K80GpJ004466@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------ ---------- ---------- ------------- ---------- ------------------------ BIT-1136 [1] Bro Robin Sommer Jon Siwek 2014-02-19 2.3 Normal Apply lib magic fix BIT-1129 [2] Bro grigorescu - 2014-02-18 2.3 Normal RADIUS Protocol Analyzer Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ---------------------------------------------- b712d64 [3] bro Bernhard Amann 2014-02-18 update 3rdparty submodule (new SQLite version) [1] BIT-1136 https://bro-tracker.atlassian.net/browse/BIT-1136 [2] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [3] b712d64 https://github.com/bro/bro/commit/b712d6436cfbfe48c0afcb0a2fd30bfd07a4687b From jira at bro-tracker.atlassian.net Thu Feb 20 07:59:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 20 Feb 2014 09:59:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569#comment-15569 ] Robin Sommer commented on BIT-1143: ----------------------------------- Just checked: the magic database covers 273 different MIME types. > Investigate replacing libmagic w/ signatures for file identificaiton > -------------------------------------------------------------------- > > Key: BIT-1143 > URL: https://bro-tracker.atlassian.net/browse/BIT-1143 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.3 > > > I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 20 07:59:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 20 Feb 2014 09:59:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568#comment-15568 ] Robin Sommer commented on BIT-1143: ----------------------------------- Agree with Seth on the verbose descriptions. While they are nice having (it's kind of cool to look at the logs and see what level of detail Bro has figured out), they don't seem worth the trouble. However I remain torn on completely replacing the MIME type detection with our own signatures. I'm concerned that we loose valuable information that way: right now, we can detect a variety of MIME types. While we don't use many of them further, even the more obscure ones get logged at least, and that seems useful. If we switch to signatures, we either have to limit the set significantly to the main cases, or we'd need to write tons of rarely used signatures that will be hard to test and maintain. Could we do a middle way: try our own signatures first and if they yield something, that's what we take. If not, use whatever libmagic reports (potentially also filtering out those cases for which we do have signatures so that libmagic won't overrule them). > Investigate replacing libmagic w/ signatures for file identificaiton > -------------------------------------------------------------------- > > Key: BIT-1143 > URL: https://bro-tracker.atlassian.net/browse/BIT-1143 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.3 > > > I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 20 11:07:38 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Thu, 20 Feb 2014 13:07:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15570#comment-15570 ] Bernhard Amann commented on BIT-1143: ------------------------------------- Just to chime in - in my experience, the more obscure mime types in libmagic are not always that helpful. Everytime I have gotten a really obscure result, it has been a misidentification... > Investigate replacing libmagic w/ signatures for file identificaiton > -------------------------------------------------------------------- > > Key: BIT-1143 > URL: https://bro-tracker.atlassian.net/browse/BIT-1143 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.3 > > > I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 20 11:59:38 2014 From: jira at bro-tracker.atlassian.net (liamrandall (JIRA)) Date: Thu, 20 Feb 2014 13:59:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571#comment-15571 ] liamrandall commented on BIT-1143: ---------------------------------- There is a really long tail of traffic here on normal networks; especially w/ new analyzers coming online. I can not speak to the accuracy of the detection of some of the more obscure types, but I can pretty easily test by doing live extractions on some production networks. > Investigate replacing libmagic w/ signatures for file identificaiton > -------------------------------------------------------------------- > > Key: BIT-1143 > URL: https://bro-tracker.atlassian.net/browse/BIT-1143 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.3 > > > I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 20 14:59:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 20 Feb 2014 16:59:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572#comment-15572 ] Jon Siwek commented on BIT-1143: -------------------------------- {quote} Could we do a middle way: try our own signatures first and if they yield something, that's what we take. If not, use whatever libmagic reports (potentially also filtering out those cases for which we do have signatures so that libmagic won't overrule them). {quote} In that case, what's gained from Bro having it's own file magic signatures instead of just using libmagic by itself? If Bro did completely switch to its own magic sigs, I think we have to do a best effort approach to porting all the current MIME magics. Tests for everything would be nice, but I don't think a test per MIME is a requirement for now. libmagic isn't exactly thoroughly tested at the moment either. We could probably just test have tests for common cases first and do obscure ones later. And I actually see keeping the dependence on libmagic as a somewhat higher maintainability cost than switching to signatures. The effort to port the magics is still unknown, but hopefully it could be done systematically or at least go fast once one understands the process of manually converting them. > Investigate replacing libmagic w/ signatures for file identificaiton > -------------------------------------------------------------------- > > Key: BIT-1143 > URL: https://bro-tracker.atlassian.net/browse/BIT-1143 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.3 > > > I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From robin at icir.org Thu Feb 20 16:01:59 2014 From: robin at icir.org (Robin Sommer) Date: Thu, 20 Feb 2014 16:01:59 -0800 Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: <20140221000159.GN84668@icir.org> On Thu, Feb 20, 2014 at 16:59 -0600, you wrote: > In that case, what's gained from Bro having it's own file magic > signatures instead of just using libmagic by itself? I was thinking better control over the matching, but I guess there's not really that much to gain in addition. > If Bro did completely switch to its own magic sigs, I think we have to > do a best effort approach to porting all the current MIME magics. Can this be (semi-)automated, i.e., converting the magic mime db into Bro regular expressions? Also, we should investigate performance: Bro's signature engine doesn't have a reputation for being the fastest in the world. :) Hard to predict how it performs compared to libmagic; but then I also don't know if it mattered much if the file type detection got slower. One more caveat, something I actually didn't think about so far: the signature engine has some depenedencies on connection state, not sure if using files as the analysis units goes without pain. > Tests for everything would be nice, but I don't think a test per MIME > is a requirement for now. Agreed. So if we can basically keep detecting all the MIME types we currently find, without hurting performance in a significant way, I'm fine fully switching. From jira at bro-tracker.atlassian.net Thu Feb 20 16:03:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 20 Feb 2014 18:03:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573#comment-15573 ] Robin Sommer commented on BIT-1143: ----------------------------------- I was thinking better control over the matching, but I guess there's not really that much to gain in addition. Can this be (semi-)automated, i.e., converting the magic mime db into Bro regular expressions? Also, we should investigate performance: Bro's signature engine doesn't have a reputation for being the fastest in the world. :) Hard to predict how it performs compared to libmagic; but then I also don't know if it mattered much if the file type detection got slower. One more caveat, something I actually didn't think about so far: the signature engine has some depenedencies on connection state, not sure if using files as the analysis units goes without pain. Agreed. So if we can basically keep detecting all the MIME types we currently find, without hurting performance in a significant way, I'm fine fully switching. > Investigate replacing libmagic w/ signatures for file identificaiton > -------------------------------------------------------------------- > > Key: BIT-1143 > URL: https://bro-tracker.atlassian.net/browse/BIT-1143 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.3 > > > I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Fri Feb 21 00:00:14 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 21 Feb 2014 00:00:14 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402210800.s1L80Ef0027133@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------ ---------- ---------- ------------- ---------- ------------------------ BIT-1136 [1] Bro Robin Sommer Jon Siwek 2014-02-19 2.3 Normal Apply lib magic fix BIT-1129 [2] Bro grigorescu - 2014-02-18 2.3 Normal RADIUS Protocol Analyzer [1] BIT-1136 https://bro-tracker.atlassian.net/browse/BIT-1136 [2] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 From jira at bro-tracker.atlassian.net Fri Feb 21 05:17:38 2014 From: jira at bro-tracker.atlassian.net (Brian Little (JIRA)) Date: Fri, 21 Feb 2014 07:17:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1144) topk_get_top returned data type In-Reply-To: References: Message-ID: Brian Little created BIT-1144: --------------------------------- Summary: topk_get_top returned data type Key: BIT-1144 URL: https://bro-tracker.atlassian.net/browse/BIT-1144 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.2 Environment: Ubuntu, both 2.2 release and 2.2.117 from git. Reporter: Brian Little Priority: Low Attachments: topk.bro I'm trying to get the top few results in a topk data type, and then loop over them to print. Running for over the results brings up an error: target to iterate over must be a table, set, vector, or string The docs say it is of type vector, and running |topk_result| brings back the correct count. example demonstration script attached -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 05:39:37 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 21 Feb 2014 07:39:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1144) topk_get_top returned data type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann reassigned BIT-1144: ----------------------------------- Assignee: Bernhard Amann > topk_get_top returned data type > ------------------------------- > > Key: BIT-1144 > URL: https://bro-tracker.atlassian.net/browse/BIT-1144 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Ubuntu, both 2.2 release and 2.2.117 from git. > Reporter: Brian Little > Assignee: Bernhard Amann > Priority: Low > Labels: topk > Attachments: topk.bro > > > I'm trying to get the top few results in a topk data type, and then loop over them to print. > Running for over the results brings up an error: > target to iterate over must be a table, set, vector, or string > The docs say it is of type vector, and running |topk_result| brings back the correct count. > example demonstration script attached -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 05:59:37 2014 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Fri, 21 Feb 2014 07:59:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1144) topk_get_top returned data type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15574#comment-15574 ] Justin Azoff commented on BIT-1144: ----------------------------------- The following works: {code} event bro_init() { local topk = topk_init(10); topk_add(topk, 1); topk_add(topk, 2); topk_add(topk, 2); topk_add(topk, 3); topk_add(topk, 3); topk_add(topk, 3); print fmt("topk size is %d", topk_size(topk)); local top2: vector of int = topk_get_top(topk, 2); print fmt("size of top2 is %d", |top2|); print top2; for (c in top2) { print top2[c]; } } {code} > topk_get_top returned data type > ------------------------------- > > Key: BIT-1144 > URL: https://bro-tracker.atlassian.net/browse/BIT-1144 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Ubuntu, both 2.2 release and 2.2.117 from git. > Reporter: Brian Little > Assignee: Bernhard Amann > Priority: Low > Labels: topk > Attachments: topk.bro > > > I'm trying to get the top few results in a topk data type, and then loop over them to print. > Running for over the results brings up an error: > target to iterate over must be a table, set, vector, or string > The docs say it is of type vector, and running |topk_result| brings back the correct count. > example demonstration script attached -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 06:03:37 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 21 Feb 2014 08:03:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1144) topk_get_top returned data type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15575#comment-15575 ] Bernhard Amann commented on BIT-1144: ------------------------------------- Yes, it is a bug, will have fixed it in a minute. > topk_get_top returned data type > ------------------------------- > > Key: BIT-1144 > URL: https://bro-tracker.atlassian.net/browse/BIT-1144 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Ubuntu, both 2.2 release and 2.2.117 from git. > Reporter: Brian Little > Assignee: Bernhard Amann > Priority: Low > Labels: topk > Attachments: topk.bro > > > I'm trying to get the top few results in a topk data type, and then loop over them to print. > Running for over the results brings up an error: > target to iterate over must be a table, set, vector, or string > The docs say it is of type vector, and running |topk_result| brings back the correct count. > example demonstration script attached -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 06:07:37 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 21 Feb 2014 08:07:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1144) topk_get_top returned data type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15576#comment-15576 ] Bernhard Amann commented on BIT-1144: ------------------------------------- Fixed in fastpath in revision 0e7d70e21924beb04cf0109e899c2b0003b55ffd. This should be merged to master soon, thank you very much for the bug report :) > topk_get_top returned data type > ------------------------------- > > Key: BIT-1144 > URL: https://bro-tracker.atlassian.net/browse/BIT-1144 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Ubuntu, both 2.2 release and 2.2.117 from git. > Reporter: Brian Little > Assignee: Bernhard Amann > Priority: Low > Labels: topk > Attachments: topk.bro > > > I'm trying to get the top few results in a topk data type, and then loop over them to print. > Running for over the results brings up an error: > target to iterate over must be a table, set, vector, or string > The docs say it is of type vector, and running |topk_result| brings back the correct count. > example demonstration script attached -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 06:07:37 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 21 Feb 2014 08:07:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1144) topk_get_top returned data type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann updated BIT-1144: -------------------------------- Status: Merge Request (was: Open) > topk_get_top returned data type > ------------------------------- > > Key: BIT-1144 > URL: https://bro-tracker.atlassian.net/browse/BIT-1144 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Ubuntu, both 2.2 release and 2.2.117 from git. > Reporter: Brian Little > Assignee: Bernhard Amann > Priority: Low > Labels: topk > Attachments: topk.bro > > > I'm trying to get the top few results in a topk data type, and then loop over them to print. > Running for over the results brings up an error: > target to iterate over must be a table, set, vector, or string > The docs say it is of type vector, and running |topk_result| brings back the correct count. > example demonstration script attached -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 07:21:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 21 Feb 2014 09:21:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1136) Apply lib magic fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1136: ------------------------------ Status: Closed (was: Merge Request) > Apply lib magic fix > ------------------- > > Key: BIT-1136 > URL: https://bro-tracker.atlassian.net/browse/BIT-1136 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Assignee: Jon Siwek > Fix For: 2.3 > > > There's a fix for libmagic that we should apply to your version. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 07:57:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 21 Feb 2014 09:57:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15577#comment-15577 ] Jon Siwek commented on BIT-1143: -------------------------------- {quote} Can this be (semi-)automated, i.e., converting the magic mime db into Bro regular expressions? {quote} That's the first approach I'd like to try. Either w/ a script to parse the mime db files or by instrumenting libmagic itself to dump Bro sigs. {quote} Also, we should investigate performance: Bro's signature engine doesn't have a reputation for being the fastest in the world. Hard to predict how it performs compared to libmagic; but then I also don't know if it mattered much if the file type detection got slower. {quote} Yeah; I planned to measure. Hopefully it's better: at a glance libmagic's matching process looked iterative so I think perf will degrade w/ number of rules; Bro's signature engine differs in that regard, right? If it's worse, then I think at least it will be worse by a predictable/consistent amount rather than being bound to libmagic's performance characteristics (which seems can vary between libmagic library releases as well as depending on the magic db content). {quote} One more caveat, something I actually didn't think about so far: the signature engine has some depenedencies on connection state, not sure if using files as the analysis units goes without pain. {quote} Yeah, the signatures are coupled with connection/analyzers (think that's why we punted on this idea last year). Currently, looking in to how much pain it is to wedge a different form of file signatures in to it. I think I understand how it might be done, though it's a bit hacky (in the sense the signature engine wasn't originally designed to accommodate this type of usage), and might be doing extra stuff that's not really necessary for "disconnected" matching. Rather than wedge it in to existing "signature" engine, another idea would be to create a new "magic" engine that parallels it. Though, I'd expect there's definitely some aspects of the signature engine that would be better to re-use existing code rather than re-invent/copy. > Investigate replacing libmagic w/ signatures for file identificaiton > -------------------------------------------------------------------- > > Key: BIT-1143 > URL: https://bro-tracker.atlassian.net/browse/BIT-1143 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.3 > > > I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 08:03:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 21 Feb 2014 10:03:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1144) topk_get_top returned data type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15578#comment-15578 ] Robin Sommer commented on BIT-1144: ----------------------------------- Getting test suite errors: {code} scripts.base.frameworks.sumstats.topk ... failed error in /home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.sumstats.topk/topk.bro, line 15: type clash in assignment (s = topk_get_top(r$topk, 5)) {code} Haven't looked into it yet. > topk_get_top returned data type > ------------------------------- > > Key: BIT-1144 > URL: https://bro-tracker.atlassian.net/browse/BIT-1144 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Ubuntu, both 2.2 release and 2.2.117 from git. > Reporter: Brian Little > Assignee: Bernhard Amann > Priority: Low > Labels: topk > Attachments: topk.bro > > > I'm trying to get the top few results in a topk data type, and then loop over them to print. > Running for over the results brings up an error: > target to iterate over must be a table, set, vector, or string > The docs say it is of type vector, and running |topk_result| brings back the correct count. > example demonstration script attached -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 08:19:37 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 21 Feb 2014 10:19:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1144) topk_get_top returned data type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann updated BIT-1144: -------------------------------- Status: Open (was: Merge Request) > topk_get_top returned data type > ------------------------------- > > Key: BIT-1144 > URL: https://bro-tracker.atlassian.net/browse/BIT-1144 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Ubuntu, both 2.2 release and 2.2.117 from git. > Reporter: Brian Little > Assignee: Bernhard Amann > Priority: Low > Labels: topk > Attachments: topk.bro > > > I'm trying to get the top few results in a topk data type, and then loop over them to print. > Running for over the results brings up an error: > target to iterate over must be a table, set, vector, or string > The docs say it is of type vector, and running |topk_result| brings back the correct count. > example demonstration script attached -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 11:45:38 2014 From: jira at bro-tracker.atlassian.net (aashish (JIRA)) Date: Fri, 21 Feb 2014 13:45:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1138) UDP scan detection generates a large number of triggers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] aashish updated BIT-1138: ------------------------- Attachment: CPU-all-scan-policies.png Memory-All-Scan-Policies.png Robin, All: Here are the graphs for a run of all scan policies (OldScan + new scan.bro, scan_udp.bro, scan_icmp.bro) from a run on a freebsd 9.1 box for approximate 3 day duration. Memory footprint continues to grow but I have noticed on other systems that memory flattens out around 11G range (after 9 day uninterrupted run). CPU is surprisingly low at on this host. (Attached graph). However on other boxes I have seen CPU being high as time progresses. It seems to me that scan_udp fix is probably working looking at this one data point. I will enable these on other DMZ boxes and lets see if we see same results. Aashish On Tue, Feb 18, 2014 at 2:41 PM, Robin Sommer (JIRA) < > UDP scan detection generates a large number of triggers > ------------------------------------------------------- > > Key: BIT-1138 > URL: https://bro-tracker.atlassian.net/browse/BIT-1138 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Fix For: 2.3 > > Attachments: CPU-all-scan-policies.png, Memory-All-Scan-Policies.png > > > These triggers then cause high CPU load. We had a fix already but I'm not sure if it has been confirmed that it solved the problem? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From asharma at lbl.gov Fri Feb 21 11:45:20 2014 From: asharma at lbl.gov (Aashish Sharma) Date: Fri, 21 Feb 2014 11:45:20 -0800 Subject: [Bro-Dev] [JIRA] (BIT-1138) UDP scan detection generates a large number of triggers In-Reply-To: References: Message-ID: Robin, All: Here are the graphs for a run of all scan policies (OldScan + new scan.bro, scan_udp.bro, scan_icmp.bro) from a run on a freebsd 9.1 box for approximate 3 day duration. Memory footprint continues to grow but I have noticed on other systems that memory flattens out around 11G range (after 9 day uninterrupted run). CPU is surprisingly low at on this host. (Attached graph). However on other boxes I have seen CPU being high as time progresses. It seems to me that scan_udp fix is probably working looking at this one data point. I will enable these on other DMZ boxes and lets see if we see same results. Aashish On Tue, Feb 18, 2014 at 2:41 PM, Robin Sommer (JIRA) < jira at bro-tracker.atlassian.net> wrote: > > [ > https://bro-tracker.atlassian.net/browse/BIT-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15562#comment-15562] > > Robin Sommer commented on BIT-1138: > ----------------------------------- > > > > > > Yeah, I keep forgetting that we don't ship that script. But still, I > would like to make sure we understand what was going on there. > > > > UDP scan detection generates a large number of triggers > > ------------------------------------------------------- > > > > Key: BIT-1138 > > URL: https://bro-tracker.atlassian.net/browse/BIT-1138 > > Project: Bro Issue Tracker > > Issue Type: Problem > > Components: Bro > > Reporter: Robin Sommer > > Fix For: 2.3 > > > > > > These triggers then cause high CPU load. We had a fix already but I'm > not sure if it has been confirmed that it solved the problem? > > > > -- > This message was sent by Atlassian JIRA > (v6.2-OD-09-036#6252) > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20140221/dcee13ac/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: Memory-All-Scan-Policies.png Type: image/png Size: 87213 bytes Desc: not available Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20140221/dcee13ac/attachment-0002.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: CPU-all-scan-policies.png Type: image/png Size: 176210 bytes Desc: not available Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20140221/dcee13ac/attachment-0003.bin From jira at bro-tracker.atlassian.net Fri Feb 21 12:09:37 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 21 Feb 2014 14:09:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1138) UDP scan detection generates a large number of triggers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15580#comment-15580 ] Jon Siwek commented on BIT-1138: -------------------------------- Aashish, can you post or link to the versions of the scripts you're running? Just for the record, and also I had some changes I tried to describe on an email thread that I don't think made it across, so if I still have any suggestions I can just modify your script and post it back to you. > UDP scan detection generates a large number of triggers > ------------------------------------------------------- > > Key: BIT-1138 > URL: https://bro-tracker.atlassian.net/browse/BIT-1138 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Fix For: 2.3 > > Attachments: CPU-all-scan-policies.png, Memory-All-Scan-Policies.png > > > These triggers then cause high CPU load. We had a fix already but I'm not sure if it has been confirmed that it solved the problem? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 12:13:37 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 21 Feb 2014 14:13:37 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1138) UDP scan detection generates a large number of triggers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581#comment-15581 ] Robin Sommer commented on BIT-1138: ----------------------------------- The CPU spikes worry me quite a bit. I can't quite tell if there's a pattern to it, i.e., if they come in regular intervals, and in particular if they align with the sumstats interval? > UDP scan detection generates a large number of triggers > ------------------------------------------------------- > > Key: BIT-1138 > URL: https://bro-tracker.atlassian.net/browse/BIT-1138 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Fix For: 2.3 > > Attachments: CPU-all-scan-policies.png, Memory-All-Scan-Policies.png > > > These triggers then cause high CPU load. We had a fix already but I'm not sure if it has been confirmed that it solved the problem? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 21 12:33:38 2014 From: jira at bro-tracker.atlassian.net (aashish (JIRA)) Date: Fri, 21 Feb 2014 14:33:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1138) UDP scan detection generates a large number of triggers In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15582#comment-15582 ] aashish commented on BIT-1138: ------------------------------ John, I am sending you the tar ball of the site-policy files in a direct email. Aashish -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Lawrence Berkeley National Laboratory http://go.lbl.gov/pgp-aashish Office: (510)-495-2680 Cell: (510)-612-7971 > UDP scan detection generates a large number of triggers > ------------------------------------------------------- > > Key: BIT-1138 > URL: https://bro-tracker.atlassian.net/browse/BIT-1138 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Fix For: 2.3 > > Attachments: CPU-all-scan-policies.png, Memory-All-Scan-Policies.png > > > These triggers then cause high CPU load. We had a fix already but I'm not sure if it has been confirmed that it solved the problem? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From robin at icir.org Fri Feb 21 13:35:15 2014 From: robin at icir.org (Robin Sommer) Date: Fri, 21 Feb 2014 13:35:15 -0800 Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: <20140221213515.GB89731@icir.org> On Fri, Feb 21, 2014 at 09:57 -0600, you wrote: > Rather than wedge it in to existing "signature" engine, another idea > would be to create a new "magic" engine that parallels it. Wedging it in is fine for now. Eventually we might end up doing a larger redesign of the signature engine to improve performance and broaden its use cases. Creating a separate magic engine now doesn't seem to be worth the effort. From jira at bro-tracker.atlassian.net Fri Feb 21 13:35:38 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 21 Feb 2014 15:35:38 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583#comment-15583 ] Robin Sommer commented on BIT-1143: ----------------------------------- Wedging it in is fine for now. Eventually we might end up doing a larger redesign of the signature engine to improve performance and broaden its use cases. Creating a separate magic engine now doesn't seem to be worth the effort. > Investigate replacing libmagic w/ signatures for file identificaiton > -------------------------------------------------------------------- > > Key: BIT-1143 > URL: https://bro-tracker.atlassian.net/browse/BIT-1143 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: Jon Siwek > Assignee: Jon Siwek > Fix For: 2.3 > > > I think it makes sense to try to make the switch from libmagic to using Bro's own signature engine for file identification before the next release. Don't want people getting used to magic file format for their own custom file identification rules. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Sat Feb 22 00:00:28 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 22 Feb 2014 00:00:28 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402220800.s1M80SPA002596@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------ BIT-1129 [1] Bro grigorescu - 2014-02-18 2.3 Normal RADIUS Protocol Analyzer Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ------------------------------------------------------------ ca2cdd8 [2] bro Bernhard Amann 2014-02-21 new TLS constants from https://tools.ietf.org/html/draft-bmo 81e561e [3] bro Bernhard Amann 2014-02-21 Revert "Correct return type of topk_get_top, addresses BIT-1 0e7d70e [4] bro Bernhard Amann 2014-02-21 Correct return type of topk_get_top, addresses BIT-1144 [1] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [2] ca2cdd8 https://github.com/bro/bro/commit/ca2cdd88615584e782564d334e703883f40f6abf [3] 81e561e https://github.com/bro/bro/commit/81e561e5dea6a00c7d70058974964434450fe292 [4] 0e7d70e https://github.com/bro/bro/commit/0e7d70e21924beb04cf0109e899c2b0003b55ffd From noreply at bro.org Sun Feb 23 00:00:42 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 23 Feb 2014 00:00:42 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402230800.s1N80g45006639@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------ BIT-1129 [1] Bro grigorescu - 2014-02-18 2.3 Normal RADIUS Protocol Analyzer Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ------------------------------------------------------------ ca2cdd8 [2] bro Bernhard Amann 2014-02-21 new TLS constants from https://tools.ietf.org/html/draft-bmo 81e561e [3] bro Bernhard Amann 2014-02-21 Revert "Correct return type of topk_get_top, addresses BIT-1 0e7d70e [4] bro Bernhard Amann 2014-02-21 Correct return type of topk_get_top, addresses BIT-1144 [1] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [2] ca2cdd8 https://github.com/bro/bro/commit/ca2cdd88615584e782564d334e703883f40f6abf [3] 81e561e https://github.com/bro/bro/commit/81e561e5dea6a00c7d70058974964434450fe292 [4] 0e7d70e https://github.com/bro/bro/commit/0e7d70e21924beb04cf0109e899c2b0003b55ffd From noreply at bro.org Mon Feb 24 00:00:34 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 24 Feb 2014 00:00:34 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402240800.s1O80Yc1012431@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------ BIT-1129 [1] Bro grigorescu - 2014-02-18 2.3 Normal RADIUS Protocol Analyzer Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ------------------------------------------------------------ ca2cdd8 [2] bro Bernhard Amann 2014-02-21 new TLS constants from https://tools.ietf.org/html/draft-bmo 81e561e [3] bro Bernhard Amann 2014-02-21 Revert "Correct return type of topk_get_top, addresses BIT-1 0e7d70e [4] bro Bernhard Amann 2014-02-21 Correct return type of topk_get_top, addresses BIT-1144 [1] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [2] ca2cdd8 https://github.com/bro/bro/commit/ca2cdd88615584e782564d334e703883f40f6abf [3] 81e561e https://github.com/bro/bro/commit/81e561e5dea6a00c7d70058974964434450fe292 [4] 0e7d70e https://github.com/bro/bro/commit/0e7d70e21924beb04cf0109e899c2b0003b55ffd From jira at bro-tracker.atlassian.net Mon Feb 24 08:19:19 2014 From: jira at bro-tracker.atlassian.net (tyler.schoenke (JIRA)) Date: Mon, 24 Feb 2014 10:19:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (TM-16) Index not working when traffic encapsulated in 802.1q trunk In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/TM-16?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15600#comment-15600 ] tyler.schoenke commented on TM-16: ---------------------------------- Hi Marek, I applied the patch, and TM appears to have stopped processing all traffic. I haven't seen any class_ files created since I restarted after the patch. The classes.timemachine.log shows no traffic being processed. The timemachine.log shows large numbers of dropped packets. Are there any configuration changes that I need to make? Here is a snippet of timemachine.log 1392643850.162819 main: TimeMachine version 0.1-4 1392643850.205792 main: Forking Daemon 1392643850.451616 main: capture started, capture thread 1392643850.452585 main: Index aggregation thread started 1392643850.452748 main: WARNING: Broccoli support not compiled in. 1392643850.452820 rmtconsole: socket ready, listening on port 42042 1392643850.534348 DROP: we dropped packets: 29181 1392644450.534820 DROP: we dropped packets: 8101887 1392645050.535250 DROP: we dropped packets: 3161247 1392645650.535675 DROP: we dropped packets: 8930749 1392646250.536146 DROP: we dropped packets: 16047665 1392646850.536640 DROP: we dropped packets: 12984575 1392647450.537066 DROP: we dropped packets: 20603271 1392648050.537473 DROP: we dropped packets: 18935480 Thanks, Tyler > Index not working when traffic encapsulated in 802.1q trunk > ----------------------------------------------------------- > > Key: TM-16 > URL: https://bro-tracker.atlassian.net/browse/TM-16 > Project: Time Machine > Issue Type: Problem > Affects Versions: git/master > Environment: Ubuntu 10.04 , pf_ring > Reporter: tyler.schoenke > Labels: 802.1Q, indexes > Attachments: tm-16.patch > > > Hi All, > When I query the time machine index, I am not receiving any results. > I just restarted time machine, and checked one of the recent class files to see there is traffic for a particular IP address. > tcpdump -e -v -n -r class_all_1385406639.023206 "vlan and host 128.138.44.198" > It shows some traffic, example: > 128.138.44.198.54014 > 74.125.225.209.443: Flags [.], cksum 0x8d2c (correct), seq 1283940799:1283940800, ack 615539104, win 16311, length 1 > 19:11:00.571731632 10:8c:cf:57:46:00 > 00:1d:09:6a:d9:a9, ethertype 802.1Q (0x8100), length 70: vlan 987, p 0, ethertype IPv4, (tos 0x0, ttl 56, id 17482, offset 0, flags [none], proto TCP (6), length 52) > When I telnet localhost 42042 and run the following command, I don't receive any results. > query to_file "128.138.44.198.pcap" index ip "128.138.44.198" > In the above tcpdump, you can see my traffic is 802.1Q trunked. I have to use the "vlan" BPF to extract it with tcpdump, and am wondering if the trunking is causing problems with indexing? > I tested the same version of time machine on non-trunked traffic, and the index works fine. > Let me know if you need any other configuration info. > Tyler -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Tue Feb 25 00:00:15 2014 From: noreply at bro.org (Merge Tracker) Date: Tue, 25 Feb 2014 00:00:15 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402250800.s1P80FBs015816@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------ BIT-1129 [1] Bro grigorescu - 2014-02-18 2.3 Normal RADIUS Protocol Analyzer Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ------------------------------------------------------------ bc75988 [2] bro Bernhard Amann 2014-02-24 More google tls extensions that are being actively used. 09c2491 [3] bro Bernhard Amann 2014-02-24 Remove unused and potentially unsafe function ListVal::Inclu [1] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [2] bc75988 https://github.com/bro/bro/commit/bc75988bd9e76bc595a086d61e2ee2fa960209a0 [3] 09c2491 https://github.com/bro/bro/commit/09c2491896971ef361e3ca339175e625c98d406e From jira at bro-tracker.atlassian.net Tue Feb 25 18:17:19 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Tue, 25 Feb 2014 20:17:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1144) topk_get_top returned data type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann updated BIT-1144: -------------------------------- Status: Merge Request (was: Open) > topk_get_top returned data type > ------------------------------- > > Key: BIT-1144 > URL: https://bro-tracker.atlassian.net/browse/BIT-1144 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Ubuntu, both 2.2 release and 2.2.117 from git. > Reporter: Brian Little > Assignee: Bernhard Amann > Priority: Low > Labels: topk > Attachments: topk.bro > > > I'm trying to get the top few results in a topk data type, and then loop over them to print. > Running for over the results brings up an error: > target to iterate over must be a table, set, vector, or string > The docs say it is of type vector, and running |topk_result| brings back the correct count. > example demonstration script attached -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Tue Feb 25 18:22:19 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Tue, 25 Feb 2014 20:22:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1144) topk_get_top returned data type In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15601#comment-15601 ] Bernhard Amann commented on BIT-1144: ------------------------------------- Second try - topic/bernhard/ticket-1144 has a new patch. I really seems to work this time and I do not think it has any side effects (all tests still pass). However, it changes parts of the basic vector implementation a bit. > topk_get_top returned data type > ------------------------------- > > Key: BIT-1144 > URL: https://bro-tracker.atlassian.net/browse/BIT-1144 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2 > Environment: Ubuntu, both 2.2 release and 2.2.117 from git. > Reporter: Brian Little > Assignee: Bernhard Amann > Priority: Low > Labels: topk > Attachments: topk.bro > > > I'm trying to get the top few results in a topk data type, and then loop over them to print. > Running for over the results brings up an error: > target to iterate over must be a table, set, vector, or string > The docs say it is of type vector, and running |topk_result| brings back the correct count. > example demonstration script attached -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Wed Feb 26 00:00:26 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 26 Feb 2014 00:00:26 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402260800.s1Q80Qmd026553@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------ -------------- ---------- ------------- ---------- ------------------------------- BIT-1144 [1] Bro Brian Little Bernhard Amann 2014-02-25 - Low topk_get_top returned data type BIT-1129 [2] Bro grigorescu - 2014-02-18 2.3 Normal RADIUS Protocol Analyzer Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ------------------------------------------------------------ bc75988 [3] bro Bernhard Amann 2014-02-24 More google tls extensions that are being actively used. 09c2491 [4] bro Bernhard Amann 2014-02-24 Remove unused and potentially unsafe function ListVal::Inclu [1] BIT-1144 https://bro-tracker.atlassian.net/browse/BIT-1144 [2] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [3] bc75988 https://github.com/bro/bro/commit/bc75988bd9e76bc595a086d61e2ee2fa960209a0 [4] 09c2491 https://github.com/bro/bro/commit/09c2491896971ef361e3ca339175e625c98d406e From jira at bro-tracker.atlassian.net Wed Feb 26 09:53:19 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Wed, 26 Feb 2014 11:53:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602#comment-15602 ] Bernhard Amann commented on BIT-700: ------------------------------------ patch to remove the packet sorter is in topic/bernhard/remove-packetsort > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.4 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 26 09:53:19 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Wed, 26 Feb 2014 11:53:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernhard Amann updated BIT-700: ------------------------------- Status: Merge Request (was: Open) > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.4 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 26 14:17:19 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 26 Feb 2014 16:17:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-123) expire-logs doesn't expire stats/* In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-123: ------------------------------ Fix Version/s: 2.3 > expire-logs doesn't expire stats/* > ---------------------------------- > > Key: BIT-123 > URL: https://bro-tracker.atlassian.net/browse/BIT-123 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Assignee: Daniel Thayer > Priority: Low > Fix For: 2.3 > > > It should however have a separate expiration period, as we might want to keep these for a different period. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 26 14:19:19 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 26 Feb 2014 16:19:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-123) expire-logs doesn't expire stats/* In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-123: ------------------------------ Status: Merge Request (was: Open) > expire-logs doesn't expire stats/* > ---------------------------------- > > Key: BIT-123 > URL: https://bro-tracker.atlassian.net/browse/BIT-123 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Assignee: Daniel Thayer > Priority: Low > Fix For: 2.3 > > > It should however have a separate expiration period, as we might want to keep these for a different period. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Wed Feb 26 14:19:19 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 26 Feb 2014 16:19:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-123) expire-logs doesn't expire stats/* In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603#comment-15603 ] Daniel Thayer commented on BIT-123: ----------------------------------- In branch topic/dnthayer/ticket123 I've added a new broctl option StatsLogExpireInterval which specifies the number of days before entries in stats.log are removed. > expire-logs doesn't expire stats/* > ---------------------------------- > > Key: BIT-123 > URL: https://bro-tracker.atlassian.net/browse/BIT-123 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Assignee: Daniel Thayer > Priority: Low > Fix For: 2.3 > > > It should however have a separate expiration period, as we might want to keep these for a different period. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Thu Feb 27 00:00:14 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 27 Feb 2014 00:00:14 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402270800.s1R80Eu2017068@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------ -------------- ---------- ------------- ---------- ---------------------------------- BIT-1144 [1] Bro Brian Little Bernhard Amann 2014-02-25 - Low topk_get_top returned data type BIT-1129 [2] Bro grigorescu - 2014-02-18 2.3 Normal RADIUS Protocol Analyzer BIT-700 [3] Bro gregor Robin Sommer 2014-02-26 2.4 Normal PacketSorter BIT-123 [4] BroControl Robin Sommer Daniel Thayer 2014-02-26 2.3 Low expire-logs doesn't expire stats/* Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ------------------------------------------------------------ 80c319b [5] bro Bernhard Amann 2014-02-26 adjust timings of a few leak tests. bc75988 [6] bro Bernhard Amann 2014-02-24 More google tls extensions that are being actively used. 09c2491 [7] bro Bernhard Amann 2014-02-24 Remove unused and potentially unsafe function ListVal::Inclu [1] BIT-1144 https://bro-tracker.atlassian.net/browse/BIT-1144 [2] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [3] BIT-700 https://bro-tracker.atlassian.net/browse/BIT-700 [4] BIT-123 https://bro-tracker.atlassian.net/browse/BIT-123 [5] 80c319b https://github.com/bro/bro/commit/80c319b522dcad8f1ee41de52d9f962fb3e615d5 [6] bc75988 https://github.com/bro/bro/commit/bc75988bd9e76bc595a086d61e2ee2fa960209a0 [7] 09c2491 https://github.com/bro/bro/commit/09c2491896971ef361e3ca339175e625c98d406e From jira at bro-tracker.atlassian.net Thu Feb 27 13:05:19 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 27 Feb 2014 15:05:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1117) Broctl base communication port should be configurable In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1117: ------------------------------- Fix Version/s: 2.3 > Broctl base communication port should be configurable > ----------------------------------------------------- > > Key: BIT-1117 > URL: https://bro-tracker.atlassian.net/browse/BIT-1117 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Reporter: Justin Azoff > Fix For: 2.3 > > > Broctl automatically assigns ports for Bro to listen on, starting with port number 47760. There is no config option to change this. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Thu Feb 27 13:07:19 2014 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 27 Feb 2014 15:07:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1117) Broctl base communication port should be configurable In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1117: ------------------------------- Status: Merge Request (was: Open) > Broctl base communication port should be configurable > ----------------------------------------------------- > > Key: BIT-1117 > URL: https://bro-tracker.atlassian.net/browse/BIT-1117 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Reporter: Justin Azoff > Fix For: 2.3 > > > Broctl automatically assigns ports for Bro to listen on, starting with port number 47760. There is no config option to change this. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From noreply at bro.org Fri Feb 28 00:00:16 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 28 Feb 2014 00:00:16 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201402280800.s1S80GSx011541@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------ -------------- ---------- ------------- ---------- ----------------------------------------------------- BIT-1144 [1] Bro Brian Little Bernhard Amann 2014-02-25 - Low topk_get_top returned data type BIT-1129 [2] Bro grigorescu - 2014-02-18 2.3 Normal RADIUS Protocol Analyzer BIT-1117 [3] BroControl Justin Azoff - 2014-02-27 2.3 Normal Broctl base communication port should be configurable BIT-700 [4] Bro gregor Robin Sommer 2014-02-26 2.4 Normal PacketSorter BIT-123 [5] BroControl Robin Sommer Daniel Thayer 2014-02-26 2.3 Low expire-logs doesn't expire stats/* Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- -------------- ---------- ------------------------------------------------------------ 80c319b [6] bro Bernhard Amann 2014-02-26 adjust timings of a few leak tests. bc75988 [7] bro Bernhard Amann 2014-02-24 More google tls extensions that are being actively used. 09c2491 [8] bro Bernhard Amann 2014-02-24 Remove unused and potentially unsafe function ListVal::Inclu [1] BIT-1144 https://bro-tracker.atlassian.net/browse/BIT-1144 [2] BIT-1129 https://bro-tracker.atlassian.net/browse/BIT-1129 [3] BIT-1117 https://bro-tracker.atlassian.net/browse/BIT-1117 [4] BIT-700 https://bro-tracker.atlassian.net/browse/BIT-700 [5] BIT-123 https://bro-tracker.atlassian.net/browse/BIT-123 [6] 80c319b https://github.com/bro/bro/commit/80c319b522dcad8f1ee41de52d9f962fb3e615d5 [7] bc75988 https://github.com/bro/bro/commit/bc75988bd9e76bc595a086d61e2ee2fa960209a0 [8] 09c2491 https://github.com/bro/bro/commit/09c2491896971ef361e3ca339175e625c98d406e From seth at icir.org Fri Feb 28 06:37:27 2014 From: seth at icir.org (Seth Hall) Date: Fri, 28 Feb 2014 09:37:27 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/bernhard/file-analysis-x509: Second try on the event interface. (7ba6bcf) In-Reply-To: <201402281104.s1SB4eKZ014903@bro-ids.icir.org> References: <201402281104.s1SB4eKZ014903@bro-ids.icir.org> Message-ID: <5E83FC66-012D-4D77-A0B5-727C68C99126@icir.org> On Feb 28, 2014, at 6:04 AM, Bernhard Amann wrote: > -event x509_extension(f: fa_file, ext: X509::Extension) > +event x509_extension(f: fa_file, cert: X509::Certificate, ext: X509::Extension) Would it make more sense to leave the cert out? Seems like state we should collect in script land instead of passing it through from the core each time. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20140228/b20b2968/attachment.bin From bernhard at ICSI.Berkeley.EDU Fri Feb 28 08:01:17 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Fri, 28 Feb 2014 08:01:17 -0800 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/bernhard/file-analysis-x509: Second try on the event interface. (7ba6bcf) In-Reply-To: <5E83FC66-012D-4D77-A0B5-727C68C99126@icir.org> References: <201402281104.s1SB4eKZ014903@bro-ids.icir.org> <5E83FC66-012D-4D77-A0B5-727C68C99126@icir.org> Message-ID: On Feb 28, 2014, at 6:37 AM, Seth Hall wrote: > > On Feb 28, 2014, at 6:04 AM, Bernhard Amann wrote: > >> -event x509_extension(f: fa_file, ext: X509::Extension) >> +event x509_extension(f: fa_file, cert: X509::Certificate, ext: X509::Extension) > > Would it make more sense to leave the cert out? Seems like state we should collect in script land instead of passing it through from the core each time. The ?cert? only is a record in the events. So - the only thing that is passed around is a ref-counted pointer. The actual certificate string is not passed to script land anymore (when I am finished you will be able to get it if you really want to, but it will not be exposed by default). An opaque type is passed around - this makes certificate verification possible without having to re-parse them with OpenSSL. I thought that that is ok. Or are you meaning something else? Bernhard From bernhard at ICSI.Berkeley.EDU Fri Feb 28 09:02:53 2014 From: bernhard at ICSI.Berkeley.EDU (Bernhard Amann) Date: Fri, 28 Feb 2014 09:02:53 -0800 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/bernhard/file-analysis-x509: Second try on the event interface. (7ba6bcf) In-Reply-To: References: <201402281104.s1SB4eKZ014903@bro-ids.icir.org> <5E83FC66-012D-4D77-A0B5-727C68C99126@icir.org> Message-ID: On Feb 28, 2014, at 8:01 AM, Bernhard Amann wrote: > > On Feb 28, 2014, at 6:37 AM, Seth Hall wrote: > >> >> On Feb 28, 2014, at 6:04 AM, Bernhard Amann wrote: >> >>> -event x509_extension(f: fa_file, ext: X509::Extension) >>> +event x509_extension(f: fa_file, cert: X509::Certificate, ext: X509::Extension) >> >> Would it make more sense to leave the cert out? Seems like state we should collect in script land instead of passing it through from the core each time. > > The ?cert? only is a record in the events. So - the only thing that is passed around is a ref-counted > pointer. The actual certificate string is not passed to script land anymore (when I am finished you > will be able to get it if you really want to, but it will not be exposed by default). > > An opaque type is passed around - this makes certificate verification possible without having to re-parse > them with OpenSSL. > > I thought that that is ok. Or are you meaning something else? Followup - Seth convinced me that I am doing it wrong :) The record will disappear from the extension events. Bernhard From seth at icir.org Fri Feb 28 09:03:38 2014 From: seth at icir.org (Seth Hall) Date: Fri, 28 Feb 2014 12:03:38 -0500 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/bernhard/file-analysis-x509: Second try on the event interface. (7ba6bcf) In-Reply-To: References: <201402281104.s1SB4eKZ014903@bro-ids.icir.org> <5E83FC66-012D-4D77-A0B5-727C68C99126@icir.org> Message-ID: On Feb 28, 2014, at 11:01 AM, Bernhard Amann wrote: > I thought that that is ok. Or are you meaning something else? Just to record it, we talked through this and it only seems like the right thing to do at the moment because we're using openssl underneath for parsing but eventually we won't and this results in a situation where the core is required to store state even if the user doesn't necessarily want the state. We also have mechanisms in script land now (record redefining) that make storing the state on-demand easy. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20140228/daa7403e/attachment.bin From jira at bro-tracker.atlassian.net Fri Feb 28 14:40:19 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 28 Feb 2014 16:40:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1129) RADIUS Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1129: ------------------------------ Status: Open (was: Merge Request) > RADIUS Protocol Analyzer > ------------------------ > > Key: BIT-1129 > URL: https://bro-tracker.atlassian.net/browse/BIT-1129 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Vlad Grigorescu > Fix For: 2.3 > > > topic/vladg/radius is ready to be merged. It's been running at CMU for a few months with no issues. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 28 14:40:19 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 28 Feb 2014 16:40:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1129) RADIUS Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1129: --------------------------------- Assignee: Vlad Grigorescu > RADIUS Protocol Analyzer > ------------------------ > > Key: BIT-1129 > URL: https://bro-tracker.atlassian.net/browse/BIT-1129 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Vlad Grigorescu > Fix For: 2.3 > > > topic/vladg/radius is ready to be merged. It's been running at CMU for a few months with no issues. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 28 14:55:19 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 28 Feb 2014 16:55:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604#comment-15604 ] Robin Sommer commented on BIT-700: ---------------------------------- What's this? :) Delete? {code} + case DLT_IEEE802_11: + { + printf("Here\n"); + exit(0); + } + {code} > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.4 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 28 14:59:19 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 28 Feb 2014 16:59:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15605#comment-15605 ] Bernhard Amann commented on BIT-700: ------------------------------------ Uh. Sorry. Und ich dachte, dass ich einmal einen problemfreien patch hinbekommen hab? das hab ich vergessen :( > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.4 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 28 14:59:19 2014 From: jira at bro-tracker.atlassian.net (Bernhard Amann (JIRA)) Date: Fri, 28 Feb 2014 16:59:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15605#comment-15605 ] Bernhard Amann edited comment on BIT-700 at 2/28/14 4:59 PM: ------------------------------------------------------------- sorry, me bad. Yes, delete. That was kind of the motivation for the patch :) was (Author: amannb): Uh. Sorry. Und ich dachte, dass ich einmal einen problemfreien patch hinbekommen hab? das hab ich vergessen :( > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.4 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 28 15:27:19 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 28 Feb 2014 17:27:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-700) PacketSorter In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-700: ----------------------------- Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > PacketSorter > ------------ > > Key: BIT-700 > URL: https://bro-tracker.atlassian.net/browse/BIT-700 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: gregor > Assignee: Robin Sommer > Labels: BroV6,, IPv6 > Fix For: 2.4 > > > (from an e-mail I sent a while ago) > Might relevant for IPv6 so setting milestone to 2.1 > Hi, > I was wondering about Bro's packet sorter. From a quick glance it > appears that it's only enabled if packet_sort_window is set to a non > zero value. When enabled it will sort packets > a) based on timestamps and > b) for TCP packets based on SEQ/ACK numbers (I presume to ensure that > ACKs are delivered after the data packet) > Note, this is independent from Bro's ability to process multiple trace > files (or multiple interfaces) in order. So I was wondering about the > use cases for PacketSorter, especially (a) > If the packet sorter is enabled Bro's behavior will slightly change: It > won't pass ARP packets to the ARP analyzer, and it won't create a weird > if it's not an IP packet. > I was just wondering whether anybody has recently used the packet > sorter. If not I'm wondering whether we should test this code path to > see whether it works correctly esp wrt IPv6. > Or, actually, whether the packet sorter is worth keeping or whether we > should remove the code. > And another question would be if the TCP sorting would better be handled > by the TCP analyzer? > Opinions? -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 28 15:27:19 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 28 Feb 2014 17:27:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-123) expire-logs doesn't expire stats/* In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-123: ----------------------------- Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > expire-logs doesn't expire stats/* > ---------------------------------- > > Key: BIT-123 > URL: https://bro-tracker.atlassian.net/browse/BIT-123 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 1.5.2 > Reporter: Robin Sommer > Assignee: Daniel Thayer > Priority: Low > Fix For: 2.3 > > > It should however have a separate expiration period, as we might want to keep these for a different period. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252) From jira at bro-tracker.atlassian.net Fri Feb 28 15:27:19 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 28 Feb 2014 17:27:19 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1117) Broctl base communication port should be configurable In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1117: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Broctl base communication port should be configurable > ----------------------------------------------------- > > Key: BIT-1117 > URL: https://bro-tracker.atlassian.net/browse/BIT-1117 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: BroControl > Reporter: Justin Azoff > Fix For: 2.3 > > > Broctl automatically assigns ports for Bro to listen on, starting with port number 47760. There is no config option to change this. -- This message was sent by Atlassian JIRA (v6.2-OD-09-036#6252)