From noreply at bro.org Sat Nov 1 00:00:44 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 1 Nov 2014 00:00:44 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411010700.sA170iSH028532@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ----------------------- BIT-1285 [1] Bro grigorescu - 2014-10-31 - Normal MySQL Protocol Analyzer [1] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 From noreply at bro.org Sun Nov 2 00:00:38 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 2 Nov 2014 00:00:38 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411020700.sA270cvX012596@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ----------------------- BIT-1285 [1] Bro grigorescu - 2014-10-31 - Normal MySQL Protocol Analyzer Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ---------------------------------------------------- 705989d [2] bro Johanna Amann 2014-11-01 add new curves from draft-ietf-tls-negotiated-ff-dhe [1] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 [2] 705989d https://github.com/bro/bro/commit/705989da39a89074849b8d1e4a2cc9588f8a3a28 From noreply at bro.org Mon Nov 3 00:00:31 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 3 Nov 2014 00:00:31 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411030800.sA380Vdj025589@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ----------------------- BIT-1285 [1] Bro grigorescu - 2014-10-31 - Normal MySQL Protocol Analyzer Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ---------------------------------------------------- 705989d [2] bro Johanna Amann 2014-11-01 add new curves from draft-ietf-tls-negotiated-ff-dhe [1] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 [2] 705989d https://github.com/bro/bro/commit/705989da39a89074849b8d1e4a2cc9588f8a3a28 From jira at bro-tracker.atlassian.net Mon Nov 3 07:33:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 3 Nov 2014 09:33:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1283) Bro crashes when using &encrypt In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18700#comment-18700 ] Jon Siwek commented on BIT-1283: -------------------------------- {quote} Bro 1.5 came with a tool bdcat that decrypts these files. I'm reopening the ticket to see if we want to bring that back. {quote} Just noticed "log_encryption_key" is marked deprecated, so maybe we should actually be removing things instead of fixing/adding ? > Bro crashes when using &encrypt > ------------------------------- > > Key: BIT-1283 > URL: https://bro-tracker.atlassian.net/browse/BIT-1283 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: bro version 2.3-263-debug > Reporter: AK > Fix For: 2.4 > > > Bro crashes when applying the &encrypt attribute when opening a file. > bro -Ci eth0 -e 'global f1: file = open("f.out") &encrypt;' -- This message was sent by Atlassian JIRA (v6.4-OD-09-005#64005) From robin at icir.org Mon Nov 3 07:54:48 2014 From: robin at icir.org (Robin Sommer) Date: Mon, 3 Nov 2014 07:54:48 -0800 Subject: [Bro-Dev] [JIRA] (BIT-1283) Bro crashes when using &encrypt In-Reply-To: References: Message-ID: <20141103155448.GB82998@icir.org> On Mon, Nov 03, 2014 at 09:33 -0600, you wrote: > Just noticed "log_encryption_key" is marked deprecated, so maybe we > should actually be removing things instead of fixing/adding ? I don't remember if we discussed this already at some point, which may have then led to the deprecation. I'm fine either way. It's a nice capability in principle, but given that files aren't our main logging mechanism anymore, it's unlikely anybody is actually using it. So in the spirit of removing complexity, maybe that's indeed the right thing to do. Indpendent of that, an item for the todo list is adding encryption support to the logging framework. But that needs a larger project, I'm hoping that we can do better than just encrypt a whole file with a single key. Would be nice to give out partial access in some form, but not quite sure how that would look like. Robin From jira at bro-tracker.atlassian.net Mon Nov 3 07:55:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 3 Nov 2014 09:55:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1283) Bro crashes when using &encrypt In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18701#comment-18701 ] Robin Sommer commented on BIT-1283: ----------------------------------- I don't remember if we discussed this already at some point, which may have then led to the deprecation. I'm fine either way. It's a nice capability in principle, but given that files aren't our main logging mechanism anymore, it's unlikely anybody is actually using it. So in the spirit of removing complexity, maybe that's indeed the right thing to do. Indpendent of that, an item for the todo list is adding encryption support to the logging framework. But that needs a larger project, I'm hoping that we can do better than just encrypt a whole file with a single key. Would be nice to give out partial access in some form, but not quite sure how that would look like. Robin > Bro crashes when using &encrypt > ------------------------------- > > Key: BIT-1283 > URL: https://bro-tracker.atlassian.net/browse/BIT-1283 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: bro version 2.3-263-debug > Reporter: AK > Fix For: 2.4 > > > Bro crashes when applying the &encrypt attribute when opening a file. > bro -Ci eth0 -e 'global f1: file = open("f.out") &encrypt;' -- This message was sent by Atlassian JIRA (v6.4-OD-09-005#64005) From jira at bro-tracker.atlassian.net Mon Nov 3 10:56:07 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Mon, 3 Nov 2014 12:56:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1286) Add policy script for Windows version detection via CryptoAPI HTTP Traffic In-Reply-To: References: Message-ID: grigorescu created BIT-1286: ------------------------------- Summary: Add policy script for Windows version detection via CryptoAPI HTTP Traffic Key: BIT-1286 URL: https://bro-tracker.atlassian.net/browse/BIT-1286 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: git/master Reporter: grigorescu Windows systems access a Microsoft Certificate Revocation List (CRL) periodically. The user agent for these requests reveals which version of Crypt32.dll installed on the system, which can uniquely identify the version of Windows that's running. This branch adds a Software framework policy script will log the version of Windows that was identified. -- This message was sent by Atlassian JIRA (v6.4-OD-09-005#64005) From jira at bro-tracker.atlassian.net Mon Nov 3 10:56:07 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Mon, 3 Nov 2014 12:56:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1286) Add policy script for Windows version detection via CryptoAPI HTTP Traffic In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] grigorescu updated BIT-1286: ---------------------------- Status: Merge Request (was: Open) > Add policy script for Windows version detection via CryptoAPI HTTP Traffic > -------------------------------------------------------------------------- > > Key: BIT-1286 > URL: https://bro-tracker.atlassian.net/browse/BIT-1286 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > > Windows systems access a Microsoft Certificate Revocation List (CRL) periodically. The user agent for these requests reveals which version of Crypt32.dll installed on the system, which can uniquely identify the version of Windows that's running. > This branch adds a Software framework policy script will log the version of Windows that was identified. -- This message was sent by Atlassian JIRA (v6.4-OD-09-005#64005) From jira at bro-tracker.atlassian.net Mon Nov 3 10:56:07 2014 From: jira at bro-tracker.atlassian.net (grigorescu (JIRA)) Date: Mon, 3 Nov 2014 12:56:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1286) Add policy script for Windows version detection via CryptoAPI HTTP Traffic In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18702#comment-18702 ] grigorescu commented on BIT-1286: --------------------------------- Forgot to mention the branch :-). It's in topic/vladg/cryptoapi > Add policy script for Windows version detection via CryptoAPI HTTP Traffic > -------------------------------------------------------------------------- > > Key: BIT-1286 > URL: https://bro-tracker.atlassian.net/browse/BIT-1286 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > > Windows systems access a Microsoft Certificate Revocation List (CRL) periodically. The user agent for these requests reveals which version of Crypt32.dll installed on the system, which can uniquely identify the version of Windows that's running. > This branch adds a Software framework policy script will log the version of Windows that was identified. -- This message was sent by Atlassian JIRA (v6.4-OD-09-005#64005) From asharma at lbl.gov Mon Nov 3 11:05:56 2014 From: asharma at lbl.gov (Aashish Sharma) Date: Mon, 3 Nov 2014 11:05:56 -0800 Subject: [Bro-Dev] [JIRA] (BIT-1286) Add policy script for Windows version detection via CryptoAPI HTTP Traffic In-Reply-To: References: Message-ID: <20141103190554.GJ23526@yaksha.lbl.gov> This is a very neat policy for sure!! On Mon, Nov 03, 2014 at 12:56:07PM -0600, grigorescu (JIRA) wrote: > > [ https://bro-tracker.atlassian.net/browse/BIT-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18702#comment-18702 ] > > grigorescu commented on BIT-1286: > --------------------------------- > > Forgot to mention the branch :-). It's in topic/vladg/cryptoapi > > > Add policy script for Windows version detection via CryptoAPI HTTP Traffic > > -------------------------------------------------------------------------- > > > > Key: BIT-1286 > > URL: https://bro-tracker.atlassian.net/browse/BIT-1286 > > Project: Bro Issue Tracker > > Issue Type: New Feature > > Components: Bro > > Affects Versions: git/master > > Reporter: grigorescu > > > > Windows systems access a Microsoft Certificate Revocation List (CRL) periodically. The user agent for these requests reveals which version of Crypt32.dll installed on the system, which can uniquely identify the version of Windows that's running. > > This branch adds a Software framework policy script will log the version of Windows that was identified. > > > > -- > This message was sent by Atlassian JIRA > (v6.4-OD-09-005#64005) > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Lawrence Berkeley National Laboratory http://go.lbl.gov/pgp-aashish Office: (510)-495-2680 Cell: (510)-612-7971 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141103/1af739bb/attachment.bin From jira at bro-tracker.atlassian.net Mon Nov 3 11:07:08 2014 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Mon, 3 Nov 2014 13:07:08 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1286) Add policy script for Windows version detection via CryptoAPI HTTP Traffic In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18703#comment-18703 ] Aashish Sharma commented on BIT-1286: ------------------------------------- This is a very neat policy for sure!! -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Lawrence Berkeley National Laboratory http://go.lbl.gov/pgp-aashish Office: (510)-495-2680 Cell: (510)-612-7971 > Add policy script for Windows version detection via CryptoAPI HTTP Traffic > -------------------------------------------------------------------------- > > Key: BIT-1286 > URL: https://bro-tracker.atlassian.net/browse/BIT-1286 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > > Windows systems access a Microsoft Certificate Revocation List (CRL) periodically. The user agent for these requests reveals which version of Crypt32.dll installed on the system, which can uniquely identify the version of Windows that's running. > This branch adds a Software framework policy script will log the version of Windows that was identified. -- This message was sent by Atlassian JIRA (v6.4-OD-09-005#64005) From robin at icir.org Mon Nov 3 13:48:17 2014 From: robin at icir.org (Robin Sommer) Date: Mon, 3 Nov 2014 13:48:17 -0800 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <20141023194704.GH97490@icir.org> <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> Message-ID: <20141103214817.GA57338@icir.org> On Thu, Oct 23, 2014 at 21:59 +0000, you wrote: > I started looking in to this a little and I?m thinking either LevelDB > or RocksDB may be good default choices to use here. I looked over them a bit, and RocksDB looks pretty cool, although also quite complex given that we won't need all of what it offers. Have you considered SQLite as an alternative? It's more than a key/value store, and slower, but it would have the advantage of not adding another dependency beyond what we already use. Not saying that's what we should do, just wondering about the pros and cons. Also, I was thinking it would be cool to have a command line tool that can inspect (and potentially even manipulate [1]), the contents of a Broker store. Say, you wanted to see what IPs are currently tracked in some table, you could just run that tool to dump it out. Robin (*) Does any of the DBs have support for modifying a table exernally while being open? Then that command line tool could even add/change entries that way. That would actually make for a nice configuration mechanism for things like whitelists or some tuning options. -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From robin at icir.org Mon Nov 3 13:50:17 2014 From: robin at icir.org (Robin Sommer) Date: Mon, 3 Nov 2014 13:50:17 -0800 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <20141023194704.GH97490@icir.org> <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> Message-ID: <20141103215017.GB57338@icir.org> Jon, When I tried compiling Broker with clang 3.5 (and it's libc++) the other day I got some compiler errors. I'll look more closely later, but was wondering what compiler are you using for development? Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jsiwek at illinois.edu Mon Nov 3 14:58:12 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Mon, 3 Nov 2014 22:58:12 +0000 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <20141103214817.GA57338@icir.org> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <20141023194704.GH97490@icir.org> <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> <20141103214817.GA57338@icir.org> Message-ID: <269C5AD6-25CC-4F17-91A9-D415214C6DCA@illinois.edu> > On Nov 3, 2014, at 3:48 PM, Robin Sommer wrote: > > > > On Thu, Oct 23, 2014 at 21:59 +0000, you wrote: > >> I started looking in to this a little and I?m thinking either LevelDB >> or RocksDB may be good default choices to use here. > > I looked over them a bit, and RocksDB looks pretty cool, although also > quite complex given that we won't need all of what it offers. > > Have you considered SQLite as an alternative? Had not thought of that. > It's more than a > key/value store, and slower, but it would have the advantage of not > adding another dependency beyond what we already use. I wasn?t that worried about adding another dependency since they?re already kind of specific, i.e. don?t expect the DB dependency to be more of a hassle than libcaf. If it is a concern, we could consider distributing (e.g. as git submodules) and building these dependencies along with Broker (analogous to Bro redistributing the sqlite amalgamation). In the end, I?m not expecting there to actually be a lot of code involved in implementing different persistent storage backends, so we wouldn?t be stuck with SQLite if that?s chosen as a default. And we could provide more than one option at a time. Maybe we could have the default be SQLite (for convenience), but optionally support RocksDB (for those in need of better performance). > Also, I was thinking it would be cool to have a command line tool that > can inspect (and potentially even manipulate [1]), the contents of a > Broker store. Say, you wanted to see what IPs are currently tracked in > some table, you could just run that tool to dump it out. > > (*) Does any of the DBs have support for modifying a table exernally > while being open? Then that command line tool could even add/change > entries that way. That would actually make for a nice configuration > mechanism for things like whitelists or some tuning options. We need to be using Broker?s data store abstraction when making changes for those updates to be correctly propagated to clones. But should be easy to write such tools using libbroker. Or once there?s Python bindings, those will probably be a natural way to do such dynamic querying and modification to data stores. - Jon From jsiwek at illinois.edu Mon Nov 3 14:59:30 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Mon, 3 Nov 2014 22:59:30 +0000 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <20141103215017.GB57338@icir.org> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <20141023194704.GH97490@icir.org> <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> <20141103215017.GB57338@icir.org> Message-ID: > On Nov 3, 2014, at 3:50 PM, Robin Sommer wrote: > > When I tried compiling Broker with clang 3.5 (and it's libc++) the > other day I got some compiler errors. I'll look more closely later, > but was wondering what compiler are you using for development? $ c++ --version Apple LLVM version 6.0 (clang-600.0.54) (based on LLVM 3.5svn) - Jon From robin at icir.org Mon Nov 3 15:02:17 2014 From: robin at icir.org (Robin Sommer) Date: Mon, 3 Nov 2014 15:02:17 -0800 Subject: [Bro-Dev] libbroker status/plans In-Reply-To: <269C5AD6-25CC-4F17-91A9-D415214C6DCA@illinois.edu> References: <32B166E7-D88F-488C-87B7-204E4405BB2E@illinois.edu> <20141023194704.GH97490@icir.org> <7DE53C59-B9BB-4F35-B793-1170B0823096@illinois.edu> <20141103214817.GA57338@icir.org> <269C5AD6-25CC-4F17-91A9-D415214C6DCA@illinois.edu> Message-ID: <20141103230217.GO82998@icir.org> On Mon, Nov 03, 2014 at 22:58 +0000, you wrote: > wouldn?t be stuck with SQLite if that?s chosen as a default. And we > could provide more than one option at a time. Maybe we could have the > default be SQLite (for convenience), but optionally support RocksDB > (for those in need of better performance). That would sound good to me. SQlite is somethign we can ship (just as with Bro), whereas RocksDB seems complex enough that leaving it external may be better. > We need to be using Broker?s data store abstraction when making > changes for those updates to be correctly propagated to clones. But > should be easy to write such tools using libbroker. Ah, good point. And yeah, working without Broker wouldn't work anyways. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From noreply at bro.org Tue Nov 4 00:00:18 2014 From: noreply at bro.org (Merge Tracker) Date: Tue, 4 Nov 2014 00:00:18 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411040800.sA480IcQ014183@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- -------------------------------------------------------------------------- BIT-1286 [1] Bro grigorescu - 2014-11-03 - Normal Add policy script for Windows version detection via CryptoAPI HTTP Traffic BIT-1285 [2] Bro grigorescu - 2014-10-31 - Normal MySQL Protocol Analyzer [1] BIT-1286 https://bro-tracker.atlassian.net/browse/BIT-1286 [2] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 From noreply at bro.org Wed Nov 5 00:00:43 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 5 Nov 2014 00:00:43 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411050800.sA580hQ4006377@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- -------------------------------------------------------------------------- BIT-1286 [1] Bro grigorescu - 2014-11-03 - Normal Add policy script for Windows version detection via CryptoAPI HTTP Traffic BIT-1285 [2] Bro grigorescu - 2014-10-31 - Normal MySQL Protocol Analyzer [1] BIT-1286 https://bro-tracker.atlassian.net/browse/BIT-1286 [2] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 From jira at bro-tracker.atlassian.net Wed Nov 5 06:28:07 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 5 Nov 2014 08:28:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1286) Add policy script for Windows version detection via CryptoAPI HTTP Traffic In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall reassigned BIT-1286: ------------------------------ Assignee: Seth Hall > Add policy script for Windows version detection via CryptoAPI HTTP Traffic > -------------------------------------------------------------------------- > > Key: BIT-1286 > URL: https://bro-tracker.atlassian.net/browse/BIT-1286 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Seth Hall > > Windows systems access a Microsoft Certificate Revocation List (CRL) periodically. The user agent for these requests reveals which version of Crypt32.dll installed on the system, which can uniquely identify the version of Windows that's running. > This branch adds a Software framework policy script will log the version of Windows that was identified. -- This message was sent by Atlassian JIRA (v6.4-OD-09-005#64005) From jira at bro-tracker.atlassian.net Wed Nov 5 06:49:07 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 5 Nov 2014 08:49:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1286) Add policy script for Windows version detection via CryptoAPI HTTP Traffic In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1286: --------------------------- Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Add policy script for Windows version detection via CryptoAPI HTTP Traffic > -------------------------------------------------------------------------- > > Key: BIT-1286 > URL: https://bro-tracker.atlassian.net/browse/BIT-1286 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Seth Hall > > Windows systems access a Microsoft Certificate Revocation List (CRL) periodically. The user agent for these requests reveals which version of Crypt32.dll installed on the system, which can uniquely identify the version of Windows that's running. > This branch adds a Software framework policy script will log the version of Windows that was identified. -- This message was sent by Atlassian JIRA (v6.4-OD-09-005#64005) From jira at bro-tracker.atlassian.net Wed Nov 5 10:52:08 2014 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 5 Nov 2014 12:52:08 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18704#comment-18704 ] Seth Hall commented on BIT-1238: -------------------------------- This issue is now being addressed through the branch named: topic/seth/files-reassembly-and-mime-updates It's going to be updates for file reassembly and the large batch of mime type identification that I have been preparing. That branch also has your x-tar issue addressed. I looked at a number of tar files and updated the pattern beyond the one that you created (it still was doing some incorrect matching for me). > High false-positive for application/x-tar signature > --------------------------------------------------- > > Key: BIT-1238 > URL: https://bro-tracker.atlassian.net/browse/BIT-1238 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Brian O'Berry > Assignee: Seth Hall > Labels: file, mime, signature > Attachments: test.tar.gz > > > The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig. > {code} > signature file-tar { > file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/ > file-mime "application/x-tar", 150 > } > {code} -- This message was sent by Atlassian JIRA (v6.4-OD-09-005#64005) From noreply at bro.org Thu Nov 6 00:00:28 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 6 Nov 2014 00:00:28 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411060800.sA680Smp015998@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ----------------------- BIT-1285 [1] Bro grigorescu - 2014-10-31 - Normal MySQL Protocol Analyzer Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [2] bro anthonykasza [3] 2014-11-06 URI Parsing Function [4] [1] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 [2] Pull Request #17 https://github.com/bro/bro/pull/17 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From noreply at bro.org Fri Nov 7 00:00:30 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 7 Nov 2014 00:00:30 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411070800.sA780UvM017367@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ----------------------- BIT-1285 [1] Bro grigorescu - 2014-10-31 - Normal MySQL Protocol Analyzer Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [2] bro anthonykasza [3] 2014-11-07 URI Parsing Function [4] [1] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 [2] Pull Request #17 https://github.com/bro/bro/pull/17 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From noreply at bro.org Sat Nov 8 00:00:26 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 8 Nov 2014 00:00:26 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411080800.sA880Q8H015274@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ----------------------- BIT-1285 [1] Bro grigorescu - 2014-10-31 - Normal MySQL Protocol Analyzer Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [2] bro anthonykasza [3] 2014-11-07 URI Parsing Function [4] [1] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 [2] Pull Request #17 https://github.com/bro/bro/pull/17 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From noreply at bro.org Sun Nov 9 00:00:31 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 9 Nov 2014 00:00:31 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411090800.sA980Vn2025712@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ----------------------- BIT-1285 [1] Bro grigorescu - 2014-10-31 - Normal MySQL Protocol Analyzer Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [2] bro anthonykasza [3] 2014-11-07 URI Parsing Function [4] [1] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 [2] Pull Request #17 https://github.com/bro/bro/pull/17 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From noreply at bro.org Mon Nov 10 00:00:29 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 10 Nov 2014 00:00:29 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411100800.sAA80TYT001346@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ----------------------- BIT-1285 [1] Bro grigorescu - 2014-10-31 - Normal MySQL Protocol Analyzer Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [2] bro anthonykasza [3] 2014-11-07 URI Parsing Function [4] [1] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 [2] Pull Request #17 https://github.com/bro/bro/pull/17 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From jira at bro-tracker.atlassian.net Mon Nov 10 15:26:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 10 Nov 2014 17:26:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1285) MySQL Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1285: --------------------------------- Assignee: Robin Sommer > MySQL Protocol Analyzer > ----------------------- > > Key: BIT-1285 > URL: https://bro-tracker.atlassian.net/browse/BIT-1285 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Robin Sommer > > topic/vladg/mysql is ready to be merged. > Note: memleak btest core.leaks.mysql is currently failing due to an issue with how regexes are initialized. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Mon Nov 10 17:52:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Mon, 10 Nov 2014 19:52:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1287) The enum type stores internally a value, but it is impossible to access it. In-Reply-To: References: Message-ID: Christian Struck created BIT-1287: ------------------------------------- Summary: The enum type stores internally a value, but it is impossible to access it. Key: BIT-1287 URL: https://bro-tracker.atlassian.net/browse/BIT-1287 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: 2.3, git/master Reporter: Christian Struck There should be a builtin function like enum_to_int() which returns the value of a specific enum_val. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Mon Nov 10 18:29:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Mon, 10 Nov 2014 20:29:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1287) The enum type stores internally a value, but it is impossible to access it. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Struck updated BIT-1287: ---------------------------------- Status: Merge Request (was: Open) > The enum type stores internally a value, but it is impossible to access it. > --------------------------------------------------------------------------- > > Key: BIT-1287 > URL: https://bro-tracker.atlassian.net/browse/BIT-1287 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master, 2.3 > Reporter: Christian Struck > Labels: language > > There should be a builtin function like enum_to_int() which returns the value of a specific enum_val. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Mon Nov 10 18:29:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Mon, 10 Nov 2014 20:29:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1287) The enum type stores internally a value, but it is impossible to access it. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18800#comment-18800 ] Christian Struck commented on BIT-1287: --------------------------------------- Added feature in branch topic/struck/BIT-1287 with tests > The enum type stores internally a value, but it is impossible to access it. > --------------------------------------------------------------------------- > > Key: BIT-1287 > URL: https://bro-tracker.atlassian.net/browse/BIT-1287 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master, 2.3 > Reporter: Christian Struck > Labels: language > > There should be a builtin function like enum_to_int() which returns the value of a specific enum_val. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From noreply at bro.org Tue Nov 11 00:00:26 2014 From: noreply at bro.org (Merge Tracker) Date: Tue, 11 Nov 2014 00:00:26 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411110800.sAB80QkS023591@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------------- ------------ ---------- ------------- ---------- --------------------------------------------------------------------------- BIT-1287 [1] Bro Christian Struck - 2014-11-10 - Normal The enum type stores internally a value, but it is impossible to access it. BIT-1285 [2] Bro grigorescu Robin Sommer 2014-11-10 - Normal MySQL Protocol Analyzer Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [3] bro anthonykasza [4] 2014-11-07 URI Parsing Function [5] [1] BIT-1287 https://bro-tracker.atlassian.net/browse/BIT-1287 [2] BIT-1285 https://bro-tracker.atlassian.net/browse/BIT-1285 [3] Pull Request #17 https://github.com/bro/bro/pull/17 [4] anthonykasza https://github.com/anthonykasza [5] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From jira at bro-tracker.atlassian.net Tue Nov 11 13:18:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Nov 2014 15:18:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1287) The enum type stores internally a value, but it is impossible to access it. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1287: --------------------------------- Assignee: Robin Sommer > The enum type stores internally a value, but it is impossible to access it. > --------------------------------------------------------------------------- > > Key: BIT-1287 > URL: https://bro-tracker.atlassian.net/browse/BIT-1287 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master, 2.3 > Reporter: Christian Struck > Assignee: Robin Sommer > Labels: language > > There should be a builtin function like enum_to_int() which returns the value of a specific enum_val. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Tue Nov 11 13:47:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Nov 2014 15:47:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1287) The enum type stores internally a value, but it is impossible to access it. In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1287: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > The enum type stores internally a value, but it is impossible to access it. > --------------------------------------------------------------------------- > > Key: BIT-1287 > URL: https://bro-tracker.atlassian.net/browse/BIT-1287 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master, 2.3 > Reporter: Christian Struck > Assignee: Robin Sommer > Labels: language > > There should be a builtin function like enum_to_int() which returns the value of a specific enum_val. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Tue Nov 11 13:50:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Nov 2014 15:50:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1285) MySQL Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1285: ------------------------------ Status: Open (was: Merge Request) > MySQL Protocol Analyzer > ----------------------- > > Key: BIT-1285 > URL: https://bro-tracker.atlassian.net/browse/BIT-1285 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Robin Sommer > > topic/vladg/mysql is ready to be merged. > Note: memleak btest core.leaks.mysql is currently failing due to an issue with how regexes are initialized. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Tue Nov 11 13:50:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 11 Nov 2014 15:50:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1285) MySQL Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18801#comment-18801 ] Robin Sommer commented on BIT-1285: ----------------------------------- Looks good, merged. A few suggestions for the scripts though: - rather than log ?ok? and ?error?, how about doing a boolean column ?success? instead? - instead of logging ?affected rows? as a string, how about adding an optional integer colum ?rows?? - Is it conceivable that a server reply could be missing? If so, the script?s state tracking would get messed up I think. Would it make sense to (1) when a new request comes in, check if one is pending and log that one first then (without reply); and (2) do the same at connection_state_remove() time if a request is still pending? Leaving the ticket open for now. > MySQL Protocol Analyzer > ----------------------- > > Key: BIT-1285 > URL: https://bro-tracker.atlassian.net/browse/BIT-1285 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Robin Sommer > > topic/vladg/mysql is ready to be merged. > Note: memleak btest core.leaks.mysql is currently failing due to an issue with how regexes are initialized. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From noreply at bro.org Wed Nov 12 00:00:24 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 12 Nov 2014 00:00:24 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411120800.sAC80OXE012807@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [1] bro anthonykasza [2] 2014-11-07 URI Parsing Function [3] [1] Pull Request #17 https://github.com/bro/bro/pull/17 [2] anthonykasza https://github.com/anthonykasza [3] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From noreply at bro.org Thu Nov 13 00:00:23 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 13 Nov 2014 00:00:23 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411130800.sAD80NV0007268@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [1] bro anthonykasza [2] 2014-11-07 URI Parsing Function [3] [1] Pull Request #17 https://github.com/bro/bro/pull/17 [2] anthonykasza https://github.com/anthonykasza [3] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From jira at bro-tracker.atlassian.net Thu Nov 13 13:21:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 13 Nov 2014 15:21:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1285) MySQL Protocol Analyzer In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1285: --------------------------------- Assignee: Vlad Grigorescu (was: Robin Sommer) > MySQL Protocol Analyzer > ----------------------- > > Key: BIT-1285 > URL: https://bro-tracker.atlassian.net/browse/BIT-1285 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: git/master > Reporter: grigorescu > Assignee: Vlad Grigorescu > > topic/vladg/mysql is ready to be merged. > Note: memleak btest core.leaks.mysql is currently failing due to an issue with how regexes are initialized. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From noreply at bro.org Fri Nov 14 00:00:40 2014 From: noreply at bro.org (Merge Tracker) Date: Fri, 14 Nov 2014 00:00:40 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411140800.sAE80etN026545@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [1] bro anthonykasza [2] 2014-11-07 URI Parsing Function [3] [1] Pull Request #17 https://github.com/bro/bro/pull/17 [2] anthonykasza https://github.com/anthonykasza [3] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From noreply at bro.org Sat Nov 15 00:00:24 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 15 Nov 2014 00:00:24 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411150800.sAF80OQ4013846@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [1] bro anthonykasza [2] 2014-11-07 URI Parsing Function [3] [1] Pull Request #17 https://github.com/bro/bro/pull/17 [2] anthonykasza https://github.com/anthonykasza [3] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From noreply at bro.org Sun Nov 16 00:00:43 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 16 Nov 2014 00:00:43 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411160800.sAG80hwA032585@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [1] bro anthonykasza [2] 2014-11-07 URI Parsing Function [3] [1] Pull Request #17 https://github.com/bro/bro/pull/17 [2] anthonykasza https://github.com/anthonykasza [3] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From noreply at bro.org Mon Nov 17 00:00:21 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 17 Nov 2014 00:00:21 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411170800.sAH80LcO013495@bro-ids.icir.org> Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [1] bro anthonykasza [2] 2014-11-07 URI Parsing Function [3] [1] Pull Request #17 https://github.com/bro/bro/pull/17 [2] anthonykasza https://github.com/anthonykasza [3] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From anthony.kasza at gmail.com Mon Nov 17 08:27:13 2014 From: anthony.kasza at gmail.com (anthony kasza) Date: Mon, 17 Nov 2014 08:27:13 -0800 Subject: [Bro-Dev] [Auto] Merge Status In-Reply-To: <201411170800.sAH80LcO013495@bro-ids.icir.org> References: <201411170800.sAH80LcO013495@bro-ids.icir.org> Message-ID: This can probably get closed. I think I forgot to create a topic branch for the feature. The function works well though. -AK On Nov 17, 2014 12:07 AM, "Merge Tracker" wrote: > > Open GitHub Pull Requests > ========================= > > Issue Component User Updated Title > ------- ----------- ---------------- ---------- > ------------------------ > #17 [1] bro anthonykasza [2] 2014-11-07 URI Parsing Function > [3] > > > [1] Pull Request #17 https://github.com/bro/bro/pull/17 > [2] anthonykasza https://github.com/anthonykasza > [3] Merge Pull Request #17 with git pull --no-ff --no-commit > https://github.com/anthonykasza/bro.git master > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20141117/3242bdcd/attachment.html From robin at icir.org Mon Nov 17 08:44:45 2014 From: robin at icir.org (Robin Sommer) Date: Mon, 17 Nov 2014 08:44:45 -0800 Subject: [Bro-Dev] [Auto] Merge Status In-Reply-To: References: <201411170800.sAH80LcO013495@bro-ids.icir.org> Message-ID: <20141117164445.GK50528@icir.org> Anthony, I'll merge it later. There was a bit of confusion who's in charge of that. :) Robin On Mon, Nov 17, 2014 at 08:27 -0800, you wrote: > This can probably get closed. I think I forgot to create a topic branch for > the feature. The function works well though. > > -AK > On Nov 17, 2014 12:07 AM, "Merge Tracker" wrote: > > > > > Open GitHub Pull Requests > > ========================= > > > > Issue Component User Updated Title > > ------- ----------- ---------------- ---------- > > ------------------------ > > #17 [1] bro anthonykasza [2] 2014-11-07 URI Parsing Function > > [3] > > > > > > [1] Pull Request #17 https://github.com/bro/bro/pull/17 > > [2] anthonykasza https://github.com/anthonykasza > > [3] Merge Pull Request #17 with git pull --no-ff --no-commit > > https://github.com/anthonykasza/bro.git master > > > > _______________________________________________ > > bro-dev mailing list > > bro-dev at bro.org > > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Mon Nov 17 11:43:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Mon, 17 Nov 2014 13:43:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1288) int: &default value has inconsistent type (0 and int) In-Reply-To: References: Message-ID: Christian Struck created BIT-1288: ------------------------------------- Summary: int: &default value has inconsistent type (0 and int) Key: BIT-1288 URL: https://bro-tracker.atlassian.net/browse/BIT-1288 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Environment: Debian 7 Reporter: Christian Struck Attachments: test.bro It seems that default >=0 for integer are not interpreted as int and therefore not assignable. for the zero value -0 works, but for -1 is not 1. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Mon Nov 17 15:05:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 17 Nov 2014 17:05:08 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1288) int: &default value has inconsistent type (0 and int) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1288: --------------------------- Fix Version/s: 2.4 > int: &default value has inconsistent type (0 and int) > ----------------------------------------------------- > > Key: BIT-1288 > URL: https://bro-tracker.atlassian.net/browse/BIT-1288 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Labels: language > Fix For: 2.4 > > Attachments: test.bro > > > It seems that default >=0 for integer are not interpreted as int and therefore not assignable. for the zero value -0 works, but for -1 is not 1. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Mon Nov 17 15:11:07 2014 From: jira at bro-tracker.atlassian.net (Christian Struck (JIRA)) Date: Mon, 17 Nov 2014 17:11:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1288) int: &default value has inconsistent type (0 and int) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18901#comment-18901 ] Christian Struck commented on BIT-1288: --------------------------------------- actually I'm using it in a record. {code:bro} type SomeType: record { ...; priority: int &default=0; }; {code} > int: &default value has inconsistent type (0 and int) > ----------------------------------------------------- > > Key: BIT-1288 > URL: https://bro-tracker.atlassian.net/browse/BIT-1288 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Labels: language > Fix For: 2.4 > > Attachments: test.bro > > > It seems that default >=0 for integer are not interpreted as int and therefore not assignable. for the zero value -0 works, but for -1 is not 1. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Mon Nov 17 15:05:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 17 Nov 2014 17:05:08 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1288) int: &default value has inconsistent type (0 and int) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18900#comment-18900 ] Jon Siwek commented on BIT-1288: -------------------------------- There's maybe two parts to this: 1) Improve the type coercion that &default attributes use. E.g. this may be enough: {noformat} diff --git a/src/Attr.cc b/src/Attr.cc index d6d0f6e..3f86238 100644 --- a/src/Attr.cc +++ b/src/Attr.cc @@ -265,6 +265,11 @@ void Attributes::CheckAttr(Attr* a) // Ok. break; + Expr* e = a->AttrExpr(); + if ( check_and_promote_expr(e, type) ) + // Ok. + break; + a->AttrExpr()->Error("&default value has inconsistent type", type); } {noformat} 2) I don't think &default currently means anything when applied to variables as the test example shows. Was that just a short way to get the parser to create some type declarations and show the error message, or can you give more context on how you're trying to use this? Wouldn't assigning a value directly be conceptually the same as &default here? > int: &default value has inconsistent type (0 and int) > ----------------------------------------------------- > > Key: BIT-1288 > URL: https://bro-tracker.atlassian.net/browse/BIT-1288 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Labels: language > Fix For: 2.4 > > Attachments: test.bro > > > It seems that default >=0 for integer are not interpreted as int and therefore not assignable. for the zero value -0 works, but for -1 is not 1. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From noreply at bro.org Tue Nov 18 00:00:23 2014 From: noreply at bro.org (Merge Tracker) Date: Tue, 18 Nov 2014 00:00:23 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411180800.sAI80NKP023603@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------------------------------------ f99bc98 [1] bro Johanna Amann 2014-11-17 for dh key exchanges, use p as the parameter for weak key ex Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ---------------- ---------- ------------------------ #17 [2] bro anthonykasza [3] 2014-11-07 URI Parsing Function [4] [1] f99bc98 https://github.com/bro/bro/commit/f99bc98800c0b6ba678da5e800adf71aaa401cca [2] Pull Request #17 https://github.com/bro/bro/pull/17 [3] anthonykasza https://github.com/anthonykasza [4] Merge Pull Request #17 with git pull --no-ff --no-commit https://github.com/anthonykasza/bro.git master From jira at bro-tracker.atlassian.net Tue Nov 18 10:45:08 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 18 Nov 2014 12:45:08 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1288) int: &default value has inconsistent type (0 and int) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1288: --------------------------- Status: Merge Request (was: Open) > int: &default value has inconsistent type (0 and int) > ----------------------------------------------------- > > Key: BIT-1288 > URL: https://bro-tracker.atlassian.net/browse/BIT-1288 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Labels: language > Fix For: 2.4 > > Attachments: test.bro > > > It seems that default >=0 for integer are not interpreted as int and therefore not assignable. for the zero value -0 works, but for -1 is not 1. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Tue Nov 18 10:45:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 18 Nov 2014 12:45:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1288) int: &default value has inconsistent type (0 and int) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18902#comment-18902 ] Jon Siwek commented on BIT-1288: -------------------------------- Fix in topic/jsiwek/bit-1288 > int: &default value has inconsistent type (0 and int) > ----------------------------------------------------- > > Key: BIT-1288 > URL: https://bro-tracker.atlassian.net/browse/BIT-1288 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Labels: language > Fix For: 2.4 > > Attachments: test.bro > > > It seems that default >=0 for integer are not interpreted as int and therefore not assignable. for the zero value -0 works, but for -1 is not 1. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Tue Nov 18 10:46:07 2014 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Tue, 18 Nov 2014 12:46:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1288) int: &default value has inconsistent type (0 and int) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Siwek updated BIT-1288: --------------------------- Assignee: Robin Sommer > int: &default value has inconsistent type (0 and int) > ----------------------------------------------------- > > Key: BIT-1288 > URL: https://bro-tracker.atlassian.net/browse/BIT-1288 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Assignee: Robin Sommer > Labels: language > Fix For: 2.4 > > Attachments: test.bro > > > It seems that default >=0 for integer are not interpreted as int and therefore not assignable. for the zero value -0 works, but for -1 is not 1. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Tue Nov 18 13:59:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 18 Nov 2014 15:59:08 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1288) int: &default value has inconsistent type (0 and int) In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1288: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > int: &default value has inconsistent type (0 and int) > ----------------------------------------------------- > > Key: BIT-1288 > URL: https://bro-tracker.atlassian.net/browse/BIT-1288 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: Debian 7 > Reporter: Christian Struck > Assignee: Robin Sommer > Labels: language > Fix For: 2.4 > > Attachments: test.bro > > > It seems that default >=0 for integer are not interpreted as int and therefore not assignable. for the zero value -0 works, but for -1 is not 1. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From robin at icir.org Tue Nov 18 15:04:03 2014 From: robin at icir.org (Robin Sommer) Date: Tue, 18 Nov 2014 15:04:03 -0800 Subject: [Bro-Dev] [Auto] Merge Status In-Reply-To: <20141117164445.GK50528@icir.org> References: <201411170800.sAH80LcO013495@bro-ids.icir.org> <20141117164445.GK50528@icir.org> Message-ID: <20141118230403.GA20998@icir.org> Anthony, I've twekaed the URI struct a bit while merging, can you take a look and see if that works for you? I've also added a regression test. If you can think of further cases to add in there, let me know. Robin On Mon, Nov 17, 2014 at 08:44 -0800, I wrote: > Anthony, I'll merge it later. There was a bit of confusion who's in > charge of that. :) > > Robin > > On Mon, Nov 17, 2014 at 08:27 -0800, you wrote: > > > This can probably get closed. I think I forgot to create a topic branch for > > the feature. The function works well though. > > > > -AK > > On Nov 17, 2014 12:07 AM, "Merge Tracker" wrote: > > > > > > > > Open GitHub Pull Requests > > > ========================= > > > > > > Issue Component User Updated Title > > > ------- ----------- ---------------- ---------- > > > ------------------------ > > > #17 [1] bro anthonykasza [2] 2014-11-07 URI Parsing Function > > > [3] > > > > > > > > > [1] Pull Request #17 https://github.com/bro/bro/pull/17 > > > [2] anthonykasza https://github.com/anthonykasza > > > [3] Merge Pull Request #17 with git pull --no-ff --no-commit > > > https://github.com/anthonykasza/bro.git master > > > > > > _______________________________________________ > > > bro-dev mailing list > > > bro-dev at bro.org > > > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > > > > > _______________________________________________ > > bro-dev mailing list > > bro-dev at bro.org > > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > > -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From anthony.kasza at gmail.com Tue Nov 18 19:17:27 2014 From: anthony.kasza at gmail.com (anthony kasza) Date: Tue, 18 Nov 2014 19:17:27 -0800 Subject: [Bro-Dev] [Auto] Merge Status In-Reply-To: <20141118230403.GA20998@icir.org> References: <201411170800.sAH80LcO013495@bro-ids.icir.org> <20141117164445.GK50528@icir.org> <20141118230403.GA20998@icir.org> Message-ID: That looks perfect. Thanks, Robin. -AK On Tue, Nov 18, 2014 at 3:04 PM, Robin Sommer wrote: > Anthony, > > I've twekaed the URI struct a bit while merging, can you take a look > and see if that works for you? > > I've also added a regression test. If you can think of further cases > to add in there, let me know. > > Robin > > On Mon, Nov 17, 2014 at 08:44 -0800, I wrote: > >> Anthony, I'll merge it later. There was a bit of confusion who's in >> charge of that. :) >> >> Robin >> >> On Mon, Nov 17, 2014 at 08:27 -0800, you wrote: >> >> > This can probably get closed. I think I forgot to create a topic branch for >> > the feature. The function works well though. >> > >> > -AK >> > On Nov 17, 2014 12:07 AM, "Merge Tracker" wrote: >> > >> > > >> > > Open GitHub Pull Requests >> > > ========================= >> > > >> > > Issue Component User Updated Title >> > > ------- ----------- ---------------- ---------- >> > > ------------------------ >> > > #17 [1] bro anthonykasza [2] 2014-11-07 URI Parsing Function >> > > [3] >> > > >> > > >> > > [1] Pull Request #17 https://github.com/bro/bro/pull/17 >> > > [2] anthonykasza https://github.com/anthonykasza >> > > [3] Merge Pull Request #17 with git pull --no-ff --no-commit >> > > https://github.com/anthonykasza/bro.git master >> > > >> > > _______________________________________________ >> > > bro-dev mailing list >> > > bro-dev at bro.org >> > > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev >> > > >> >> > _______________________________________________ >> > bro-dev mailing list >> > bro-dev at bro.org >> > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev >> >> >> > > > -- > Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Wed Nov 19 09:02:07 2014 From: jira at bro-tracker.atlassian.net (Steve Egbert (JIRA)) Date: Wed, 19 Nov 2014 11:02:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1289) Ascii Stream Reader not freeing up memory In-Reply-To: References: Message-ID: Steve Egbert created BIT-1289: --------------------------------- Summary: Ascii Stream Reader not freeing up memory Key: BIT-1289 URL: https://bro-tracker.atlassian.net/browse/BIT-1289 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.3, 2.2 Environment: CentOS 6.6 distro x86_64 gcc 4.4.7 20120313 (Red Hat 4.4.7-4) ld 2.20.51.0.2-5.36.el6 20100205 glibc 2.12 1.32.el6_5.4 libunwind 1.1 2.el6 libstdc++ 4.4.7 4.el6 libm (via glibc-devel 2.12 1.07.el6_4.4) Reporter: Steve Egbert A triple leak occurred whenever an ASCII file has its modify time changed {{DoUpdate}} as well as once during {{DoInit}} initialization which called {{DoUpdate}}. When tracing the EVENT_STREAM message from reading of an input file using Ascii framework, but I spotted a memory leak that impacts both Bro 2.2 (and possibly 2.3) as reported by gperftool/tcmalloc {{HEAPCHECK}} output: Using local file /usr/bin/bro. Leak of 5148576 bytes in 160893 objects allocated from: @ 7ab2b5 AsciiFormatter::ParseValue @ 7d9404 input::reader::Ascii::DoUpdate @ 7d665c input::ReaderBackend::Update @ 7d97ba input::reader::Ascii::DoHeartbeat @ 7d674a input::ReaderBackend::OnHeartbeat @ 7b0e95 threading::HeartbeatMessage::Process @ 7b085a threading::MsgThread::Run @ 7ad264 threading::BasicThread::launcher @ 37340079d1 start_thread @ 37338e886d __clone @ 0 _init Leak of 5148480 bytes in 160890 objects allocated from: @ 7d91e2 input::reader::Ascii::DoUpdate @ 7d665c input::ReaderBackend::Update @ 7d97ba input::reader::Ascii::DoHeartbeat @ 7d674a input::ReaderBackend::OnHeartbeat @ 7b0e95 threading::HeartbeatMessage::Process @ 7b085a threading::MsgThread::Run @ 7ad264 threading::BasicThread::launcher @ 37340079d1 start_thread @ 37338e886d __clone @ 0 _init Leak of 2062752 bytes in 160854 objects allocated from: @ 673735 copy_string @ 7ab399 AsciiFormatter::ParseValue @ 7d9404 input::reader::Ascii::DoUpdate @ 7d665c input::ReaderBackend::Update @ 7d97ba input::reader::Ascii::DoHeartbeat @ 7d674a input::ReaderBackend::OnHeartbeat @ 7b0e95 threading::HeartbeatMessage::Process @ 7b085a threading::MsgThread::Run @ 7ad264 threading::BasicThread::launcher @ 37340079d1 start_thread @ 37338e886d __clone @ 0 _init In short, the leakage was found to occur in the Manager during processing incoming EVENT_STREAM messages that was sent by its child thread(s) via {{queue_out}} queue. Manager used {{RetrieveOut()}} of getting these messages from the {{queue_out}} and eventually calling {{SendEntry()}} for final processing and disposal of these messages (note the {{delete_vals_ptr_array()}} at the end of the {{SendEntry()}} function). But the error was in miscomputing of {[readFields}} for its actual number of fields associating with the msg/vals/fields. In {{SendEntry()}} (of {{Manager.cc}}), note the out-of-sync between EVENT_STREAM's {{readField}} assignment and the original {{stream->num_fields}} waaaay back in when the message was first created by the {{DoInit}}. So, when it comes to counting number of columns/fields, it becomes an issue of whether to go by what was in the first line of the text file describing the columns or what was given in the data line afterwards. Perhaps a better value is to use {{(EventStream *)i->num_fields}} instead, but only for EVENT_STREAM? I've demonstrated removal of these leaks using this field instead. We've already lost the {{columnMap}} details at file read time which would have made for a better comparator value. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Wed Nov 19 09:21:07 2014 From: jira at bro-tracker.atlassian.net (Steve Egbert (JIRA)) Date: Wed, 19 Nov 2014 11:21:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1289) Ascii Stream Reader not freeing up memory In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Egbert updated BIT-1289: ------------------------------ Attachment: bro-ascii-stream-event-leak.patch > Ascii Stream Reader not freeing up memory > ------------------------------------------ > > Key: BIT-1289 > URL: https://bro-tracker.atlassian.net/browse/BIT-1289 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2, 2.3 > Environment: CentOS 6.6 distro x86_64 > gcc 4.4.7 20120313 (Red Hat 4.4.7-4) > ld 2.20.51.0.2-5.36.el6 20100205 > glibc 2.12 1.32.el6_5.4 > libunwind 1.1 2.el6 > libstdc++ 4.4.7 4.el6 > libm (via glibc-devel 2.12 1.07.el6_4.4) > Reporter: Steve Egbert > Labels: input-framework > Attachments: bro-ascii-stream-event-leak.patch > > > A triple leak occurred whenever an ASCII file has its modify time changed {{DoUpdate}} as well as once during {{DoInit}} initialization which called {{DoUpdate}}. > When tracing the EVENT_STREAM message from reading of an input file using Ascii framework, but I spotted a memory leak that impacts both Bro 2.2 (and possibly 2.3) as reported by gperftool/tcmalloc {{HEAPCHECK}} output: > Using local file /usr/bin/bro. > Leak of 5148576 bytes in 160893 objects allocated from: > @ 7ab2b5 AsciiFormatter::ParseValue > @ 7d9404 input::reader::Ascii::DoUpdate > @ 7d665c input::ReaderBackend::Update > @ 7d97ba input::reader::Ascii::DoHeartbeat > @ 7d674a input::ReaderBackend::OnHeartbeat > @ 7b0e95 threading::HeartbeatMessage::Process > @ 7b085a threading::MsgThread::Run > @ 7ad264 threading::BasicThread::launcher > @ 37340079d1 start_thread > @ 37338e886d __clone > @ 0 _init > Leak of 5148480 bytes in 160890 objects allocated from: > @ 7d91e2 input::reader::Ascii::DoUpdate > @ 7d665c input::ReaderBackend::Update > @ 7d97ba input::reader::Ascii::DoHeartbeat > @ 7d674a input::ReaderBackend::OnHeartbeat > @ 7b0e95 threading::HeartbeatMessage::Process > @ 7b085a threading::MsgThread::Run > @ 7ad264 threading::BasicThread::launcher > @ 37340079d1 start_thread > @ 37338e886d __clone > @ 0 _init > Leak of 2062752 bytes in 160854 objects allocated from: > @ 673735 copy_string > @ 7ab399 AsciiFormatter::ParseValue > @ 7d9404 input::reader::Ascii::DoUpdate > @ 7d665c input::ReaderBackend::Update > @ 7d97ba input::reader::Ascii::DoHeartbeat > @ 7d674a input::ReaderBackend::OnHeartbeat > @ 7b0e95 threading::HeartbeatMessage::Process > @ 7b085a threading::MsgThread::Run > @ 7ad264 threading::BasicThread::launcher > @ 37340079d1 start_thread > @ 37338e886d __clone > @ 0 _init > In short, the leakage was found to occur in the Manager during processing incoming EVENT_STREAM messages that was sent by its child thread(s) via {{queue_out}} queue. Manager used {{RetrieveOut()}} of getting these messages from the {{queue_out}} and eventually calling {{SendEntry()}} for final processing and disposal of these messages (note the {{delete_vals_ptr_array()}} at the end of the {{SendEntry()}} function). But the error was in miscomputing of {[readFields}} for its actual number of fields associating with the msg/vals/fields. > In {{SendEntry()}} (of {{Manager.cc}}), note the out-of-sync between EVENT_STREAM's {{readField}} assignment and the original {{stream->num_fields}} waaaay back in when the message was first created by the {{DoInit}}. > So, when it comes to counting number of columns/fields, it becomes an issue of whether to go by what was in the first line of the text file describing the columns or what was given in the data line afterwards. > Perhaps a better value is to use {{(EventStream *)i->num_fields}} instead, but only for EVENT_STREAM? I've demonstrated removal of these leaks using this field instead. > We've already lost the {{columnMap}} details at file read time which would have made for a better comparator value. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Wed Nov 19 09:23:08 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Wed, 19 Nov 2014 11:23:08 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1289) Ascii Stream Reader not freeing up memory In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann reassigned BIT-1289: ---------------------------------- Assignee: Johanna Amann > Ascii Stream Reader not freeing up memory > ------------------------------------------ > > Key: BIT-1289 > URL: https://bro-tracker.atlassian.net/browse/BIT-1289 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2, 2.3 > Environment: CentOS 6.6 distro x86_64 > gcc 4.4.7 20120313 (Red Hat 4.4.7-4) > ld 2.20.51.0.2-5.36.el6 20100205 > glibc 2.12 1.32.el6_5.4 > libunwind 1.1 2.el6 > libstdc++ 4.4.7 4.el6 > libm (via glibc-devel 2.12 1.07.el6_4.4) > Reporter: Steve Egbert > Assignee: Johanna Amann > Labels: input-framework > Attachments: bro-ascii-stream-event-leak.patch > > > A triple leak occurred whenever an ASCII file has its modify time changed {{DoUpdate}} as well as once during {{DoInit}} initialization which called {{DoUpdate}}. > When tracing the EVENT_STREAM message from reading of an input file using Ascii framework, but I spotted a memory leak that impacts both Bro 2.2 (and possibly 2.3) as reported by gperftool/tcmalloc {{HEAPCHECK}} output: > Using local file /usr/bin/bro. > Leak of 5148576 bytes in 160893 objects allocated from: > @ 7ab2b5 AsciiFormatter::ParseValue > @ 7d9404 input::reader::Ascii::DoUpdate > @ 7d665c input::ReaderBackend::Update > @ 7d97ba input::reader::Ascii::DoHeartbeat > @ 7d674a input::ReaderBackend::OnHeartbeat > @ 7b0e95 threading::HeartbeatMessage::Process > @ 7b085a threading::MsgThread::Run > @ 7ad264 threading::BasicThread::launcher > @ 37340079d1 start_thread > @ 37338e886d __clone > @ 0 _init > Leak of 5148480 bytes in 160890 objects allocated from: > @ 7d91e2 input::reader::Ascii::DoUpdate > @ 7d665c input::ReaderBackend::Update > @ 7d97ba input::reader::Ascii::DoHeartbeat > @ 7d674a input::ReaderBackend::OnHeartbeat > @ 7b0e95 threading::HeartbeatMessage::Process > @ 7b085a threading::MsgThread::Run > @ 7ad264 threading::BasicThread::launcher > @ 37340079d1 start_thread > @ 37338e886d __clone > @ 0 _init > Leak of 2062752 bytes in 160854 objects allocated from: > @ 673735 copy_string > @ 7ab399 AsciiFormatter::ParseValue > @ 7d9404 input::reader::Ascii::DoUpdate > @ 7d665c input::ReaderBackend::Update > @ 7d97ba input::reader::Ascii::DoHeartbeat > @ 7d674a input::ReaderBackend::OnHeartbeat > @ 7b0e95 threading::HeartbeatMessage::Process > @ 7b085a threading::MsgThread::Run > @ 7ad264 threading::BasicThread::launcher > @ 37340079d1 start_thread > @ 37338e886d __clone > @ 0 _init > In short, the leakage was found to occur in the Manager during processing incoming EVENT_STREAM messages that was sent by its child thread(s) via {{queue_out}} queue. Manager used {{RetrieveOut()}} of getting these messages from the {{queue_out}} and eventually calling {{SendEntry()}} for final processing and disposal of these messages (note the {{delete_vals_ptr_array()}} at the end of the {{SendEntry()}} function). But the error was in miscomputing of {[readFields}} for its actual number of fields associating with the msg/vals/fields. > In {{SendEntry()}} (of {{Manager.cc}}), note the out-of-sync between EVENT_STREAM's {{readField}} assignment and the original {{stream->num_fields}} waaaay back in when the message was first created by the {{DoInit}}. > So, when it comes to counting number of columns/fields, it becomes an issue of whether to go by what was in the first line of the text file describing the columns or what was given in the data line afterwards. > Perhaps a better value is to use {{(EventStream *)i->num_fields}} instead, but only for EVENT_STREAM? I've demonstrated removal of these leaks using this field instead. > We've already lost the {{columnMap}} details at file read time which would have made for a better comparator value. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jsiwek at illinois.edu Wed Nov 19 09:23:03 2014 From: jsiwek at illinois.edu (Siwek, Jon) Date: Wed, 19 Nov 2014 17:23:03 +0000 Subject: [Bro-Dev] ninja builds Message-ID: <41D68C59-0BAB-40C9-B3E4-7ACAF2DCB6DE@illinois.edu> Ninja builds (https://martine.github.io/ninja/) should now work with Bro?s master branch. After installing it, Bro can use it like: ./configure --generator=Ninja cd build && ninja # May want to use ?builddir if you want generated files to go somewhere besides ./build The benefit over generating standard Makefiles is it checks for out-of-date targets quicker, which I?ve found to consume a majority of the build time for just small source code changes (e.g. a few .cc files). Examples: A ?no-op? build: $ time ninja >/dev/null real 0m0.162s user 0m0.119s sys 0m0.083s $ time make >/dev/null real 0m8.280s user 0m4.191s sys 0m3.029s Single source file changed: $ touch src/Attr.cc $ time ninja >/dev/null real 0m1.715s user 0m1.541s sys 0m0.273s $ time make >/dev/null real 0m9.725s user 0m5.723s sys 0m3.223s - Jon From jira at bro-tracker.atlassian.net Wed Nov 19 10:12:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Wed, 19 Nov 2014 12:12:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1289) Ascii Stream Reader not freeing up memory In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1289: ------------------------------- Resolution: Works for Me Status: Closed (was: Open) This should have been fixed in 2.3 -- the return value of SendEventStreamEvent was changed to stream->num_fields, which is equivalent to your patch. There also were a number of other fixes in the input framework, so you should consider upgrading. Closing the bug, if this does not solve your problem please reopen it. > Ascii Stream Reader not freeing up memory > ------------------------------------------ > > Key: BIT-1289 > URL: https://bro-tracker.atlassian.net/browse/BIT-1289 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.2, 2.3 > Environment: CentOS 6.6 distro x86_64 > gcc 4.4.7 20120313 (Red Hat 4.4.7-4) > ld 2.20.51.0.2-5.36.el6 20100205 > glibc 2.12 1.32.el6_5.4 > libunwind 1.1 2.el6 > libstdc++ 4.4.7 4.el6 > libm (via glibc-devel 2.12 1.07.el6_4.4) > Reporter: Steve Egbert > Assignee: Johanna Amann > Labels: input-framework > Attachments: bro-ascii-stream-event-leak.patch > > > A triple leak occurred whenever an ASCII file has its modify time changed {{DoUpdate}} as well as once during {{DoInit}} initialization which called {{DoUpdate}}. > When tracing the EVENT_STREAM message from reading of an input file using Ascii framework, but I spotted a memory leak that impacts both Bro 2.2 (and possibly 2.3) as reported by gperftool/tcmalloc {{HEAPCHECK}} output: > Using local file /usr/bin/bro. > Leak of 5148576 bytes in 160893 objects allocated from: > @ 7ab2b5 AsciiFormatter::ParseValue > @ 7d9404 input::reader::Ascii::DoUpdate > @ 7d665c input::ReaderBackend::Update > @ 7d97ba input::reader::Ascii::DoHeartbeat > @ 7d674a input::ReaderBackend::OnHeartbeat > @ 7b0e95 threading::HeartbeatMessage::Process > @ 7b085a threading::MsgThread::Run > @ 7ad264 threading::BasicThread::launcher > @ 37340079d1 start_thread > @ 37338e886d __clone > @ 0 _init > Leak of 5148480 bytes in 160890 objects allocated from: > @ 7d91e2 input::reader::Ascii::DoUpdate > @ 7d665c input::ReaderBackend::Update > @ 7d97ba input::reader::Ascii::DoHeartbeat > @ 7d674a input::ReaderBackend::OnHeartbeat > @ 7b0e95 threading::HeartbeatMessage::Process > @ 7b085a threading::MsgThread::Run > @ 7ad264 threading::BasicThread::launcher > @ 37340079d1 start_thread > @ 37338e886d __clone > @ 0 _init > Leak of 2062752 bytes in 160854 objects allocated from: > @ 673735 copy_string > @ 7ab399 AsciiFormatter::ParseValue > @ 7d9404 input::reader::Ascii::DoUpdate > @ 7d665c input::ReaderBackend::Update > @ 7d97ba input::reader::Ascii::DoHeartbeat > @ 7d674a input::ReaderBackend::OnHeartbeat > @ 7b0e95 threading::HeartbeatMessage::Process > @ 7b085a threading::MsgThread::Run > @ 7ad264 threading::BasicThread::launcher > @ 37340079d1 start_thread > @ 37338e886d __clone > @ 0 _init > In short, the leakage was found to occur in the Manager during processing incoming EVENT_STREAM messages that was sent by its child thread(s) via {{queue_out}} queue. Manager used {{RetrieveOut()}} of getting these messages from the {{queue_out}} and eventually calling {{SendEntry()}} for final processing and disposal of these messages (note the {{delete_vals_ptr_array()}} at the end of the {{SendEntry()}} function). But the error was in miscomputing of {[readFields}} for its actual number of fields associating with the msg/vals/fields. > In {{SendEntry()}} (of {{Manager.cc}}), note the out-of-sync between EVENT_STREAM's {{readField}} assignment and the original {{stream->num_fields}} waaaay back in when the message was first created by the {{DoInit}}. > So, when it comes to counting number of columns/fields, it becomes an issue of whether to go by what was in the first line of the text file describing the columns or what was given in the data line afterwards. > Perhaps a better value is to use {{(EventStream *)i->num_fields}} instead, but only for EVENT_STREAM? I've demonstrated removal of these leaks using this field instead. > We've already lost the {{columnMap}} details at file read time which would have made for a better comparator value. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From seth at icir.org Wed Nov 19 12:16:13 2014 From: seth at icir.org (Seth Hall) Date: Wed, 19 Nov 2014 15:16:13 -0500 Subject: [Bro-Dev] ninja builds In-Reply-To: <41D68C59-0BAB-40C9-B3E4-7ACAF2DCB6DE@illinois.edu> References: <41D68C59-0BAB-40C9-B3E4-7ACAF2DCB6DE@illinois.edu> Message-ID: <1E742155-BBDC-4DF3-9267-C911252A4D66@icir.org> > On Nov 19, 2014, at 12:23 PM, Siwek, Jon wrote: > > $ touch src/Attr.cc > > $ time ninja >/dev/null > real 0m1.715s > user 0m1.541s > sys 0m0.273s > > $ time make >/dev/null > real 0m9.725s > user 0m5.723s > sys 0m3.223s Ooh! That is really nice! /me heads off to install ninja .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From jira at bro-tracker.atlassian.net Fri Nov 21 06:25:08 2014 From: jira at bro-tracker.atlassian.net (=?UTF-8?Q?Ra=C3=BAl_Benencia_=28JIRA=29?=) Date: Fri, 21 Nov 2014 08:25:08 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-856) more documentation for utilities would be cool In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ra?l Benencia updated BIT-856: ------------------------------ Status: Open (was: Merge Request) > more documentation for utilities would be cool > ---------------------------------------------- > > Key: BIT-856 > URL: https://bro-tracker.atlassian.net/browse/BIT-856 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: bro-aux > Affects Versions: git/master > Reporter: Vern Paxson > Fix For: 2.4 > > > Utilities like bro-cut only supply \--help documentation, as far as I can tell. Man pages would be handy. (In particular, I was looking for some sort of statement of exactly to what degree bro-cut can munch on the concatenation of multiple log files that have different column layouts.) -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Fri Nov 21 06:25:07 2014 From: jira at bro-tracker.atlassian.net (=?UTF-8?Q?Ra=C3=BAl_Benencia_=28JIRA=29?=) Date: Fri, 21 Nov 2014 08:25:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-856) more documentation for utilities would be cool In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ra?l Benencia updated BIT-856: ------------------------------ Status: Merge Request (was: Open) > more documentation for utilities would be cool > ---------------------------------------------- > > Key: BIT-856 > URL: https://bro-tracker.atlassian.net/browse/BIT-856 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: bro-aux > Affects Versions: git/master > Reporter: Vern Paxson > Fix For: 2.4 > > > Utilities like bro-cut only supply \--help documentation, as far as I can tell. Man pages would be handy. (In particular, I was looking for some sort of statement of exactly to what degree bro-cut can munch on the concatenation of multiple log files that have different column layouts.) -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Fri Nov 21 06:27:08 2014 From: jira at bro-tracker.atlassian.net (=?UTF-8?Q?Ra=C3=BAl_Benencia_=28JIRA=29?=) Date: Fri, 21 Nov 2014 08:27:08 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-856) more documentation for utilities would be cool In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ra?l Benencia updated BIT-856: ------------------------------ Attachment: bro.8 broctl.8 bro-cut.1 trace-summary.1 Hi, I'm attaching the man pages I've created for the Debian project. They're heavily based in the Bro documentation and the "--help" option of each of the tools. Thanks. > more documentation for utilities would be cool > ---------------------------------------------- > > Key: BIT-856 > URL: https://bro-tracker.atlassian.net/browse/BIT-856 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: bro-aux > Affects Versions: git/master > Reporter: Vern Paxson > Fix For: 2.4 > > Attachments: bro.8, broctl.8, bro-cut.1, trace-summary.1 > > > Utilities like bro-cut only supply \--help documentation, as far as I can tell. Man pages would be handy. (In particular, I was looking for some sort of statement of exactly to what degree bro-cut can munch on the concatenation of multiple log files that have different column layouts.) -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Fri Nov 21 06:28:07 2014 From: jira at bro-tracker.atlassian.net (=?UTF-8?Q?Ra=C3=BAl_Benencia_=28JIRA=29?=) Date: Fri, 21 Nov 2014 08:28:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-856) more documentation for utilities would be cool In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ra?l Benencia updated BIT-856: ------------------------------ Status: In Progress (was: Open) > more documentation for utilities would be cool > ---------------------------------------------- > > Key: BIT-856 > URL: https://bro-tracker.atlassian.net/browse/BIT-856 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: bro-aux > Affects Versions: git/master > Reporter: Vern Paxson > Fix For: 2.4 > > Attachments: bro.8, broctl.8, bro-cut.1, trace-summary.1 > > > Utilities like bro-cut only supply \--help documentation, as far as I can tell. Man pages would be handy. (In particular, I was looking for some sort of statement of exactly to what degree bro-cut can munch on the concatenation of multiple log files that have different column layouts.) -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Fri Nov 21 06:28:07 2014 From: jira at bro-tracker.atlassian.net (=?UTF-8?Q?Ra=C3=BAl_Benencia_=28JIRA=29?=) Date: Fri, 21 Nov 2014 08:28:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-856) more documentation for utilities would be cool In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ra?l Benencia updated BIT-856: ------------------------------ Status: Merge Request (was: In Progress) > more documentation for utilities would be cool > ---------------------------------------------- > > Key: BIT-856 > URL: https://bro-tracker.atlassian.net/browse/BIT-856 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: bro-aux > Affects Versions: git/master > Reporter: Vern Paxson > Fix For: 2.4 > > Attachments: bro.8, broctl.8, bro-cut.1, trace-summary.1 > > > Utilities like bro-cut only supply \--help documentation, as far as I can tell. Man pages would be handy. (In particular, I was looking for some sort of statement of exactly to what degree bro-cut can munch on the concatenation of multiple log files that have different column layouts.) -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Fri Nov 21 06:39:07 2014 From: jira at bro-tracker.atlassian.net (=?UTF-8?Q?Ra=C3=BAl_Benencia_=28JIRA=29?=) Date: Fri, 21 Nov 2014 08:39:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1290) Permissions of some helpers scripts In-Reply-To: References: Message-ID: Ra?l Benencia created BIT-1290: ---------------------------------- Summary: Permissions of some helpers scripts Key: BIT-1290 URL: https://bro-tracker.atlassian.net/browse/BIT-1290 Project: Bro Issue Tracker Issue Type: Improvement Components: BroControl Affects Versions: 2.3 Reporter: Ra?l Benencia Attachments: 01-properly-install-script-helpers.patch Hi, Some broctl helpers are incorrectly installed with +x permissions. The affected files are {{aux/broctl/bin/helpers/to-bytes.awk}} and {{/aux/broctl/bin/set-bro-path}}. I'm attaching a patch I've used for the Debian package in order to solve this issue. Thanks. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Fri Nov 21 06:49:07 2014 From: jira at bro-tracker.atlassian.net (=?UTF-8?Q?Ra=C3=BAl_Benencia_=28JIRA=29?=) Date: Fri, 21 Nov 2014 08:49:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1291) Source contains prebuilt Python pbject In-Reply-To: References: Message-ID: Ra?l Benencia created BIT-1291: ---------------------------------- Summary: Source contains prebuilt Python pbject Key: BIT-1291 URL: https://bro-tracker.atlassian.net/browse/BIT-1291 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Reporter: Ra?l Benencia Priority: Trivial Hi, The source tarball contains a prebuilt Python object. They are usually left by mistake when generating the tarball by not cleaning the source directory first. The affected files are {{doc/ext/bro_lexer/bro.pyc}} and {{doc/ext/bro_lexer/__init__.pyc}}. It would be great if you could clean them up. Thanks -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Fri Nov 21 10:50:07 2014 From: jira at bro-tracker.atlassian.net (=?UTF-8?Q?Ra=C3=BAl_Benencia_=28JIRA=29?=) Date: Fri, 21 Nov 2014 12:50:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1292) During distclean, clean testing tmp and log files In-Reply-To: References: Message-ID: Ra?l Benencia created BIT-1292: ---------------------------------- Summary: During distclean, clean testing tmp and log files Key: BIT-1292 URL: https://bro-tracker.atlassian.net/browse/BIT-1292 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro Reporter: Ra?l Benencia Attachments: 04-clean-test-files.patch Hi, Before building Bro Debian package, a {{make distclean}} is launched in order to clean the source tree. After that, the build will fail if the source tree is left in a different state than when it was decompressed from the tar.gz. Temporary and log files generated during the tests aren't cleaned when {{distclean}} rule is invoked. Attached to this issue you'll find a patch, which I'm going to use for the Debian package, that takes care of this problem. Thanks! -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From noreply at bro.org Sat Nov 22 00:00:27 2014 From: noreply at bro.org (Merge Tracker) Date: Sat, 22 Nov 2014 00:00:27 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411220800.sAM80Rus002211@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ----------- ---------- ---------- ------------- ---------- ---------------------------------------------- BIT-856 [1] bro-aux Vern Paxson - 2014-11-21 2.4 Normal more documentation for utilities would be cool [1] BIT-856 https://bro-tracker.atlassian.net/browse/BIT-856 From noreply at bro.org Sun Nov 23 00:00:32 2014 From: noreply at bro.org (Merge Tracker) Date: Sun, 23 Nov 2014 00:00:32 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411230800.sAN80WJE013209@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ----------- ---------- ---------- ------------- ---------- ---------------------------------------------- BIT-856 [1] bro-aux Vern Paxson - 2014-11-21 2.4 Normal more documentation for utilities would be cool [1] BIT-856 https://bro-tracker.atlassian.net/browse/BIT-856 From noreply at bro.org Mon Nov 24 00:00:26 2014 From: noreply at bro.org (Merge Tracker) Date: Mon, 24 Nov 2014 00:00:26 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411240800.sAO80Q9J021204@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ----------- ---------- ---------- ------------- ---------- ---------------------------------------------- BIT-856 [1] bro-aux Vern Paxson - 2014-11-21 2.4 Normal more documentation for utilities would be cool [1] BIT-856 https://bro-tracker.atlassian.net/browse/BIT-856 From jira at bro-tracker.atlassian.net Mon Nov 24 07:28:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 24 Nov 2014 09:28:08 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-856) more documentation for utilities would be cool In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=19000#comment-19000 ] Robin Sommer commented on BIT-856: ---------------------------------- Daniel, up for integrating these? > more documentation for utilities would be cool > ---------------------------------------------- > > Key: BIT-856 > URL: https://bro-tracker.atlassian.net/browse/BIT-856 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: bro-aux > Affects Versions: git/master > Reporter: Vern Paxson > Assignee: Daniel Thayer > Fix For: 2.4 > > Attachments: bro.8, broctl.8, bro-cut.1, trace-summary.1 > > > Utilities like bro-cut only supply \--help documentation, as far as I can tell. Man pages would be handy. (In particular, I was looking for some sort of statement of exactly to what degree bro-cut can munch on the concatenation of multiple log files that have different column layouts.) -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Mon Nov 24 07:28:08 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 24 Nov 2014 09:28:08 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-856) more documentation for utilities would be cool In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-856: ----------------------------- Assignee: Daniel Thayer > more documentation for utilities would be cool > ---------------------------------------------- > > Key: BIT-856 > URL: https://bro-tracker.atlassian.net/browse/BIT-856 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: bro-aux > Affects Versions: git/master > Reporter: Vern Paxson > Assignee: Daniel Thayer > Fix For: 2.4 > > Attachments: bro.8, broctl.8, bro-cut.1, trace-summary.1 > > > Utilities like bro-cut only supply \--help documentation, as far as I can tell. Man pages would be handy. (In particular, I was looking for some sort of statement of exactly to what degree bro-cut can munch on the concatenation of multiple log files that have different column layouts.) -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From noreply at bro.org Tue Nov 25 00:00:29 2014 From: noreply at bro.org (Merge Tracker) Date: Tue, 25 Nov 2014 00:00:29 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411250800.sAP80TuD011922@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ----------- ------------- ---------- ------------- ---------- ---------------------------------------------- BIT-856 [1] bro-aux Vern Paxson Daniel Thayer 2014-11-24 2.4 Normal more documentation for utilities would be cool [1] BIT-856 https://bro-tracker.atlassian.net/browse/BIT-856 From jira at bro-tracker.atlassian.net Tue Nov 25 15:04:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Tue, 25 Nov 2014 17:04:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1293) Merge topic/johanna/ssl-fail-earlier In-Reply-To: References: Message-ID: Johanna Amann created BIT-1293: ---------------------------------- Summary: Merge topic/johanna/ssl-fail-earlier Key: BIT-1293 URL: https://bro-tracker.atlassian.net/browse/BIT-1293 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Reporter: Johanna Amann Fix For: 2.4 Please merge topic/johanna/ssl-fail-earlier. It makes some of the failures in the SSL analyzer more final (meaning it does not attempt to parse any further packets in the trace). This should have been the behavior in most of the cases where we rise ProtocolViolations. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Tue Nov 25 15:04:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Tue, 25 Nov 2014 17:04:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1293) Merge topic/johanna/ssl-fail-earlier In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1293: ------------------------------- Status: Merge Request (was: Open) > Merge topic/johanna/ssl-fail-earlier > ------------------------------------ > > Key: BIT-1293 > URL: https://bro-tracker.atlassian.net/browse/BIT-1293 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Johanna Amann > Fix For: 2.4 > > > Please merge topic/johanna/ssl-fail-earlier. It makes some of the failures in the SSL analyzer more final (meaning it does not attempt to parse any further packets in the trace). > This should have been the behavior in most of the cases where we rise ProtocolViolations. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Tue Nov 25 15:04:07 2014 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Tue, 25 Nov 2014 17:04:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1293) Merge topic/johanna/ssl-fail-earlier In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1293: ------------------------------- Assignee: Robin Sommer > Merge topic/johanna/ssl-fail-earlier > ------------------------------------ > > Key: BIT-1293 > URL: https://bro-tracker.atlassian.net/browse/BIT-1293 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Johanna Amann > Assignee: Robin Sommer > Fix For: 2.4 > > > Please merge topic/johanna/ssl-fail-earlier. It makes some of the failures in the SSL analyzer more final (meaning it does not attempt to parse any further packets in the trace). > This should have been the behavior in most of the cases where we rise ProtocolViolations. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From jira at bro-tracker.atlassian.net Tue Nov 25 17:44:07 2014 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 25 Nov 2014 19:44:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1293) Merge topic/johanna/ssl-fail-earlier In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1293: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > Merge topic/johanna/ssl-fail-earlier > ------------------------------------ > > Key: BIT-1293 > URL: https://bro-tracker.atlassian.net/browse/BIT-1293 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Reporter: Johanna Amann > Assignee: Robin Sommer > Fix For: 2.4 > > > Please merge topic/johanna/ssl-fail-earlier. It makes some of the failures in the SSL analyzer more final (meaning it does not attempt to parse any further packets in the trace). > This should have been the behavior in most of the cases where we rise ProtocolViolations. -- This message was sent by Atlassian JIRA (v6.4-OD-09-008#64005) From noreply at bro.org Wed Nov 26 00:00:40 2014 From: noreply at bro.org (Merge Tracker) Date: Wed, 26 Nov 2014 00:00:40 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411260800.sAQ80e7j010122@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ----------- ------------- ---------- ------------- ---------- ---------------------------------------------- BIT-856 [1] bro-aux Vern Paxson Daniel Thayer 2014-11-24 2.4 Normal more documentation for utilities would be cool [1] BIT-856 https://bro-tracker.atlassian.net/browse/BIT-856 From noreply at bro.org Thu Nov 27 00:00:24 2014 From: noreply at bro.org (Merge Tracker) Date: Thu, 27 Nov 2014 00:00:24 -0800 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201411270800.sAR80OL9032720@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ----------- ----------- ----------- ------------- ---------- ------------- ---------- ---------------------------------------------- BIT-856 [1] bro-aux Vern Paxson Daniel Thayer 2014-11-24 2.4 Normal more documentation for utilities would be cool Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------- ---------- ---------------------------------------------------------------- #18 [2] bro hillu [3] 2014-11-26 BIFScanner: Make filename->symbol transformation more robust [4] [1] BIT-856 https://bro-tracker.atlassian.net/browse/BIT-856 [2] Pull Request #18 https://github.com/bro/bro/pull/18 [3] hillu https://github.com/hillu [4] Merge Pull Request #18 with git pull --no-ff --no-commit https://github.com/hillu/bro.git master From jira at bro-tracker.atlassian.net Thu Nov 27 07:50:07 2014 From: jira at bro-tracker.atlassian.net (Wouter Clarie (JIRA)) Date: Thu, 27 Nov 2014 09:50:07 -0600 (CST) Subject: [Bro-Dev] [JIRA] (BIT-1294) Input framework: table name truncated at underscore In-Reply-To: References: Message-ID: Wouter Clarie created BIT-1294: ---------------------------------- Summary: Input framework: table name truncated at underscore Key: BIT-1294 URL: https://bro-tracker.atlassian.net/browse/BIT-1294 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Reporter: Wouter Clarie When loading a table in the input framework, e.g.: Input::add_table([ $source="/tmp/sdb_servers", $name="sdb_servers", $idx=sdb_servers_idx, $val=sdb_servers_val, $destination=sdb_servers, $mode=Input::REREAD ]); And later on reporting when the input has been fully read: event Input::end_of_data(name: string, source: string) { Reporter::info(fmt("Input stream name %s source %s ready", name, source)); } This is what ends up in the reporter log: 0.000000 Reporter::INFO Input stream name sdb source /tmp/sdb_servers ready