From noreply at bro.org Mon Jun 1 00:00:27 2015 From: noreply at bro.org (Merge Tracker) Date: Mon, 1 Jun 2015 00:00:27 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506010700.t5170RFi024927@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-05-29 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From robin at icir.org Mon Jun 1 08:10:39 2015 From: robin at icir.org (Robin Sommer) Date: Mon, 1 Jun 2015 08:10:39 -0700 Subject: [Bro-Dev] [JIRA] (BIT-1407) -f silently fails if base/frameworks/packet-filter isn't loaded In-Reply-To: References: Message-ID: <20150601151039.GL72788@icir.org> On Sat, May 30, 2015 at 15:31 -0500, you wrote: > Jira, I couldn't find the thread). But to revisit: the "-f filter" > option silently does nothing if base/frameworks/packet-filter isn't > loaded (so the scenario here is using -b to suppress its automatic > loading). This can lead to seriously confusing behavior. Yeah, I can see that. I think the main problem is the interaction between the command-line option and script, something that's rare (i.e., that the command-line option is tight that closely to a script being loaded). I would actually suggest we remove the command-line option altogether and instead work with a global: "bro -i eth0 PacketFilter::filter=XXXX" (I believe we have a global with that effect already, otherwise we could add it). Robin From jira at bro-tracker.atlassian.net Mon Jun 1 08:12:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 1 Jun 2015 10:12:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1407) -f silently fails if base/frameworks/packet-filter isn't loaded In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20900#comment-20900 ] Robin Sommer commented on BIT-1407: ----------------------------------- Yeah, I can see that. I think the main problem is the interaction between the command-line option and script, something that's rare (i.e., that the command-line option is tight that closely to a script being loaded). I would actually suggest we remove the command-line option altogether and instead work with a global: "bro -i eth0 PacketFilter::filter=XXXX" (I believe we have a global with that effect already, otherwise we could add it). Robin > -f silently fails if base/frameworks/packet-filter isn't loaded > --------------------------------------------------------------- > > Key: BIT-1407 > URL: https://bro-tracker.atlassian.net/browse/BIT-1407 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Vern Paxson > > I know we've been through this before (though searching the tickets in Jira, I couldn't find the thread). But to revisit: the "-f filter" option silently does nothing if base/frameworks/packet-filter isn't loaded (so the scenario here is using -b to suppress its automatic loading). This can lead to seriously confusing behavior. It would be preferable if there's either an error message indicating that the option won't be supported, or if it forced loading of packet-filter. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From seth at icir.org Mon Jun 1 08:18:26 2015 From: seth at icir.org (Seth Hall) Date: Mon, 1 Jun 2015 11:18:26 -0400 Subject: [Bro-Dev] [JIRA] (BIT-1407) -f silently fails if base/frameworks/packet-filter isn't loaded In-Reply-To: <20150601151039.GL72788@icir.org> References: <20150601151039.GL72788@icir.org> Message-ID: <86689609-F965-4A9F-B627-8AA4E2C9739C@icir.org> > On Jun 1, 2015, at 11:10 AM, Robin Sommer wrote: > > Yeah, I can see that. I think the main problem is the interaction > between the command-line option and script, something that's rare > (i.e., that the command-line option is tight that closely to a script > being loaded). I would actually suggest we remove the command-line > option altogether and instead work with a global: "bro -i eth0 > PacketFilter::filter=XXXX" (I believe we have a global with that > effect already, otherwise we could add it). We could alternately take the additional step of moving command line argument processing into Bro scripts. I don?t know how exactly we?d solve this particular problem, but it should be reasonably solvable that way. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150601/7258d761/attachment.bin From jira at bro-tracker.atlassian.net Mon Jun 1 08:20:00 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Mon, 1 Jun 2015 10:20:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1407) -f silently fails if base/frameworks/packet-filter isn't loaded In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1407: --------------------------- Attachment: signature.asc We could alternately take the additional step of moving command line argument processing into Bro scripts. I don?t know how exactly we?d solve this particular problem, but it should be reasonably solvable that way. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ > -f silently fails if base/frameworks/packet-filter isn't loaded > --------------------------------------------------------------- > > Key: BIT-1407 > URL: https://bro-tracker.atlassian.net/browse/BIT-1407 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Vern Paxson > Attachments: signature.asc > > > I know we've been through this before (though searching the tickets in Jira, I couldn't find the thread). But to revisit: the "-f filter" option silently does nothing if base/frameworks/packet-filter isn't loaded (so the scenario here is using -b to suppress its automatic loading). This can lead to seriously confusing behavior. It would be preferable if there's either an error message indicating that the option won't be supported, or if it forced loading of packet-filter. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 08:27:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 1 Jun 2015 10:27:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1408) Broker I/O loop issue In-Reply-To: References: Message-ID: Robin Sommer created BIT-1408: --------------------------------- Summary: Broker I/O loop issue Key: BIT-1408 URL: https://bro-tracker.atlassian.net/browse/BIT-1408 Project: Bro Issue Tracker Issue Type: Problem Components: Broker Reporter: Robin Sommer Fix For: 2.4 Attachments: patch.txt This script from Johanna doesn't terminate: {code} redef exit_only_after_terminate = T; event terminate_me() { print "terminating"; terminate(); } event bro_init() { BrokerComm::enable(); schedule 1sec { terminate_me() }; } {code} It works once the {{enable()}} call is removed. Attached patch seems solve the problem, but it might not be quite the right fix, not sure yet. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From robin at icir.org Mon Jun 1 08:34:54 2015 From: robin at icir.org (Robin Sommer) Date: Mon, 1 Jun 2015 08:34:54 -0700 Subject: [Bro-Dev] [JIRA] (BIT-1407) -f silently fails if base/frameworks/packet-filter isn't loaded In-Reply-To: References: Message-ID: <20150601153454.GQ72788@icir.org> On Mon, Jun 01, 2015 at 10:20 -0500, you wrote: > We could alternately take the additional step of moving command line > argument processing into Bro scripts. I don?t know how exactly we?d > solve this particular problem, but it should be reasonably solvable > that way. If we had that, the '-f' switch would probably be added by the packet filter script, so it'd be available only when loaded. I'd like to move argument processing into script-land as well. Generally, however, I think it would still be good to avoid arguments controlling script behaviour as much as possible. Main reason is argument inflation: -f has been historically around but there are plenty other scripts that in principle could take command-line arguments as well, which would get messy. And we have the X=Y syntax already to take care of that. From jira at bro-tracker.atlassian.net Mon Jun 1 08:36:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 1 Jun 2015 10:36:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1407) -f silently fails if base/frameworks/packet-filter isn't loaded In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20902#comment-20902 ] Robin Sommer commented on BIT-1407: ----------------------------------- If we had that, the '-f' switch would probably be added by the packet filter script, so it'd be available only when loaded. I'd like to move argument processing into script-land as well. Generally, however, I think it would still be good to avoid arguments controlling script behaviour as much as possible. Main reason is argument inflation: -f has been historically around but there are plenty other scripts that in principle could take command-line arguments as well, which would get messy. And we have the X=Y syntax already to take care of that. > -f silently fails if base/frameworks/packet-filter isn't loaded > --------------------------------------------------------------- > > Key: BIT-1407 > URL: https://bro-tracker.atlassian.net/browse/BIT-1407 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Vern Paxson > Attachments: signature.asc > > > I know we've been through this before (though searching the tickets in Jira, I couldn't find the thread). But to revisit: the "-f filter" option silently does nothing if base/frameworks/packet-filter isn't loaded (so the scenario here is using -b to suppress its automatic loading). This can lead to seriously confusing behavior. It would be preferable if there's either an error message indicating that the option won't be supported, or if it forced loading of packet-filter. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 08:40:01 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Mon, 1 Jun 2015 10:40:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1404) decompose_uri() builtin throws errors on URIs with select parameters In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1404: --------------------------- Resolution: Fixed Status: Closed (was: Open) > decompose_uri() builtin throws errors on URIs with select parameters > -------------------------------------------------------------------- > > Key: BIT-1404 > URL: https://bro-tracker.atlassian.net/browse/BIT-1404 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Stephen Hosom > Assignee: Seth Hall > Fix For: 2.4 > > > URIs with odd query strings cause errors in reporter.log. > For example: > local something = decompose_uri("dfasjdfasdfasdf?asd"); > results in: > error in /usr/local/bro-master/share/bro/base/utils/urls.bro, line 79: no such index (parts[1]) > http://try.bro.org/#/trybro/saved/8505 demonstrates a pretty alright example. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 11:34:01 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Mon, 1 Jun 2015 13:34:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1407) -f silently fails if base/frameworks/packet-filter isn't loaded In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20903#comment-20903 ] Vern Paxson commented on BIT-1407: ---------------------------------- While there's an appeal to processing arguments in script-land, because some arguments control basic script processing (e.g., -p), I'm not sure this can be done in a coherent fashion without some significant under-the-hood kludges. Regarding replacing -f with PacketFilter::filter=XXXX, yuck - I wouldn't want to have to remember that, and there's no ready way for a user to discover this. I'd be happy settling for -f warning (or exiting) with a statement that without the packet filtering framework loaded, it's a no-op. Though I have a forboding that you're going to tell me that that's actually hard to implement :-P. > -f silently fails if base/frameworks/packet-filter isn't loaded > --------------------------------------------------------------- > > Key: BIT-1407 > URL: https://bro-tracker.atlassian.net/browse/BIT-1407 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Vern Paxson > Attachments: signature.asc > > > I know we've been through this before (though searching the tickets in Jira, I couldn't find the thread). But to revisit: the "-f filter" option silently does nothing if base/frameworks/packet-filter isn't loaded (so the scenario here is using -b to suppress its automatic loading). This can lead to seriously confusing behavior. It would be preferable if there's either an error message indicating that the option won't be supported, or if it forced loading of packet-filter. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 11:36:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Mon, 1 Jun 2015 13:36:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1407) -f silently fails if base/frameworks/packet-filter isn't loaded In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20904#comment-20904 ] Vern Paxson commented on BIT-1407: ---------------------------------- Also, regarding the policy script adding the -f flag: how would the user discover that that's what they need to do (have the policy script loaded) to get the flag? I can see quite a bit of perplexment if Bro just tells the user that the flag doesn't exist without explaining why, if they don't have the script loaded. > -f silently fails if base/frameworks/packet-filter isn't loaded > --------------------------------------------------------------- > > Key: BIT-1407 > URL: https://bro-tracker.atlassian.net/browse/BIT-1407 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Vern Paxson > Attachments: signature.asc > > > I know we've been through this before (though searching the tickets in Jira, I couldn't find the thread). But to revisit: the "-f filter" option silently does nothing if base/frameworks/packet-filter isn't loaded (so the scenario here is using -b to suppress its automatic loading). This can lead to seriously confusing behavior. It would be preferable if there's either an error message indicating that the option won't be supported, or if it forced loading of packet-filter. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 11:42:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Mon, 1 Jun 2015 13:42:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1409) DNS ZoneTransfer notice missing In-Reply-To: References: Message-ID: Vern Paxson created BIT-1409: -------------------------------- Summary: DNS ZoneTransfer notice missing Key: BIT-1409 URL: https://bro-tracker.atlassian.net/browse/BIT-1409 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Vern Paxson Assignee: Seth Hall The DNS analyzer used to generate a ZoneTransfer notice that can be handy for some operational settings. This fell by the wayside, apparently (per Seth) for no particular reason, so would be nice to restore it. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 11:44:00 2015 From: jira at bro-tracker.atlassian.net (Ali Hadi (JIRA)) Date: Mon, 1 Jun 2015 13:44:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1410) tx_hosts and rx_hosts switched in files.log In-Reply-To: References: Message-ID: Ali Hadi created BIT-1410: ----------------------------- Summary: tx_hosts and rx_hosts switched in files.log Key: BIT-1410 URL: https://bro-tracker.atlassian.net/browse/BIT-1410 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Environment: Linux Ubuntu Reporter: Ali Hadi Priority: High Hi, _Based on Robin's request I opened this ticket. _ If you use the PCAP below and analyze it using Bro: https://www.bro.org/static/traces/email.pcap Then when checking the files.log, the tx_hosts is supposed to show the host who transmitted the file, and rx_hosts is for the host who received the file based on Bro's documentation: https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html If you do the following: cat files.log | bro-cut fuid tx_hosts rx_hosts | grep You'll get that the TX Host IP (SrcIP) is 192.168.121.176 and not 192.168.121.179 !!! It seems that Bro switched their positions in the output. I found this in an assignment given to my students, and one of them gave me a result completely different. So when I double checked with Wireshark, I found that the IPs have been switched by Bro. Hope this helps. Ali -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 12:26:00 2015 From: jira at bro-tracker.atlassian.net (Ali Hadi (JIRA)) Date: Mon, 1 Jun 2015 14:26:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1410) tx_hosts and rx_hosts switched in files.log In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20905#comment-20905 ] Ali Hadi commented on BIT-1410: ------------------------------- Kindly note the version used of Bro is 2.4-beta. > tx_hosts and rx_hosts switched in files.log > ------------------------------------------- > > Key: BIT-1410 > URL: https://bro-tracker.atlassian.net/browse/BIT-1410 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: Linux Ubuntu > Reporter: Ali Hadi > Priority: High > Labels: analyzer > > Hi, > _Based on Robin's request I opened this ticket. > _ > If you use the PCAP below and analyze it using Bro: > https://www.bro.org/static/traces/email.pcap > Then when checking the files.log, the tx_hosts is supposed to show the host who transmitted the file, and rx_hosts is for the host who received the file based on Bro's documentation: https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html > If you do the following: > cat files.log | bro-cut fuid tx_hosts rx_hosts | grep > You'll get that the TX Host IP (SrcIP) is 192.168.121.176 and not 192.168.121.179 !!! > It seems that Bro switched their positions in the output. I found this in an assignment given to my students, and one of them gave me a result completely different. So when I double checked with Wireshark, I found that the IPs have been switched by Bro. > Hope this helps. > Ali -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 13:28:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 1 Jun 2015 15:28:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1410) tx_hosts and rx_hosts switched in files.log In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20906#comment-20906 ] Robin Sommer commented on BIT-1410: ----------------------------------- >From Vlad on the mailing list: Looks like this comes from the assumption made here: https://github.com/bro/bro/blob/master/src/analyzer/protocol/mime/MIME.cc#L1459 > tx_hosts and rx_hosts switched in files.log > ------------------------------------------- > > Key: BIT-1410 > URL: https://bro-tracker.atlassian.net/browse/BIT-1410 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: Linux Ubuntu > Reporter: Ali Hadi > Priority: High > Labels: analyzer > Fix For: 2.4 > > > Hi, > _Based on Robin's request I opened this ticket. > _ > If you use the PCAP below and analyze it using Bro: > https://www.bro.org/static/traces/email.pcap > Then when checking the files.log, the tx_hosts is supposed to show the host who transmitted the file, and rx_hosts is for the host who received the file based on Bro's documentation: https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html > If you do the following: > cat files.log | bro-cut fuid tx_hosts rx_hosts | grep > You'll get that the TX Host IP (SrcIP) is 192.168.121.176 and not 192.168.121.179 !!! > It seems that Bro switched their positions in the output. I found this in an assignment given to my students, and one of them gave me a result completely different. So when I double checked with Wireshark, I found that the IPs have been switched by Bro. > Hope this helps. > Ali -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 13:28:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 1 Jun 2015 15:28:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1410) tx_hosts and rx_hosts switched in files.log In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1410: ------------------------------ Fix Version/s: 2.4 > tx_hosts and rx_hosts switched in files.log > ------------------------------------------- > > Key: BIT-1410 > URL: https://bro-tracker.atlassian.net/browse/BIT-1410 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: Linux Ubuntu > Reporter: Ali Hadi > Priority: High > Labels: analyzer > Fix For: 2.4 > > > Hi, > _Based on Robin's request I opened this ticket. > _ > If you use the PCAP below and analyze it using Bro: > https://www.bro.org/static/traces/email.pcap > Then when checking the files.log, the tx_hosts is supposed to show the host who transmitted the file, and rx_hosts is for the host who received the file based on Bro's documentation: https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html > If you do the following: > cat files.log | bro-cut fuid tx_hosts rx_hosts | grep > You'll get that the TX Host IP (SrcIP) is 192.168.121.176 and not 192.168.121.179 !!! > It seems that Bro switched their positions in the output. I found this in an assignment given to my students, and one of them gave me a result completely different. So when I double checked with Wireshark, I found that the IPs have been switched by Bro. > Hope this helps. > Ali -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 13:28:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 1 Jun 2015 15:28:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1410) tx_hosts and rx_hosts switched in files.log In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20907#comment-20907 ] Robin Sommer commented on BIT-1410: ----------------------------------- Setting it to 2.4, let's see if we can get this fixed still. > tx_hosts and rx_hosts switched in files.log > ------------------------------------------- > > Key: BIT-1410 > URL: https://bro-tracker.atlassian.net/browse/BIT-1410 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: Linux Ubuntu > Reporter: Ali Hadi > Priority: High > Labels: analyzer > Fix For: 2.4 > > > Hi, > _Based on Robin's request I opened this ticket. > _ > If you use the PCAP below and analyze it using Bro: > https://www.bro.org/static/traces/email.pcap > Then when checking the files.log, the tx_hosts is supposed to show the host who transmitted the file, and rx_hosts is for the host who received the file based on Bro's documentation: https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html > If you do the following: > cat files.log | bro-cut fuid tx_hosts rx_hosts | grep > You'll get that the TX Host IP (SrcIP) is 192.168.121.176 and not 192.168.121.179 !!! > It seems that Bro switched their positions in the output. I found this in an assignment given to my students, and one of them gave me a result completely different. So when I double checked with Wireshark, I found that the IPs have been switched by Bro. > Hope this helps. > Ali -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 14:11:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Mon, 1 Jun 2015 16:11:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1411) SQL_Injection_Victim is a misleading name In-Reply-To: References: Message-ID: Vern Paxson created BIT-1411: -------------------------------- Summary: SQL_Injection_Victim is a misleading name Key: BIT-1411 URL: https://bro-tracker.atlassian.net/browse/BIT-1411 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Vern Paxson I suggest changing the name of this notice to {{SQL_Injection_Target}}. Having "victim" in the name implies to me that the attack succeeded, which is not what the associated logic is about. Indeed, I even wonder if this notice is useful. The information should be directly available from {{SQL_Injection_Attacker}} notices (though it doesn't appear to be currently set up to provide this - why not?). -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 14:17:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Mon, 1 Jun 2015 16:17:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1412) Documentation/control of Jira markup shortcuts? In-Reply-To: References: Message-ID: Vern Paxson created BIT-1412: -------------------------------- Summary: Documentation/control of Jira markup shortcuts? Key: BIT-1412 URL: https://bro-tracker.atlassian.net/browse/BIT-1412 Project: Bro Issue Tracker Issue Type: Problem Components: TicketTracker Reporter: Vern Paxson I find that some of the keystroke bindings for markup when typing in a Description (like this one!) are counterintuitive for me and would like to change them. However, searching the Jira documentation I haven't been able to find where these shortcuts are documented (as opposed to more general ones that aren't about markup). Where do I look for info like this? -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 15:02:01 2015 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Mon, 1 Jun 2015 17:02:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1412) Documentation/control of Jira markup shortcuts? In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20908#comment-20908 ] Vlad Grigorescu commented on BIT-1412: -------------------------------------- I don't think they're modifiable, but you can disable them if you'd prefer. You can view the shortcuts by using the "?" shortcut, or by clicking on the question mark at the upper right, and then selecting "Keyboard Shortcuts." Once the shortcut modal is displayed, there's an option to disable them at the bottom. > Documentation/control of Jira markup shortcuts? > ----------------------------------------------- > > Key: BIT-1412 > URL: https://bro-tracker.atlassian.net/browse/BIT-1412 > Project: Bro Issue Tracker > Issue Type: Problem > Components: TicketTracker > Reporter: Vern Paxson > > I find that some of the keystroke bindings for markup when typing in a Description (like this one!) are counterintuitive for me and would like to change them. However, searching the Jira documentation I haven't been able to find where these shortcuts are documented (as opposed to more general ones that aren't about markup). Where do I look for info like this? -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 18:04:01 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Mon, 1 Jun 2015 20:04:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1412) Documentation/control of Jira markup shortcuts? In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20909#comment-20909 ] Vern Paxson commented on BIT-1412: ---------------------------------- Yes I see those but those aren't the shortcuts enabled for Description/Comment. For example, if right now I try to use ctrl-B to back up, instead I get *bold*, which isn't even mentioned on that menu. > Documentation/control of Jira markup shortcuts? > ----------------------------------------------- > > Key: BIT-1412 > URL: https://bro-tracker.atlassian.net/browse/BIT-1412 > Project: Bro Issue Tracker > Issue Type: Problem > Components: TicketTracker > Reporter: Vern Paxson > > I find that some of the keystroke bindings for markup when typing in a Description (like this one!) are counterintuitive for me and would like to change them. However, searching the Jira documentation I haven't been able to find where these shortcuts are documented (as opposed to more general ones that aren't about markup). Where do I look for info like this? -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 18:24:00 2015 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Mon, 1 Jun 2015 20:24:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1412) Documentation/control of Jira markup shortcuts? In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20910#comment-20910 ] Vlad Grigorescu commented on BIT-1412: -------------------------------------- Ah, my mistake. I believe the editor shortcuts are a subset of those listed here: https://confluence.atlassian.com/display/ConfCloud/Keyboard+Shortcuts#KeyboardShortcuts-Intheeditor If I disable the keyboard shortcuts, the editor shortcuts get disabled as well, if that helps... > Documentation/control of Jira markup shortcuts? > ----------------------------------------------- > > Key: BIT-1412 > URL: https://bro-tracker.atlassian.net/browse/BIT-1412 > Project: Bro Issue Tracker > Issue Type: Problem > Components: TicketTracker > Reporter: Vern Paxson > > I find that some of the keystroke bindings for markup when typing in a Description (like this one!) are counterintuitive for me and would like to change them. However, searching the Jira documentation I haven't been able to find where these shortcuts are documented (as opposed to more general ones that aren't about markup). Where do I look for info like this? -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 19:17:01 2015 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Mon, 1 Jun 2015 21:17:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1410) tx_hosts and rx_hosts switched in files.log In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Grigorescu updated BIT-1410: --------------------------------- Status: Merge Request (was: Open) > tx_hosts and rx_hosts switched in files.log > ------------------------------------------- > > Key: BIT-1410 > URL: https://bro-tracker.atlassian.net/browse/BIT-1410 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: Linux Ubuntu > Reporter: Ali Hadi > Priority: High > Labels: analyzer > Fix For: 2.4 > > > Hi, > _Based on Robin's request I opened this ticket. > _ > If you use the PCAP below and analyze it using Bro: > https://www.bro.org/static/traces/email.pcap > Then when checking the files.log, the tx_hosts is supposed to show the host who transmitted the file, and rx_hosts is for the host who received the file based on Bro's documentation: https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html > If you do the following: > cat files.log | bro-cut fuid tx_hosts rx_hosts | grep > You'll get that the TX Host IP (SrcIP) is 192.168.121.176 and not 192.168.121.179 !!! > It seems that Bro switched their positions in the output. I found this in an assignment given to my students, and one of them gave me a result completely different. So when I double checked with Wireshark, I found that the IPs have been switched by Bro. > Hope this helps. > Ali -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 1 19:17:00 2015 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Mon, 1 Jun 2015 21:17:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1410) tx_hosts and rx_hosts switched in files.log In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20911#comment-20911 ] Vlad Grigorescu commented on BIT-1410: -------------------------------------- Fix is in branch topic/vladg/bit-1410 in bro, bro-testing and bro-testing-private. I just propagated is_orig down to MIME_Mail. The POP3 analyzer was also touched, but since it's missing btests, I didn't write any tests for that change. > tx_hosts and rx_hosts switched in files.log > ------------------------------------------- > > Key: BIT-1410 > URL: https://bro-tracker.atlassian.net/browse/BIT-1410 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: Linux Ubuntu > Reporter: Ali Hadi > Priority: High > Labels: analyzer > Fix For: 2.4 > > > Hi, > _Based on Robin's request I opened this ticket. > _ > If you use the PCAP below and analyze it using Bro: > https://www.bro.org/static/traces/email.pcap > Then when checking the files.log, the tx_hosts is supposed to show the host who transmitted the file, and rx_hosts is for the host who received the file based on Bro's documentation: https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html > If you do the following: > cat files.log | bro-cut fuid tx_hosts rx_hosts | grep > You'll get that the TX Host IP (SrcIP) is 192.168.121.176 and not 192.168.121.179 !!! > It seems that Bro switched their positions in the output. I found this in an assignment given to my students, and one of them gave me a result completely different. So when I double checked with Wireshark, I found that the IPs have been switched by Bro. > Hope this helps. > Ali -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Tue Jun 2 00:00:17 2015 From: noreply at bro.org (Merge Tracker) Date: Tue, 2 Jun 2015 00:00:17 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506020700.t5270HbN012629@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------- BIT-1410 [1] Bro Ali Hadi - 2015-06-01 2.4 High tx_hosts and rx_hosts switched in files.log BIT-1399 [2] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [3] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [4] bro yunzheng [5] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [6] #30 [7] bro jsbarber [8] 2015-05-29 Use a common Packet format and preserve layer 2 information [9] #1 [10] bro-plugins jsbarber [11] 2015-05-23 Use a common Packet format and preserve layer 2 information [12] [1] BIT-1410 https://bro-tracker.atlassian.net/browse/BIT-1410 [2] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [3] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [4] Pull Request #31 https://github.com/bro/bro/pull/31 [5] yunzheng https://github.com/yunzheng [6] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [7] Pull Request #30 https://github.com/bro/bro/pull/30 [8] jsbarber https://github.com/jsbarber [9] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [10] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [11] jsbarber https://github.com/jsbarber [12] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From robin at icir.org Tue Jun 2 09:11:32 2015 From: robin at icir.org (Robin Sommer) Date: Tue, 2 Jun 2015 09:11:32 -0700 Subject: [Bro-Dev] [Bro-Commits] [git/bro] topic/dnthayer/doc-improvements-2.4: Add missing documentation on the "Bro Package Index" page (45caf8d) In-Reply-To: <201506021503.t52F3WPK025079@bro-ids.icir.org> References: <201506021503.t52F3WPK025079@bro-ids.icir.org> Message-ID: <20150602161132.GA94675@icir.org> Daniel, do you mind if I just go ahead and keep merging this branch as you make updates? The less last-minute changes, the better. Robin On Tue, Jun 02, 2015 at 10:00 -0500, Daniel Thayer wrote: > Repository : ssh://git at bro-ids.icir.org/bro > > On branch : topic/dnthayer/doc-improvements-2.4 > Link : https://github.com/bro/bro/commit/45caf8d2c1a655a7d0613feeb7e6e818e216d83e > > >--------------------------------------------------------------- > > commit 45caf8d2c1a655a7d0613feeb7e6e818e216d83e > Author: Daniel Thayer > Date: Tue Jun 2 10:00:00 2015 -0500 > > Add missing documentation on the "Bro Package Index" page > > > >--------------------------------------------------------------- > > 45caf8d2c1a655a7d0613feeb7e6e818e216d83e > scripts/base/files/pe/README | 1 + > scripts/base/frameworks/broker/README | 2 ++ > scripts/base/protocols/krb/README | 1 + > scripts/base/protocols/mysql/README | 1 + > scripts/base/protocols/radius/README | 1 + > scripts/base/protocols/rdp/README | 1 + > scripts/base/protocols/sip/README | 1 + > scripts/base/protocols/ssh/README | 1 + > 8 files changed, 9 insertions(+) > > diff --git a/scripts/base/files/pe/README b/scripts/base/files/pe/README > new file mode 100644 > index 0000000..3ba2354 > --- /dev/null > +++ b/scripts/base/files/pe/README > @@ -0,0 +1 @@ > +Support for Portable Executable (PE) file analysis. > diff --git a/scripts/base/frameworks/broker/README b/scripts/base/frameworks/broker/README > new file mode 100644 > index 0000000..11c2479 > --- /dev/null > +++ b/scripts/base/frameworks/broker/README > @@ -0,0 +1,2 @@ > +The Broker communication framework facilitates connecting to remote Bro > +instances to share state and transfer events. > diff --git a/scripts/base/protocols/krb/README b/scripts/base/protocols/krb/README > new file mode 100644 > index 0000000..66c3228 > --- /dev/null > +++ b/scripts/base/protocols/krb/README > @@ -0,0 +1 @@ > +Support for Kerberos protocol analysis. > diff --git a/scripts/base/protocols/mysql/README b/scripts/base/protocols/mysql/README > new file mode 100644 > index 0000000..9f642b7 > --- /dev/null > +++ b/scripts/base/protocols/mysql/README > @@ -0,0 +1 @@ > +Support for MySQL protocol analysis. > diff --git a/scripts/base/protocols/radius/README b/scripts/base/protocols/radius/README > new file mode 100644 > index 0000000..5248f62 > --- /dev/null > +++ b/scripts/base/protocols/radius/README > @@ -0,0 +1 @@ > +Support for RADIUS protocol analysis. > diff --git a/scripts/base/protocols/rdp/README b/scripts/base/protocols/rdp/README > new file mode 100644 > index 0000000..19903d0 > --- /dev/null > +++ b/scripts/base/protocols/rdp/README > @@ -0,0 +1 @@ > +Support for Remote Desktop Protocol (RDP) analysis. > diff --git a/scripts/base/protocols/sip/README b/scripts/base/protocols/sip/README > new file mode 100644 > index 0000000..6de9e6c > --- /dev/null > +++ b/scripts/base/protocols/sip/README > @@ -0,0 +1 @@ > +Support for Session Initiation Protocol (SIP) analysis. > diff --git a/scripts/base/protocols/ssh/README b/scripts/base/protocols/ssh/README > new file mode 100644 > index 0000000..54357e1 > --- /dev/null > +++ b/scripts/base/protocols/ssh/README > @@ -0,0 +1 @@ > +Support for SSH protocol analysis. > > > _______________________________________________ > bro-commits mailing list > bro-commits at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits > -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Tue Jun 2 09:12:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 2 Jun 2015 11:12:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1410) tx_hosts and rx_hosts switched in files.log In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1410: --------------------------------- Assignee: Robin Sommer > tx_hosts and rx_hosts switched in files.log > ------------------------------------------- > > Key: BIT-1410 > URL: https://bro-tracker.atlassian.net/browse/BIT-1410 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: Linux Ubuntu > Reporter: Ali Hadi > Assignee: Robin Sommer > Priority: High > Labels: analyzer > Fix For: 2.4 > > > Hi, > _Based on Robin's request I opened this ticket. > _ > If you use the PCAP below and analyze it using Bro: > https://www.bro.org/static/traces/email.pcap > Then when checking the files.log, the tx_hosts is supposed to show the host who transmitted the file, and rx_hosts is for the host who received the file based on Bro's documentation: https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html > If you do the following: > cat files.log | bro-cut fuid tx_hosts rx_hosts | grep > You'll get that the TX Host IP (SrcIP) is 192.168.121.176 and not 192.168.121.179 !!! > It seems that Bro switched their positions in the output. I found this in an assignment given to my students, and one of them gave me a result completely different. So when I double checked with Wireshark, I found that the IPs have been switched by Bro. > Hope this helps. > Ali -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Tue Jun 2 09:26:00 2015 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Tue, 2 Jun 2015 11:26:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1413) README files misidentified by GitHub In-Reply-To: References: Message-ID: Vlad Grigorescu created BIT-1413: ------------------------------------ Summary: README files misidentified by GitHub Key: BIT-1413 URL: https://bro-tracker.atlassian.net/browse/BIT-1413 Project: Bro Issue Tracker Issue Type: Problem Components: Documentation Reporter: Vlad Grigorescu If a README file doesn't have an extension, GitHub will parse it as Markdown. Because our README files are ReST, this results in some ugly (and not very useful) READMEs when visiting the repository on GitHub. For example, see: https://github.com/bro/btest#readme There are two options we could take to fix this: rename README to README.rst, or create a symlink. I tried out the symlink option here, and I think the result is much more useful: https://github.com/grigorescu/btest#readme The affected repos are: binpac bro bro-aux bro-plugins bro-scripts broccoli broccoli-perl broccoli-python broccoli-ruby broctl (broctl's README just instructs users to see doc/broctl.rst. This could just be a symlink) broker bromagic (this can probably be deleted?) btest capstats time-machine trace-summary -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Tue Jun 2 10:58:00 2015 From: jira at bro-tracker.atlassian.net (Jeff (JIRA)) Date: Tue, 2 Jun 2015 12:58:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1414) Make PIE option availalbe during compiling In-Reply-To: References: Message-ID: Jeff created BIT-1414: ------------------------- Summary: Make PIE option availalbe during compiling Key: BIT-1414 URL: https://bro-tracker.atlassian.net/browse/BIT-1414 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: 2.3 Environment: We would like to request PIE support be built in and available in the Bro binary. Reporter: Jeff -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Tue Jun 2 11:09:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 2 Jun 2015 13:09:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1410) tx_hosts and rx_hosts switched in files.log In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1410: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > tx_hosts and rx_hosts switched in files.log > ------------------------------------------- > > Key: BIT-1410 > URL: https://bro-tracker.atlassian.net/browse/BIT-1410 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Environment: Linux Ubuntu > Reporter: Ali Hadi > Assignee: Robin Sommer > Priority: High > Labels: analyzer > Fix For: 2.4 > > > Hi, > _Based on Robin's request I opened this ticket. > _ > If you use the PCAP below and analyze it using Bro: > https://www.bro.org/static/traces/email.pcap > Then when checking the files.log, the tx_hosts is supposed to show the host who transmitted the file, and rx_hosts is for the host who received the file based on Bro's documentation: https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html > If you do the following: > cat files.log | bro-cut fuid tx_hosts rx_hosts | grep > You'll get that the TX Host IP (SrcIP) is 192.168.121.176 and not 192.168.121.179 !!! > It seems that Bro switched their positions in the output. I found this in an assignment given to my students, and one of them gave me a result completely different. So when I double checked with Wireshark, I found that the IPs have been switched by Bro. > Hope this helps. > Ali -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Tue Jun 2 11:26:00 2015 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Tue, 2 Jun 2015 13:26:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1414) Make PIE option availalbe during compiling In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20912#comment-20912 ] Vlad Grigorescu commented on BIT-1414: -------------------------------------- It worked just fine for me. What issues were you having, specifically? > Make PIE option availalbe during compiling > ------------------------------------------ > > Key: BIT-1414 > URL: https://bro-tracker.atlassian.net/browse/BIT-1414 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.3 > Environment: We would like to request PIE support be built in and available in the Bro binary. > Reporter: Jeff > -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Wed Jun 3 00:00:24 2015 From: noreply at bro.org (Merge Tracker) Date: Wed, 3 Jun 2015 00:00:24 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506030700.t5370O77025233@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------------------------------------ bc8eb0c [3] bro-aux Daniel Thayer 2015-06-02 Fix replace_version_in_rst function in update-changes script fbf1fc7 [4] bro-aux Daniel Thayer 2015-06-02 Portability fix for plugin configure script b7c1e2c [5] bro-aux Daniel Thayer 2015-06-02 Fix minor typo in init-plugin error message 4cdba1c [6] bro-aux Daniel Thayer 2015-06-02 Fix replace_version_in_rst function in update-changes script Open GitHub Pull Requests ========================= Issue Component User Updated Title -------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [7] bro yunzheng [8] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [9] #30 [10] bro jsbarber [11] 2015-06-02 Use a common Packet format and preserve layer 2 information [12] #1 [13] bro-plugins jsbarber [14] 2015-05-23 Use a common Packet format and preserve layer 2 information [15] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] bc8eb0c https://github.com/bro/bro-aux/commit/bc8eb0c4ab120a818528c24d4c492301d4f72c8d [4] fbf1fc7 https://github.com/bro/bro-aux/commit/fbf1fc7e67ff90fa5a7dd10523078f7b3ad018a1 [5] b7c1e2c https://github.com/bro/bro-aux/commit/b7c1e2ca686f8c694c45148e10f5fb8f5df7e5af [6] 4cdba1c https://github.com/bro/bro-aux/commit/4cdba1c09d724fa7647be84e47873e4e7fcb16f3 [7] Pull Request #31 https://github.com/bro/bro/pull/31 [8] yunzheng https://github.com/yunzheng [9] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [10] Pull Request #30 https://github.com/bro/bro/pull/30 [11] jsbarber https://github.com/jsbarber [12] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [13] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [14] jsbarber https://github.com/jsbarber [15] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Wed Jun 3 05:10:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 3 Jun 2015 07:10:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1413) README files misidentified by GitHub In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20913#comment-20913 ] Jon Siwek commented on BIT-1413: -------------------------------- Probably doesn't matter much which way to it's done, but I remember going with the symlink README.rst -> README in pysubnettree after a user complained about the github formatting there. For the question about deleting the bromagic repo, unfortunately I think it's better to have it stick around -- there are some commits in Bro that refer to it as a submodule, so if bromagic just goes away they will point to nothing and that part of Bro's git history becomes broken (and I don't recall if a release version of Bro ever used bromagic, but that could make deleting it a bigger deal if so). > README files misidentified by GitHub > ------------------------------------ > > Key: BIT-1413 > URL: https://bro-tracker.atlassian.net/browse/BIT-1413 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Documentation > Reporter: Vlad Grigorescu > > If a README file doesn't have an extension, GitHub will parse it as Markdown. Because our README files are ReST, this results in some ugly (and not very useful) READMEs when visiting the repository on GitHub. > For example, see: https://github.com/bro/btest#readme > There are two options we could take to fix this: rename README to README.rst, or create a symlink. I tried out the symlink option here, and I think the result is much more useful: https://github.com/grigorescu/btest#readme > The affected repos are: > binpac > bro > bro-aux > bro-plugins > bro-scripts > broccoli > broccoli-perl > broccoli-python > broccoli-ruby > broctl (broctl's README just instructs users to see doc/broctl.rst. This could just be a symlink) > broker > bromagic (this can probably be deleted?) > btest > capstats > time-machine > trace-summary -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Wed Jun 3 06:43:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Wed, 3 Jun 2015 08:43:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1408) Broker I/O loop issue In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20914#comment-20914 ] Jon Siwek commented on BIT-1408: -------------------------------- topic/jsiwek/bit-1408 has an idea for a fix. * The changes to bro_broker::Manager I think make more sense in the context of how the IOSource API is claimed to work -- it's now always considered idle and relies only on select() to trigger processing. The timestamp associated with the work to be processed is always the last time select() indicated readiness for the broker IOSource. * Historically, the stdin file descriptor got added to all IOSource's fd_set before doing select() if the particular fd_set was empty. I never really figured out why this was there, but for some reason it reduced cpu usage when reading live traffic and removing it caused poorer packet capture performance (IIRC, for at least one user that complained). However, adding in stdin to bro_broker::Manager's fd_sets basically defeats the goal of the changes mentioned in the last bullet point, so I've just moved the "hack" into PktSrc, the IOSource where it seems to matter. Ultimately, I think this is just a hack and needs to be completely removed, but I don't understand what the purpose of it was in the main I/O loop to begin with or what change needs to be done in its place to improve the cpu usage and capture performance. * I haven't extensively tested the changes in this branch -- not that concerned about the Broker IOSource changes, but I know how easy it is to break stuff in the main I/O loop... > Broker I/O loop issue > --------------------- > > Key: BIT-1408 > URL: https://bro-tracker.atlassian.net/browse/BIT-1408 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Broker > Reporter: Robin Sommer > Fix For: 2.4 > > Attachments: patch.txt > > > This script from Johanna doesn't terminate: > {code} > redef exit_only_after_terminate = T; > event terminate_me() { > print "terminating"; > terminate(); > } > event bro_init() { > BrokerComm::enable(); > schedule 1sec { terminate_me() }; > } > {code} > It works once the {{enable()}} call is removed. > Attached patch seems solve the problem, but it might not be quite the right fix, not sure yet. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Wed Jun 3 09:55:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Wed, 3 Jun 2015 11:55:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1408) Broker I/O loop issue In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20915#comment-20915 ] Robin Sommer commented on BIT-1408: ----------------------------------- instead of moving the fd_set pieces into PktSrc, what do you think of this: {code} +++ b/src/iosource/Manager.cc @@ -118,9 +118,17 @@ IOSource* Manager::FindSoonest(double* ts) src->Clear(); src->src->GetFds(&src->fd_read, &src->fd_write, &src->fd_except); - if ( src->fd_read.Empty() ) src->fd_read.Insert(0); - if ( src->fd_write.Empty() ) src->fd_write.Insert(0); - if ( src->fd_except.Empty() ) src->fd_except.Insert(0); + + if ( Size() ) + { + // TODO: This seems like a hack that should be removed, but doing so + // causes the main run loop to spin more frequently and increase cpu usage. + // See also commit 9cd85be308. + if ( src->fd_read.Empty() ) src->fd_read.Insert(0); + if ( src->fd_write.Empty() ) src->fd_write.Insert(0); + if ( src->fd_except.Empty() ) src->fd_except.Insert(0); + }; + {code} That passes the test-suite as well as Johanna's script. > Broker I/O loop issue > --------------------- > > Key: BIT-1408 > URL: https://bro-tracker.atlassian.net/browse/BIT-1408 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Broker > Reporter: Robin Sommer > Fix For: 2.4 > > Attachments: patch.txt > > > This script from Johanna doesn't terminate: > {code} > redef exit_only_after_terminate = T; > event terminate_me() { > print "terminating"; > terminate(); > } > event bro_init() { > BrokerComm::enable(); > schedule 1sec { terminate_me() }; > } > {code} > It works once the {{enable()}} call is removed. > Attached patch seems solve the problem, but it might not be quite the right fix, not sure yet. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From asharma at lbl.gov Wed Jun 3 15:01:02 2015 From: asharma at lbl.gov (Aashish Sharma) Date: Wed, 3 Jun 2015 15:01:02 -0700 Subject: [Bro-Dev] some Broker questions Message-ID: <20150603220100.GI24432@yaksha.lbl.gov> I am trying using BrokerStore with a master and a clone setup. Where by I was thinking of using master on manager and all the workers are clones. However, I am somewhat confused at a few things - attaching the sample policies used: 1) I see that stores-listener.bro has clone created into it and store-connector.bro has master in it. Does that mean the idea is to have workers run listener and manager run connector ? Which fundamentally means manager connects to the workers ? Or this is open to 'case-by-case' basis ? 2) What exactly does "bro/event/ready" mean ? Is idea here to compartmentalize various events for various policies ? something like bro/event/tor-ban/balh ? 2b) Is it right to understand that with auto_event the event will be automatically called on workers if called on manager ? 2c) How do I trigger a clone to update the master (how often or can I trigger updates on certain conditions ? ) 3) Since all the action happens in "event BrokerComm::outgoing_connection_established" I don't see way to pass data to it. Do I need to create global variables and then use them in this event ? I mean whats a good way to "pass"/use data to this event ? 3b) How is BrokerComm::outgoing_connection_established event triggered ? Does using BrokerStore::insert in some other event also trigger the updates to master from the clone ? 4) Somewhat whimsical issue: Why is peer_address of string type when we have peer_port as port data type. Shouldn't peer_address be address data type ? I was hoping may be one can use dns-names thats why but I cannot seem to get that working ? 4b) Shouldn't this event be better off as : event BrokerComm::outgoing_connection_established(p: peer) Oh also, I see that it supports sets but seems like doesn't support tables ? I am really liking Broker from what my current understanding is so far. Its tremendously powerful. Thanks, Aashish -------------- next part -------------- const broker_port= 9999/tcp &redef ; redef exit_only_after_terminate = T ; global h: opaque of BrokerStore::Handle ; function dv(d: BrokerComm::Data): BrokerComm::DataVector { local rval: BrokerComm::DataVector ; rval[0] = d; return rval ; } global ready: event(); event BrokerComm::outgoing_connection_broken(peer_address: string, peer_port: port) { terminate(); } event BrokerComm::outgoing_connection_established(peer_address: string, peer_port: port, peer_name: string) { local myset: set[string] = ["a", "b", "c", "d" ]; local myvec: vector of string = [ "alpha", "beta", "gamma", "theta"] ; h = BrokerStore::create_master("mystore"); BrokerStore::insert(h, BrokerComm::data("one"), BrokerComm::data(110)); BrokerStore::insert(h, BrokerComm::data("two"), BrokerComm::data(223)); BrokerStore::insert(h, BrokerComm::data("myset"), BrokerComm::data(myset)); BrokerStore::insert(h, BrokerComm::data("myvec"), BrokerComm::data(myvec)); BrokerStore::increment(h, BrokerComm::data("one")); BrokerStore::increment(h, BrokerComm::data("two")); BrokerStore::add_to_set(h, BrokerComm::data("myset"), BrokerComm::data("e")); BrokerStore::remove_from_set(h, BrokerComm::data("myset"), BrokerComm::data("b")); BrokerStore::push_left(h, BrokerComm::data("myvec"), dv(BrokerComm::data("delta"))); BrokerStore::push_right(h, BrokerComm::data("myvec"), dv(BrokerComm::data("omega"))); when (local res = BrokerStore::size(h) ) { print "master size", res; event ready(); } timeout 10 sec { print "timeout" ; } } event bro_init() { BrokerComm::enable(); BrokerComm::connect("127.0.0.1", broker_port, 1 secs); BrokerComm::auto_event("bro/event/ready", ready); } -------------- next part -------------- const broker_port: port 9999/tcp &redef; redef exit_only_after_terminate = T ; global h: opaque of BrokerStore::Handle; global expected_key_count = 4; global key_count = 0 ; function do_lookup(key: string) { when (local res = BrokerStore::lookup(h, BrokerComm::data(key)) ) { ++key_count ; print "lookup", key, res ; if (key_count == expected_key_count) terminate(); } timeout 10 sec { print "timeout",key; } } event ready() { h = BrokerStore::create_clone("mystore"); when (local res = BrokerStore::keys(h) ) { print "clone keys", res ; do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result,0))) ; do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result,1))) ; do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result,2))) ; do_lookup(BrokerComm::refine_to_string(BrokerComm::vector_lookup(res$result,3))) ; } timeout 10 sec { print "timeout"; } } event bro_init() { BrokerComm::enable(); BrokerComm::subscribe_to_events("bro/event/ready"); BrokerComm::listen(broker_port, "127.0.0.1"); } From noreply at bro.org Thu Jun 4 00:00:23 2015 From: noreply at bro.org (Merge Tracker) Date: Thu, 4 Jun 2015 00:00:23 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506040700.t5470NQf013772@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------------------------------------ bc8eb0c [3] bro-aux Daniel Thayer 2015-06-02 Fix replace_version_in_rst function in update-changes script fbf1fc7 [4] bro-aux Daniel Thayer 2015-06-02 Portability fix for plugin configure script b7c1e2c [5] bro-aux Daniel Thayer 2015-06-02 Fix minor typo in init-plugin error message 4cdba1c [6] bro-aux Daniel Thayer 2015-06-02 Fix replace_version_in_rst function in update-changes script Open GitHub Pull Requests ========================= Issue Component User Updated Title -------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [7] bro yunzheng [8] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [9] #30 [10] bro jsbarber [11] 2015-06-02 Use a common Packet format and preserve layer 2 information [12] #1 [13] bro-plugins jsbarber [14] 2015-05-23 Use a common Packet format and preserve layer 2 information [15] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] bc8eb0c https://github.com/bro/bro-aux/commit/bc8eb0c4ab120a818528c24d4c492301d4f72c8d [4] fbf1fc7 https://github.com/bro/bro-aux/commit/fbf1fc7e67ff90fa5a7dd10523078f7b3ad018a1 [5] b7c1e2c https://github.com/bro/bro-aux/commit/b7c1e2ca686f8c694c45148e10f5fb8f5df7e5af [6] 4cdba1c https://github.com/bro/bro-aux/commit/4cdba1c09d724fa7647be84e47873e4e7fcb16f3 [7] Pull Request #31 https://github.com/bro/bro/pull/31 [8] yunzheng https://github.com/yunzheng [9] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [10] Pull Request #30 https://github.com/bro/bro/pull/30 [11] jsbarber https://github.com/jsbarber [12] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [13] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [14] jsbarber https://github.com/jsbarber [15] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Thu Jun 4 04:55:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Thu, 4 Jun 2015 06:55:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1408) Broker I/O loop issue In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20916#comment-20916 ] Jon Siwek commented on BIT-1408: -------------------------------- Would that mean that if you have two IOSources, broker and a pktsrc, that the broker IOSource would always be eligible for processing even if it didn't actually have anything to do (since stdin is put into the fd_set for writing, I think that most always triggers the select()) ? That behavior doesn't seem quite right either. > Broker I/O loop issue > --------------------- > > Key: BIT-1408 > URL: https://bro-tracker.atlassian.net/browse/BIT-1408 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Broker > Reporter: Robin Sommer > Fix For: 2.4 > > Attachments: patch.txt > > > This script from Johanna doesn't terminate: > {code} > redef exit_only_after_terminate = T; > event terminate_me() { > print "terminating"; > terminate(); > } > event bro_init() { > BrokerComm::enable(); > schedule 1sec { terminate_me() }; > } > {code} > It works once the {{enable()}} call is removed. > Attached patch seems solve the problem, but it might not be quite the right fix, not sure yet. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jsiwek at illinois.edu Thu Jun 4 06:22:27 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 4 Jun 2015 13:22:27 +0000 Subject: [Bro-Dev] some Broker questions In-Reply-To: <20150603220100.GI24432@yaksha.lbl.gov> References: <20150603220100.GI24432@yaksha.lbl.gov> Message-ID: <6CA4D412-DCB5-4017-BDC4-AAFC01BA4287@illinois.edu> > On Jun 3, 2015, at 5:01 PM, Aashish Sharma wrote: > > 1) I see that stores-listener.bro has clone created into it and store-connector.bro has master in it. > > Does that mean the idea is to have workers run listener and manager run connector ? Which fundamentally means manager connects to the workers ? > Or this is open to 'case-by-case' basis ? You?re free to chose which endpoints do listen and which do connect and there?s no restrictions related to messaging patterns or data stores related to which one is chosen for a given endpoint. > 2) What exactly does "bro/event/ready" mean ? Is idea here to compartmentalize various events for various policies ? > something like bro/event/tor-ban/balh ? It?s an arbitrary choice of a topic string that other endpoints may chose to subscribe to. It doesn?t have to be in a hierarchical/directory-like format, but that model probably works nice for things you?d want to do. For example you could have an endpoint subscribe to the prefix ?bro/event? and it will receive all events that peers publish that use that prefix (and are willing to send to others). Or it could use ?bro/event/http? to get all http related events, or it can match a topic string exactly if it only cares about just that one event, etc. The degree of specificity is left up to the subscriber. > 2b) Is it right to understand that with auto_event the event will be automatically called on workers if called on manager ? For the most part, yes. There?s other conditions that matter like if the manager allowed the event to be published (the default is ?yes?) and if the worker subscribed to the event (they need to explicitly set up the subscriptions they are interested in). > 2c) How do I trigger a clone to update the master (how often or can I trigger updates on certain conditions ? ) A clone attempts to be as close a copy of the master data store as it can. Any modifications done to the master or via the clone will automatically trigger updates to be propagated to all clones and you don?t have control over that. The primary goal of the clone is to keep a local copy of the master data store so that queries can use that local cache and maybe eliminate some latency. If that?s not a concern for your use-case, you could just create a plain ?frontend? to the master data store instead of a clone. With a plain frontend, queries always are made against the master store, there?s no local cache. > 3) Since all the action happens in "event BrokerComm::outgoing_connection_established" I don't see way to pass data to it. > > Do I need to create global variables and then use them in this event ? I mean whats a good way to "pass"/use data to this event ? You might keep the store handle as a global variable and initialize it in bro_init(), but typically I don?t think you?d actually want to do any data store operations in that event. You can do the operations inside any event depending on what you want the data store to actually do, e.g. handle http_request instead and do store operations there. > 3b) How is BrokerComm::outgoing_connection_established event triggered ? It?s raised automatically when a connection to a peer has completed. If you start trying to send messages before you?ve seen the connection has been established, there?s no guarantee the other side will actually see them. > Does using BrokerStore::insert in some other event also trigger the updates to master from the clone ? Yes, you can perform data store operations inside any event and modifications to a master store are automatically propagated to all clones of it. > 4) Somewhat whimsical issue: Why is peer_address of string type when we have peer_port as port data type. Shouldn't peer_address be address data type ? I was hoping may be one can use dns-names thats why but I cannot seem to get that working ? Yeah, the idea was that either names or IPs should work and a string is a good way to represent either. Can you give more detail on what?s not working? > 4b) Shouldn't this event be better off as : event BrokerComm::outgoing_connection_established(p: peer) Possibly, but the available information for incoming/outgoing and established/broken is different enough that providing a single ?peer? record would make it hard to tell what each one actually can provide. > Oh also, I see that it supports sets but seems like doesn't support tables ? Tables are a supported data type, but broker data stores don?t support any in-place modification operations on them. Maybe if you are wanting to do a lot of modifications (that aren?t simply whole value replacements) to a table stored inside a broker data store, it?s worth considering mapping the keys in the table to keys in the data store instead. - Jon From jira at bro-tracker.atlassian.net Thu Jun 4 07:58:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 4 Jun 2015 09:58:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1408) Broker I/O loop issue In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20917#comment-20917 ] Robin Sommer commented on BIT-1408: ----------------------------------- Ah, I had missed the difference between the two solutions in that regard. Alright, I'm going with your solution. Really hope we get to overhauling the I/O loop sometime soon. > Broker I/O loop issue > --------------------- > > Key: BIT-1408 > URL: https://bro-tracker.atlassian.net/browse/BIT-1408 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Broker > Reporter: Robin Sommer > Fix For: 2.4 > > Attachments: patch.txt > > > This script from Johanna doesn't terminate: > {code} > redef exit_only_after_terminate = T; > event terminate_me() { > print "terminating"; > terminate(); > } > event bro_init() { > BrokerComm::enable(); > schedule 1sec { terminate_me() }; > } > {code} > It works once the {{enable()}} call is removed. > Attached patch seems solve the problem, but it might not be quite the right fix, not sure yet. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Thu Jun 4 14:50:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 4 Jun 2015 16:50:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1408) Broker I/O loop issue In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1408: ------------------------------ Resolution: Merged Status: Closed (was: Open) > Broker I/O loop issue > --------------------- > > Key: BIT-1408 > URL: https://bro-tracker.atlassian.net/browse/BIT-1408 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Broker > Reporter: Robin Sommer > Fix For: 2.4 > > Attachments: patch.txt > > > This script from Johanna doesn't terminate: > {code} > redef exit_only_after_terminate = T; > event terminate_me() { > print "terminating"; > terminate(); > } > event bro_init() { > BrokerComm::enable(); > schedule 1sec { terminate_me() }; > } > {code} > It works once the {{enable()}} call is removed. > Attached patch seems solve the problem, but it might not be quite the right fix, not sure yet. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Fri Jun 5 00:00:18 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 5 Jun 2015 00:00:18 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506050700.t5570ILV014662@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From nb.nospam at gmail.com Fri Jun 5 00:16:22 2015 From: nb.nospam at gmail.com (N B) Date: Fri, 5 Jun 2015 00:16:22 -0700 Subject: [Bro-Dev] HTTPS Analyzer Message-ID: Hello, I am quite new to Bro and need some help. I did go through some of the documentation and some source code but still not clear whether its possible to achieve what we are trying to do. In a nutshell, we are trying to write an HTTPS analyzer for on the fly decryption of the SSL stream and then feed it to the built in HTTP Analyzer. We will use a crypto library + server keys to achieve the decryption. Is it possible at all do this in Bro? The high level idea is to derive the HTTPS_Analyzer from the current HTTP_Analyzer, feed the stream from TCP_Analyzer into the HTTPS_Analyzer and utilize the HTTP_Analyzer calls for the remainder of the functionality. Thanks for your help, NB -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150605/68251395/attachment.html From jdopheid at illinois.edu Fri Jun 5 12:41:54 2015 From: jdopheid at illinois.edu (Dopheide, Jeannette M) Date: Fri, 5 Jun 2015 19:41:54 +0000 Subject: [Bro-Dev] HTTPS Analyzer In-Reply-To: References: Message-ID: Hello NB. This email alias is for tracking development tickets. Your odds of receiving help are much better if you join our mailing list: http://mailman.icsi.berkeley.edu/mailman/listinfo/bro Thanks, Jeannette ------ Jeannette Dopheide Bro Outreach Coordinator National Center for Supercomputing Applications University of Illinois at Urbana-Champaign From: N B > Date: Friday, June 5, 2015 at 2:16 AM To: "bro-dev at bro.org" > Subject: [Bro-Dev] HTTPS Analyzer Hello, I am quite new to Bro and need some help. I did go through some of the documentation and some source code but still not clear whether its possible to achieve what we are trying to do. In a nutshell, we are trying to write an HTTPS analyzer for on the fly decryption of the SSL stream and then feed it to the built in HTTP Analyzer. We will use a crypto library + server keys to achieve the decryption. Is it possible at all do this in Bro? The high level idea is to derive the HTTPS_Analyzer from the current HTTP_Analyzer, feed the stream from TCP_Analyzer into the HTTPS_Analyzer and utilize the HTTP_Analyzer calls for the remainder of the functionality. Thanks for your help, NB -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150605/ac22f7ad/attachment.html From jira at bro-tracker.atlassian.net Fri Jun 5 13:47:01 2015 From: jira at bro-tracker.atlassian.net (Bill Parker (JIRA)) Date: Fri, 5 Jun 2015 15:47:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1415) Lack of Sanity Checking in file patricia.c in Bro-2.3.2 In-Reply-To: References: Message-ID: Bill Parker created BIT-1415: -------------------------------- Summary: Lack of Sanity Checking in file patricia.c in Bro-2.3.2 Key: BIT-1415 URL: https://bro-tracker.atlassian.net/browse/BIT-1415 Project: Bro Issue Tracker Issue Type: Patch Components: bro-aux Affects Versions: 2.3 Environment: Unix/Linux/Windows (lack of sanity checking) Reporter: Bill Parker Attachments: patricia.c.patch Hello All, In reviewing source code in Bro-2.3.2, I found several instances of missing sanity checks for calls to calloc() in file 'patricia.c' in directory 'aux/broctl/aux/pysubnettree', where calls to calloc() are not checked for a return value of NULL, indicating failure. The patch file below corrects/addresses these issues: --- patricia.c.orig 2015-06-05 13:25:12.749964570 -0700 +++ patricia.c 2015-06-05 13:36:05.432917217 -0700 @@ -265,7 +265,10 @@ //prefix4_t size incorrect on NT prefix = calloc(1, sizeof (prefix_t)); #endif /* NT */ - + if (prefix == NULL) { /* we tried to allocate memory again, and failed... */ + fprintf(stderr, "Unable to allocate memory for prefix...\n"); + return (prefix); /* can we return NULL here? */ + } dynamic_allocated++; } memcpy (&prefix->add.sin, dest, 4); @@ -396,6 +399,10 @@ New_Patricia (int maxbits) { patricia_tree_t *patricia = calloc(1, sizeof *patricia); + if (patricia == NULL) { /* oops, calloc() failed, now what? */ + fprintf(stderr, "Unable to allocate memory in New_Patricia...\n"); + return (patricia); /* can we return NULL here? */ + } patricia->maxbits = maxbits; patricia->head = NULL; @@ -665,6 +672,10 @@ if (patricia->head == NULL) { node = calloc(1, sizeof *node); + if (node == NULL) { /* oops, memory allocation failed... */ + fprintf(stderr, "Unable to allocate memory for patricia_lookup...\n"); + return NULL; /* can we return NULL here??? */ + } node->bit = prefix->bitlen; node->prefix = Ref_Prefix (prefix); node->parent = NULL; @@ -776,6 +787,11 @@ } new_node = calloc(1, sizeof *new_node); + if (new_node == NULL) { /* oops, unable to allocate memory for new_node */ + fprintf(stderr, "Unable to allocate memory for new_node in patricia_lookup...\n"); + free(node); + return (NULL); /* can we return NULL here? */ + } new_node->bit = prefix->bitlen; new_node->prefix = Ref_Prefix (prefix); new_node->parent = NULL; @@ -828,6 +844,12 @@ } else { glue = calloc(1, sizeof *glue); + if (glue == NULL) { /* oops, unable to allocate memory for glue... */ + fprintf(stderr, "Unable to allocate memory for glue in patricia_lookup...\n"); + free(new_node); + free(node); + return (glue); /* can we return NULL here? */ + } glue->bit = differ_bit; glue->prefix = NULL; glue->parent = node->parent; -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Fri Jun 5 13:53:00 2015 From: jira at bro-tracker.atlassian.net (Bill Parker (JIRA)) Date: Fri, 5 Jun 2015 15:53:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1416) Lack of Sanity Checking in file nfcollector.c in Bro-2.3.2 In-Reply-To: References: Message-ID: Bill Parker created BIT-1416: -------------------------------- Summary: Lack of Sanity Checking in file nfcollector.c in Bro-2.3.2 Key: BIT-1416 URL: https://bro-tracker.atlassian.net/browse/BIT-1416 Project: Bro Issue Tracker Issue Type: Patch Components: bro-aux Affects Versions: 2.3 Environment: Unix/Linux/Windows/All (OS) Reporter: Bill Parker Attachments: nfcollector.c.patch Hello All, In reviewing code in Bro-2.3.2, file 'nfcollector.c', in directory 'aux/bro-aux/nftools', I found a call to malloc() without a check for a return value of NULL, indicating failure. The patch file below should correct/address this issue: --- nfcollector.c.orig 2015-06-05 13:13:50.404241937 -0700 +++ nfcollector.c 2015-06-05 13:16:10.305022607 -0700 @@ -41,6 +41,10 @@ switch (opt) { case 'o': outfile = malloc (strlen(optarg) + 1); + if (outfile == NULL) { + fprintf(stderr, " Unable to allocate memory for output file I/O, exiting...\n"); + pleave(1, "Out of Memory"); + } strcpy (outfile, optarg); break; case 'p': I am attaching the patch file to this bug report Bill Parker (wp02855 at gmail dot com) -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Fri Jun 5 16:03:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Fri, 5 Jun 2015 18:03:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1417) FTP_UnexpectedConn notice has gone away In-Reply-To: References: Message-ID: Vern Paxson created BIT-1417: -------------------------------- Summary: FTP_UnexpectedConn notice has gone away Key: BIT-1417 URL: https://bro-tracker.atlassian.net/browse/BIT-1417 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Vern Paxson This notice went away during The Great Policy Script Rewrite. It would be good to reintroduce, even though it's not so straightforward to do so given it requires cross-connection analysis. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Fri Jun 5 16:06:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Fri, 5 Jun 2015 18:06:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1418) SSH::Login_By_Password_Guesser is not implemented In-Reply-To: References: Message-ID: Vern Paxson created BIT-1418: -------------------------------- Summary: SSH::Login_By_Password_Guesser is not implemented Key: BIT-1418 URL: https://bro-tracker.atlassian.net/browse/BIT-1418 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Reporter: Vern Paxson While that tag for this notice is defined, it's commented as not implemented. Seth indicated this is because it requires drawing upon distributed information, which doesn't have an apt framework yet. But this is precisely the sort of good-value, non-trivial-behavior notice that Bro should support. Presumably this can be done without requiring extensive inter-cluster-node communication. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Fri Jun 5 16:07:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Fri, 5 Jun 2015 18:07:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1419) HTTPProxyFound notice has gone away In-Reply-To: References: Message-ID: Vern Paxson created BIT-1419: -------------------------------- Summary: HTTPProxyFound notice has gone away Key: BIT-1419 URL: https://bro-tracker.atlassian.net/browse/BIT-1419 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Reporter: Vern Paxson This one also went away during The Great Policy Script Rewrite, but unlike FTP_UnexpectedConn really ought to be pretty straightforward to support again. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Sat Jun 6 00:00:23 2015 From: noreply at bro.org (Merge Tracker) Date: Sat, 6 Jun 2015 00:00:23 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506060700.t5670NC8004217@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Sat Jun 6 13:53:00 2015 From: jira at bro-tracker.atlassian.net (Bill Parker (JIRA)) Date: Sat, 6 Jun 2015 15:53:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1420) Replace bzero() with memset() in broccoli/test/broping.c In-Reply-To: References: Message-ID: Bill Parker created BIT-1420: -------------------------------- Summary: Replace bzero() with memset() in broccoli/test/broping.c Key: BIT-1420 URL: https://bro-tracker.atlassian.net/browse/BIT-1420 Project: Bro Issue Tracker Issue Type: Patch Components: Broccoli Affects Versions: 2.3 Environment: Operating System (Linux/Unix/Windows/All) Reporter: Bill Parker Attachments: broping.c.patch Hello, In reviewing code for file 'broping.c' in directory 'broccoli/test', I found an instance of a call to bzero() which is deprecated per POSIX/C99 standards, which should be replaced with memset(). The patch file which changes this is below: --- broping.c.orig 2015-06-06 09:43:16.694378874 -0700 +++ broping.c 2015-06-06 09:44:06.625724891 -0700 @@ -224,7 +224,7 @@ exit(-1); } - bzero(&server, sizeof(server)); + memset(&server, 0, sizeof(server)); server.sin_family = AF_INET; server.sin_port = htons(port); server.sin_addr.s_addr = 0; I am attaching the patch file to this bug report. Bill Parker (wp02855 at gmail dot com) -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Sat Jun 6 13:59:00 2015 From: jira at bro-tracker.atlassian.net (Bill Parker (JIRA)) Date: Sat, 6 Jun 2015 15:59:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1421) Lack of Sanity Check in file 'bro_type.c' in directory aux/broccoli/src In-Reply-To: References: Message-ID: Bill Parker created BIT-1421: -------------------------------- Summary: Lack of Sanity Check in file 'bro_type.c' in directory aux/broccoli/src Key: BIT-1421 URL: https://bro-tracker.atlassian.net/browse/BIT-1421 Project: Bro Issue Tracker Issue Type: Patch Components: bro-aux Affects Versions: 2.3 Environment: Operating System (Linux/Unix/Windows/All) Reporter: Bill Parker Attachments: bro_type.c.patch Hello, In reviewing code in file 'bro_type.c' in directory 'aux/broccoli/src', I found a(n) instance where calloc() is called without a corresponding test for NULL, indicating failure. The patch file below addresses/corrects this issue: --- bro_type.c.orig 2015-06-06 09:36:11.857384277 -0700 +++ bro_type.c 2015-06-06 09:37:58.675960368 -0700 @@ -1479,6 +1479,9 @@ while (len--) { BroString name; uint64 *val = (uint64*) calloc(1, sizeof(uint64)); + if (val == NULL) { /* Unable to allocate memory... */ + D_RETURN_(FALSE); + } if (! __bro_buf_read_string(bc->rx_buf, &name) || ! __bro_buf_read_int64(bc->rx_buf, val)) I am attaching the patch file to this bug report. Bill Parker (wp02855 at gmail dot com) -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Sat Jun 6 14:05:00 2015 From: jira at bro-tracker.atlassian.net (Bill Parker (JIRA)) Date: Sat, 6 Jun 2015 16:05:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1422) Lack of Sanity Check in file 'broccoli_intern.i' In-Reply-To: References: Message-ID: Bill Parker created BIT-1422: -------------------------------- Summary: Lack of Sanity Check in file 'broccoli_intern.i' Key: BIT-1422 URL: https://bro-tracker.atlassian.net/browse/BIT-1422 Project: Bro Issue Tracker Issue Type: Patch Components: broccoli-python Affects Versions: 2.3 Environment: Operating System (Linux/Unix/Windows/All) Reporter: Bill Parker Attachments: broccoli_intern.i.patch Hello All, In file 'broccoli_intern.i', in directory 'aux/broccoli/bindings/broccoli-python', I found a number of instances where calls to malloc() are made without a corresponding check for a return value of NULL, indicating failure. The patch file below corrects/addresses this issue: --- broccoli_intern.i.orig 2015-06-06 09:02:11.949122426 -0700 +++ broccoli_intern.i 2015-06-06 09:23:00.187767139 -0700 @@ -229,6 +229,11 @@ case BRO_TYPE_BOOL: case BRO_TYPE_INT: { int64_t* tmp = (int64_t *)malloc(sizeof(int64_t)); + if (tmp == NULL) { /* memory allocation failed... */ + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro BOOL/INT"); + return 0; /* should we return ENOMEM here instead? */ + } + *tmp = PyInt_AsLong(val); *data = tmp; break; @@ -237,6 +242,10 @@ case BRO_TYPE_COUNT: case BRO_TYPE_COUNTER: { uint64_t* tmp = (uint64_t *)malloc(sizeof(uint64_t)); + if (tmp == NULL) { /* memory allocation failed... */ + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro COUNT/COUNTER"); + return 0; /* should we return ENOMEM here instead? */ + } *tmp = PyInt_AsLong(val); *data = tmp; break; @@ -247,6 +256,10 @@ return 0; BroAddr* addr = (BroAddr*)malloc(sizeof(BroAddr)); + if (addr == NULL) { /* memory allocation failed... */ + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE_IPADDR"); + return 0; /* should we return ENOMEM here instead? */ + } parseAddrTuple(val, addr); *data = addr; break; @@ -256,6 +269,10 @@ case BRO_TYPE_TIME: case BRO_TYPE_INTERVAL: { double* tmp = (double *)malloc(sizeof(double)); + if (tmp == NULL) { /* memory allocation failed... */ + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE DOUBLE/TIME/INTERVAL"); + return 0; /* should we return ENOMEM here instead? */ + } *tmp = PyFloat_AsDouble(val); *data = tmp; break; @@ -269,6 +286,10 @@ return 0; str = (BroString *)malloc(sizeof(BroString)); + if (str == NULL) { /* memory allocation failed... */ + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE_STRING"); + return 0; /* should we return ENOMEM here instead? */ + } str->str_len = strlen(tmp); str->str_val = (uchar*)strdup(tmp); *data = str; @@ -282,6 +303,10 @@ } int* tmp = (int *)malloc(sizeof(int)); + if (tmp == NULL) { /* memory allocation failed... */ + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE_ENUM"); + return 0; /* should we return ENOMEM here instead? */ + } *tmp = PyInt_AsLong(PyTuple_GetItem(val, 0)); *data = tmp; @@ -300,6 +325,10 @@ } BroPort* port = (BroPort *)malloc(sizeof(BroPort)); + if (port == NULL) { /* memory allocation failed... */ + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE_PORT"); + return 0; /* should we return ENOMEM here instead? */ + } port->port_num = PyInt_AsLong(PyTuple_GetItem(val, 0)); port->port_proto = PyInt_AsLong(PyTuple_GetItem(val, 1)); *data = port; @@ -316,6 +345,10 @@ return 0; BroSubnet* subnet = (BroSubnet *)malloc(sizeof(BroSubnet)); + if (subnet == NULL) { /* memory allocation failed... */ + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE_SUBNET"); + return 0; + } parseAddrTuple(addr, &subnet->sn_net); I am attaching the patch file to this bug report... Bill Parker (wp02855 at gmail dot com) -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Sun Jun 7 00:00:22 2015 From: noreply at bro.org (Merge Tracker) Date: Sun, 7 Jun 2015 00:00:22 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506070700.t5770M2o028664@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From noreply at bro.org Mon Jun 8 00:00:23 2015 From: noreply at bro.org (Merge Tracker) Date: Mon, 8 Jun 2015 00:00:23 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506080700.t5870NgF011890@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Mon Jun 8 08:13:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 8 Jun 2015 10:13:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1421) Lack of Sanity Check in file 'bro_type.c' in directory aux/broccoli/src In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1421: ------------------------------ Fix Version/s: 2.5 > Lack of Sanity Check in file 'bro_type.c' in directory aux/broccoli/src > ----------------------------------------------------------------------- > > Key: BIT-1421 > URL: https://bro-tracker.atlassian.net/browse/BIT-1421 > Project: Bro Issue Tracker > Issue Type: Patch > Components: bro-aux > Affects Versions: 2.3 > Environment: Operating System (Linux/Unix/Windows/All) > Reporter: Bill Parker > Labels: Check, Sanity > Fix For: 2.5 > > Attachments: bro_type.c.patch > > > Hello, > In reviewing code in file 'bro_type.c' in directory 'aux/broccoli/src', I found a(n) instance where calloc() is called without a corresponding test for NULL, indicating failure. The patch file below addresses/corrects this issue: > --- bro_type.c.orig 2015-06-06 09:36:11.857384277 -0700 > +++ bro_type.c 2015-06-06 09:37:58.675960368 -0700 > @@ -1479,6 +1479,9 @@ > while (len--) { > BroString name; > uint64 *val = (uint64*) calloc(1, sizeof(uint64)); > + if (val == NULL) { /* Unable to allocate memory... */ > + D_RETURN_(FALSE); > + } > > if (! __bro_buf_read_string(bc->rx_buf, &name) || > ! __bro_buf_read_int64(bc->rx_buf, val)) > I am attaching the patch file to this bug report. > Bill Parker (wp02855 at gmail dot com) -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 8 08:13:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 8 Jun 2015 10:13:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1422) Lack of Sanity Check in file 'broccoli_intern.i' In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1422: ------------------------------ Fix Version/s: 2.5 > Lack of Sanity Check in file 'broccoli_intern.i' > ------------------------------------------------ > > Key: BIT-1422 > URL: https://bro-tracker.atlassian.net/browse/BIT-1422 > Project: Bro Issue Tracker > Issue Type: Patch > Components: broccoli-python > Affects Versions: 2.3 > Environment: Operating System (Linux/Unix/Windows/All) > Reporter: Bill Parker > Labels: Checking, Sanity > Fix For: 2.5 > > Attachments: broccoli_intern.i.patch > > > Hello All, > In file 'broccoli_intern.i', in directory 'aux/broccoli/bindings/broccoli-python', I found a number of instances where calls to malloc() are made without a corresponding check for a return value of NULL, indicating failure. The patch file below corrects/addresses this issue: > --- broccoli_intern.i.orig 2015-06-06 09:02:11.949122426 -0700 > +++ broccoli_intern.i 2015-06-06 09:23:00.187767139 -0700 > @@ -229,6 +229,11 @@ > case BRO_TYPE_BOOL: > case BRO_TYPE_INT: { > int64_t* tmp = (int64_t *)malloc(sizeof(int64_t)); > + if (tmp == NULL) { /* memory allocation failed... */ > + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro BOOL/INT"); > + return 0; /* should we return ENOMEM here instead? */ > + } > + > *tmp = PyInt_AsLong(val); > *data = tmp; > break; > @@ -237,6 +242,10 @@ > case BRO_TYPE_COUNT: > case BRO_TYPE_COUNTER: { > uint64_t* tmp = (uint64_t *)malloc(sizeof(uint64_t)); > + if (tmp == NULL) { /* memory allocation failed... */ > + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro COUNT/COUNTER"); > + return 0; /* should we return ENOMEM here instead? */ > + } > *tmp = PyInt_AsLong(val); > *data = tmp; > break; > @@ -247,6 +256,10 @@ > return 0; > > BroAddr* addr = (BroAddr*)malloc(sizeof(BroAddr)); > + if (addr == NULL) { /* memory allocation failed... */ > + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE_IPADDR"); > + return 0; /* should we return ENOMEM here instead? */ > + } > parseAddrTuple(val, addr); > *data = addr; > break; > @@ -256,6 +269,10 @@ > case BRO_TYPE_TIME: > case BRO_TYPE_INTERVAL: { > double* tmp = (double *)malloc(sizeof(double)); > + if (tmp == NULL) { /* memory allocation failed... */ > + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE DOUBLE/TIME/INTERVAL"); > + return 0; /* should we return ENOMEM here instead? */ > + } > *tmp = PyFloat_AsDouble(val); > *data = tmp; > break; > @@ -269,6 +286,10 @@ > return 0; > > str = (BroString *)malloc(sizeof(BroString)); > + if (str == NULL) { /* memory allocation failed... */ > + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE_STRING"); > + return 0; /* should we return ENOMEM here instead? */ > + } > str->str_len = strlen(tmp); > str->str_val = (uchar*)strdup(tmp); > *data = str; > @@ -282,6 +303,10 @@ > } > > int* tmp = (int *)malloc(sizeof(int)); > + if (tmp == NULL) { /* memory allocation failed... */ > + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE_ENUM"); > + return 0; /* should we return ENOMEM here instead? */ > + } > *tmp = PyInt_AsLong(PyTuple_GetItem(val, 0)); > *data = tmp; > > @@ -300,6 +325,10 @@ > } > > BroPort* port = (BroPort *)malloc(sizeof(BroPort)); > + if (port == NULL) { /* memory allocation failed... */ > + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE_PORT"); > + return 0; /* should we return ENOMEM here instead? */ > + } > port->port_num = PyInt_AsLong(PyTuple_GetItem(val, 0)); > port->port_proto = PyInt_AsLong(PyTuple_GetItem(val, 1)); > *data = port; > @@ -316,6 +345,10 @@ > return 0; > > BroSubnet* subnet = (BroSubnet *)malloc(sizeof(BroSubnet)); > + if (subnet == NULL) { /* memory allocation failed... */ > + PyErr_SetString(PyExc_RuntimeError, "Unable to allocate memory for Bro TYPE_SUBNET"); > + return 0; > + } > > parseAddrTuple(addr, &subnet->sn_net); > > I am attaching the patch file to this bug report... > Bill Parker (wp02855 at gmail dot com) -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 8 08:14:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 8 Jun 2015 10:14:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1420) Replace bzero() with memset() in broccoli/test/broping.c In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1420: ------------------------------ Fix Version/s: 2.5 > Replace bzero() with memset() in broccoli/test/broping.c > -------------------------------------------------------- > > Key: BIT-1420 > URL: https://bro-tracker.atlassian.net/browse/BIT-1420 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Broccoli > Affects Versions: 2.3 > Environment: Operating System (Linux/Unix/Windows/All) > Reporter: Bill Parker > Labels: obsolete/deprecated > Fix For: 2.5 > > Attachments: broping.c.patch > > > Hello, > In reviewing code for file 'broping.c' in directory 'broccoli/test', I found an instance of > a call to bzero() which is deprecated per POSIX/C99 standards, which should be replaced > with memset(). The patch file which changes this is below: > --- broping.c.orig 2015-06-06 09:43:16.694378874 -0700 > +++ broping.c 2015-06-06 09:44:06.625724891 -0700 > @@ -224,7 +224,7 @@ > exit(-1); > } > > - bzero(&server, sizeof(server)); > + memset(&server, 0, sizeof(server)); > server.sin_family = AF_INET; > server.sin_port = htons(port); > server.sin_addr.s_addr = 0; > I am attaching the patch file to this bug report. > Bill Parker (wp02855 at gmail dot com) -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 8 08:14:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 8 Jun 2015 10:14:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1416) Lack of Sanity Checking in file nfcollector.c in Bro-2.3.2 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1416: ------------------------------ Fix Version/s: 2.5 > Lack of Sanity Checking in file nfcollector.c in Bro-2.3.2 > ---------------------------------------------------------- > > Key: BIT-1416 > URL: https://bro-tracker.atlassian.net/browse/BIT-1416 > Project: Bro Issue Tracker > Issue Type: Patch > Components: bro-aux > Affects Versions: 2.3 > Environment: Unix/Linux/Windows/All (OS) > Reporter: Bill Parker > Labels: cleanup > Fix For: 2.5 > > Attachments: nfcollector.c.patch > > > Hello All, > In reviewing code in Bro-2.3.2, file 'nfcollector.c', in directory 'aux/bro-aux/nftools', I found a call to malloc() without a check for a return value of NULL, indicating failure. The patch file below should correct/address this issue: > --- nfcollector.c.orig 2015-06-05 13:13:50.404241937 -0700 > +++ nfcollector.c 2015-06-05 13:16:10.305022607 -0700 > @@ -41,6 +41,10 @@ > switch (opt) { > case 'o': > outfile = malloc (strlen(optarg) + 1); > + if (outfile == NULL) { > + fprintf(stderr, " Unable to allocate memory for output file I/O, exiting...\n"); > + pleave(1, "Out of Memory"); > + } > strcpy (outfile, optarg); > break; > case 'p': > I am attaching the patch file to this bug report > Bill Parker (wp02855 at gmail dot com) -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 8 08:14:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 8 Jun 2015 10:14:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1413) README files misidentified by GitHub In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1413: ------------------------------ Fix Version/s: 2.5 > README files misidentified by GitHub > ------------------------------------ > > Key: BIT-1413 > URL: https://bro-tracker.atlassian.net/browse/BIT-1413 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Documentation > Reporter: Vlad Grigorescu > Fix For: 2.5 > > > If a README file doesn't have an extension, GitHub will parse it as Markdown. Because our README files are ReST, this results in some ugly (and not very useful) READMEs when visiting the repository on GitHub. > For example, see: https://github.com/bro/btest#readme > There are two options we could take to fix this: rename README to README.rst, or create a symlink. I tried out the symlink option here, and I think the result is much more useful: https://github.com/grigorescu/btest#readme > The affected repos are: > binpac > bro > bro-aux > bro-plugins > bro-scripts > broccoli > broccoli-perl > broccoli-python > broccoli-ruby > broctl (broctl's README just instructs users to see doc/broctl.rst. This could just be a symlink) > broker > bromagic (this can probably be deleted?) > btest > capstats > time-machine > trace-summary -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 8 08:14:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 8 Jun 2015 10:14:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1415) Lack of Sanity Checking in file patricia.c in Bro-2.3.2 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1415: ------------------------------ Fix Version/s: 2.5 > Lack of Sanity Checking in file patricia.c in Bro-2.3.2 > ------------------------------------------------------- > > Key: BIT-1415 > URL: https://bro-tracker.atlassian.net/browse/BIT-1415 > Project: Bro Issue Tracker > Issue Type: Patch > Components: bro-aux > Affects Versions: 2.3 > Environment: Unix/Linux/Windows (lack of sanity checking) > Reporter: Bill Parker > Labels: broctl > Fix For: 2.5 > > Attachments: patricia.c.patch > > > Hello All, > In reviewing source code in Bro-2.3.2, I found several instances of missing sanity checks > for calls to calloc() in file 'patricia.c' in directory 'aux/broctl/aux/pysubnettree', where calls > to calloc() are not checked for a return value of NULL, indicating failure. The patch file below corrects/addresses these issues: > --- patricia.c.orig 2015-06-05 13:25:12.749964570 -0700 > +++ patricia.c 2015-06-05 13:36:05.432917217 -0700 > @@ -265,7 +265,10 @@ > //prefix4_t size incorrect on NT > prefix = calloc(1, sizeof (prefix_t)); > #endif /* NT */ > - > + if (prefix == NULL) { /* we tried to allocate memory again, and failed... */ > + fprintf(stderr, "Unable to allocate memory for prefix...\n"); > + return (prefix); /* can we return NULL here? */ > + } > dynamic_allocated++; > } > memcpy (&prefix->add.sin, dest, 4); > @@ -396,6 +399,10 @@ > New_Patricia (int maxbits) > { > patricia_tree_t *patricia = calloc(1, sizeof *patricia); > + if (patricia == NULL) { /* oops, calloc() failed, now what? */ > + fprintf(stderr, "Unable to allocate memory in New_Patricia...\n"); > + return (patricia); /* can we return NULL here? */ > + } > > patricia->maxbits = maxbits; > patricia->head = NULL; > @@ -665,6 +672,10 @@ > > if (patricia->head == NULL) { > node = calloc(1, sizeof *node); > + if (node == NULL) { /* oops, memory allocation failed... */ > + fprintf(stderr, "Unable to allocate memory for patricia_lookup...\n"); > + return NULL; /* can we return NULL here??? */ > + } > node->bit = prefix->bitlen; > node->prefix = Ref_Prefix (prefix); > node->parent = NULL; > @@ -776,6 +787,11 @@ > } > > new_node = calloc(1, sizeof *new_node); > + if (new_node == NULL) { /* oops, unable to allocate memory for new_node */ > + fprintf(stderr, "Unable to allocate memory for new_node in patricia_lookup...\n"); > + free(node); > + return (NULL); /* can we return NULL here? */ > + } > new_node->bit = prefix->bitlen; > new_node->prefix = Ref_Prefix (prefix); > new_node->parent = NULL; > @@ -828,6 +844,12 @@ > } > else { > glue = calloc(1, sizeof *glue); > + if (glue == NULL) { /* oops, unable to allocate memory for glue... */ > + fprintf(stderr, "Unable to allocate memory for glue in patricia_lookup...\n"); > + free(new_node); > + free(node); > + return (glue); /* can we return NULL here? */ > + } > glue->bit = differ_bit; > glue->prefix = NULL; > glue->parent = node->parent; -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 8 11:05:01 2015 From: jira at bro-tracker.atlassian.net (Bill Parker (JIRA)) Date: Mon, 8 Jun 2015 13:05:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1423) Add power of 2 test to file 'cq.c', test for overflow in 'nb_dns.c' In-Reply-To: References: Message-ID: Bill Parker created BIT-1423: -------------------------------- Summary: Add power of 2 test to file 'cq.c', test for overflow in 'nb_dns.c' Key: BIT-1423 URL: https://bro-tracker.atlassian.net/browse/BIT-1423 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: 2.3 Environment: Source Code Requested Fixes Reporter: Bill Parker Attachments: nb_dns.c.patch Hello All, Here is a hunk of code which is a FIXME to the following statement: /* XXX could check that nbuckets is a power of 2 */ In directory 'src', file 'cq.c' The patch file which adds this test is below: --- cq.c.orig 2015-06-06 19:01:58.220926680 -0700 +++ cq.c 2015-06-06 19:13:03.233446352 -0700 @@ -444,6 +444,9 @@ /* XXX could check that nbuckets is a power of 2 */ + if ((nbuckets % 2) != 0) { /* modulus of nbuckets and 2 isn't zero, not a power of 2 */ + return (-1); /* should we send error message to stderr? */ + } size = sizeof(*buckets) * nbuckets; buckets = (struct cq_bucket *)malloc(size); memory_allocation += size; If the modulus returned is zero, then nbuckets is some power of 2... Upon further review, this is actually incorrect, and should be implemented as a lookup table for actual powers of 2, since any even value will return a modulus of zero. Here is a link which will implement the request properly (my bad): http://www.exploringbinary.com/ten-ways-to-check-if-an-integer-is-a-power-of-two-in-c/ ==================================================================== In directory 'src', file 'nb_dns.c', there is a XXX comment/request to check for overflow in function 'nb_dns_activity', the patch file below implements the test for overflow (which should be correct from review of T_TXT code above this): --- nb_dns.c.orig 2015-06-06 19:29:49.447330962 -0700 +++ nb_dns.c 2015-06-06 19:32:14.693791040 -0700 @@ -614,6 +614,12 @@ } he->h_name = bp; /* XXX check for overflow */ + if (bp + n >= ep) { + snprintf(errstr, NB_DNS_ERRSIZE, + "nb dns activity(): overflow 1 for ptr"); + nr->host_errno = NO_RECOVERY; + return (-1); + } bp += n; /* returned len includes EOS */ /* "Find first satisfactory answer" */ I am attaching the patch file(s) to this bug report Bill Parker (wp02855 at gmail dot com) -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Tue Jun 9 00:00:24 2015 From: noreply at bro.org (Merge Tracker) Date: Tue, 9 Jun 2015 00:00:24 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506090700.t5970OeQ001498@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Tue Jun 9 07:05:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 9 Jun 2015 09:05:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1423) Add power of 2 test to file 'cq.c', test for overflow in 'nb_dns.c' In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1423: ------------------------------ Fix Version/s: 2.5 > Add power of 2 test to file 'cq.c', test for overflow in 'nb_dns.c' > -------------------------------------------------------------------- > > Key: BIT-1423 > URL: https://bro-tracker.atlassian.net/browse/BIT-1423 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.3 > Environment: Source Code Requested Fixes > Reporter: Bill Parker > Labels: Enhancement > Fix For: 2.5 > > Attachments: nb_dns.c.patch > > > Hello All, > Here is a hunk of code which is a FIXME to the following statement: > /* XXX could check that nbuckets is a power of 2 */ > In directory 'src', file 'cq.c' > The patch file which adds this test is below: > --- cq.c.orig 2015-06-06 19:01:58.220926680 -0700 > +++ cq.c 2015-06-06 19:13:03.233446352 -0700 > @@ -444,6 +444,9 @@ > > /* XXX could check that nbuckets is a power of 2 */ > > + if ((nbuckets % 2) != 0) { /* modulus of nbuckets and 2 isn't zero, not a power of 2 */ > + return (-1); /* should we send error message to stderr? */ > + } > size = sizeof(*buckets) * nbuckets; > buckets = (struct cq_bucket *)malloc(size); > memory_allocation += size; > > If the modulus returned is zero, then nbuckets is some power of 2... > Upon further review, this is actually incorrect, and should be implemented as a lookup table for actual powers of 2, since any even value will return a modulus of zero. Here is a link which will implement the request properly (my bad): > http://www.exploringbinary.com/ten-ways-to-check-if-an-integer-is-a-power-of-two-in-c/ > ==================================================================== > In directory 'src', file 'nb_dns.c', there is a XXX comment/request > to check for overflow in function 'nb_dns_activity', the patch file > below implements the test for overflow (which should be correct > from review of T_TXT code above this): > --- nb_dns.c.orig 2015-06-06 19:29:49.447330962 -0700 > +++ nb_dns.c 2015-06-06 19:32:14.693791040 -0700 > @@ -614,6 +614,12 @@ > } > he->h_name = bp; > /* XXX check for overflow */ > + if (bp + n >= ep) { > + snprintf(errstr, NB_DNS_ERRSIZE, > + "nb dns activity(): overflow 1 for ptr"); > + nr->host_errno = NO_RECOVERY; > + return (-1); > + } > bp += n; /* returned len includes EOS */ > > /* "Find first satisfactory answer" */ > > I am attaching the patch file(s) to this bug report > Bill Parker (wp02855 at gmail dot com) -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Tue Jun 9 10:12:00 2015 From: jira at bro-tracker.atlassian.net (Bill Parker (JIRA)) Date: Tue, 9 Jun 2015 12:12:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1424) Add power of 2 test to file(s) 'cq.c/cq.h' (revises BIT-1423) In-Reply-To: References: Message-ID: Bill Parker created BIT-1424: -------------------------------- Summary: Add power of 2 test to file(s) 'cq.c/cq.h' (revises BIT-1423) Key: BIT-1424 URL: https://bro-tracker.atlassian.net/browse/BIT-1424 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Affects Versions: 2.3 Environment: Enhancement Request in file cq.c Reporter: Bill Parker Attachments: cq.c.patch, cq.h.patch Subj: Add power of 2 test to file(s) 'cq.c/cq.h' (revised) *This replaces the cq.c code/patch in BIT-1423..*. Hello All, Here is a hunk of code which is a FIXME to the following statement: /* XXX could check that nbuckets is a power of 2 */ In directory 'src', file 'cq.c' The patch file which adds this test is below: --- cq.c.orig 2015-06-06 19:01:58.220926680 -0700 +++ cq.c 2015-06-08 18:36:37.323755402 -0700 @@ -414,6 +414,15 @@ return hp->max_qlen; } +int cq_IsPowerOfTwo(unsigned int x) +{ + /* This function returns one (1) if val is a power of 2, i.e. */ + /* 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024...2^31 */ + /* or zero (0) if it isn't a power of two */ + + return ((x != 0) && ((x & (~x + 1)) == x)); +} + /* Return without doing anything if we fail to allocate a new bucket array */ static int cq_resize(register struct cq_handle *hp, register int grow) @@ -422,6 +431,7 @@ register size_t size; register struct cq_bucket *bp, *bp2, *buckets, *oldbuckets; struct cq_handle savedhandle; + int power_of_2_result; /* for power of two call */ if (hp->noresize) return (0); @@ -444,6 +454,11 @@ /* XXX could check that nbuckets is a power of 2 */ + power_of_2_result = cq_IsPowerOfTwo((unsigned int)nbuckets); + + if (power_of_2_result == 0) /* If this is zero, nbuckets is NOT a power of 2 */ + return (-1); /* do we need to print a warning/error here as well? */ + size = sizeof(*buckets) * nbuckets; buckets = (struct cq_bucket *)malloc(size); memory_allocation += size; ==================================================================== The function above is a one-liner that can be found on the Web. The first half of the expression ensures that x is a positive integer. The second half of the expression, (x & (~x + 1)) == x, is true only when x is a power of two. It compares x with its two?s complement. The two?s complement of x is computed with ~x + 1, which inverts the bits of x and adds 1 (~x + 1 is equivalent to -x, but negation is technically illegal for an unsigned integer). Let n be the position of the leftmost 1 bit if x. If x is a power of two, its lone 1 bit is in position n. This means ~x has a 0 in position n and 1s everywhere else. When 1 is added to ~x, all positions below n become 0 and the 0 at position n becomes 1. In other words, the carry propagates all the way to position n. So what happens is this: negating x inverts all its bits, but adding 1 inverts them back, from position n on down. So, (x & (~x + 1)) == x is true. ==================================================================== Here is the modification to file 'cq.h' which adds the function prototype for cq_IsPowerOfTwo: --- cq.h.orig 2015-06-09 08:35:29.001007785 -0700 +++ cq.h 2015-06-09 08:36:08.194989138 -0700 @@ -5,6 +5,7 @@ void *cq_remove(struct cq_handle *, double, void *); int cq_size(struct cq_handle *); int cq_max_size(struct cq_handle *); +int cq_IsPowerOfTwo(unsigned int); unsigned int cq_memory_allocation(void); #ifdef DEBUG void cq_debug(struct cq_handle *, int); ==================================================================== I am attaching the patch file(s) to this bug report Bill Parker (wp02855 at gmail dot com) -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Wed Jun 10 00:00:21 2015 From: noreply at bro.org (Merge Tracker) Date: Wed, 10 Jun 2015 00:00:21 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506100700.t5A70LIr016196@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From robin at icir.org Wed Jun 10 20:28:18 2015 From: robin at icir.org (Robin Sommer) Date: Wed, 10 Jun 2015 20:28:18 -0700 Subject: [Bro-Dev] State of TCPStats analyzer? Message-ID: <20150611032818.GO68577@icir.org> Does anybody remember what the state of the TCPStats analyzer is? We have this in scripts/base/frameworks/analyzer/main.bro: ## A set of analyzers to disable by default at startup. The default set ## contains legacy analyzers that are no longer supported. global disabled_analyzers: set[Analyzer::Tag] = { ANALYZER_INTERCONN, ANALYZER_STEPPINGSTONE, ANALYZER_BACKDOOR, ANALYZER_TCPSTATS, } &redef; I understand why the first three are in there, but I don't recall anything about TCPSTATS. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From noreply at bro.org Thu Jun 11 00:00:18 2015 From: noreply at bro.org (Merge Tracker) Date: Thu, 11 Jun 2015 00:00:18 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506110700.t5B70Ibg030043@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From noreply at bro.org Fri Jun 12 00:00:19 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 12 Jun 2015 00:00:19 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506120700.t5C70JY3007568@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Fri Jun 12 16:10:01 2015 From: jira at bro-tracker.atlassian.net (Jonathan Ganz (JIRA)) Date: Fri, 12 Jun 2015 18:10:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1425) BroString::Set() Attempts Allocation of Negative-Length Memory In-Reply-To: References: Message-ID: Jonathan Ganz created BIT-1425: ---------------------------------- Summary: BroString::Set() Attempts Allocation of Negative-Length Memory Key: BIT-1425 URL: https://bro-tracker.atlassian.net/browse/BIT-1425 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.3, 2.4 Environment: Linux Mint 17.1 (Ubuntu 14.04) on bare metal and in a VirtualBox VM. Mac OS X 10.10.3 Reporter: Jonathan Ganz Attachments: lbl-internal.20041215-1142.port004.dump.anon, memory_trace.log, negativeMemory.bro When the tcp_packet() event is used, Bro may attempt to allocate memory that is negative in length (i.e. -6 bytes). Bro crashes with the following output: tcmalloc: large alloc 0 bytes == (nil) @ 0x7f6abeaefc73 0x7f6abeb111c3 0x765e81 0x765b24 0x872562 0xaddc2f 0xaded94 0xb7aeca 0x775180 0x84105b 0x83f5c0 0x83f39d 0x7fb1bc 0xb3cde6 0x7fb3d9 0x750e98 0x7f6abdaf4ec5 0x72e553 (nil) out of memory in new. 1103139821.634774 fatal error: out of memory in new. The attached pcap file and bro script cause such a crash when run with the following command: /usr/local/bro/bin/bro -r lbl-internal.20041215-1142.port004.dump.anon /usr/local/bro/share/bro/site/negativeMemory.bro A core file is not being generated for me, despite following the directions for reporting problems (https://www.bro.org/support/reporting-problems.html#getting-more-information-after-acrash). The file named memory_trace.log shows an alternatively formatted traceback of the stack when the error occurs. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Sat Jun 13 00:00:18 2015 From: noreply at bro.org (Merge Tracker) Date: Sat, 13 Jun 2015 00:00:18 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506130700.t5D70IPC002904@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From noreply at bro.org Sun Jun 14 00:00:27 2015 From: noreply at bro.org (Merge Tracker) Date: Sun, 14 Jun 2015 00:00:27 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506140700.t5E70RHx009517@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Sun Jun 14 09:47:01 2015 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Sun, 14 Jun 2015 11:47:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20918#comment-20918 ] Aashish Sharma commented on BIT-1396: ------------------------------------- Issue Remains. I am not sure what specific crashes of bro is causing it but yes logs are not getting archived. While, I have not manually been able to reproduce this, there is quite a few of this events which happened automatically since Jun 1st: Logs got moved to ~/spool/tmp but never got archived: 36G post-terminate-2015-06-02-13-50-24-6473-crash 9.4G post-terminate-2015-06-03-15-05-04-18332-crash 11G post-terminate-2015-06-05-15-05-05-12274-crash 9.4G post-terminate-2015-06-08-15-05-45-71408-crash 11G post-terminate-2015-06-11-15-05-45-5191-crash > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Sun Jun 14 09:47:01 2015 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Sun, 14 Jun 2015 11:47:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aashish Sharma updated BIT-1396: -------------------------------- Status: Reopened (was: Closed) Resolution: (was: Cannot Reproduce) > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Mon Jun 15 00:00:21 2015 From: noreply at bro.org (Merge Tracker) Date: Mon, 15 Jun 2015 00:00:21 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506150700.t5F70LiW005089@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Mon Jun 15 08:43:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Mon, 15 Jun 2015 10:43:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1425) BroString::Set() Attempts Allocation of Negative-Length Memory In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1425: ------------------------------ Fix Version/s: 2.5 > BroString::Set() Attempts Allocation of Negative-Length Memory > -------------------------------------------------------------- > > Key: BIT-1425 > URL: https://bro-tracker.atlassian.net/browse/BIT-1425 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3, 2.4 > Environment: Linux Mint 17.1 (Ubuntu 14.04) on bare metal and in a VirtualBox VM. > Mac OS X 10.10.3 > Reporter: Jonathan Ganz > Labels: analyzer > Fix For: 2.5 > > Attachments: lbl-internal.20041215-1142.port004.dump.anon, memory_trace.log, negativeMemory.bro > > > When the tcp_packet() event is used, Bro may attempt to allocate memory that is negative in length (i.e. -6 bytes). Bro crashes with the following output: > tcmalloc: large alloc 0 bytes == (nil) @ 0x7f6abeaefc73 0x7f6abeb111c3 0x765e81 0x765b24 0x872562 0xaddc2f 0xaded94 0xb7aeca 0x775180 0x84105b 0x83f5c0 0x83f39d 0x7fb1bc 0xb3cde6 0x7fb3d9 0x750e98 0x7f6abdaf4ec5 0x72e553 (nil) > out of memory in new. > 1103139821.634774 fatal error: out of memory in new. > The attached pcap file and bro script cause such a crash when run with the following command: > /usr/local/bro/bin/bro -r lbl-internal.20041215-1142.port004.dump.anon /usr/local/bro/share/bro/site/negativeMemory.bro > A core file is not being generated for me, despite following the directions for reporting problems (https://www.bro.org/support/reporting-problems.html#getting-more-information-after-acrash). The file named memory_trace.log shows an alternatively formatted traceback of the stack when the error occurs. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 15 13:18:00 2015 From: jira at bro-tracker.atlassian.net (Jeff (JIRA)) Date: Mon, 15 Jun 2015 15:18:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1414) Make PIE option availalbe during compiling In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21000#comment-21000 ] Jeff commented on BIT-1414: --------------------------- What was the process you used to compile it? any specific actions or flags I'm missing? > Make PIE option availalbe during compiling > ------------------------------------------ > > Key: BIT-1414 > URL: https://bro-tracker.atlassian.net/browse/BIT-1414 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.3 > Environment: We would like to request PIE support be built in and available in the Bro binary. > Reporter: Jeff > -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 15 15:09:01 2015 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Mon, 15 Jun 2015 17:09:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1414) Make PIE option availalbe during compiling In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21001#comment-21001 ] Vlad Grigorescu commented on BIT-1414: -------------------------------------- There are two compiler/linker flags you can use, fpic and pie. configure can take additional flags via env vars, so the only change you should need to make in your compilation process is: {{ CFLAGS="-fpic -pie" CXXFLAGS="-fpic -pie" ./configure }} > Make PIE option availalbe during compiling > ------------------------------------------ > > Key: BIT-1414 > URL: https://bro-tracker.atlassian.net/browse/BIT-1414 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.3 > Environment: We would like to request PIE support be built in and available in the Bro binary. > Reporter: Jeff > -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Mon Jun 15 15:09:01 2015 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Mon, 15 Jun 2015 17:09:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1414) Make PIE option availalbe during compiling In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21001#comment-21001 ] Vlad Grigorescu edited comment on BIT-1414 at 6/15/15 5:08 PM: --------------------------------------------------------------- There are two compiler/linker flags you can use, fpic and pie. configure can take additional flags via env vars, so the only change you should need to make in your compilation process is: {{CFLAGS="-fpic -pie" CXXFLAGS="-fpic -pie" ./configure}} was (Author: grigorescu): There are two compiler/linker flags you can use, fpic and pie. configure can take additional flags via env vars, so the only change you should need to make in your compilation process is: {{ CFLAGS="-fpic -pie" CXXFLAGS="-fpic -pie" ./configure }} > Make PIE option availalbe during compiling > ------------------------------------------ > > Key: BIT-1414 > URL: https://bro-tracker.atlassian.net/browse/BIT-1414 > Project: Bro Issue Tracker > Issue Type: New Feature > Components: Bro > Affects Versions: 2.3 > Environment: We would like to request PIE support be built in and available in the Bro binary. > Reporter: Jeff > -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Tue Jun 16 00:00:19 2015 From: noreply at bro.org (Merge Tracker) Date: Tue, 16 Jun 2015 00:00:19 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506160700.t5G70JFL012812@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From james.swaro at gmail.com Tue Jun 16 19:22:23 2015 From: james.swaro at gmail.com (James Swaro) Date: Tue, 16 Jun 2015 21:22:23 -0500 Subject: [Bro-Dev] Trouble with getting Bro 2.2 private analyzer to write logs on current master Message-ID: I have a TCP analyzer that I wrote for my master thesis which I'm trying to update to the latest version of Bro. After rebasing to the trunk, I observed only a few collisions. I resolved the collisions and but something seems to have changed with how the logs are written. Are there changes in the logging framework between Bro 2.2 and the current master which could influence how events are generated? Could this be a change in how packets are delivered to TCP child/support/application analyzers? I am only guessing at things as I haven't had much time to debug why the logs aren't being generated. From some quick debug, I can see that the analyzer is still being added to TCP as a child analyzer, so it seems related to either delivery or event generation. I know this is little information to go on. I can provide more information as needed. -- James Swaro Internetworking Research Group Ohio University -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150616/995c6795/attachment.html From vlad at grigorescu.org Tue Jun 16 21:56:36 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Tue, 16 Jun 2015 23:56:36 -0500 Subject: [Bro-Dev] Trouble with getting Bro 2.2 private analyzer to write logs on current master In-Reply-To: References: Message-ID: Just a guess, but it could be related to this: https://github.com/bro/bro/blob/master/CHANGES#L1578 ints changed to uint64s. As an example, you can see how the HTTP analyzer was modified here: https://github.com/bro/bro/commit/96bcc2d69d72c21f5f4eff0c88cd8d43613bee22#diff-978a30a2ac40a10fbf3c8b5500d3a9f3 The other big change was moving to plugins, but if you're seeing it added as a child analyzer, that doesn't sound like it'd be the issue. Was this analyzer written in BinPAC, or in C++? --Vlad On Tue, Jun 16, 2015 at 9:22 PM, James Swaro wrote: > I have a TCP analyzer that I wrote for my master thesis which I'm trying > to update to the latest version of Bro. After rebasing to the trunk, I > observed only a few collisions. I resolved the collisions and but something > seems to have changed with how the logs are written. Are there changes in > the logging framework between Bro 2.2 and the current master which could > influence how events are generated? Could this be a change in how packets > are delivered to TCP child/support/application analyzers? > > I am only guessing at things as I haven't had much time to debug why the > logs aren't being generated. From some quick debug, I can see that the > analyzer is still being added to TCP as a child analyzer, so it seems > related to either delivery or event generation. > > I know this is little information to go on. I can provide more information > as needed. > > -- > James Swaro > Internetworking Research Group > Ohio University > > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150616/b4d913b6/attachment.html From noreply at bro.org Wed Jun 17 00:00:31 2015 From: noreply at bro.org (Merge Tracker) Date: Wed, 17 Jun 2015 00:00:31 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506170700.t5H70VI5028769@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From vlad at grigorescu.org Wed Jun 17 08:10:27 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Wed, 17 Jun 2015 10:10:27 -0500 Subject: [Bro-Dev] Trouble with getting Bro 2.2 private analyzer to write logs on current master In-Reply-To: References: Message-ID: On Wed, Jun 17, 2015 at 9:45 AM, James Swaro wrote: > > Just a guess, but it could be related to this: > https://github.com/bro/bro/blob/master/CHANGES#L1578 > I'm looking, but nothing seems to pop out at me. > > > The other big change was moving to plugins, but if you're seeing it > added as a child analyzer, that doesn't sound like it'd be the issue. > It seems to be ok. Did data delivery change from DeliverPacket to > something else? > > > Was this analyzer written in BinPAC, or in C++? > It was written in C++. > Well, what I meant with that change was that the functions used for data delivery changed. Specifically: Analyzer::{NextPacket, NextUndelivered, ForwardPacket, ForwardUndelivered, DeliverPacket, Undelivered} were modified to change the int seq parameter to a uint64. If your functions aren't updated, and are expecting a plain old int for the sequence number, I've seen the scenario you describe: the analyzer attaches, but doesn't function. --Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150617/92d9bcec/attachment.html From james.swaro at gmail.com Wed Jun 17 08:30:05 2015 From: james.swaro at gmail.com (James Swaro) Date: Wed, 17 Jun 2015 10:30:05 -0500 Subject: [Bro-Dev] Trouble with getting Bro 2.2 private analyzer to write logs on current master In-Reply-To: References: Message-ID: If I understand the patch correctly, it would only cause problems for connections with over 2GB of data payload, but I think it should work fine for a small trace of say 200KB. I'm not seeing any events at all, nor am I seeing the log files that should be created when using the analyzer. I'll correct the functions and test it out though. On Wed, Jun 17, 2015 at 10:10 AM, Vlad Grigorescu wrote: > On Wed, Jun 17, 2015 at 9:45 AM, James Swaro > wrote: > >> > Just a guess, but it could be related to this: >> https://github.com/bro/bro/blob/master/CHANGES#L1578 >> I'm looking, but nothing seems to pop out at me. >> >> > The other big change was moving to plugins, but if you're seeing it >> added as a child analyzer, that doesn't sound like it'd be the issue. >> It seems to be ok. Did data delivery change from DeliverPacket to >> something else? >> >> > Was this analyzer written in BinPAC, or in C++? >> It was written in C++. >> > > Well, what I meant with that change was that the functions used for data > delivery changed. Specifically: > > Analyzer::{NextPacket, NextUndelivered, ForwardPacket, ForwardUndelivered, > DeliverPacket, Undelivered} were modified to change the int seq parameter > to a uint64. If your functions aren't updated, and are expecting a plain > old int for the sequence number, I've seen the scenario you describe: the > analyzer attaches, but doesn't function. > > --Vlad > > -- James Swaro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150617/7a7cee7c/attachment.html From jira at bro-tracker.atlassian.net Wed Jun 17 10:00:00 2015 From: jira at bro-tracker.atlassian.net (Jonathan Ganz (JIRA)) Date: Wed, 17 Jun 2015 12:00:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1425) BroString::Set() Attempts Allocation of Negative-Length Memory In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ganz updated BIT-1425: ------------------------------- Attachment: backtrace.log > BroString::Set() Attempts Allocation of Negative-Length Memory > -------------------------------------------------------------- > > Key: BIT-1425 > URL: https://bro-tracker.atlassian.net/browse/BIT-1425 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3, 2.4 > Environment: Linux Mint 17.1 (Ubuntu 14.04) on bare metal and in a VirtualBox VM. > Mac OS X 10.10.3 > Reporter: Jonathan Ganz > Labels: analyzer > Fix For: 2.5 > > Attachments: backtrace.log, lbl-internal.20041215-1142.port004.dump.anon, memory_trace.log, negativeMemory.bro > > > When the tcp_packet() event is used, Bro may attempt to allocate memory that is negative in length (i.e. -6 bytes). Bro crashes with the following output: > tcmalloc: large alloc 0 bytes == (nil) @ 0x7f6abeaefc73 0x7f6abeb111c3 0x765e81 0x765b24 0x872562 0xaddc2f 0xaded94 0xb7aeca 0x775180 0x84105b 0x83f5c0 0x83f39d 0x7fb1bc 0xb3cde6 0x7fb3d9 0x750e98 0x7f6abdaf4ec5 0x72e553 (nil) > out of memory in new. > 1103139821.634774 fatal error: out of memory in new. > The attached pcap file and bro script cause such a crash when run with the following command: > /usr/local/bro/bin/bro -r lbl-internal.20041215-1142.port004.dump.anon /usr/local/bro/share/bro/site/negativeMemory.bro > A core file is not being generated for me, despite following the directions for reporting problems (https://www.bro.org/support/reporting-problems.html#getting-more-information-after-acrash). The file named memory_trace.log shows an alternatively formatted traceback of the stack when the error occurs. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Wed Jun 17 10:08:00 2015 From: jira at bro-tracker.atlassian.net (Jonathan Ganz (JIRA)) Date: Wed, 17 Jun 2015 12:08:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1425) BroString::Set() Attempts Allocation of Negative-Length Memory In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21002#comment-21002 ] Jonathan Ganz commented on BIT-1425: ------------------------------------ I have added the gdb backtrace (backtrace.log). It indicates that the (packet?) length passed is -6, which causes issues when attempting to allocate that much memory. A simple workaround I found was to take the absolute value of len in function BroString::Set() (bro-2.4/src/BroString.cc:125). This might cause other problems though (it assumes that the the negative value should just be a positive value of the same magnitude) and does not address the root cause. It may be that the packet is malformed, but Bro should be able to fail more gracefully than this. > BroString::Set() Attempts Allocation of Negative-Length Memory > -------------------------------------------------------------- > > Key: BIT-1425 > URL: https://bro-tracker.atlassian.net/browse/BIT-1425 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3, 2.4 > Environment: Linux Mint 17.1 (Ubuntu 14.04) on bare metal and in a VirtualBox VM. > Mac OS X 10.10.3 > Reporter: Jonathan Ganz > Labels: analyzer > Fix For: 2.5 > > Attachments: backtrace.log, lbl-internal.20041215-1142.port004.dump.anon, memory_trace.log, negativeMemory.bro > > > When the tcp_packet() event is used, Bro may attempt to allocate memory that is negative in length (i.e. -6 bytes). Bro crashes with the following output: > tcmalloc: large alloc 0 bytes == (nil) @ 0x7f6abeaefc73 0x7f6abeb111c3 0x765e81 0x765b24 0x872562 0xaddc2f 0xaded94 0xb7aeca 0x775180 0x84105b 0x83f5c0 0x83f39d 0x7fb1bc 0xb3cde6 0x7fb3d9 0x750e98 0x7f6abdaf4ec5 0x72e553 (nil) > out of memory in new. > 1103139821.634774 fatal error: out of memory in new. > The attached pcap file and bro script cause such a crash when run with the following command: > /usr/local/bro/bin/bro -r lbl-internal.20041215-1142.port004.dump.anon /usr/local/bro/share/bro/site/negativeMemory.bro > A core file is not being generated for me, despite following the directions for reporting problems (https://www.bro.org/support/reporting-problems.html#getting-more-information-after-acrash). The file named memory_trace.log shows an alternatively formatted traceback of the stack when the error occurs. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Wed Jun 17 10:09:00 2015 From: jira at bro-tracker.atlassian.net (Jonathan Ganz (JIRA)) Date: Wed, 17 Jun 2015 12:09:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1425) BroString::Set() Attempts Allocation of Negative-Length Memory In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21002#comment-21002 ] Jonathan Ganz edited comment on BIT-1425 at 6/17/15 12:08 PM: -------------------------------------------------------------- I have added the gdb backtrace (backtrace.log). It indicates that the (payload?) length passed is -6, which causes issues when attempting to allocate that much memory. A simple workaround I found was to take the absolute value of len in function BroString::Set() (bro-2.4/src/BroString.cc:125). This might cause other problems though (it assumes that the the negative value should just be a positive value of the same magnitude) and does not address the root cause. It may be that the packet is malformed, but Bro should be able to fail more gracefully than this. was (Author: ganz): I have added the gdb backtrace (backtrace.log). It indicates that the (packet?) length passed is -6, which causes issues when attempting to allocate that much memory. A simple workaround I found was to take the absolute value of len in function BroString::Set() (bro-2.4/src/BroString.cc:125). This might cause other problems though (it assumes that the the negative value should just be a positive value of the same magnitude) and does not address the root cause. It may be that the packet is malformed, but Bro should be able to fail more gracefully than this. > BroString::Set() Attempts Allocation of Negative-Length Memory > -------------------------------------------------------------- > > Key: BIT-1425 > URL: https://bro-tracker.atlassian.net/browse/BIT-1425 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3, 2.4 > Environment: Linux Mint 17.1 (Ubuntu 14.04) on bare metal and in a VirtualBox VM. > Mac OS X 10.10.3 > Reporter: Jonathan Ganz > Labels: analyzer > Fix For: 2.5 > > Attachments: backtrace.log, lbl-internal.20041215-1142.port004.dump.anon, memory_trace.log, negativeMemory.bro > > > When the tcp_packet() event is used, Bro may attempt to allocate memory that is negative in length (i.e. -6 bytes). Bro crashes with the following output: > tcmalloc: large alloc 0 bytes == (nil) @ 0x7f6abeaefc73 0x7f6abeb111c3 0x765e81 0x765b24 0x872562 0xaddc2f 0xaded94 0xb7aeca 0x775180 0x84105b 0x83f5c0 0x83f39d 0x7fb1bc 0xb3cde6 0x7fb3d9 0x750e98 0x7f6abdaf4ec5 0x72e553 (nil) > out of memory in new. > 1103139821.634774 fatal error: out of memory in new. > The attached pcap file and bro script cause such a crash when run with the following command: > /usr/local/bro/bin/bro -r lbl-internal.20041215-1142.port004.dump.anon /usr/local/bro/share/bro/site/negativeMemory.bro > A core file is not being generated for me, despite following the directions for reporting problems (https://www.bro.org/support/reporting-problems.html#getting-more-information-after-acrash). The file named memory_trace.log shows an alternatively formatted traceback of the stack when the error occurs. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From james.swaro at gmail.com Wed Jun 17 11:26:30 2015 From: james.swaro at gmail.com (James Swaro) Date: Wed, 17 Jun 2015 13:26:30 -0500 Subject: [Bro-Dev] Trouble with getting Bro 2.2 private analyzer to write logs on current master In-Reply-To: References: Message-ID: In Analyzer.cc, there is a quick check for 'if (skip)' . How does this variable get set? On Wed, Jun 17, 2015 at 10:30 AM, James Swaro wrote: > If I understand the patch correctly, it would only cause problems for > connections with over 2GB of data payload, but I think it should work fine > for a small trace of say 200KB. I'm not seeing any events at all, nor am I > seeing the log files that should be created when using the analyzer. > > I'll correct the functions and test it out though. > > On Wed, Jun 17, 2015 at 10:10 AM, Vlad Grigorescu > wrote: > >> On Wed, Jun 17, 2015 at 9:45 AM, James Swaro >> wrote: >> >>> > Just a guess, but it could be related to this: >>> https://github.com/bro/bro/blob/master/CHANGES#L1578 >>> I'm looking, but nothing seems to pop out at me. >>> >>> > The other big change was moving to plugins, but if you're seeing it >>> added as a child analyzer, that doesn't sound like it'd be the issue. >>> It seems to be ok. Did data delivery change from DeliverPacket to >>> something else? >>> >>> > Was this analyzer written in BinPAC, or in C++? >>> It was written in C++. >>> >> >> Well, what I meant with that change was that the functions used for data >> delivery changed. Specifically: >> >> Analyzer::{NextPacket, NextUndelivered, ForwardPacket, >> ForwardUndelivered, DeliverPacket, Undelivered} were modified to change the >> int seq parameter to a uint64. If your functions aren't updated, and are >> expecting a plain old int for the sequence number, I've seen the scenario >> you describe: the analyzer attaches, but doesn't function. >> >> --Vlad >> >> > > > -- > James Swaro > > > -- James Swaro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150617/82b3a895/attachment.html From jira at bro-tracker.atlassian.net Wed Jun 17 13:05:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 17 Jun 2015 15:05:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1396: ------------------------------- Component/s: (was: Bro) BroControl > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Wed Jun 17 13:08:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 17 Jun 2015 15:08:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21003#comment-21003 ] Daniel Thayer commented on BIT-1396: ------------------------------------ Have you tried upgrading to the 2.4 release? (all but one of the timestamps in your comment are from before the official release of 2.4) > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From james.swaro at gmail.com Wed Jun 17 14:10:19 2015 From: james.swaro at gmail.com (James Swaro) Date: Wed, 17 Jun 2015 16:10:19 -0500 Subject: [Bro-Dev] Trouble with getting Bro 2.2 private analyzer to write logs on current master In-Reply-To: References: Message-ID: 1285862632.803262/1434571577.132267 [dpd] TCPRS[101422] DeliverPacket(0, T, 9005, 0x7fff0d41bf80, 0) [] 1285862632.803262/1434571577.132274 [dpd] TCP_ApplicationAnalyzer ignoring DeliverPacket(0, T, 9005, 0x7fff0d41bf80, 0) [] Are these two lines related? I'm stuck. I've run bro with GDB attached using a simple trace file and TCPRS_Analyzer::DeliverPacket never seems to be entered. On Wed, Jun 17, 2015 at 1:26 PM, James Swaro wrote: > In Analyzer.cc, there is a quick check for 'if (skip)' . How does this > variable get set? > > On Wed, Jun 17, 2015 at 10:30 AM, James Swaro > wrote: > >> If I understand the patch correctly, it would only cause problems for >> connections with over 2GB of data payload, but I think it should work fine >> for a small trace of say 200KB. I'm not seeing any events at all, nor am I >> seeing the log files that should be created when using the analyzer. >> >> I'll correct the functions and test it out though. >> >> On Wed, Jun 17, 2015 at 10:10 AM, Vlad Grigorescu >> wrote: >> >>> On Wed, Jun 17, 2015 at 9:45 AM, James Swaro >>> wrote: >>> >>>> > Just a guess, but it could be related to this: >>>> https://github.com/bro/bro/blob/master/CHANGES#L1578 >>>> I'm looking, but nothing seems to pop out at me. >>>> >>>> > The other big change was moving to plugins, but if you're seeing it >>>> added as a child analyzer, that doesn't sound like it'd be the issue. >>>> It seems to be ok. Did data delivery change from DeliverPacket to >>>> something else? >>>> >>>> > Was this analyzer written in BinPAC, or in C++? >>>> It was written in C++. >>>> >>> >>> Well, what I meant with that change was that the functions used for data >>> delivery changed. Specifically: >>> >>> Analyzer::{NextPacket, NextUndelivered, ForwardPacket, >>> ForwardUndelivered, DeliverPacket, Undelivered} were modified to change the >>> int seq parameter to a uint64. If your functions aren't updated, and are >>> expecting a plain old int for the sequence number, I've seen the scenario >>> you describe: the analyzer attaches, but doesn't function. >>> >>> --Vlad >>> >>> >> >> >> -- >> James Swaro >> >> >> > > > -- > James Swaro > > > -- James Swaro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150617/7f89d67b/attachment.html From vlad at grigorescu.org Wed Jun 17 17:13:17 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Wed, 17 Jun 2015 19:13:17 -0500 Subject: [Bro-Dev] Trouble with getting Bro 2.2 private analyzer to write logs on current master In-Reply-To: References: Message-ID: On Wed, Jun 17, 2015 at 10:30 AM, James Swaro wrote: > If I understand the patch correctly, it would only cause problems for > connections with over 2GB of data payload, but I think it should work fine > for a small trace of say 200KB. I'm not seeing any events at all, nor am I > seeing the log files that should be created when using the analyzer. > That was the point of that change, yes, but the breaking modification was that the function signatures are now different. Specifically, those are virtual functions that you're inheriting. If your parameters don't match exactly, you're just defining a new virtual function as opposed to redefining the existing function. > I'll correct the functions and test it out though. > When you said that you're not seeing DeliverPacket be entered, was that after making the uint64 change? --Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150617/7b4eae37/attachment.html From james.swaro at gmail.com Wed Jun 17 20:26:09 2015 From: james.swaro at gmail.com (James Swaro) Date: Wed, 17 Jun 2015 22:26:09 -0500 Subject: [Bro-Dev] Trouble with getting Bro 2.2 private analyzer to write logs on current master In-Reply-To: References: Message-ID: > > On Wed, Jun 17, 2015 at 10:30 AM, James Swaro > wrote: > >> If I understand the patch correctly, it would only cause problems for >> connections with over 2GB of data payload, but I think it should work fine >> for a small trace of say 200KB. I'm not seeing any events at all, nor am I >> seeing the log files that should be created when using the analyzer. >> > > That was the point of that change, yes, but the breaking modification was > that the function signatures are now different. Specifically, those are > virtual functions that you're inheriting. If your parameters don't match > exactly, you're just defining a new virtual function as opposed to > redefining the existing function. > Good point. > > >> I'll correct the functions and test it out though. >> > > When you said that you're not seeing DeliverPacket be entered, was that > after making the uint64 change? > I thought I had corrected it, but seems like the definition was still slightly off. I'm getting all of my logs now as expected. Thanks. On Wed, Jun 17, 2015 at 7:13 PM, Vlad Grigorescu wrote: > On Wed, Jun 17, 2015 at 10:30 AM, James Swaro > wrote: > >> If I understand the patch correctly, it would only cause problems for >> connections with over 2GB of data payload, but I think it should work fine >> for a small trace of say 200KB. I'm not seeing any events at all, nor am I >> seeing the log files that should be created when using the analyzer. >> > > That was the point of that change, yes, but the breaking modification was > that the function signatures are now different. Specifically, those are > virtual functions that you're inheriting. If your parameters don't match > exactly, you're just defining a new virtual function as opposed to > redefining the existing function. > > >> I'll correct the functions and test it out though. >> > > When you said that you're not seeing DeliverPacket be entered, was that > after making the uint64 change? > > --Vlad > > -- James Swaro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150617/418ed0ee/attachment-0001.html From noreply at bro.org Thu Jun 18 00:00:58 2015 From: noreply at bro.org (Merge Tracker) Date: Thu, 18 Jun 2015 00:00:58 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506180700.t5I70wcv005161@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-06-02 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From balint.martina at gmail.com Thu Jun 18 03:19:38 2015 From: balint.martina at gmail.com (Martina Balintova) Date: Thu, 18 Jun 2015 11:19:38 +0100 Subject: [Bro-Dev] Bro's option In-Reply-To: References: Message-ID: Hi, Just a really small typo in Bro 2.4 release - in src/main.cc program option, you removed -G from option listing, but kept case 'G'. M -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150618/7070e0fb/attachment.html From life.130815 at gmail.com Thu Jun 18 08:28:47 2015 From: life.130815 at gmail.com (Mo Jia) Date: Thu, 18 Jun 2015 23:28:47 +0800 Subject: [Bro-Dev] translate bro scripts to c or c++ or broc like pyc? Message-ID: If I write some scripts, I don't want deploy them in bro scripts, Is there some ways like nuitka in python (translate python to c++ ), In my opinion the bro scripts at end was called like c++ , how about we translate all scripts to c++ and link them at compile time? (Of course we can debug using scripts) or something like pyc in python: #cat test.py print "helloworld" # python -m compileall test.py # python test.pyc helloworld Do we already have tools like it? If not, Is there chance that it will be offered by bro team in the future? From dnthayer at illinois.edu Thu Jun 18 10:45:10 2015 From: dnthayer at illinois.edu (Daniel Thayer) Date: Thu, 18 Jun 2015 12:45:10 -0500 Subject: [Bro-Dev] Bro's option In-Reply-To: References: Message-ID: <558303A6.3060301@illinois.edu> On 06/18/2015 05:19 AM, Martina Balintova wrote: > Hi, > Just a really small typo in Bro 2.4 release - in src/main.cc program > option, you removed -G from option listing, but kept case 'G'. > M > In Bro 2.3.2, the "-G" option didn't work (it didn't accept an argument) and it was never listed in Bro's usage output ("bro -h"). From jira at bro-tracker.atlassian.net Thu Jun 18 13:30:01 2015 From: jira at bro-tracker.atlassian.net (Aaron Brown (JIRA)) Date: Thu, 18 Jun 2015 15:30:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1425) BroString::Set() Attempts Allocation of Negative-Length Memory In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21004#comment-21004 ] Aaron Brown commented on BIT-1425: ---------------------------------- Looks like ExtractTCP_Header makes sure that the captured length is greater than the struct tcphdr size instead of the actual header length (th_off * 4). It then subtracts the actual header length, so you end up with a negative number (caplen is 36, but the len and header length is 40) instead of getting a "truncated header" error. The easiest fix would be to just replace this in TCP.cc at line ~442: sizeof(struct tcphdr) > uint32(caplen) ) with: tcp_hdr_len > uint32(caplen) ) But i'm not positive there isn't some better way this check should be done. > BroString::Set() Attempts Allocation of Negative-Length Memory > -------------------------------------------------------------- > > Key: BIT-1425 > URL: https://bro-tracker.atlassian.net/browse/BIT-1425 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3, 2.4 > Environment: Linux Mint 17.1 (Ubuntu 14.04) on bare metal and in a VirtualBox VM. > Mac OS X 10.10.3 > Reporter: Jonathan Ganz > Labels: analyzer > Fix For: 2.5 > > Attachments: backtrace.log, lbl-internal.20041215-1142.port004.dump.anon, memory_trace.log, negativeMemory.bro > > > When the tcp_packet() event is used, Bro may attempt to allocate memory that is negative in length (i.e. -6 bytes). Bro crashes with the following output: > tcmalloc: large alloc 0 bytes == (nil) @ 0x7f6abeaefc73 0x7f6abeb111c3 0x765e81 0x765b24 0x872562 0xaddc2f 0xaded94 0xb7aeca 0x775180 0x84105b 0x83f5c0 0x83f39d 0x7fb1bc 0xb3cde6 0x7fb3d9 0x750e98 0x7f6abdaf4ec5 0x72e553 (nil) > out of memory in new. > 1103139821.634774 fatal error: out of memory in new. > The attached pcap file and bro script cause such a crash when run with the following command: > /usr/local/bro/bin/bro -r lbl-internal.20041215-1142.port004.dump.anon /usr/local/bro/share/bro/site/negativeMemory.bro > A core file is not being generated for me, despite following the directions for reporting problems (https://www.bro.org/support/reporting-problems.html#getting-more-information-after-acrash). The file named memory_trace.log shows an alternatively formatted traceback of the stack when the error occurs. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Fri Jun 19 00:00:21 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 19 Jun 2015 00:00:21 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506190700.t5J70LmY010121@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------------------ 6c812bd [3] bro Daniel Thayer 2015-06-18 Put cmd-line options in alphabetical order Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [4] bro yunzheng [5] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [6] #30 [7] bro jsbarber [8] 2015-06-02 Use a common Packet format and preserve layer 2 information [9] #1 [10] bro-plugins jsbarber [11] 2015-05-23 Use a common Packet format and preserve layer 2 information [12] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] 6c812bd https://github.com/bro/bro/commit/6c812bd5d66a26279ccd1bbb2d96cacc3268471a [4] Pull Request #31 https://github.com/bro/bro/pull/31 [5] yunzheng https://github.com/yunzheng [6] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [7] Pull Request #30 https://github.com/bro/bro/pull/30 [8] jsbarber https://github.com/jsbarber [9] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [10] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [11] jsbarber https://github.com/jsbarber [12] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Fri Jun 19 07:43:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 19 Jun 2015 09:43:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1425) BroString::Set() Attempts Allocation of Negative-Length Memory In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1425: --------------------------------- Assignee: Robin Sommer > BroString::Set() Attempts Allocation of Negative-Length Memory > -------------------------------------------------------------- > > Key: BIT-1425 > URL: https://bro-tracker.atlassian.net/browse/BIT-1425 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3, 2.4 > Environment: Linux Mint 17.1 (Ubuntu 14.04) on bare metal and in a VirtualBox VM. > Mac OS X 10.10.3 > Reporter: Jonathan Ganz > Assignee: Robin Sommer > Labels: analyzer > Fix For: 2.5 > > Attachments: backtrace.log, lbl-internal.20041215-1142.port004.dump.anon, memory_trace.log, negativeMemory.bro > > > When the tcp_packet() event is used, Bro may attempt to allocate memory that is negative in length (i.e. -6 bytes). Bro crashes with the following output: > tcmalloc: large alloc 0 bytes == (nil) @ 0x7f6abeaefc73 0x7f6abeb111c3 0x765e81 0x765b24 0x872562 0xaddc2f 0xaded94 0xb7aeca 0x775180 0x84105b 0x83f5c0 0x83f39d 0x7fb1bc 0xb3cde6 0x7fb3d9 0x750e98 0x7f6abdaf4ec5 0x72e553 (nil) > out of memory in new. > 1103139821.634774 fatal error: out of memory in new. > The attached pcap file and bro script cause such a crash when run with the following command: > /usr/local/bro/bin/bro -r lbl-internal.20041215-1142.port004.dump.anon /usr/local/bro/share/bro/site/negativeMemory.bro > A core file is not being generated for me, despite following the directions for reporting problems (https://www.bro.org/support/reporting-problems.html#getting-more-information-after-acrash). The file named memory_trace.log shows an alternatively formatted traceback of the stack when the error occurs. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Fri Jun 19 08:23:00 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Fri, 19 Jun 2015 10:23:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1426) Fix an issue with the modbus protocol never being confirmed In-Reply-To: References: Message-ID: Seth Hall created BIT-1426: ------------------------------ Summary: Fix an issue with the modbus protocol never being confirmed Key: BIT-1426 URL: https://bro-tracker.atlassian.net/browse/BIT-1426 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.5 Reporter: Seth Hall The modbus analyzer now calls ConfirmProtocol after it successfully parses a PDU from both sides of a conversation which causes the conn.log to now identify "modbus" as the attached analyzer. This is ready for merging with a test update in the topic/seth/modbus_dpd_fix branch. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Fri Jun 19 08:23:00 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Fri, 19 Jun 2015 10:23:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1426) Fix an issue with the modbus protocol never being confirmed In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1426: --------------------------- Status: Merge Request (was: Open) > Fix an issue with the modbus protocol never being confirmed > ----------------------------------------------------------- > > Key: BIT-1426 > URL: https://bro-tracker.atlassian.net/browse/BIT-1426 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.5 > Reporter: Seth Hall > > The modbus analyzer now calls ConfirmProtocol after it successfully parses a PDU from both sides of a conversation which causes the conn.log to now identify "modbus" as the attached analyzer. > This is ready for merging with a test update in the topic/seth/modbus_dpd_fix branch. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From robin at icir.org Fri Jun 19 09:10:21 2015 From: robin at icir.org (Robin Sommer) Date: Fri, 19 Jun 2015 09:10:21 -0700 Subject: [Bro-Dev] translate bro scripts to c or c++ or broc like pyc? In-Reply-To: References: Message-ID: <20150619161021.GH2299@icir.org> On Thu, Jun 18, 2015 at 23:28 +0800, you wrote: > translate all scripts to c++ and link them at compile time? (Of course > we can debug using scripts) We actually have a proof-of-concept script compiler, see this paper: http://www.icir.org/robin/papers/imc14-hilti.pdf However, there's a lot of work left until this becomes ready for production usage. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From robin at icir.org Fri Jun 19 09:14:50 2015 From: robin at icir.org (Robin Sommer) Date: Fri, 19 Jun 2015 09:14:50 -0700 Subject: [Bro-Dev] Bro's option In-Reply-To: <558303A6.3060301@illinois.edu> References: <558303A6.3060301@illinois.edu> Message-ID: <20150619161450.GI2299@icir.org> On Thu, Jun 18, 2015 at 12:45 -0500, you wrote: > In Bro 2.3.2, the "-G" option didn't work (it didn't accept an argument) > and it was never listed in Bro's usage output ("bro -h"). I'll remove the dead 'G' code. Thanks, Martina. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Fri Jun 19 13:38:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 19 Jun 2015 15:38:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1426) Fix an issue with the modbus protocol never being confirmed In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1426: --------------------------------- Assignee: Robin Sommer > Fix an issue with the modbus protocol never being confirmed > ----------------------------------------------------------- > > Key: BIT-1426 > URL: https://bro-tracker.atlassian.net/browse/BIT-1426 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.5 > Reporter: Seth Hall > Assignee: Robin Sommer > > The modbus analyzer now calls ConfirmProtocol after it successfully parses a PDU from both sides of a conversation which causes the conn.log to now identify "modbus" as the attached analyzer. > This is ready for merging with a test update in the topic/seth/modbus_dpd_fix branch. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Sat Jun 20 00:00:25 2015 From: noreply at bro.org (Merge Tracker) Date: Sat, 20 Jun 2015 00:00:25 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506200700.t5K70P5J002423@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ----------------------------------------------------------- BIT-1426 [1] Bro Seth Hall Robin Sommer 2015-06-19 - Normal Fix an issue with the modbus protocol never being confirmed BIT-1399 [2] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [3] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------------------ 6c812bd [4] bro Daniel Thayer 2015-06-18 Put cmd-line options in alphabetical order Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [5] bro yunzheng [6] 2015-06-19 Fix BIT-1314: Detect "quantum insert" type of attacks [7] #30 [8] bro jsbarber [9] 2015-06-19 Use a common Packet format and preserve layer 2 information [10] #1 [11] bro-plugins jsbarber [12] 2015-05-23 Use a common Packet format and preserve layer 2 information [13] [1] BIT-1426 https://bro-tracker.atlassian.net/browse/BIT-1426 [2] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [3] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [4] 6c812bd https://github.com/bro/bro/commit/6c812bd5d66a26279ccd1bbb2d96cacc3268471a [5] Pull Request #31 https://github.com/bro/bro/pull/31 [6] yunzheng https://github.com/yunzheng [7] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [8] Pull Request #30 https://github.com/bro/bro/pull/30 [9] jsbarber https://github.com/jsbarber [10] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [11] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [12] jsbarber https://github.com/jsbarber [13] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Sat Jun 20 11:30:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sat, 20 Jun 2015 13:30:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1427) rare SSH successful login heuristic FPs In-Reply-To: References: Message-ID: Vern Paxson created BIT-1427: -------------------------------- Summary: rare SSH successful login heuristic FPs Key: BIT-1427 URL: https://bro-tracker.atlassian.net/browse/BIT-1427 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.3 Reporter: Vern Paxson During a bruteforce attack that made 27M attempted logins, 2 were flagged as successful by one instance of Bro monitoring the traffic, but not by another running an identical config on the same traffic stream. I wasn't able to reproduce the FPs from bulk traces of the event. Both instances were associated with two Weirds, "SYN_after_close" and "excessive_data_without_further_acks" that were otherwise quite rare in the traffic. This suggests that there's a flaw in the heuristic whereby it's analyzing traffic streams that have confused state. Perhaps an adequate fix is to track whether a given flow has experienced those Weirds, and if so, don't apply the heuristic to it. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Sat Jun 20 11:49:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sat, 20 Jun 2015 13:49:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1428) Customizable email subject lines In-Reply-To: References: Message-ID: Vern Paxson created BIT-1428: -------------------------------- Summary: Customizable email subject lines Key: BIT-1428 URL: https://bro-tracker.atlassian.net/browse/BIT-1428 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Reporter: Vern Paxson There should be a hook of some sort to allow customizing email Subject lines. In particular, I want emails sent for alarm summaries to include the hostname of the Bro that's sending them (since at ICSI we run two concurrent Bros). Looking at *pp_send* in *base/frameworks/notice/actions/pp-alarms.bro* I don't see any way to do this currently. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Sun Jun 21 00:00:26 2015 From: noreply at bro.org (Merge Tracker) Date: Sun, 21 Jun 2015 00:00:26 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506210700.t5L70Q2q012356@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ----------------------------------------------------------- BIT-1426 [1] Bro Seth Hall Robin Sommer 2015-06-19 - Normal Fix an issue with the modbus protocol never being confirmed BIT-1399 [2] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [3] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------------------ 6c812bd [4] bro Daniel Thayer 2015-06-18 Put cmd-line options in alphabetical order Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [5] bro yunzheng [6] 2015-06-19 Fix BIT-1314: Detect "quantum insert" type of attacks [7] #30 [8] bro jsbarber [9] 2015-06-19 Use a common Packet format and preserve layer 2 information [10] #1 [11] bro-plugins jsbarber [12] 2015-05-23 Use a common Packet format and preserve layer 2 information [13] [1] BIT-1426 https://bro-tracker.atlassian.net/browse/BIT-1426 [2] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [3] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [4] 6c812bd https://github.com/bro/bro/commit/6c812bd5d66a26279ccd1bbb2d96cacc3268471a [5] Pull Request #31 https://github.com/bro/bro/pull/31 [6] yunzheng https://github.com/yunzheng [7] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [8] Pull Request #30 https://github.com/bro/bro/pull/30 [9] jsbarber https://github.com/jsbarber [10] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [11] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [12] jsbarber https://github.com/jsbarber [13] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Sun Jun 21 08:52:01 2015 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Sun, 21 Jun 2015 10:52:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1427) rare SSH successful login heuristic FPs In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21005#comment-21005 ] Vlad Grigorescu commented on BIT-1427: -------------------------------------- The heuristic was removed in Bro 2.4 and replaced by analysis of the cleartext handshake phase of the protocol, and the subsequent packet sizes. Part of the reason for this change was because I had seen similar cases of the heuristic being thrown off by unexpected states (e.g. BIT-947). I imagine what happened was that one instance of Bro dropped a couple of packets which caused it to reach a confused state, while the other Bro instance and the bulk packet recorder didn't drop those packets. We could try editing the traces to cause those weirds, and see if the 2.4 SSH analyzer deals with it properly, but at that point the trace might be too synthetic to be of real value. > rare SSH successful login heuristic FPs > --------------------------------------- > > Key: BIT-1427 > URL: https://bro-tracker.atlassian.net/browse/BIT-1427 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Vern Paxson > > During a bruteforce attack that made 27M attempted logins, 2 were flagged as successful by one instance of Bro monitoring the traffic, but not by another running an identical config on the same traffic stream. I wasn't able to reproduce the FPs from bulk traces of the event. Both instances were associated with two Weirds, "SYN_after_close" and "excessive_data_without_further_acks" that were otherwise quite rare in the traffic. This suggests that there's a flaw in the heuristic whereby it's analyzing traffic streams that have confused state. Perhaps an adequate fix is to track whether a given flow has experienced those Weirds, and if so, don't apply the heuristic to it. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Sun Jun 21 09:19:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sun, 21 Jun 2015 11:19:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1427) rare SSH successful login heuristic FPs In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21006#comment-21006 ] Vern Paxson commented on BIT-1427: ---------------------------------- Thanks, Vlad. I'll close this. Once we upgrade to 2.4, surely the Internet will provide another opportunity to test this :-P. > rare SSH successful login heuristic FPs > --------------------------------------- > > Key: BIT-1427 > URL: https://bro-tracker.atlassian.net/browse/BIT-1427 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Vern Paxson > > During a bruteforce attack that made 27M attempted logins, 2 were flagged as successful by one instance of Bro monitoring the traffic, but not by another running an identical config on the same traffic stream. I wasn't able to reproduce the FPs from bulk traces of the event. Both instances were associated with two Weirds, "SYN_after_close" and "excessive_data_without_further_acks" that were otherwise quite rare in the traffic. This suggests that there's a flaw in the heuristic whereby it's analyzing traffic streams that have confused state. Perhaps an adequate fix is to track whether a given flow has experienced those Weirds, and if so, don't apply the heuristic to it. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Sun Jun 21 09:20:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sun, 21 Jun 2015 11:20:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1427) rare SSH successful login heuristic FPs In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1427?focusedWorklogId=10100&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-10100 ] Vern Paxson logged work on BIT-1427: ------------------------------------ Author: Vern Paxson Created on: 21/Jun/15 11:19 AM Start Date: 21/Jun/15 11:19 AM Worklog Time Spent: 5 minutes Work Description: Already presumed fixed in 2.4. Issue Time Tracking ------------------- Worklog Id: (was: 10100) Time Spent: 5 minutes Remaining Estimate: 0 minutes > rare SSH successful login heuristic FPs > --------------------------------------- > > Key: BIT-1427 > URL: https://bro-tracker.atlassian.net/browse/BIT-1427 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Vern Paxson > Fix For: 2.4 > > Time Spent: 5 minutes > Remaining Estimate: 0 minutes > > During a bruteforce attack that made 27M attempted logins, 2 were flagged as successful by one instance of Bro monitoring the traffic, but not by another running an identical config on the same traffic stream. I wasn't able to reproduce the FPs from bulk traces of the event. Both instances were associated with two Weirds, "SYN_after_close" and "excessive_data_without_further_acks" that were otherwise quite rare in the traffic. This suggests that there's a flaw in the heuristic whereby it's analyzing traffic streams that have confused state. Perhaps an adequate fix is to track whether a given flow has experienced those Weirds, and if so, don't apply the heuristic to it. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From jira at bro-tracker.atlassian.net Sun Jun 21 09:20:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sun, 21 Jun 2015 11:20:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1427) rare SSH successful login heuristic FPs In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vern Paxson updated BIT-1427: ----------------------------- Resolution: Fixed Fix Version/s: 2.4 Status: Closed (was: Open) Already presumed fixed in 2.4. > rare SSH successful login heuristic FPs > --------------------------------------- > > Key: BIT-1427 > URL: https://bro-tracker.atlassian.net/browse/BIT-1427 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Reporter: Vern Paxson > Fix For: 2.4 > > > During a bruteforce attack that made 27M attempted logins, 2 were flagged as successful by one instance of Bro monitoring the traffic, but not by another running an identical config on the same traffic stream. I wasn't able to reproduce the FPs from bulk traces of the event. Both instances were associated with two Weirds, "SYN_after_close" and "excessive_data_without_further_acks" that were otherwise quite rare in the traffic. This suggests that there's a flaw in the heuristic whereby it's analyzing traffic streams that have confused state. Perhaps an adequate fix is to track whether a given flow has experienced those Weirds, and if so, don't apply the heuristic to it. -- This message was sent by Atlassian JIRA (v6.5-OD-05-041#65001) From noreply at bro.org Mon Jun 22 00:00:17 2015 From: noreply at bro.org (Merge Tracker) Date: Mon, 22 Jun 2015 00:00:17 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506220700.t5M70HJ2024261@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ----------------------------------------------------------- BIT-1426 [1] Bro Seth Hall Robin Sommer 2015-06-19 - Normal Fix an issue with the modbus protocol never being confirmed BIT-1399 [2] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [3] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [4] bro yunzheng [5] 2015-06-19 Fix BIT-1314: Detect "quantum insert" type of attacks [6] #30 [7] bro jsbarber [8] 2015-06-19 Use a common Packet format and preserve layer 2 information [9] #1 [10] bro-plugins jsbarber [11] 2015-05-23 Use a common Packet format and preserve layer 2 information [12] [1] BIT-1426 https://bro-tracker.atlassian.net/browse/BIT-1426 [2] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [3] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [4] Pull Request #31 https://github.com/bro/bro/pull/31 [5] yunzheng https://github.com/yunzheng [6] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [7] Pull Request #30 https://github.com/bro/bro/pull/30 [8] jsbarber https://github.com/jsbarber [9] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [10] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [11] jsbarber https://github.com/jsbarber [12] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From seth at icir.org Mon Jun 22 13:50:13 2015 From: seth at icir.org (Seth Hall) Date: Mon, 22 Jun 2015 16:50:13 -0400 Subject: [Bro-Dev] Find filtered traces? Message-ID: I?ve been noticing this message... 1232039469.548925 warning in ~/bro/scripts/base/misc/find-filtered-trace.bro, line 48: The analyzed trace file was determined to contain only TCP control packets, which may indicate it's been pre-filtered. By default, Bro reports the missing segments for this type of trace, but the 'detect_filtered_trace' option may be toggled if that's not desired. I have looked at the script yet, but I?ve seen it often enough with traces that I generally think of as ?normal? that I suspect there is something buggy in the script. Anyone have any ideas? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150622/d87d2c4a/attachment.bin From noreply at bro.org Tue Jun 23 00:00:29 2015 From: noreply at bro.org (Merge Tracker) Date: Tue, 23 Jun 2015 00:00:29 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506230700.t5N70TRe032703@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ----------------------------------------------------------- BIT-1426 [1] Bro Seth Hall Robin Sommer 2015-06-19 - Normal Fix an issue with the modbus protocol never being confirmed BIT-1399 [2] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [3] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------------- ---------- ---------------------------------------------------------------- #31 [4] bro yunzheng [5] 2015-06-19 Fix BIT-1314: Detect "quantum insert" type of attacks [6] #30 [7] bro jsbarber [8] 2015-06-19 Use a common Packet format and preserve layer 2 information [9] #1 [10] bro-plugins jsbarber [11] 2015-05-23 Use a common Packet format and preserve layer 2 information [12] #1 [13] btest grigorescu [14] 2015-06-22 Allow testbase overriding in the config [15] [1] BIT-1426 https://bro-tracker.atlassian.net/browse/BIT-1426 [2] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [3] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [4] Pull Request #31 https://github.com/bro/bro/pull/31 [5] yunzheng https://github.com/yunzheng [6] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [7] Pull Request #30 https://github.com/bro/bro/pull/30 [8] jsbarber https://github.com/jsbarber [9] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [10] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [11] jsbarber https://github.com/jsbarber [12] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets [13] Pull Request #1 https://github.com/bro/btest/pull/1 [14] grigorescu https://github.com/grigorescu [15] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/grigorescu/btest.git topic/vladg/config_file_testbase From jsiwek at illinois.edu Tue Jun 23 09:22:16 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Tue, 23 Jun 2015 16:22:16 +0000 Subject: [Bro-Dev] Find filtered traces? In-Reply-To: References: Message-ID: > On Jun 22, 2015, at 3:50 PM, Seth Hall wrote: > > I?ve been noticing this message... > > 1232039469.548925 warning in ~/bro/scripts/base/misc/find-filtered-trace.bro, line 48: The analyzed trace file was determined to contain only TCP control packets, which may indicate it's been pre-filtered. By default, Bro reports the missing segments for this type of trace, but the 'detect_filtered_trace' option may be toggled if that's not desired. > > I have looked at the script yet, but I?ve seen it often enough with traces that I generally think of as ?normal? that I suspect there is something buggy in the script. Anyone have any ideas? Here?s some history of the decision to add that script if that?s what you?re looking for: https://bro-tracker.atlassian.net/browse/BIT-1119 But as far as whether the script actually miss-detects that situation, I also didn?t look closely enough to know ? feel free to send pcaps if you still find the behavior fishy/not-obvious. - Jon From noreply at bro.org Wed Jun 24 00:00:24 2015 From: noreply at bro.org (Merge Tracker) Date: Wed, 24 Jun 2015 00:00:24 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506240700.t5O70OXX007738@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ----------------------------------------------------------- BIT-1426 [1] Bro Seth Hall Robin Sommer 2015-06-19 - Normal Fix an issue with the modbus protocol never being confirmed BIT-1399 [2] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [3] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ---------------------------------------- 368c146 [4] bro Daniel Thayer 2015-06-23 Restore the --load-seeds cmd-line option Open GitHub Pull Requests ========================= Issue Component User Updated Title -------- ----------- --------------- ---------- ---------------------------------------------------------------- #33 [5] bro jswaro [6] 2015-06-23 Initial commit of the TCPRS analyzer [7] #31 [8] bro yunzheng [9] 2015-06-19 Fix BIT-1314: Detect "quantum insert" type of attacks [10] #30 [11] bro jsbarber [12] 2015-06-19 Use a common Packet format and preserve layer 2 information [13] #1 [14] bro-plugins jsbarber [15] 2015-05-23 Use a common Packet format and preserve layer 2 information [16] #1 [17] btest grigorescu [18] 2015-06-22 Allow testbase overriding in the config [19] [1] BIT-1426 https://bro-tracker.atlassian.net/browse/BIT-1426 [2] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [3] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [4] 368c146 https://github.com/bro/bro/commit/368c1463abe4aa03b3302e2ed6f9ab31dc551752 [5] Pull Request #33 https://github.com/bro/bro/pull/33 [6] jswaro https://github.com/jswaro [7] Merge Pull Request #33 with git pull --no-ff --no-commit https://github.com/jswaro/bro.git topic/jswaro/feature/initial-tcprs-release [8] Pull Request #31 https://github.com/bro/bro/pull/31 [9] yunzheng https://github.com/yunzheng [10] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [11] Pull Request #30 https://github.com/bro/bro/pull/30 [12] jsbarber https://github.com/jsbarber [13] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [14] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [15] jsbarber https://github.com/jsbarber [16] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets [17] Pull Request #1 https://github.com/bro/btest/pull/1 [18] grigorescu https://github.com/grigorescu [19] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/grigorescu/btest.git topic/vladg/config_file_testbase From jira at bro-tracker.atlassian.net Wed Jun 24 00:15:01 2015 From: jira at bro-tracker.atlassian.net (Garanews (JIRA)) Date: Wed, 24 Jun 2015 02:15:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1383) memory leak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garanews updated BIT-1383: -------------------------- Attachment: memory usage 2 weeks.png > memory leak > ----------- > > Key: BIT-1383 > URL: https://bro-tracker.atlassian.net/browse/BIT-1383 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Ubuntu 12.04.5 LTS > Reporter: Garanews > Labels: leak, memory > Attachments: log.tar.gz, memory usage 2 weeks.png, screenshot-1.png, threads.png > > > BRO is taking almos all resources availables (CPUs and 192GBRAM +50GB Swap) : > top - 12:26:56 up 1 day, 1:40, 2 users, load average: 28.10, 28.69, 27.94 > Tasks: 365 total, 16 running, 349 sleeping, 0 stopped, 0 zombie > Cpu(s): 28.7%us, 18.4%sy, 1.4%ni, 48.3%id, 2.4%wa, 0.0%hi, 0.7%si, 0.0%st > Mem: 198059808k total, 197549384k used, 510424k free, 1384k buffers > Swap: 201289980k total, 50575440k used, 150714540k free, 10596k cached > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 13551 root 20 0 1341m 880m 517m R 100 0.5 993:31.62 bro > 13596 root 20 0 1173m 882m 517m R 100 0.5 1001:27 bro > 13617 root 20 0 8472m 898m 517m R 100 0.5 990:51.10 bro > 13659 root 20 0 1469m 889m 517m R 100 0.5 990:53.08 bro > 13679 root 20 0 1390m 885m 517m R 100 0.5 981:09.51 bro > 13681 root 20 0 2229m 1.8g 517m R 100 1.0 952:03.55 bro > 13653 root 20 0 2953m 881m 517m R 99 0.5 988:00.31 bro > 13685 root 20 0 1106m 888m 517m R 99 0.5 988:19.98 bro > 13641 root 20 0 1122m 884m 517m R 64 0.5 987:33.61 bro > 13672 root 20 0 10.1g 7.5g 517m R 56 4.0 987:49.47 bro > 13696 root 20 0 1161m 881m 517m R 53 0.5 982:57.39 bro > 13668 root 20 0 1149m 883m 517m S 51 0.5 989:52.19 bro > 13691 root 20 0 1989m 884m 517m R 50 0.5 1012:40 bro > 13692 root 20 0 1190m 885m 517m S 48 0.5 995:16.83 bro > 13677 root 20 0 1188m 889m 517m S 44 0.5 993:21.39 bro > 13687 root 20 0 5340m 2.7g 517m S 43 1.5 978:51.70 bro > 8906 root 25 5 294m 111m 672 S 40 0.1 526:31.84 bro > 6545 root 25 5 1235m 859m 624 S 29 0.4 482:23.79 bro > 4645 root 20 0 198g 160g 1832 S 24 85.0 254:50.36 bro > 8238 root 20 0 1755m 114m 1716 S 19 0.1 129:58.95 bro > 15196 root 25 5 685m 539m 515m S 16 0.3 219:22.30 bro > 15149 root 25 5 678m 535m 515m S 16 0.3 210:17.67 bro > 15166 root 25 5 678m 544m 515m S 16 0.3 208:54.70 bro > 15200 root 25 5 686m 543m 515m S 16 0.3 210:56.13 bro > 15148 root 25 5 678m 546m 515m R 16 0.3 211:22.38 bro > 15186 root 25 5 685m 537m 515m S 16 0.3 208:55.80 bro > 15187 root 25 5 685m 545m 515m R 16 0.3 211:02.48 bro > 15188 root 25 5 685m 536m 515m R 16 0.3 207:31.37 bro > 15197 root 25 5 692m 536m 515m S 16 0.3 208:34.05 bro > 15201 root 25 5 692m 547m 515m S 16 0.3 209:19.18 bro > 15165 root 25 5 685m 543m 515m S 15 0.3 207:28.37 bro > 15147 root 25 5 692m 536m 515m S 15 0.3 210:45.36 bro > 15185 root 25 5 686m 544m 515m S 15 0.3 210:45.12 bro > 15198 root 25 5 678m 546m 515m S 15 0.3 203:51.90 bro > 15150 root 25 5 678m 537m 515m S 15 0.3 207:34.16 bro > 15199 root 25 5 685m 537m 515m S 14 0.3 204:45.21 bro -- This message was sent by Atlassian JIRA (v6.5-OD-07-005#65007) From jira at bro-tracker.atlassian.net Wed Jun 24 00:16:00 2015 From: jira at bro-tracker.atlassian.net (Garanews (JIRA)) Date: Wed, 24 Jun 2015 02:16:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1383) memory leak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garanews updated BIT-1383: -------------------------- Attachment: screenshot-2.png > memory leak > ----------- > > Key: BIT-1383 > URL: https://bro-tracker.atlassian.net/browse/BIT-1383 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Ubuntu 12.04.5 LTS > Reporter: Garanews > Labels: leak, memory > Attachments: log.tar.gz, memory usage 2 weeks.png, screenshot-1.png, screenshot-2.png, threads.png > > > BRO is taking almos all resources availables (CPUs and 192GBRAM +50GB Swap) : > top - 12:26:56 up 1 day, 1:40, 2 users, load average: 28.10, 28.69, 27.94 > Tasks: 365 total, 16 running, 349 sleeping, 0 stopped, 0 zombie > Cpu(s): 28.7%us, 18.4%sy, 1.4%ni, 48.3%id, 2.4%wa, 0.0%hi, 0.7%si, 0.0%st > Mem: 198059808k total, 197549384k used, 510424k free, 1384k buffers > Swap: 201289980k total, 50575440k used, 150714540k free, 10596k cached > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 13551 root 20 0 1341m 880m 517m R 100 0.5 993:31.62 bro > 13596 root 20 0 1173m 882m 517m R 100 0.5 1001:27 bro > 13617 root 20 0 8472m 898m 517m R 100 0.5 990:51.10 bro > 13659 root 20 0 1469m 889m 517m R 100 0.5 990:53.08 bro > 13679 root 20 0 1390m 885m 517m R 100 0.5 981:09.51 bro > 13681 root 20 0 2229m 1.8g 517m R 100 1.0 952:03.55 bro > 13653 root 20 0 2953m 881m 517m R 99 0.5 988:00.31 bro > 13685 root 20 0 1106m 888m 517m R 99 0.5 988:19.98 bro > 13641 root 20 0 1122m 884m 517m R 64 0.5 987:33.61 bro > 13672 root 20 0 10.1g 7.5g 517m R 56 4.0 987:49.47 bro > 13696 root 20 0 1161m 881m 517m R 53 0.5 982:57.39 bro > 13668 root 20 0 1149m 883m 517m S 51 0.5 989:52.19 bro > 13691 root 20 0 1989m 884m 517m R 50 0.5 1012:40 bro > 13692 root 20 0 1190m 885m 517m S 48 0.5 995:16.83 bro > 13677 root 20 0 1188m 889m 517m S 44 0.5 993:21.39 bro > 13687 root 20 0 5340m 2.7g 517m S 43 1.5 978:51.70 bro > 8906 root 25 5 294m 111m 672 S 40 0.1 526:31.84 bro > 6545 root 25 5 1235m 859m 624 S 29 0.4 482:23.79 bro > 4645 root 20 0 198g 160g 1832 S 24 85.0 254:50.36 bro > 8238 root 20 0 1755m 114m 1716 S 19 0.1 129:58.95 bro > 15196 root 25 5 685m 539m 515m S 16 0.3 219:22.30 bro > 15149 root 25 5 678m 535m 515m S 16 0.3 210:17.67 bro > 15166 root 25 5 678m 544m 515m S 16 0.3 208:54.70 bro > 15200 root 25 5 686m 543m 515m S 16 0.3 210:56.13 bro > 15148 root 25 5 678m 546m 515m R 16 0.3 211:22.38 bro > 15186 root 25 5 685m 537m 515m S 16 0.3 208:55.80 bro > 15187 root 25 5 685m 545m 515m R 16 0.3 211:02.48 bro > 15188 root 25 5 685m 536m 515m R 16 0.3 207:31.37 bro > 15197 root 25 5 692m 536m 515m S 16 0.3 208:34.05 bro > 15201 root 25 5 692m 547m 515m S 16 0.3 209:19.18 bro > 15165 root 25 5 685m 543m 515m S 15 0.3 207:28.37 bro > 15147 root 25 5 692m 536m 515m S 15 0.3 210:45.36 bro > 15185 root 25 5 686m 544m 515m S 15 0.3 210:45.12 bro > 15198 root 25 5 678m 546m 515m S 15 0.3 203:51.90 bro > 15150 root 25 5 678m 537m 515m S 15 0.3 207:34.16 bro > 15199 root 25 5 685m 537m 515m S 14 0.3 204:45.21 bro -- This message was sent by Atlassian JIRA (v6.5-OD-07-005#65007) From jira at bro-tracker.atlassian.net Wed Jun 24 00:17:00 2015 From: jira at bro-tracker.atlassian.net (Garanews (JIRA)) Date: Wed, 24 Jun 2015 02:17:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1383) memory leak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Garanews updated BIT-1383: -------------------------- Attachment: threads 2 weeks.png > memory leak > ----------- > > Key: BIT-1383 > URL: https://bro-tracker.atlassian.net/browse/BIT-1383 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Ubuntu 12.04.5 LTS > Reporter: Garanews > Labels: leak, memory > Attachments: log.tar.gz, memory usage 2 weeks.png, screenshot-1.png, screenshot-2.png, threads 2 weeks.png, threads.png > > > BRO is taking almos all resources availables (CPUs and 192GBRAM +50GB Swap) : > top - 12:26:56 up 1 day, 1:40, 2 users, load average: 28.10, 28.69, 27.94 > Tasks: 365 total, 16 running, 349 sleeping, 0 stopped, 0 zombie > Cpu(s): 28.7%us, 18.4%sy, 1.4%ni, 48.3%id, 2.4%wa, 0.0%hi, 0.7%si, 0.0%st > Mem: 198059808k total, 197549384k used, 510424k free, 1384k buffers > Swap: 201289980k total, 50575440k used, 150714540k free, 10596k cached > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 13551 root 20 0 1341m 880m 517m R 100 0.5 993:31.62 bro > 13596 root 20 0 1173m 882m 517m R 100 0.5 1001:27 bro > 13617 root 20 0 8472m 898m 517m R 100 0.5 990:51.10 bro > 13659 root 20 0 1469m 889m 517m R 100 0.5 990:53.08 bro > 13679 root 20 0 1390m 885m 517m R 100 0.5 981:09.51 bro > 13681 root 20 0 2229m 1.8g 517m R 100 1.0 952:03.55 bro > 13653 root 20 0 2953m 881m 517m R 99 0.5 988:00.31 bro > 13685 root 20 0 1106m 888m 517m R 99 0.5 988:19.98 bro > 13641 root 20 0 1122m 884m 517m R 64 0.5 987:33.61 bro > 13672 root 20 0 10.1g 7.5g 517m R 56 4.0 987:49.47 bro > 13696 root 20 0 1161m 881m 517m R 53 0.5 982:57.39 bro > 13668 root 20 0 1149m 883m 517m S 51 0.5 989:52.19 bro > 13691 root 20 0 1989m 884m 517m R 50 0.5 1012:40 bro > 13692 root 20 0 1190m 885m 517m S 48 0.5 995:16.83 bro > 13677 root 20 0 1188m 889m 517m S 44 0.5 993:21.39 bro > 13687 root 20 0 5340m 2.7g 517m S 43 1.5 978:51.70 bro > 8906 root 25 5 294m 111m 672 S 40 0.1 526:31.84 bro > 6545 root 25 5 1235m 859m 624 S 29 0.4 482:23.79 bro > 4645 root 20 0 198g 160g 1832 S 24 85.0 254:50.36 bro > 8238 root 20 0 1755m 114m 1716 S 19 0.1 129:58.95 bro > 15196 root 25 5 685m 539m 515m S 16 0.3 219:22.30 bro > 15149 root 25 5 678m 535m 515m S 16 0.3 210:17.67 bro > 15166 root 25 5 678m 544m 515m S 16 0.3 208:54.70 bro > 15200 root 25 5 686m 543m 515m S 16 0.3 210:56.13 bro > 15148 root 25 5 678m 546m 515m R 16 0.3 211:22.38 bro > 15186 root 25 5 685m 537m 515m S 16 0.3 208:55.80 bro > 15187 root 25 5 685m 545m 515m R 16 0.3 211:02.48 bro > 15188 root 25 5 685m 536m 515m R 16 0.3 207:31.37 bro > 15197 root 25 5 692m 536m 515m S 16 0.3 208:34.05 bro > 15201 root 25 5 692m 547m 515m S 16 0.3 209:19.18 bro > 15165 root 25 5 685m 543m 515m S 15 0.3 207:28.37 bro > 15147 root 25 5 692m 536m 515m S 15 0.3 210:45.36 bro > 15185 root 25 5 686m 544m 515m S 15 0.3 210:45.12 bro > 15198 root 25 5 678m 546m 515m S 15 0.3 203:51.90 bro > 15150 root 25 5 678m 537m 515m S 15 0.3 207:34.16 bro > 15199 root 25 5 685m 537m 515m S 14 0.3 204:45.21 bro -- This message was sent by Atlassian JIRA (v6.5-OD-07-005#65007) From jira at bro-tracker.atlassian.net Wed Jun 24 00:19:00 2015 From: jira at bro-tracker.atlassian.net (Garanews (JIRA)) Date: Wed, 24 Jun 2015 02:19:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1383) memory leak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21100#comment-21100 ] Garanews commented on BIT-1383: ------------------------------- Hello, thanks to your tip I found a bottle neck on disk I/O. Now seems better: !screenshot-2.png|thumbnail! Threads number is bigger than previously: !threads 2 weeks.png|thumbnail! After 1 week I rebooted the BRO service, can I ask you if I sould schedule this kind of task and how often? Thanks > memory leak > ----------- > > Key: BIT-1383 > URL: https://bro-tracker.atlassian.net/browse/BIT-1383 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Ubuntu 12.04.5 LTS > Reporter: Garanews > Labels: leak, memory > Attachments: log.tar.gz, memory usage 2 weeks.png, screenshot-1.png, screenshot-2.png, threads 2 weeks.png, threads.png > > > BRO is taking almos all resources availables (CPUs and 192GBRAM +50GB Swap) : > top - 12:26:56 up 1 day, 1:40, 2 users, load average: 28.10, 28.69, 27.94 > Tasks: 365 total, 16 running, 349 sleeping, 0 stopped, 0 zombie > Cpu(s): 28.7%us, 18.4%sy, 1.4%ni, 48.3%id, 2.4%wa, 0.0%hi, 0.7%si, 0.0%st > Mem: 198059808k total, 197549384k used, 510424k free, 1384k buffers > Swap: 201289980k total, 50575440k used, 150714540k free, 10596k cached > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 13551 root 20 0 1341m 880m 517m R 100 0.5 993:31.62 bro > 13596 root 20 0 1173m 882m 517m R 100 0.5 1001:27 bro > 13617 root 20 0 8472m 898m 517m R 100 0.5 990:51.10 bro > 13659 root 20 0 1469m 889m 517m R 100 0.5 990:53.08 bro > 13679 root 20 0 1390m 885m 517m R 100 0.5 981:09.51 bro > 13681 root 20 0 2229m 1.8g 517m R 100 1.0 952:03.55 bro > 13653 root 20 0 2953m 881m 517m R 99 0.5 988:00.31 bro > 13685 root 20 0 1106m 888m 517m R 99 0.5 988:19.98 bro > 13641 root 20 0 1122m 884m 517m R 64 0.5 987:33.61 bro > 13672 root 20 0 10.1g 7.5g 517m R 56 4.0 987:49.47 bro > 13696 root 20 0 1161m 881m 517m R 53 0.5 982:57.39 bro > 13668 root 20 0 1149m 883m 517m S 51 0.5 989:52.19 bro > 13691 root 20 0 1989m 884m 517m R 50 0.5 1012:40 bro > 13692 root 20 0 1190m 885m 517m S 48 0.5 995:16.83 bro > 13677 root 20 0 1188m 889m 517m S 44 0.5 993:21.39 bro > 13687 root 20 0 5340m 2.7g 517m S 43 1.5 978:51.70 bro > 8906 root 25 5 294m 111m 672 S 40 0.1 526:31.84 bro > 6545 root 25 5 1235m 859m 624 S 29 0.4 482:23.79 bro > 4645 root 20 0 198g 160g 1832 S 24 85.0 254:50.36 bro > 8238 root 20 0 1755m 114m 1716 S 19 0.1 129:58.95 bro > 15196 root 25 5 685m 539m 515m S 16 0.3 219:22.30 bro > 15149 root 25 5 678m 535m 515m S 16 0.3 210:17.67 bro > 15166 root 25 5 678m 544m 515m S 16 0.3 208:54.70 bro > 15200 root 25 5 686m 543m 515m S 16 0.3 210:56.13 bro > 15148 root 25 5 678m 546m 515m R 16 0.3 211:22.38 bro > 15186 root 25 5 685m 537m 515m S 16 0.3 208:55.80 bro > 15187 root 25 5 685m 545m 515m R 16 0.3 211:02.48 bro > 15188 root 25 5 685m 536m 515m R 16 0.3 207:31.37 bro > 15197 root 25 5 692m 536m 515m S 16 0.3 208:34.05 bro > 15201 root 25 5 692m 547m 515m S 16 0.3 209:19.18 bro > 15165 root 25 5 685m 543m 515m S 15 0.3 207:28.37 bro > 15147 root 25 5 692m 536m 515m S 15 0.3 210:45.36 bro > 15185 root 25 5 686m 544m 515m S 15 0.3 210:45.12 bro > 15198 root 25 5 678m 546m 515m S 15 0.3 203:51.90 bro > 15150 root 25 5 678m 537m 515m S 15 0.3 207:34.16 bro > 15199 root 25 5 685m 537m 515m S 14 0.3 204:45.21 bro -- This message was sent by Atlassian JIRA (v6.5-OD-07-005#65007) From jira at bro-tracker.atlassian.net Wed Jun 24 07:19:00 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 24 Jun 2015 09:19:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1383) memory leak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21101#comment-21101 ] Seth Hall commented on BIT-1383: -------------------------------- We typically treat the need to reboot on some regular schedule as a bug. You shouldn't need to do it. It sounds like this case was all a local issue related to disk I/O so I'm going to close the ticket. Thanks. > memory leak > ----------- > > Key: BIT-1383 > URL: https://bro-tracker.atlassian.net/browse/BIT-1383 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Ubuntu 12.04.5 LTS > Reporter: Garanews > Labels: leak, memory > Attachments: log.tar.gz, memory usage 2 weeks.png, screenshot-1.png, screenshot-2.png, threads 2 weeks.png, threads.png > > > BRO is taking almos all resources availables (CPUs and 192GBRAM +50GB Swap) : > top - 12:26:56 up 1 day, 1:40, 2 users, load average: 28.10, 28.69, 27.94 > Tasks: 365 total, 16 running, 349 sleeping, 0 stopped, 0 zombie > Cpu(s): 28.7%us, 18.4%sy, 1.4%ni, 48.3%id, 2.4%wa, 0.0%hi, 0.7%si, 0.0%st > Mem: 198059808k total, 197549384k used, 510424k free, 1384k buffers > Swap: 201289980k total, 50575440k used, 150714540k free, 10596k cached > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 13551 root 20 0 1341m 880m 517m R 100 0.5 993:31.62 bro > 13596 root 20 0 1173m 882m 517m R 100 0.5 1001:27 bro > 13617 root 20 0 8472m 898m 517m R 100 0.5 990:51.10 bro > 13659 root 20 0 1469m 889m 517m R 100 0.5 990:53.08 bro > 13679 root 20 0 1390m 885m 517m R 100 0.5 981:09.51 bro > 13681 root 20 0 2229m 1.8g 517m R 100 1.0 952:03.55 bro > 13653 root 20 0 2953m 881m 517m R 99 0.5 988:00.31 bro > 13685 root 20 0 1106m 888m 517m R 99 0.5 988:19.98 bro > 13641 root 20 0 1122m 884m 517m R 64 0.5 987:33.61 bro > 13672 root 20 0 10.1g 7.5g 517m R 56 4.0 987:49.47 bro > 13696 root 20 0 1161m 881m 517m R 53 0.5 982:57.39 bro > 13668 root 20 0 1149m 883m 517m S 51 0.5 989:52.19 bro > 13691 root 20 0 1989m 884m 517m R 50 0.5 1012:40 bro > 13692 root 20 0 1190m 885m 517m S 48 0.5 995:16.83 bro > 13677 root 20 0 1188m 889m 517m S 44 0.5 993:21.39 bro > 13687 root 20 0 5340m 2.7g 517m S 43 1.5 978:51.70 bro > 8906 root 25 5 294m 111m 672 S 40 0.1 526:31.84 bro > 6545 root 25 5 1235m 859m 624 S 29 0.4 482:23.79 bro > 4645 root 20 0 198g 160g 1832 S 24 85.0 254:50.36 bro > 8238 root 20 0 1755m 114m 1716 S 19 0.1 129:58.95 bro > 15196 root 25 5 685m 539m 515m S 16 0.3 219:22.30 bro > 15149 root 25 5 678m 535m 515m S 16 0.3 210:17.67 bro > 15166 root 25 5 678m 544m 515m S 16 0.3 208:54.70 bro > 15200 root 25 5 686m 543m 515m S 16 0.3 210:56.13 bro > 15148 root 25 5 678m 546m 515m R 16 0.3 211:22.38 bro > 15186 root 25 5 685m 537m 515m S 16 0.3 208:55.80 bro > 15187 root 25 5 685m 545m 515m R 16 0.3 211:02.48 bro > 15188 root 25 5 685m 536m 515m R 16 0.3 207:31.37 bro > 15197 root 25 5 692m 536m 515m S 16 0.3 208:34.05 bro > 15201 root 25 5 692m 547m 515m S 16 0.3 209:19.18 bro > 15165 root 25 5 685m 543m 515m S 15 0.3 207:28.37 bro > 15147 root 25 5 692m 536m 515m S 15 0.3 210:45.36 bro > 15185 root 25 5 686m 544m 515m S 15 0.3 210:45.12 bro > 15198 root 25 5 678m 546m 515m S 15 0.3 203:51.90 bro > 15150 root 25 5 678m 537m 515m S 15 0.3 207:34.16 bro > 15199 root 25 5 685m 537m 515m S 14 0.3 204:45.21 bro -- This message was sent by Atlassian JIRA (v6.5-OD-07-005#65007) From jira at bro-tracker.atlassian.net Wed Jun 24 07:20:00 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Wed, 24 Jun 2015 09:20:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1383) memory leak In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1383: --------------------------- Resolution: Invalid Status: Closed (was: Open) > memory leak > ----------- > > Key: BIT-1383 > URL: https://bro-tracker.atlassian.net/browse/BIT-1383 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Ubuntu 12.04.5 LTS > Reporter: Garanews > Labels: leak, memory > Attachments: log.tar.gz, memory usage 2 weeks.png, screenshot-1.png, screenshot-2.png, threads 2 weeks.png, threads.png > > > BRO is taking almos all resources availables (CPUs and 192GBRAM +50GB Swap) : > top - 12:26:56 up 1 day, 1:40, 2 users, load average: 28.10, 28.69, 27.94 > Tasks: 365 total, 16 running, 349 sleeping, 0 stopped, 0 zombie > Cpu(s): 28.7%us, 18.4%sy, 1.4%ni, 48.3%id, 2.4%wa, 0.0%hi, 0.7%si, 0.0%st > Mem: 198059808k total, 197549384k used, 510424k free, 1384k buffers > Swap: 201289980k total, 50575440k used, 150714540k free, 10596k cached > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 13551 root 20 0 1341m 880m 517m R 100 0.5 993:31.62 bro > 13596 root 20 0 1173m 882m 517m R 100 0.5 1001:27 bro > 13617 root 20 0 8472m 898m 517m R 100 0.5 990:51.10 bro > 13659 root 20 0 1469m 889m 517m R 100 0.5 990:53.08 bro > 13679 root 20 0 1390m 885m 517m R 100 0.5 981:09.51 bro > 13681 root 20 0 2229m 1.8g 517m R 100 1.0 952:03.55 bro > 13653 root 20 0 2953m 881m 517m R 99 0.5 988:00.31 bro > 13685 root 20 0 1106m 888m 517m R 99 0.5 988:19.98 bro > 13641 root 20 0 1122m 884m 517m R 64 0.5 987:33.61 bro > 13672 root 20 0 10.1g 7.5g 517m R 56 4.0 987:49.47 bro > 13696 root 20 0 1161m 881m 517m R 53 0.5 982:57.39 bro > 13668 root 20 0 1149m 883m 517m S 51 0.5 989:52.19 bro > 13691 root 20 0 1989m 884m 517m R 50 0.5 1012:40 bro > 13692 root 20 0 1190m 885m 517m S 48 0.5 995:16.83 bro > 13677 root 20 0 1188m 889m 517m S 44 0.5 993:21.39 bro > 13687 root 20 0 5340m 2.7g 517m S 43 1.5 978:51.70 bro > 8906 root 25 5 294m 111m 672 S 40 0.1 526:31.84 bro > 6545 root 25 5 1235m 859m 624 S 29 0.4 482:23.79 bro > 4645 root 20 0 198g 160g 1832 S 24 85.0 254:50.36 bro > 8238 root 20 0 1755m 114m 1716 S 19 0.1 129:58.95 bro > 15196 root 25 5 685m 539m 515m S 16 0.3 219:22.30 bro > 15149 root 25 5 678m 535m 515m S 16 0.3 210:17.67 bro > 15166 root 25 5 678m 544m 515m S 16 0.3 208:54.70 bro > 15200 root 25 5 686m 543m 515m S 16 0.3 210:56.13 bro > 15148 root 25 5 678m 546m 515m R 16 0.3 211:22.38 bro > 15186 root 25 5 685m 537m 515m S 16 0.3 208:55.80 bro > 15187 root 25 5 685m 545m 515m R 16 0.3 211:02.48 bro > 15188 root 25 5 685m 536m 515m R 16 0.3 207:31.37 bro > 15197 root 25 5 692m 536m 515m S 16 0.3 208:34.05 bro > 15201 root 25 5 692m 547m 515m S 16 0.3 209:19.18 bro > 15165 root 25 5 685m 543m 515m S 15 0.3 207:28.37 bro > 15147 root 25 5 692m 536m 515m S 15 0.3 210:45.36 bro > 15185 root 25 5 686m 544m 515m S 15 0.3 210:45.12 bro > 15198 root 25 5 678m 546m 515m S 15 0.3 203:51.90 bro > 15150 root 25 5 678m 537m 515m S 15 0.3 207:34.16 bro > 15199 root 25 5 685m 537m 515m S 14 0.3 204:45.21 bro -- This message was sent by Atlassian JIRA (v6.5-OD-07-005#65007) From jira at bro-tracker.atlassian.net Wed Jun 24 23:17:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Thu, 25 Jun 2015 01:17:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1429) The SMTP logs should include CC: addresses as well as To: addresses In-Reply-To: References: Message-ID: Vern Paxson created BIT-1429: -------------------------------- Summary: The SMTP logs should include CC: addresses as well as To: addresses Key: BIT-1429 URL: https://bro-tracker.atlassian.net/browse/BIT-1429 Project: Bro Issue Tracker Issue Type: New Feature Components: Bro Reporter: Vern Paxson Assignee: Seth Hall Including CC's would enable more complete analysis of email sending patterns. -- This message was sent by Atlassian JIRA (v6.5-OD-07-005#65007) From noreply at bro.org Thu Jun 25 00:00:31 2015 From: noreply at bro.org (Merge Tracker) Date: Thu, 25 Jun 2015 00:00:31 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506250700.t5P70Vml029537@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ----------------------------------------------------------- BIT-1426 [1] Bro Seth Hall Robin Sommer 2015-06-19 - Normal Fix an issue with the modbus protocol never being confirmed BIT-1399 [2] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [3] Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------ ---------- ---------------------------------------------- 5c060f3 [4] bro Justin Azoff 2015-06-24 Correct the name used in the header identifier Open GitHub Pull Requests ========================= Issue Component User Updated Title -------- ----------- --------------- ---------- ---------------------------------------------------------------- #33 [5] bro jswaro [6] 2015-06-25 Initial commit of the TCPRS analyzer [7] #31 [8] bro yunzheng [9] 2015-06-19 Fix BIT-1314: Detect "quantum insert" type of attacks [10] #30 [11] bro jsbarber [12] 2015-06-19 Use a common Packet format and preserve layer 2 information [13] #1 [14] bro-plugins jsbarber [15] 2015-05-23 Use a common Packet format and preserve layer 2 information [16] #1 [17] btest grigorescu [18] 2015-06-22 Allow testbase overriding in the config [19] [1] BIT-1426 https://bro-tracker.atlassian.net/browse/BIT-1426 [2] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [3] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [4] 5c060f3 https://github.com/bro/bro/commit/5c060f302e8ad0298705e1c0ab411e7fdb96f412 [5] Pull Request #33 https://github.com/bro/bro/pull/33 [6] jswaro https://github.com/jswaro [7] Merge Pull Request #33 with git pull --no-ff --no-commit https://github.com/jswaro/bro.git topic/jswaro/feature/initial-tcprs-release [8] Pull Request #31 https://github.com/bro/bro/pull/31 [9] yunzheng https://github.com/yunzheng [10] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [11] Pull Request #30 https://github.com/bro/bro/pull/30 [12] jsbarber https://github.com/jsbarber [13] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [14] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [15] jsbarber https://github.com/jsbarber [16] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets [17] Pull Request #1 https://github.com/bro/btest/pull/1 [18] grigorescu https://github.com/grigorescu [19] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/grigorescu/btest.git topic/vladg/config_file_testbase From codyit at gmail.com Thu Jun 25 04:38:53 2015 From: codyit at gmail.com (Cody) Date: Thu, 25 Jun 2015 19:38:53 +0800 Subject: [Bro-Dev] Making cross compile easier Message-ID: Hi, Would you guys consider making cross compile easier on your roadmap? The platform I tried to have it running was OpenWRT, which has its own problems such as the libresolv is a stub in the most popular C library choice uClibc, that is causing some troubles. What I mean is, even without these problems, any cross compile would fail because of of the use of bicfl. Apparently somebody had gone through the painful process 10 years ago: http://mailman.icsi.berkeley.edu/pipermail/bro/2005-July/001318.html This fella has created a set of patches of CMake files for 2.3.1-2: http://inspirated.com/2015/06/08/release-bro-2-3-1-2-on-openwrt Which is hosted on this github page: https://github.com/krkhan/openwrt-bro/tree/master/bro/patches The build he created would fail because old versions are not being hosted on your site, but more importantly the patches are obsolete as of 2.4. Those patches are not huge, it would be great if you guys would consider making it easier. Thanks, Cody From seth at icir.org Thu Jun 25 05:10:18 2015 From: seth at icir.org (Seth Hall) Date: Thu, 25 Jun 2015 08:10:18 -0400 Subject: [Bro-Dev] Making cross compile easier In-Reply-To: References: Message-ID: > On Jun 25, 2015, at 7:38 AM, Cody wrote: > > Would you guys consider making cross compile easier on your roadmap? I think that?d be nice. We honestly haven?t ever gotten many requests for it, but conceptually it shouldn?t be hard to add and maintain support for. > Which is hosted on this github page: > https://github.com/krkhan/openwrt-bro/tree/master/bro/patches Ah, great! I?ll file a ticket. I skimmed these patches and seem fairly unintrusive to the build system. Thanks, .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150625/f5974ce8/attachment.bin From jira at bro-tracker.atlassian.net Thu Jun 25 05:13:00 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Thu, 25 Jun 2015 07:13:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1430) Cross compilation support In-Reply-To: References: Message-ID: Seth Hall created BIT-1430: ------------------------------ Summary: Cross compilation support Key: BIT-1430 URL: https://bro-tracker.atlassian.net/browse/BIT-1430 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Seth Hall >From an email Cody sent to the bro-dev list:: {quote}Hi, Would you guys consider making cross compile easier on your roadmap? The platform I tried to have it running was OpenWRT, which has its own problems such as the libresolv is a stub in the most popular C library choice uClibc, that is causing some troubles. What I mean is, even without these problems, any cross compile would fail because of of the use of bicfl. Apparently somebody had gone through the painful process 10 years ago: http://mailman.icsi.berkeley.edu/pipermail/bro/2005-July/001318.html This fella has created a set of patches of CMake files for 2.3.1-2: http://inspirated.com/2015/06/08/release-bro-2-3-1-2-on-openwrt Which is hosted on this github page: https://github.com/krkhan/openwrt-bro/tree/master/bro/patches The build he created would fail because old versions are not being hosted on your site, but more importantly the patches are obsolete as of 2.4. Those patches are not huge, it would be great if you guys would consider making it easier. Thanks, Cody {quote} -- This message was sent by Atlassian JIRA (v6.5-OD-07-005#65007) From robin at icir.org Thu Jun 25 07:17:23 2015 From: robin at icir.org (Robin Sommer) Date: Thu, 25 Jun 2015 07:17:23 -0700 Subject: [Bro-Dev] Making cross compile easier In-Reply-To: References: Message-ID: <20150625141723.GG73843@icir.org> On Thu, Jun 25, 2015 at 08:10 -0400, you wrote: > Ah, great! I?ll file a ticket. I skimmed these patches and seem > fairly unintrusive to the build system. It would certainly be a nice to have that integrated, I have actually heard that request a couple of times recently. One thing we need to think about is regression testing, though, so that we don't break it going forward. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From codyit at gmail.com Thu Jun 25 08:28:36 2015 From: codyit at gmail.com (Cody) Date: Thu, 25 Jun 2015 23:28:36 +0800 Subject: [Bro-Dev] Making cross compile easier In-Reply-To: <20150625141723.GG73843@icir.org> References: <20150625141723.GG73843@icir.org> Message-ID: On Thu, Jun 25, 2015 at 10:17 PM, Robin Sommer wrote: > > > On Thu, Jun 25, 2015 at 08:10 -0400, you wrote: > >> Ah, great! I?ll file a ticket. I skimmed these patches and seem >> fairly unintrusive to the build system. Awesome! Thanks, Cody From noreply at bro.org Fri Jun 26 00:00:29 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 26 Jun 2015 00:00:29 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506260700.t5Q70Tm6013281@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ----------------------------------------------------------- BIT-1426 [1] Bro Seth Hall Robin Sommer 2015-06-19 - Normal Fix an issue with the modbus protocol never being confirmed BIT-1399 [2] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [3] Open GitHub Pull Requests ========================= Issue Component User Updated Title -------- ----------- --------------- ---------- ---------------------------------------------------------------- #33 [4] bro jswaro [5] 2015-06-25 Initial commit of the TCPRS analyzer [6] #31 [7] bro yunzheng [8] 2015-06-19 Fix BIT-1314: Detect "quantum insert" type of attacks [9] #30 [10] bro jsbarber [11] 2015-06-19 Use a common Packet format and preserve layer 2 information [12] #1 [13] bro-plugins jsbarber [14] 2015-05-23 Use a common Packet format and preserve layer 2 information [15] #1 [16] btest grigorescu [17] 2015-06-22 Allow testbase overriding in the config [18] [1] BIT-1426 https://bro-tracker.atlassian.net/browse/BIT-1426 [2] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [3] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [4] Pull Request #33 https://github.com/bro/bro/pull/33 [5] jswaro https://github.com/jswaro [6] Merge Pull Request #33 with git pull --no-ff --no-commit https://github.com/jswaro/bro.git topic/jswaro/feature/initial-tcprs-release [7] Pull Request #31 https://github.com/bro/bro/pull/31 [8] yunzheng https://github.com/yunzheng [9] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [10] Pull Request #30 https://github.com/bro/bro/pull/30 [11] jsbarber https://github.com/jsbarber [12] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [13] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [14] jsbarber https://github.com/jsbarber [15] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets [16] Pull Request #1 https://github.com/bro/btest/pull/1 [17] grigorescu https://github.com/grigorescu [18] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/grigorescu/btest.git topic/vladg/config_file_testbase From noreply at bro.org Sat Jun 27 00:00:22 2015 From: noreply at bro.org (Merge Tracker) Date: Sat, 27 Jun 2015 00:00:22 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506270700.t5R70M8x001560@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ----------------------------------------------------------- BIT-1426 [1] Bro Seth Hall Robin Sommer 2015-06-19 - Normal Fix an issue with the modbus protocol never being confirmed BIT-1399 [2] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [3] Open GitHub Pull Requests ========================= Issue Component User Updated Title -------- ----------- --------------- ---------- ---------------------------------------------------------------- #33 [4] bro jswaro [5] 2015-06-27 Initial commit of the TCPRS analyzer [6] #31 [7] bro yunzheng [8] 2015-06-19 Fix BIT-1314: Detect "quantum insert" type of attacks [9] #30 [10] bro jsbarber [11] 2015-06-19 Use a common Packet format and preserve layer 2 information [12] #1 [13] bro-plugins jsbarber [14] 2015-05-23 Use a common Packet format and preserve layer 2 information [15] #1 [16] btest grigorescu [17] 2015-06-22 Allow testbase overriding in the config [18] [1] BIT-1426 https://bro-tracker.atlassian.net/browse/BIT-1426 [2] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [3] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [4] Pull Request #33 https://github.com/bro/bro/pull/33 [5] jswaro https://github.com/jswaro [6] Merge Pull Request #33 with git pull --no-ff --no-commit https://github.com/jswaro/bro.git topic/jswaro/feature/initial-tcprs-release [7] Pull Request #31 https://github.com/bro/bro/pull/31 [8] yunzheng https://github.com/yunzheng [9] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [10] Pull Request #30 https://github.com/bro/bro/pull/30 [11] jsbarber https://github.com/jsbarber [12] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [13] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [14] jsbarber https://github.com/jsbarber [15] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets [16] Pull Request #1 https://github.com/bro/btest/pull/1 [17] grigorescu https://github.com/grigorescu [18] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/grigorescu/btest.git topic/vladg/config_file_testbase From jira at bro-tracker.atlassian.net Sat Jun 27 18:34:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Sat, 27 Jun 2015 20:34:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1426) Fix an issue with the modbus protocol never being confirmed In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1426: ------------------------------ Status: Closed (was: Merge Request) > Fix an issue with the modbus protocol never being confirmed > ----------------------------------------------------------- > > Key: BIT-1426 > URL: https://bro-tracker.atlassian.net/browse/BIT-1426 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.5 > Reporter: Seth Hall > Assignee: Robin Sommer > > The modbus analyzer now calls ConfirmProtocol after it successfully parses a PDU from both sides of a conversation which causes the conn.log to now identify "modbus" as the attached analyzer. > This is ready for merging with a test update in the topic/seth/modbus_dpd_fix branch. -- This message was sent by Atlassian JIRA (v6.5-OD-07-005#65007) From noreply at bro.org Sun Jun 28 00:00:32 2015 From: noreply at bro.org (Merge Tracker) Date: Sun, 28 Jun 2015 00:00:32 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506280700.t5S70WVA031780@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------------- ---------- ---------------------------------------------------------------- #33 [3] bro jswaro [4] 2015-06-27 Initial commit of the TCPRS analyzer [5] #31 [6] bro yunzheng [7] 2015-06-19 Fix BIT-1314: Detect "quantum insert" type of attacks [8] #30 [9] bro jsbarber [10] 2015-06-19 Use a common Packet format and preserve layer 2 information [11] #1 [12] bro-plugins jsbarber [13] 2015-05-23 Use a common Packet format and preserve layer 2 information [14] #1 [15] btest grigorescu [16] 2015-06-22 Allow testbase overriding in the config [17] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #33 https://github.com/bro/bro/pull/33 [4] jswaro https://github.com/jswaro [5] Merge Pull Request #33 with git pull --no-ff --no-commit https://github.com/jswaro/bro.git topic/jswaro/feature/initial-tcprs-release [6] Pull Request #31 https://github.com/bro/bro/pull/31 [7] yunzheng https://github.com/yunzheng [8] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [9] Pull Request #30 https://github.com/bro/bro/pull/30 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [12] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [13] jsbarber https://github.com/jsbarber [14] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets [15] Pull Request #1 https://github.com/bro/btest/pull/1 [16] grigorescu https://github.com/grigorescu [17] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/grigorescu/btest.git topic/vladg/config_file_testbase From noreply at bro.org Mon Jun 29 00:00:23 2015 From: noreply at bro.org (Merge Tracker) Date: Mon, 29 Jun 2015 00:00:23 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506290700.t5T70NTp016827@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------------- ---------- ---------------------------------------------------------------- #33 [3] bro jswaro [4] 2015-06-27 Initial commit of the TCPRS analyzer [5] #31 [6] bro yunzheng [7] 2015-06-19 Fix BIT-1314: Detect "quantum insert" type of attacks [8] #30 [9] bro jsbarber [10] 2015-06-19 Use a common Packet format and preserve layer 2 information [11] #1 [12] bro-plugins jsbarber [13] 2015-05-23 Use a common Packet format and preserve layer 2 information [14] #1 [15] btest grigorescu [16] 2015-06-22 Allow testbase overriding in the config [17] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #33 https://github.com/bro/bro/pull/33 [4] jswaro https://github.com/jswaro [5] Merge Pull Request #33 with git pull --no-ff --no-commit https://github.com/jswaro/bro.git topic/jswaro/feature/initial-tcprs-release [6] Pull Request #31 https://github.com/bro/bro/pull/31 [7] yunzheng https://github.com/yunzheng [8] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [9] Pull Request #30 https://github.com/bro/bro/pull/30 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [12] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [13] jsbarber https://github.com/jsbarber [14] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets [15] Pull Request #1 https://github.com/bro/btest/pull/1 [16] grigorescu https://github.com/grigorescu [17] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/grigorescu/btest.git topic/vladg/config_file_testbase From noreply at bro.org Tue Jun 30 00:00:22 2015 From: noreply at bro.org (Merge Tracker) Date: Tue, 30 Jun 2015 00:00:22 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201506300700.t5U70MUh022766@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- --------------- ---------- ---------------------------------------------------------------- #33 [3] bro jswaro [4] 2015-06-27 Initial commit of the TCPRS analyzer [5] #31 [6] bro yunzheng [7] 2015-06-19 Fix BIT-1314: Detect "quantum insert" type of attacks [8] #30 [9] bro jsbarber [10] 2015-06-19 Use a common Packet format and preserve layer 2 information [11] #1 [12] bro-plugins jsbarber [13] 2015-05-23 Use a common Packet format and preserve layer 2 information [14] #1 [15] btest grigorescu [16] 2015-06-22 Allow testbase overriding in the config [17] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #33 https://github.com/bro/bro/pull/33 [4] jswaro https://github.com/jswaro [5] Merge Pull Request #33 with git pull --no-ff --no-commit https://github.com/jswaro/bro.git topic/jswaro/feature/initial-tcprs-release [6] Pull Request #31 https://github.com/bro/bro/pull/31 [7] yunzheng https://github.com/yunzheng [8] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [9] Pull Request #30 https://github.com/bro/bro/pull/30 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [12] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [13] jsbarber https://github.com/jsbarber [14] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets [15] Pull Request #1 https://github.com/bro/btest/pull/1 [16] grigorescu https://github.com/grigorescu [17] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/grigorescu/btest.git topic/vladg/config_file_testbase