From daniel.guerra69 at gmail.com Tue Mar 1 00:18:43 2016 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Tue, 1 Mar 2016 09:18:43 +0100 Subject: [Bro] How use logs-to-elasticsearch.bro In-Reply-To: <000d01d17387$0c4f2e30$24ed8a90$@126.com> References: <000d01d17387$0c4f2e30$24ed8a90$@126.com> Message-ID: Hi, There is a problem with elasticsearch 2.0 and higher. It doesn?t accept dots in field names and there are some timestamp issues. Check https://hub.docker.com/r/danielguerra/bro-debian-elasticsearch/ or https://github.com/danielguerra69/bro-debian-elasticsearch (check the patch dir) Regards, Daniel > On 01 Mar 2016, at 07:53, mz wrote: > > Dear all > I would like to use logs-to-elasticsearch.bro this script to log the Bro Elasticsearch? > > My Bro Version: 2.4.1 > > 1?Use this script is not you do not need logstash, Bro will be sent directly to the log Elasticsearch? > > 2?I follow the official document: https: //www.bro.org/sphinx/components/bro-plugins/elasticsearch/README.html is configured in /usr/local/bro/share/bro/site/local. bro added @load bro/ElasticSearch/logs-to-elasticsearch.bro. But it was not successful, in addition to the configuration of the document still need additional configuration? > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160301/b2ccfc8d/attachment-0001.html From cbarbaro at cert.unlp.edu.ar Tue Mar 1 04:06:38 2016 From: cbarbaro at cert.unlp.edu.ar (Cristian Daniel Barbaro) Date: Tue, 1 Mar 2016 09:06:38 -0300 Subject: [Bro] About Bro Cluster Configuration In-Reply-To: References: <56D48BC8.40107@cert.unlp.edu.ar> Message-ID: <56D585CE.80004@cert.unlp.edu.ar> Perfect. I'll try it. Thanks. El 29/02/16 a las 16:07, Vlad Grigorescu escribi?: > Yes, this should be fine. The standard architecture is meant to > provide load-balancing for monitoring points that are too large for a > single system to monitor (> 4-5 Gbps with modern, beefy hardware). As > long as each Bro worker is seeing both the upflow and downflow of each > connection it sees, the cluster doesn't care about which worker sees > which subset of the overall traffic. > > --Vlad > > Cristian Daniel Barbaro writes: > >> Hello, I have a question about Bro Cluster architecture. By default, the >> cluster architecture has a frontend listening to a high-speed link; >> spliting traffic to each worker and to finally all workers information >> be administered by a manager using a proxy, etc. >> >> What we want to do is to have several workers analysing different >> networks segments and that each of those workers communicate with a >> manager, who will be responsible for managing all information and of >> course, enabling a centralized administration of workers configuration. >> >> Is it possible to do this? >> >> Thanks and regards. >> >> -- >> Cristian Daniel Barbaro >> CERTUNLP >> -- >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Cristian Daniel Barbaro CERTUNLP -- From Blake.Mackey at rmc.ca Tue Mar 1 04:35:14 2016 From: Blake.Mackey at rmc.ca (Blake Mackey) Date: Tue, 1 Mar 2016 12:35:14 +0000 Subject: [Bro] How use logs-to-elasticsearch.bro In-Reply-To: References: <000d01d17387$0c4f2e30$24ed8a90$@126.com> Message-ID: If you are using the elk stack, check out : https://github.com/BrashEndeavours/logstash-input-bro Respectfully, Blake Mackey, CD SLt | ens 1 Royal Military College of Canada | Coll?ge militaire royal du Canada (613)331-6438 On Mar 1, 2016, at 03:18, Daniel Guerra > wrote: Hi, There is a problem with elasticsearch 2.0 and higher. It doesn?t accept dots in field names and there are some timestamp issues. Check https://hub.docker.com/r/danielguerra/bro-debian-elasticsearch/ or https://github.com/danielguerra69/bro-debian-elasticsearch (check the patch dir) Regards, Daniel On 01 Mar 2016, at 07:53, mz > wrote: Dear all I would like to use logs-to-elasticsearch.bro this script to log the Bro Elasticsearch? My Bro Version: 2.4.1 1?Use this script is not you do not need logstash, Bro will be sent directly to the log Elasticsearch? 2?I follow the official document: https: //www.bro.org/sphinx/components/bro-plugins/elasticsearch/README.html is configured in /usr/local/bro/share/bro/site/local. bro added @load bro/ElasticSearch/logs-to-elasticsearch.bro. But it was not successful, in addition to the configuration of the document still need additional configuration? _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160301/c8b13015/attachment.html From seth at icir.org Tue Mar 1 07:51:03 2016 From: seth at icir.org (Seth Hall) Date: Tue, 1 Mar 2016 10:51:03 -0500 Subject: [Bro] How use logs-to-elasticsearch.bro In-Reply-To: References: <000d01d17387$0c4f2e30$24ed8a90$@126.com> Message-ID: <9757F21F-FBD3-4E6C-B0BF-C1C11DC06AE0@icir.org> > On Mar 1, 2016, at 3:18 AM, Daniel Guerra wrote: > > There is a problem with elasticsearch 2.0 and higher. > It doesn?t accept dots in field names and there are > some timestamp issues. I know this discussion has been going on for a while and unfortunately I've been a bit behind the curve on keeping up with it closely. As someone who seems to have been coping with this problem for a while, what do you recommend? Would it be best if we could do nested json documents in the json output? i.e.... {"ts":1223421341234.1234, "id": {"orig_h": "1.2.3.4", "orig_p":1234.......etc }} .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From shirkdog.bsd at gmail.com Tue Mar 1 08:29:29 2016 From: shirkdog.bsd at gmail.com (Michael Shirk) Date: Tue, 1 Mar 2016 11:29:29 -0500 Subject: [Bro] How use logs-to-elasticsearch.bro In-Reply-To: <9757F21F-FBD3-4E6C-B0BF-C1C11DC06AE0@icir.org> References: <000d01d17387$0c4f2e30$24ed8a90$@126.com> <9757F21F-FBD3-4E6C-B0BF-C1C11DC06AE0@icir.org> Message-ID: I am happy this came up, as I have been going through the same issues for testing Brownian vs. ELK with Bro filters If it is not supported in Bro's JSON output, it would be nice to be able to configure it, as there may already be some parsing of the default JSON output of Bro with tools like Splunk. -- Michael Shirk Daemon Security, Inc. http://www.daemon-security.com On Mar 1, 2016 11:06, "Seth Hall" wrote: > > > On Mar 1, 2016, at 3:18 AM, Daniel Guerra > wrote: > > > > There is a problem with elasticsearch 2.0 and higher. > > It doesn?t accept dots in field names and there are > > some timestamp issues. > > I know this discussion has been going on for a while and unfortunately > I've been a bit behind the curve on keeping up with it closely. As someone > who seems to have been coping with this problem for a while, what do you > recommend? Would it be best if we could do nested json documents in the > json output? i.e.... > > {"ts":1223421341234.1234, "id": {"orig_h": "1.2.3.4", > "orig_p":1234.......etc }} > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160301/7a3c3587/attachment-0001.html From macochran0 at gmail.com Tue Mar 1 09:35:55 2016 From: macochran0 at gmail.com (Michael Cochran) Date: Tue, 1 Mar 2016 12:35:55 -0500 Subject: [Bro] Renaming carved files Message-ID: I'm trying to find a simple way to rename a carved file back to it's original file name using bro-script rather than having bash try to rip it out of the files.log file. I have seen the mime type analyzers on git that re-add the extension based on known mime types, but I'd rather be able to immediately identify the original file name as it came across the wire. I don't need the unique session identifier because by the time I'm using bro file analysis I already have the individual session pcap isolated. I'm guessing there should be a way to capture the files.log table data in broscript, match the unique file identifier then rename the file with that filename string from files.log. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160301/553d4608/attachment.html From anthony.kasza at gmail.com Tue Mar 1 10:48:49 2016 From: anthony.kasza at gmail.com (anthony kasza) Date: Tue, 1 Mar 2016 10:48:49 -0800 Subject: [Bro] Renaming carved files In-Reply-To: References: Message-ID: This is a tricky thing to do regardless of how you do it. What happens when the file was transfered over something besides protocols with URLs? Or, what if the file is a PE and includes an original name in its manifest but resides at a different URL? -AK On Mar 1, 2016 9:51 AM, "Michael Cochran" wrote: > I'm trying to find a simple way to rename a carved file back to it's > original file name using bro-script rather than having bash try to rip it > out of the files.log file. I have seen the mime type analyzers on git that > re-add the extension based on known mime types, but I'd rather be able to > immediately identify the original file name as it came across the wire. I > don't need the unique session identifier because by the time I'm using bro > file analysis I already have the individual session pcap isolated. > > I'm guessing there should be a way to capture the files.log table data in > broscript, match the unique file identifier then rename the file with that > filename string from files.log. > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160301/0320f0ef/attachment.html From macochran0 at gmail.com Tue Mar 1 11:11:07 2016 From: macochran0 at gmail.com (Michael Cochran) Date: Tue, 1 Mar 2016 14:11:07 -0500 Subject: [Bro] Renaming carved files In-Reply-To: References: Message-ID: I'm not expecting there to be a filename associated with every file, but if the filename was in the pcap, for SMTP attachments, FTP file transfers, or HTTP sessions this shouldn't be a complicated thing to do. I'm looking at this from a network analyst point of view in making it more efficient for them to quickly disseminate information. Maybe the fact that there is no filename for the extracted data makes it more/less interesting depending on the situation. I'm not looking for bro to try to make up a filename based on URI, but rather just get the information from the HTTP header if the filename is present (which I think is how bro gets the filename in files.log for HTTP sessions). In which case just ripping it out of files.log would be the right thing to do. I guess the real question is, is it possible to do that in bro-script? Or is it just more realistic to do that using python/shell? On Tue, Mar 1, 2016 at 1:48 PM, anthony kasza wrote: > This is a tricky thing to do regardless of how you do it. What happens > when the file was transfered over something besides protocols with URLs? > Or, what if the file is a PE and includes an original name in its manifest > but resides at a different URL? > > -AK > On Mar 1, 2016 9:51 AM, "Michael Cochran" wrote: > >> I'm trying to find a simple way to rename a carved file back to it's >> original file name using bro-script rather than having bash try to rip it >> out of the files.log file. I have seen the mime type analyzers on git that >> re-add the extension based on known mime types, but I'd rather be able to >> immediately identify the original file name as it came across the wire. I >> don't need the unique session identifier because by the time I'm using bro >> file analysis I already have the individual session pcap isolated. >> >> I'm guessing there should be a way to capture the files.log table data in >> broscript, match the unique file identifier then rename the file with that >> filename string from files.log. >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160301/4724f997/attachment.html From daniel.guerra69 at gmail.com Tue Mar 1 11:17:31 2016 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Tue, 1 Mar 2016 20:17:31 +0100 Subject: [Bro] Renaming carved files In-Reply-To: References: Message-ID: <03BB65F8-EAAD-483A-AA4E-47CBA33A0C14@gmail.com> Maybe this is useful securityonion-bro-scripts/file-extraction/extract.bro Regards, Daniel > On 01 Mar 2016, at 18:35, Michael Cochran wrote: > > I'm trying to find a simple way to rename a carved file back to it's original file name using bro-script rather than having bash try to rip it out of the files.log file. I have seen the mime type analyzers on git that re-add the extension based on known mime types, but I'd rather be able to immediately identify the original file name as it came across the wire. I don't need the unique session identifier because by the time I'm using bro file analysis I already have the individual session pcap isolated. > > I'm guessing there should be a way to capture the files.log table data in broscript, match the unique file identifier then rename the file with that filename string from files.log. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From daniel.guerra69 at gmail.com Tue Mar 1 11:18:38 2016 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Tue, 1 Mar 2016 20:18:38 +0100 Subject: [Bro] Renaming carved files In-Reply-To: References: Message-ID: <7B799BD7-6EFB-434A-88BD-9C399FDA90A3@gmail.com> https://github.com/Security-Onion-Solutions/securityonion-bro-scripts/blob/master/file-extraction/extract.bro > On 01 Mar 2016, at 18:35, Michael Cochran wrote: > > I'm trying to find a simple way to rename a carved file back to it's original file name using bro-script rather than having bash try to rip it out of the files.log file. I have seen the mime type analyzers on git that re-add the extension based on known mime types, but I'd rather be able to immediately identify the original file name as it came across the wire. I don't need the unique session identifier because by the time I'm using bro file analysis I already have the individual session pcap isolated. > > I'm guessing there should be a way to capture the files.log table data in broscript, match the unique file identifier then rename the file with that filename string from files.log. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160301/dcb8ad89/attachment-0001.html From tgdesrochers at gmail.com Tue Mar 1 11:38:23 2016 From: tgdesrochers at gmail.com (Tim Desrochers) Date: Tue, 1 Mar 2016 14:38:23 -0500 Subject: [Bro] How use logs-to-elasticsearch.bro In-Reply-To: References: <000d01d17387$0c4f2e30$24ed8a90$@126.com> <9757F21F-FBD3-4E6C-B0BF-C1C11DC06AE0@icir.org> Message-ID: I use bro with ELK in production and it works great. I use bro to json and all my logs are in json. Then use logstash to pick up the logs and the good folks at elastic have created a plugin for de_dot. It's not perfect but with some mutates it works fine for the time being. Kibana is a fine interface to build dashboards and query the data. Bro and ELK integration works great with a little tweaking. I'm happy to share come configs if you're interested. On Mar 1, 2016 11:31, "Michael Shirk" wrote: > I am happy this came up, as I have been going through the same issues for > testing Brownian vs. ELK with Bro filters > > If it is not supported in Bro's JSON output, it would be nice to be able > to configure it, as there may already be some parsing of the default JSON > output of Bro with tools like Splunk. > > -- > Michael Shirk > Daemon Security, Inc. > http://www.daemon-security.com > On Mar 1, 2016 11:06, "Seth Hall" wrote: > >> >> > On Mar 1, 2016, at 3:18 AM, Daniel Guerra >> wrote: >> > >> > There is a problem with elasticsearch 2.0 and higher. >> > It doesn?t accept dots in field names and there are >> > some timestamp issues. >> >> I know this discussion has been going on for a while and unfortunately >> I've been a bit behind the curve on keeping up with it closely. As someone >> who seems to have been coping with this problem for a while, what do you >> recommend? Would it be best if we could do nested json documents in the >> json output? i.e.... >> >> {"ts":1223421341234.1234, "id": {"orig_h": "1.2.3.4", >> "orig_p":1234.......etc }} >> >> .Seth >> >> -- >> Seth Hall >> International Computer Science Institute >> (Bro) because everyone has a network >> http://www.bro.org/ >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160301/962e626e/attachment.html From seth at icir.org Tue Mar 1 14:20:03 2016 From: seth at icir.org (Seth Hall) Date: Tue, 1 Mar 2016 17:20:03 -0500 Subject: [Bro] How use logs-to-elasticsearch.bro In-Reply-To: <2064EDC8-15C4-4B0A-A231-31877FCAE326@gmail.com> References: <000d01d17387$0c4f2e30$24ed8a90$@126.com> <9757F21F-FBD3-4E6C-B0BF-C1C11DC06AE0@icir.org> <2064EDC8-15C4-4B0A-A231-31877FCAE326@gmail.com> Message-ID: > On Mar 1, 2016, at 11:47 AM, Derek Ditch wrote: > > I actually ran into this again this morning. I patched the Elasticsearch writer and I?m still testing it [1]. It uses some code adapted from ?g-clef? that built it into a Kafka output plugin. I'm not sure about that mechanism, but I think it should be integrated deeper into Bro, like probably into the json formatter and then exposed in the writers as a configuration option. Alternately, I could see having a configuration option for the writers (that flows through into the json formatter) which provides the structured output instead of flattened output as is done now. > Seth, I know some people prefer different timestamp formats, it might be best to parameterize that so that it can be modified in script land using the existing Bro formatting libraries. I?ve found that TS_8601 works extremely well with the Elasticsearch Joda parsing library. Instead of making the change as you've specified, can you add ISO8601 output as a config option as the ascii logger does? You can even set the default to be ISO8601, but I think there is some value in having that be configurable in the same way that it is in other parts of Bro. > I also changed the ts field name to @timestamp, since it?s almost universal now for the standard field for use in Elasticsearch data (used by Fluentd, Logstash, and even Spark). That's actually part of a larger change that I would like to address soon . The timestamps in each log right now are protocol specific. You definitely don't always want to use the ts field as the @timestamp field (to keep things concrete). We need to add a new field that is displayed whenever json output is enabled and I could even see a justification for adding it to the ascii logs that represents when the log was written. It's basically metadata about the log line. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From derek at criticalstack.com Tue Mar 1 14:53:14 2016 From: derek at criticalstack.com (Derek Ditch) Date: Tue, 1 Mar 2016 16:53:14 -0600 Subject: [Bro] How use logs-to-elasticsearch.bro In-Reply-To: References: <000d01d17387$0c4f2e30$24ed8a90$@126.com> <9757F21F-FBD3-4E6C-B0BF-C1C11DC06AE0@icir.org> Message-ID: <72B89271-EBC0-42A1-9ECA-B91AAE5AF1CC@criticalstack.com> I would also add, that I use ELK almost exclusively for Bro logs, but I go through a Kafka output plugin. There?s an easy setup using Chef to automate for a simple test environment over at http://rocknsm.io/ . Disclaimer, I?m one of the authors of that open source project. ? Derek Ditch dcode at rocknsm.io GPG: 0x2543A3B5 > On 01Mar 2016, at 13:38, Tim Desrochers wrote: > > I use bro with ELK in production and it works great. I use bro to json and all my logs are in json. Then use logstash to pick up the logs and the good folks at elastic have created a plugin for de_dot. It's not perfect but with some mutates it works fine for the time being. Kibana is a fine interface to build dashboards and query the data. > > Bro and ELK integration works great with a little tweaking. I'm happy to share come configs if you're interested. > > On Mar 1, 2016 11:31, "Michael Shirk" > wrote: > I am happy this came up, as I have been going through the same issues for testing Brownian vs. ELK with Bro filters > > If it is not supported in Bro's JSON output, it would be nice to be able to configure it, as there may already be some parsing of the default JSON output of Bro with tools like Splunk. > > -- > Michael Shirk > Daemon Security, Inc. > http://www.daemon-security.com > On Mar 1, 2016 11:06, "Seth Hall" > wrote: > > > On Mar 1, 2016, at 3:18 AM, Daniel Guerra > wrote: > > > > There is a problem with elasticsearch 2.0 and higher. > > It doesn?t accept dots in field names and there are > > some timestamp issues. > > I know this discussion has been going on for a while and unfortunately I've been a bit behind the curve on keeping up with it closely. As someone who seems to have been coping with this problem for a while, what do you recommend? Would it be best if we could do nested json documents in the json output? i.e.... > > {"ts":1223421341234.1234, "id": {"orig_h": "1.2.3.4", "orig_p":1234.......etc }} > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160301/fbb0d880/attachment.html From mgill6 at student.concordia.ab.ca Tue Mar 1 17:07:24 2016 From: mgill6 at student.concordia.ab.ca (Manmeet Gill) Date: Tue, 1 Mar 2016 18:07:24 -0700 Subject: [Bro] Can Bro detect a traffic difference, according to days and time. In-Reply-To: References: Message-ID: Hi everybody, I will give a scenario let me know is it possible using Bro ids or not. If there is a traffic of tcp,udp,icmp,https,smtp and dns, 80%,50%,30%,70%,80% and 60% respectively during working days(mon-fri)(from 10am-6pm) which we can say is a normal traffic. and if these traffic differs with 10% below or above for each protocol. then alarm should be triggered, similarly with (off hours 7pm to 9am) if we see same amount of traffic, alarm should be triggered. Is it possible with Bro to make this type of scenario detectable. --manmeet singh -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160301/f9b8c95a/attachment.html From mz89924 at 126.com Tue Mar 1 19:14:43 2016 From: mz89924 at 126.com (mz) Date: Wed, 2 Mar 2016 11:14:43 +0800 Subject: [Bro] What is the difference Snort and Bro Message-ID: <000d01d17431$ab3b3a20$01b1ae60$@126.com> Dear all 1. Bro and Snort what difference? 2. Both of who is stronger? 3. Who is better suited for large enterprise environments? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160302/75803e7b/attachment.html From macochran0 at gmail.com Wed Mar 2 07:15:44 2016 From: macochran0 at gmail.com (Michael Cochran) Date: Wed, 2 Mar 2016 10:15:44 -0500 Subject: [Bro] Renaming carved files In-Reply-To: <7B799BD7-6EFB-434A-88BD-9C399FDA90A3@gmail.com> References: <7B799BD7-6EFB-434A-88BD-9C399FDA90A3@gmail.com> Message-ID: So the problem I'm running into with this extraction script is here (I've already got a script that handles the extracted metadata mime types): local fname = fmt("/nsm/bro/extracted/%s-%s.%s", f$source, f$id, ext); I don't need f$source or f$id in the filename. What I'm searching for is being generated here in main.bro. I just need a way to grab this information and add it to the extract.bro script to rename extracted file. https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html#type-Files::Info Files::Info filename: string &log &optional A filename for the file if one is available from the source for the file. These will frequently come from ?Content-Disposition? headers in network protocols The logic (forgive my terrible syntax) should be along the lines of if f$filename is not empty, local fname = fmt(outputdir, f$filename, ext); else local fname = fmt("outputdir", f$source, f$id, ext); On Tue, Mar 1, 2016 at 2:18 PM, Daniel Guerra wrote: > > > https://github.com/Security-Onion-Solutions/securityonion-bro-scripts/blob/master/file-extraction/extract.bro > > > On 01 Mar 2016, at 18:35, Michael Cochran wrote: > > I'm trying to find a simple way to rename a carved file back to it's > original file name using bro-script rather than having bash try to rip it > out of the files.log file. I have seen the mime type analyzers on git that > re-add the extension based on known mime types, but I'd rather be able to > immediately identify the original file name as it came across the wire. I > don't need the unique session identifier because by the time I'm using bro > file analysis I already have the individual session pcap isolated. > > I'm guessing there should be a way to capture the files.log table data in > broscript, match the unique file identifier then rename the file with that > filename string from files.log. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160302/7f4eea16/attachment-0001.html From derek at criticalstack.com Wed Mar 2 07:52:34 2016 From: derek at criticalstack.com (Derek Ditch) Date: Wed, 2 Mar 2016 09:52:34 -0600 Subject: [Bro] Renaming carved files In-Reply-To: References: <7B799BD7-6EFB-434A-88BD-9C399FDA90A3@gmail.com> Message-ID: <4EED1C85-96C4-4D47-B81B-BE46554C61CB@criticalstack.com> Michael, I haven?t tested this other than validate syntax, but I think the logic you?re looking for is below. You of course have to add in the dynamic extension mapping and maybe make the outputdir configurable w/ an export {} block. Basically, you have to check to see if the filename is set. I would caution you, that there are many instances where it is not set, however. If you?re looking for a more robust file extraction strategy, I would recommend [1]. There?s some additional overhead in moving files around, but it allows you to store files by hash once extraction is complete. This should greatly reduce your disk usage and processing overhead of any follow on processing. event file_sniff(f: fa_file, meta: fa_metadata) { local fname = ""; local outputdir = "/data/bro/extracted_files/"; local ext = ".out"; # .. logic here to generate ext (with starting .) and outputdir (with ending /) if ( f?$info && f$info?$filename ) fname = cat(outputdir, f$info$filename, ext); else fname = cat(outputdir, f$source, f$id, ext); Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]); } [1] https://github.com/hosom/bro-file-extraction ? Derek Ditch derek at criticalstack.com GPG: 0x2543A3B5 > On 02Mar 2016, at 09:15, Michael Cochran wrote: > > So the problem I'm running into with this extraction script is here (I've already got a script that handles the extracted metadata mime types): > > local fname = fmt("/nsm/bro/extracted/%s-%s.%s", f$source, f$id, ext); > > I don't need f$source or f$id in the filename. What I'm searching for is being generated here in main.bro. I just need a way to grab this information and add it to the extract.bro script to rename extracted file. > > https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html#type-Files::Info > Files::Info > filename: string &log &optional > A filename for the file if one is available from the source for the file. These will frequently come from ?Content-Disposition? headers in network protocols > > The logic (forgive my terrible syntax) should be along the lines of > if f$filename is not empty, > local fname = fmt(outputdir, f$filename, ext); > else > local fname = fmt("outputdir", f$source, f$id, ext); > > > > On Tue, Mar 1, 2016 at 2:18 PM, Daniel Guerra wrote: > > https://github.com/Security-Onion-Solutions/securityonion-bro-scripts/blob/master/file-extraction/extract.bro > > >> On 01 Mar 2016, at 18:35, Michael Cochran wrote: >> >> I'm trying to find a simple way to rename a carved file back to it's original file name using bro-script rather than having bash try to rip it out of the files.log file. I have seen the mime type analyzers on git that re-add the extension based on known mime types, but I'd rather be able to immediately identify the original file name as it came across the wire. I don't need the unique session identifier because by the time I'm using bro file analysis I already have the individual session pcap isolated. >> >> I'm guessing there should be a way to capture the files.log table data in broscript, match the unique file identifier then rename the file with that filename string from files.log. >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From macochran0 at gmail.com Wed Mar 2 11:25:26 2016 From: macochran0 at gmail.com (Michael Cochran) Date: Wed, 2 Mar 2016 14:25:26 -0500 Subject: [Bro] Renaming carved files In-Reply-To: <29487F03-041A-406A-9222-DE3F8089BD91@gmail.com> References: <7B799BD7-6EFB-434A-88BD-9C399FDA90A3@gmail.com> <29487F03-041A-406A-9222-DE3F8089BD91@gmail.com> Message-ID: Derek, This is nearly spot on. Here's what I have in main.bro from the git link you provided that almost works, but is missing some sort of syntax, as it's giving me errors. If I comment out the If/else statement f$info$filename gives me the content-disposition extracted filename from the protocol. But I need a check placed in line to see if f$info$filename is empty, it's empty it should go ahead and try to figure out a mime-type extension. Very close, and it's probably something very obvious I'm looking over. @load ./file-extensions module FileExtraction; export { ## Path to store files const path: string = "" &redef; ## Hook to include files in extraction global extract: hook(f: fa_file, meta: fa_metadata); ## Hook to exclude files from extraction global ignore: hook(f: fa_file, meta: fa_metadata); } event file_sniff(f: fa_file, meta: fa_metadata) { if ( meta?$mime_type && !hook FileExtraction::extract(f, meta) ) { if ( !hook FileExtraction::ignore(f, meta) ) return; if ( meta$mime_type in mime_to_ext ) local fext = mime_to_ext[meta$mime_type]; else fext = split_string(meta$mime_type, /\//)[1]; if ( f$info$filename != "" ) local fname = cat("%s%s-%s", path, f$source, f$info$filename); else local fname = cat("%s%s-%s.%s", path, f$source, f$id, fext); Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]); } } error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line 26 and /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line 28: already defined (FileExtraction::fname) error in /opt/bro/share/bro/base/frameworks/files/./main.bro, lines 18-28 and /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line 30: incompatible record types (Files::AnalyzerArgs and [$extract_filename=FileExtraction::fname]) error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line 30 and /opt/bro/share/bro/base/frameworks/files/./main.bro, lines 18-28: type mismatch ([$extract_filename=FileExtraction::fname] and Files::AnalyzerArgs) error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, lines 29-30: argument type mismatch in function call (Files::add_analyzer(FileExtraction::f, Files::ANALYZER_EXTRACT, [$extract_filename=FileExtraction::fname])) warning in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line 30: expression value ignored (Files::add_analyzer(FileExtraction::f, Files::ANALYZER_EXTRACT, [$extract_filename=FileExtraction::fname])) On Wed, Mar 2, 2016 at 10:51 AM, Derek Ditch wrote: > Michael, > > I haven?t tested this other than validate syntax, but I think the logic > you?re looking for is below. You of course have to add in the dynamic > extension mapping and maybe make the outputdir configurable w/ an export {} > block. Basically, you have to check to see if the filename is set. I would > caution you, that there are many instances where it is not set, however. If > you?re looking for a more robust file extraction strategy, I would > recommend [1]. There?s some additional overhead in moving files around, but > it allows you to store files by hash once extraction is complete. This > should greatly reduce your disk usage and processing overhead of any follow > on processing. > > > event file_sniff(f: fa_file, meta: fa_metadata) > { > local fname = ""; > local outputdir = "/data/bro/extracted_files/"; > local ext = ".out"; > > # .. logic here to generate ext (with starting .) and outputdir (with > ending /) > if ( f?$info && f$info?$filename ) > fname = cat(outputdir, f$info$filename, ext); > else > fname = cat(outputdir, f$source, f$id, ext); > > Files::add_analyzer(f, Files::ANALYZER_EXTRACT, > [$extract_filename=fname]); > } > > [1] https://github.com/hosom/bro-file-extraction > ? > Derek Ditch > derek.ditch at gmail.com > GPG: 0x2543A3B5 > > > On 02Mar 2016, at 09:15, Michael Cochran wrote: > > > > So the problem I'm running into with this extraction script is here > (I've already got a script that handles the extracted metadata mime types): > > > > local fname = fmt("/nsm/bro/extracted/%s-%s.%s", f$source, f$id, ext); > > > > I don't need f$source or f$id in the filename. What I'm searching for is > being generated here in main.bro. I just need a way to grab this > information and add it to the extract.bro script to rename extracted file. > > > > > https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html#type-Files::Info > > Files::Info > > filename: string &log &optional > > A filename for the file if one is available from the source for the > file. These will frequently come from ?Content-Disposition? headers in > network protocols > > > > The logic (forgive my terrible syntax) should be along the lines of > > if f$filename is not empty, > > local fname = fmt(outputdir, f$filename, ext); > > else > > local fname = fmt("outputdir", f$source, f$id, ext); > > > > > > > > On Tue, Mar 1, 2016 at 2:18 PM, Daniel Guerra > wrote: > > > > > https://github.com/Security-Onion-Solutions/securityonion-bro-scripts/blob/master/file-extraction/extract.bro > > > > > >> On 01 Mar 2016, at 18:35, Michael Cochran wrote: > >> > >> I'm trying to find a simple way to rename a carved file back to it's > original file name using bro-script rather than having bash try to rip it > out of the files.log file. I have seen the mime type analyzers on git that > re-add the extension based on known mime types, but I'd rather be able to > immediately identify the original file name as it came across the wire. I > don't need the unique session identifier because by the time I'm using bro > file analysis I already have the individual session pcap isolated. > >> > >> I'm guessing there should be a way to capture the files.log table data > in broscript, match the unique file identifier then rename the file with > that filename string from files.log. > >> _______________________________________________ > >> Bro mailing list > >> bro at bro-ids.org > >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160302/8fa1292e/attachment.html From macochran0 at gmail.com Wed Mar 2 11:40:45 2016 From: macochran0 at gmail.com (Michael Cochran) Date: Wed, 2 Mar 2016 14:40:45 -0500 Subject: [Bro] Renaming carved files In-Reply-To: References: <7B799BD7-6EFB-434A-88BD-9C399FDA90A3@gmail.com> <29487F03-041A-406A-9222-DE3F8089BD91@gmail.com> Message-ID: Disregard last, the correct answer was to not go off on my own and try to use an != "" Also used fmt instead of cat, and removed the unnecessary local statement. Thank you to everyone that lent a hand in this. The correct script (which now works...) @load ./file-extensions module FileExtraction; export { ## Path to store files const path: string = "" &redef; ## Hook to include files in extraction global extract: hook(f: fa_file, meta: fa_metadata); ## Hook to exclude files from extraction global ignore: hook(f: fa_file, meta: fa_metadata); } event file_sniff(f: fa_file, meta: fa_metadata) { if ( meta?$mime_type && !hook FileExtraction::extract(f, meta) ) { if ( !hook FileExtraction::ignore(f, meta) ) return; if ( meta$mime_type in mime_to_ext ) local fext = mime_to_ext[meta$mime_type]; else fext = split_string(meta$mime_type, /\//)[1]; if ( f?$info && f$info?$filename ) local fname = fmt("%s%s-%s", path, f$source, f$info$filename); else fname = fmt("%s%s-%s.%s", path, f$source, f$id, fext); Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]); } } On Wed, Mar 2, 2016 at 2:25 PM, Michael Cochran wrote: > Derek, > > This is nearly spot on. Here's what I have in main.bro from the git link > you provided that almost works, but is missing some sort of syntax, as it's > giving me errors. If I comment out the If/else statement f$info$filename > gives me the content-disposition extracted filename from the protocol. But > I need a check placed in line to see if f$info$filename is empty, it's > empty it should go ahead and try to figure out a mime-type extension. Very > close, and it's probably something very obvious I'm looking over. > > > @load ./file-extensions > > module FileExtraction; > > export { > ## Path to store files > const path: string = "" &redef; > ## Hook to include files in extraction > global extract: hook(f: fa_file, meta: fa_metadata); > ## Hook to exclude files from extraction > global ignore: hook(f: fa_file, meta: fa_metadata); > } > > event file_sniff(f: fa_file, meta: fa_metadata) > { > if ( meta?$mime_type && !hook FileExtraction::extract(f, meta) ) > { > if ( !hook FileExtraction::ignore(f, meta) ) > return; > if ( meta$mime_type in mime_to_ext ) > local fext = mime_to_ext[meta$mime_type]; > else > fext = split_string(meta$mime_type, /\//)[1]; > > if ( f$info$filename != "" ) > local fname = cat("%s%s-%s", path, f$source, > f$info$filename); > else > local fname = cat("%s%s-%s.%s", path, f$source, > f$id, fext); > Files::add_analyzer(f, Files::ANALYZER_EXTRACT, > [$extract_filename=fname]); > } > } > > > > error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, > line 26 and > /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line 28: > already defined (FileExtraction::fname) > error in /opt/bro/share/bro/base/frameworks/files/./main.bro, lines 18-28 > and /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line > 30: incompatible record types (Files::AnalyzerArgs and > [$extract_filename=FileExtraction::fname]) > error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, > line 30 and /opt/bro/share/bro/base/frameworks/files/./main.bro, lines > 18-28: type mismatch ([$extract_filename=FileExtraction::fname] and > Files::AnalyzerArgs) > error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, > lines 29-30: argument type mismatch in function call > (Files::add_analyzer(FileExtraction::f, Files::ANALYZER_EXTRACT, > [$extract_filename=FileExtraction::fname])) > warning in > /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line 30: > expression value ignored (Files::add_analyzer(FileExtraction::f, > Files::ANALYZER_EXTRACT, [$extract_filename=FileExtraction::fname])) > > On Wed, Mar 2, 2016 at 10:51 AM, Derek Ditch > wrote: > >> Michael, >> >> I haven?t tested this other than validate syntax, but I think the logic >> you?re looking for is below. You of course have to add in the dynamic >> extension mapping and maybe make the outputdir configurable w/ an export {} >> block. Basically, you have to check to see if the filename is set. I would >> caution you, that there are many instances where it is not set, however. If >> you?re looking for a more robust file extraction strategy, I would >> recommend [1]. There?s some additional overhead in moving files around, but >> it allows you to store files by hash once extraction is complete. This >> should greatly reduce your disk usage and processing overhead of any follow >> on processing. >> >> >> event file_sniff(f: fa_file, meta: fa_metadata) >> { >> local fname = ""; >> local outputdir = "/data/bro/extracted_files/"; >> local ext = ".out"; >> >> # .. logic here to generate ext (with starting .) and outputdir (with >> ending /) >> if ( f?$info && f$info?$filename ) >> fname = cat(outputdir, f$info$filename, ext); >> else >> fname = cat(outputdir, f$source, f$id, ext); >> >> Files::add_analyzer(f, Files::ANALYZER_EXTRACT, >> [$extract_filename=fname]); >> } >> >> [1] https://github.com/hosom/bro-file-extraction >> ? >> Derek Ditch >> derek.ditch at gmail.com >> GPG: 0x2543A3B5 >> >> > On 02Mar 2016, at 09:15, Michael Cochran wrote: >> > >> > So the problem I'm running into with this extraction script is here >> (I've already got a script that handles the extracted metadata mime types): >> > >> > local fname = fmt("/nsm/bro/extracted/%s-%s.%s", f$source, f$id, ext); >> > >> > I don't need f$source or f$id in the filename. What I'm searching for >> is being generated here in main.bro. I just need a way to grab this >> information and add it to the extract.bro script to rename extracted file. >> > >> > >> https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html#type-Files::Info >> > Files::Info >> > filename: string &log &optional >> > A filename for the file if one is available from the source for the >> file. These will frequently come from ?Content-Disposition? headers in >> network protocols >> > >> > The logic (forgive my terrible syntax) should be along the lines of >> > if f$filename is not empty, >> > local fname = fmt(outputdir, f$filename, ext); >> > else >> > local fname = fmt("outputdir", f$source, f$id, ext); >> > >> > >> > >> > On Tue, Mar 1, 2016 at 2:18 PM, Daniel Guerra < >> daniel.guerra69 at gmail.com> wrote: >> > >> > >> https://github.com/Security-Onion-Solutions/securityonion-bro-scripts/blob/master/file-extraction/extract.bro >> > >> > >> >> On 01 Mar 2016, at 18:35, Michael Cochran >> wrote: >> >> >> >> I'm trying to find a simple way to rename a carved file back to it's >> original file name using bro-script rather than having bash try to rip it >> out of the files.log file. I have seen the mime type analyzers on git that >> re-add the extension based on known mime types, but I'd rather be able to >> immediately identify the original file name as it came across the wire. I >> don't need the unique session identifier because by the time I'm using bro >> file analysis I already have the individual session pcap isolated. >> >> >> >> I'm guessing there should be a way to capture the files.log table data >> in broscript, match the unique file identifier then rename the file with >> that filename string from files.log. >> >> _______________________________________________ >> >> Bro mailing list >> >> bro at bro-ids.org >> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > >> > >> > _______________________________________________ >> > Bro mailing list >> > bro at bro-ids.org >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160302/ab724d8c/attachment-0001.html From chrisoname at gmail.com Wed Mar 2 13:01:12 2016 From: chrisoname at gmail.com (Christopher De Jesus) Date: Wed, 2 Mar 2016 17:01:12 -0400 Subject: [Bro] Bro Cluster using Vagrant Issues Message-ID: Hello, my name is Christopher and I?m doing undergrad research using Bro to make an IDS for one of the servers of my mentor. One of my first approaches was to start doing this video: https://www.youtube.com/watch?v=it7SZli61ZM in which it setups a cluster using Vagrant. I?ve been struggling for at least 5 days trying to make it work. I?ve changed permissions, ssh-keys, etc. The thing is that every time is use ?broctl? and do ?install? or ?deploy? on it.. it says this: Permission denied (publickey,password). Error: cannot create (some of the) directories /nsm/bro,/nsm/bro/logs,/nsm/bro/spool,/nsm/bro/spool/tmp on node worker-1 I used this site to install bro on Ubuntu of Vagrant: http://knowm.org/how-to-install-bro-network-security-monitor-on-ubuntu/ So yeah, apparently my host doesn?t have control over its slaves even tho? I could ping to it. I used 10.2.2.10 for the host and 10.2.2.11 for the slave in the Vagrantfile. Any other information that you guys need, let me know. This is pretty much the overview of what i?ve done. Thank you in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160302/643cf9e7/attachment.html From lists at g-clef.net Wed Mar 2 13:42:12 2016 From: lists at g-clef.net (Aaron Gee-Clough) Date: Wed, 02 Mar 2016 16:42:12 -0500 Subject: [Bro] How use logs-to-elasticsearch.bro References: <000d01d17387$0c4f2e30$24ed8a90$@126.com> <9757F21F-FBD3-4E6C-B0BF-C1C11DC06AE0@icir.org> <72B89271-EBC0-42A1-9ECA-B91AAE5AF1CC@criticalstack.com> Message-ID: <56D75E34.1070109@g-clef.net> Hi, all. (sorry for missing this conversation yesterday) I'm the author of that Bro Kafka logging plugin ( https://github.com/g-clef/KafkaLogger ). If folks have any questions about it or issues with it, please let me know. (Happy to fix bugs if people hit them.) The way I'm using it right now is that bro logs to a Kafka topic, then Logstash pulls the events off Kafka for insertion into Elasticsearch. That's working quite well for me at the moment (several thousand events per second going to Kafka with no noticeable impact to bro or kafka). Aaron On 03/01/2016 05:53 PM, Derek Ditch wrote: > I would also add, that I use ELK almost exclusively for Bro logs, but > I go through a Kafka output plugin. There?s an easy setup using Chef > to automate for a simple test environment over at http://rocknsm.io/. > > Disclaimer, I?m one of the authors of that open source project. > ? > Derek Ditch > dcode at rocknsm.io > GPG: 0x2543A3B5 > >> On 01Mar 2016, at 13:38, Tim Desrochers > > wrote: >> >> I use bro with ELK in production and it works great. I use bro to >> json and all my logs are in json. Then use logstash to pick up the >> logs and the good folks at elastic have created a plugin for de_dot. >> It's not perfect but with some mutates it works fine for the time >> being. Kibana is a fine interface to build dashboards and query the >> data. >> >> Bro and ELK integration works great with a little tweaking. I'm happy >> to share come configs if you're interested. >> >> On Mar 1, 2016 11:31, "Michael Shirk" > > wrote: >> >> I am happy this came up, as I have been going through the same >> issues for testing Brownian vs. ELK with Bro filters >> >> If it is not supported in Bro's JSON output, it would be nice to >> be able to configure it, as there may already be some parsing of >> the default JSON output of Bro with tools like Splunk. >> >> -- >> Michael Shirk >> Daemon Security, Inc. >> http://www.daemon-security.com >> >> On Mar 1, 2016 11:06, "Seth Hall" > > wrote: >> >> >> > On Mar 1, 2016, at 3:18 AM, Daniel Guerra >> > > wrote: >> > >> > There is a problem with elasticsearch 2.0 and higher. >> > It doesn?t accept dots in field names and there are >> > some timestamp issues. >> >> I know this discussion has been going on for a while and >> unfortunately I've been a bit behind the curve on keeping up >> with it closely. As someone who seems to have been coping >> with this problem for a while, what do you recommend? Would >> it be best if we could do nested json documents in the json >> output? i.e.... >> >> {"ts":1223421341234.1234, "id": {"orig_h": "1.2.3.4", >> "orig_p":1234.......etc }} >> >> .Seth >> >> -- >> Seth Hall >> International Computer Science Institute >> (Bro) because everyone has a network >> http://www.bro.org/ >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From cbarbaro at cert.unlp.edu.ar Wed Mar 2 16:09:18 2016 From: cbarbaro at cert.unlp.edu.ar (Cristian Daniel Barbaro) Date: Wed, 2 Mar 2016 21:09:18 -0300 Subject: [Bro] Bro Cluster using Vagrant Issues In-Reply-To: References: Message-ID: <56D780AE.1070607@cert.unlp.edu.ar> Do you add ssh key of master in authorized_keys file in workers into .ssh folder? Maybe here is the problem. Sorry for my english. El 02/03/16 a las 18:01, Christopher De Jesus escribi?: > > Hello, my name is Christopher and I?m doing undergrad research using > Bro to make an IDS for one of the servers of my mentor. > > One of my first approaches was to start doing this > video: https://www.youtube.com/watch?v=it7SZli61ZM > > in which it setups a cluster using Vagrant. I?ve been struggling for > at least 5 days trying to make it work. I?ve changed > permissions, ssh-keys, etc. > > The thing is that every time is use ?broctl? and do ?install? > or ?deploy? on it.. it says this: > > > Permission denied (publickey,password). > > Error: cannot create (some of the) directories > /nsm/bro,/nsm/bro/logs,/nsm/bro/spool,/nsm/bro/spool/tmp on node worker-1 > > > I used this site to install bro on Ubuntu of Vagrant: > > http://knowm.org/how-to-install-bro-network-security-monitor-on-ubuntu/ > > > So yeah, apparently my host doesn?t have control over its slaves even > tho? I could ping to it. I used 10.2.2.10 for the host and 10.2.2.11 > for the slave in the Vagrantfile. > > > > Any other information that you guys need, let me know. This is pretty > much the overview of what i?ve done. Thank you in advance. > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Cristian Daniel Barbaro CERTUNLP -- -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160302/0960393f/attachment.html From cbarbaro at cert.unlp.edu.ar Wed Mar 2 16:09:51 2016 From: cbarbaro at cert.unlp.edu.ar (Cristian Daniel Barbaro) Date: Wed, 2 Mar 2016 21:09:51 -0300 Subject: [Bro] Bro Cluster using Vagrant Issues In-Reply-To: References: Message-ID: <56D780CF.1030408@cert.unlp.edu.ar> Do you add ssh key of master in authorized_keys file in workers into .ssh folder? Maybe here is the problem. Sorry for my english. El 02/03/16 a las 18:01, Christopher De Jesus escribi?: > > Hello, my name is Christopher and I?m doing undergrad research using > Bro to make an IDS for one of the servers of my mentor. > > One of my first approaches was to start doing this > video: https://www.youtube.com/watch?v=it7SZli61ZM > > in which it setups a cluster using Vagrant. I?ve been struggling for > at least 5 days trying to make it work. I?ve changed > permissions, ssh-keys, etc. > > The thing is that every time is use ?broctl? and do ?install? > or ?deploy? on it.. it says this: > > > Permission denied (publickey,password). > > Error: cannot create (some of the) directories > /nsm/bro,/nsm/bro/logs,/nsm/bro/spool,/nsm/bro/spool/tmp on node worker-1 > > > I used this site to install bro on Ubuntu of Vagrant: > > http://knowm.org/how-to-install-bro-network-security-monitor-on-ubuntu/ > > > So yeah, apparently my host doesn?t have control over its slaves even > tho? I could ping to it. I used 10.2.2.10 for the host and 10.2.2.11 > for the slave in the Vagrantfile. > > > > Any other information that you guys need, let me know. This is pretty > much the overview of what i?ve done. Thank you in advance. > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Cristian Daniel Barbaro CERTUNLP -- -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160302/755c0599/attachment.html From seth at icir.org Wed Mar 2 21:21:11 2016 From: seth at icir.org (Seth Hall) Date: Thu, 3 Mar 2016 00:21:11 -0500 Subject: [Bro] Renaming carved files In-Reply-To: References: Message-ID: <4F710A0A-5714-4816-AB41-AE1D20BA24D4@icir.org> > On Mar 1, 2016, at 12:35 PM, Michael Cochran wrote: > > I'm trying to find a simple way to rename a carved file back to it's original file name using bro-script rather than having bash try to rip it out of the files.log file. I actually had this fully implemented a long time ago (naming files as they were named on the wire), but then I ripped it all out because it gave attackers the ability to control files being written on your file system. FireEye just got caught doing nearly this same thing recently and it turned out to be an evasion for them. I generally would not recommend going down the path of letting attackers control file names on your disk because you're likely to open a much larger hole than an evasion if you aren't extremely careful.  I am curious why you would like to do that though? Is it purely for convenience when you are doing analysis? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From macochran0 at gmail.com Thu Mar 3 05:32:58 2016 From: macochran0 at gmail.com (Michael Cochran) Date: Thu, 3 Mar 2016 08:32:58 -0500 Subject: [Bro] Renaming carved files In-Reply-To: <4F710A0A-5714-4816-AB41-AE1D20BA24D4@icir.org> References: <4F710A0A-5714-4816-AB41-AE1D20BA24D4@icir.org> Message-ID: This is pretty common practice among forensic network analysis tools. The page preview function is one of the reasons Netwitness is so popular with analysts. Dangerous as well, it will attempt to render entire pages of HTTP based off of carved files. I've recommended the analysts just look in files.log if they want to see the original file name. From my perspective, the best solution is the mime type file analysis. To take it a step further a simple check to see if the mime type matches the file extension seen in the content-disposition header. On Thu, Mar 3, 2016 at 12:21 AM, Seth Hall wrote: > > > On Mar 1, 2016, at 12:35 PM, Michael Cochran > wrote: > > > > I'm trying to find a simple way to rename a carved file back to it's > original file name using bro-script rather than having bash try to rip it > out of the files.log file. > > I actually had this fully implemented a long time ago (naming files as > they were named on the wire), but then I ripped it all out because it gave > attackers the ability to control files being written on your file system. > FireEye just got caught doing nearly this same thing recently and it turned > out to be an evasion for them. I generally would not recommend going down > the path of letting attackers control file names on your disk because > you're likely to open a much larger hole than an evasion if you aren't > extremely careful. > > I am curious why you would like to do that though? Is it purely for > convenience when you are doing analysis? > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160303/c5bcf00c/attachment.html From seth at icir.org Thu Mar 3 06:54:53 2016 From: seth at icir.org (Seth Hall) Date: Thu, 3 Mar 2016 09:54:53 -0500 Subject: [Bro] Renaming carved files In-Reply-To: References: <4F710A0A-5714-4816-AB41-AE1D20BA24D4@icir.org> Message-ID: <5383627E-D740-44FE-8C7F-80FB5C08DA72@icir.org> > On Mar 3, 2016, at 8:32 AM, Michael Cochran wrote: > > This is pretty common practice among forensic network analysis tools. The page preview function is one of the reasons Netwitness is so popular with analysts. Dangerous as well, it will attempt to render entire pages of HTTP based off of carved files. I've recommended the analysts just look in files.log if they want to see the original file name. I've never used netwitness, but wow. I suppose you're saying that you need the files named as they were on the remote server so the page display works? I would expect more html/css munging to be required even with the files named in the same way though, so you might as well just name the files in another way. :) > From my perspective, the best solution is the mime type file analysis. To take it a step further a simple check to see if the mime type matches the file extension seen in the content-disposition header. I'd be curious to see how many files don't match their declared mime types, I bet a lot. I thought about writing a script to do this once, but then stopped myself because at the very least, there are lots of favicon files that are jpegs and gifs, but the remote server even declares in the header that it's actually an icon file (since servers typically just base on the file extension). I would still be interested to see what people's experiences are if anyone ever takes it on though (i.e., does it catch anything worth following). Thanks, .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From pawelec93 at googlemail.com Thu Mar 3 07:35:53 2016 From: pawelec93 at googlemail.com (=?UTF-8?Q?Pawe=C5=82_Piszczatowski?=) Date: Thu, 3 Mar 2016 15:35:53 +0000 Subject: [Bro] Monitoring traffic on VPC Message-ID: I've got a cluster set up in the cloud with a Master and two workers all in separate VPC. They are talking using VPN and I can see the traffic from the workers in the the master. What I'm trying to do is to have the worker monitor the whole VPC as there will be other VMs such as honeypots etc. I have tried port forwarding (forwarding all the traffic from the other instances into the bro worker) however with no luck as AWS doesn't allow port forwarding apparently. My question is can Bro monitor whole subnets? Or is there a better solution to monitor all of the traffic in a VPC? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160303/d65fd2a6/attachment.html From chrisoname at gmail.com Thu Mar 3 08:35:31 2016 From: chrisoname at gmail.com (Christopher De Jesus) Date: Thu, 3 Mar 2016 12:35:31 -0400 Subject: [Bro] Bro Cluster using Vagrant Issues In-Reply-To: <56D780CF.1030408@cert.unlp.edu.ar> References: <56D780CF.1030408@cert.unlp.edu.ar> Message-ID: Well, that's the thing. I did it. not sure if I did it right, but I did what i know of RSA. I made the sss-keygen, used that same key and copied it into the shared folder and with the slave I took it and put it in the .ssh. Is there another way of doing it? Besides, every time I tried to do 'scp' it asked for the root password of the slave which I think it's a private key. On Wed, Mar 2, 2016 at 8:09 PM, Cristian Daniel Barbaro < cbarbaro at cert.unlp.edu.ar> wrote: > Do you add ssh key of master in authorized_keys file in workers into .ssh > folder? > Maybe here is the problem. > > Sorry for my english. > > El 02/03/16 a las 18:01, Christopher De Jesus escribi?: > > Hello, my name is Christopher and I?m doing undergrad research using Bro > to make an IDS for one of the servers of my mentor. > > One of my first approaches was to start doing this video: > > https://www.youtube.com/watch?v=it7SZli61ZM > > in which it setups a cluster using Vagrant. I?ve been struggling for at > least 5 days trying to make it work. I?ve changed permissions, ssh-keys, > etc. > > The thing is that every time is use ?broctl? and do ?install? or ?deploy? > on it.. it says this: > > > Permission denied (publickey,password). > > Error: cannot create (some of the) directories > /nsm/bro,/nsm/bro/logs,/nsm/bro/spool,/nsm/bro/spool/tmp on node worker-1 > > > I used this site to install bro on Ubuntu of Vagrant: > > http://knowm.org/how-to-install-bro-network-security-monitor-on-ubuntu/ > > > So yeah, apparently my host doesn?t have control over its slaves even > tho? I could ping to it. I used 10.2.2.10 for the host and 10.2.2.11 for > the slave in the Vagrantfile. > > > > Any other information that you guys need, let me know. This is pretty much > the overview of what i?ve done. Thank you in advance. > > > _______________________________________________ > Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > -- > Cristian Daniel Barbaro > CERTUNLP > -- > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160303/f3c74a17/attachment.html From michalpurzynski1 at gmail.com Thu Mar 3 14:28:13 2016 From: michalpurzynski1 at gmail.com (=?utf-8?Q?Micha=C5=82_Purzy=C5=84ski?=) Date: Thu, 3 Mar 2016 23:28:13 +0100 Subject: [Bro] Renaming carved files In-Reply-To: <5383627E-D740-44FE-8C7F-80FB5C08DA72@icir.org> References: <4F710A0A-5714-4816-AB41-AE1D20BA24D4@icir.org> <5383627E-D740-44FE-8C7F-80FB5C08DA72@icir.org> Message-ID: <229AF808-3A67-40B6-88FE-D305CEC1437C@gmail.com> > I'd be curious to see how many files don't match their declared mime types, I bet a lot. I thought about writing a script to do this once, but then stopped myself because at the very least, there are lots of favicon files that are jpegs and gifs, but the remote server even declares in the header that it's actually an icon file (since servers typically just base on the file extension). I would still be interested to see what people's experiences are if anyone ever takes it on though (i.e., does it catch anything worth following). > That's exactly the kind of script I've just written. Will send an update how it behaves in a week or so. It's going to be deployed in several busy offices. > Thanks, > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From justin at justinthomas.name Thu Mar 3 14:33:59 2016 From: justin at justinthomas.name (Justin Thomas) Date: Thu, 3 Mar 2016 14:33:59 -0800 Subject: [Bro] Monitoring traffic on VPC In-Reply-To: References: Message-ID: I tackled this problem in AWS (using Suricata and Bro) by forcing all data through a handful of NAT instances. That allowed me to centralize the data flows and install VTUN and daemonlogger at those points to transfer the network traffic to a few dedicated IDS instances. Amazon's routing makes even this challenging, and I can get in to more detail about that directly if you'd like. There are many downsides to that approach, but it worked reliably for my needs (providing IDS services in AWS and complying with regulations). On Thu, Mar 3, 2016 at 7:35 AM, Pawe? Piszczatowski < pawelec93 at googlemail.com> wrote: > I've got a cluster set up in the cloud with a Master and two workers all > in separate VPC. They are talking using VPN and I can see the traffic from > the workers in the the master. What I'm trying to do is to have the worker > monitor the whole VPC as there will be other VMs such as honeypots etc. > I have tried port forwarding (forwarding all the traffic from the other > instances into the bro worker) however with no luck as AWS doesn't allow > port forwarding apparently. > > My question is can Bro monitor whole subnets? Or is there a better > solution to monitor all of the traffic in a VPC? > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160303/d539c4f8/attachment.html From raj at bivio.net Thu Mar 3 15:50:44 2016 From: raj at bivio.net (Raj Srinivasan) Date: Thu, 3 Mar 2016 23:50:44 +0000 Subject: [Bro] Question on proxy configuration Message-ID: The documentation does mention that workers are shared equally between proxies, but I wanted to make sure I wasn't missing anything before increasing the number of proxies (and workers) at a particular installation. Does the manager assign a worker to a proxy (and/or vice versa) automatically? Is there a way to explicitly define an association between a worker and a proxy? I am assuming that the order of the worker and proxy declarations in the node.cfg file, and the IP addresses of the nodes running the processes/threads don't matter. Thanks! Raj -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160303/e465cd5a/attachment.html From cbarbaro at cert.unlp.edu.ar Fri Mar 4 05:21:04 2016 From: cbarbaro at cert.unlp.edu.ar (Cristian Daniel Barbaro) Date: Fri, 4 Mar 2016 10:21:04 -0300 Subject: [Bro] Bro Cluster using Vagrant Issues In-Reply-To: References: <56D780CF.1030408@cert.unlp.edu.ar> Message-ID: <56D98BC0.4040607@cert.unlp.edu.ar> You could make cat .ssh / id_rsa.pub of master's file and then paste it into .ssh / authorized_keys file of each worker (if not exists, create it). Then you make sure if you can connect via ssh to each worker from master. Everything I do with root (in /root folder). Saludos. El 03/03/16 a las 13:35, Christopher De Jesus escribi?: > Well, that's the thing. I did it. not sure if I did it right, but I > did what i know of RSA. I made the sss-keygen, used that same key and > copied it into the shared folder and with the slave I took it and put > it in the .ssh. Is there another way of doing it? Besides, every time > I tried to do 'scp' it asked for the root password of the slave which > I think it's a private key. > > On Wed, Mar 2, 2016 at 8:09 PM, Cristian Daniel Barbaro > > wrote: > > Do you add ssh key of master in authorized_keys file in workers > into .ssh folder? > Maybe here is the problem. > > Sorry for my english. > > El 02/03/16 a las 18:01, Christopher De Jesus escribi?: >> >> Hello, my name is Christopher and I?m doing undergrad research >> using Bro to make an IDS for one of the servers of my mentor. >> >> One of my first approaches was to start doing this >> video: https://www.youtube.com/watch?v=it7SZli61ZM >> >> in which it setups a cluster using Vagrant. I?ve been struggling >> for at least 5 days trying to make it work. I?ve changed >> permissions, ssh-keys, etc. >> >> The thing is that every time is use ?broctl? and do ?install? >> or ?deploy? on it.. it says this: >> >> >> Permission denied (publickey,password). >> >> Error: cannot create (some of the) directories >> /nsm/bro,/nsm/bro/logs,/nsm/bro/spool,/nsm/bro/spool/tmp on node >> worker-1 >> >> >> I used this site to install bro on Ubuntu of Vagrant: >> >> http://knowm.org/how-to-install-bro-network-security-monitor-on-ubuntu/ >> >> >> So yeah, apparently my host doesn?t have control over its slaves >> even tho? I could ping to it. I used 10.2.2.10 for the host and >> 10.2.2.11 for the slave in the Vagrantfile. >> >> >> >> Any other information that you guys need, let me know. This is >> pretty much the overview of what i?ve done. Thank you in advance. >> >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -- > Cristian Daniel Barbaro > CERTUNLP > -- > > -- Cristian Daniel Barbaro CERTUNLP -- -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160304/65db14b4/attachment-0001.html From valerio.click at gmx.com Sun Mar 6 11:01:26 2016 From: valerio.click at gmx.com (Valerio) Date: Sun, 6 Mar 2016 20:01:26 +0100 Subject: [Bro] SIP Analyzer and SDP payload Message-ID: <56DC7E86.4040406@gmx.com> Hi all, I am exploring how the SIP protocol analyzer works and it seems that SIP analyzer does not extract SDP payload out of SIP messages. I am trying to extend the SIP analyzer to extract such info and print it in an additional field of the sip.log file. Unfortunately, I am not able to see where the boolean variable "sip_reply" accessed in sip-analyzer.pac:37 is actually defined and set. The same holds for sip_request (sip-analyzer.pac:22), sip_header (sip-analyzer.pac:53). Can someone point me where and how these variables are populated? best regards, Valerio From grahambridgeland at yahoo.co.uk Mon Mar 7 14:56:33 2016 From: grahambridgeland at yahoo.co.uk (Graham Bridgeland) Date: Mon, 7 Mar 2016 22:56:33 +0000 (UTC) Subject: [Bro] Port Scanning Detection advice References: <1234132513.2966070.1457391393031.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <1234132513.2966070.1457391393031.JavaMail.yahoo@mail.yahoo.com> Hello Wondering if anyone could shed some light on the best way to handle port scanning tasks within Bro. I'm particularly interested in creating a basic script to react when a threshold is met i.e. when X attacks are detected within a Y time window. Courting the attacks is fine but its how to relate to the time window I'm stuck on. With a start and end time I can create a duration but as time is continuous I don't know the best method to decide when to start and when to stop. I'm studying the scan.bro from the \misc folder but can't work out how it handles this time-window dilemma. Are there basic notes on these scripts other than the comments with them? Not sure if anyone can help but thought I'd ask. ThanksGraham -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160307/501fea9d/attachment.html From mahimjamal360 at hotmail.com Mon Mar 7 15:49:32 2016 From: mahimjamal360 at hotmail.com (Jamal Tarik) Date: Mon, 7 Mar 2016 18:49:32 -0500 Subject: [Bro] Port Scanning Detection advice In-Reply-To: <1234132513.2966070.1457391393031.JavaMail.yahoo@mail.yahoo.com> References: <1234132513.2966070.1457391393031.JavaMail.yahoo.ref@mail.yahoo.com>, <1234132513.2966070.1457391393031.JavaMail.yahoo@mail.yahoo.com> Message-ID: Are you aware of this law computer FRAUD and abuse ACT.? Updated you earlier in regards to this item also there are some new things added to it be advised further. Thanks again for all the best regards. To address ING this FOOLISH for OF misinformation whichever angle they try to create is covered should be resolved as soon as possible to get them wrong bits out of the way. Date: Mon, 7 Mar 2016 22:56:33 +0000 From: grahambridgeland at yahoo.co.uk To: bro at bro.org Subject: [Bro] Port Scanning Detection advice Hello Wondering if anyone could shed some light on the best way to handle port scanning tasks within Bro. I'm particularly interested in creating a basic script to react when a threshold is met i.e. when X attacks are detected within a Y time window. Courting the attacks is fine but its how to relate to the time window I'm stuck on. With a start and end time I can create a duration but as time is continuous I don't know the best method to decide when to start and when to stop. I'm studying the scan.bro from the \misc folder but can't work out how it handles this time-window dilemma. Are there basic notes on these scripts other than the comments with them? Not sure if anyone can help but thought I'd ask. ThanksGraham _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160307/92ff0a6d/attachment.html From belongtorobby at gmail.com Mon Mar 7 16:12:28 2016 From: belongtorobby at gmail.com (Lizzie Chandler) Date: Mon, 7 Mar 2016 18:12:28 -0600 Subject: [Bro] Port Scanning Detection advice In-Reply-To: <1234132513.2966070.1457391393031.JavaMail.yahoo@mail.yahoo.com> References: <1234132513.2966070.1457391393031.JavaMail.yahoo.ref@mail.yahoo.com> <1234132513.2966070.1457391393031.JavaMail.yahoo@mail.yahoo.com> Message-ID: I saw the original question sent in, and I am / was interested in the same. The given response has left me more than a not befuddled. Clarification? On Mar 7, 2016 5:04 PM, "Graham Bridgeland" wrote: > Hello > > Wondering if anyone could shed some light on the best way to handle port > scanning tasks within Bro. I'm particularly interested in creating a basic > script to react when a threshold is met i.e. when X attacks are detected > within a Y time window. Courting the attacks is fine but its how to relate > to the time window I'm stuck on. With a start and end time I can create a > duration but as time is continuous I don't know the best method to decide > when to start and when to stop. > > I'm studying the scan.bro from the \misc folder but can't work out how it > handles this time-window dilemma. Are there basic notes on these scripts > other than the comments with them? Not sure if anyone can help but thought > I'd ask. > > Thanks > Graham > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160307/e349586d/attachment.html From belongtorobby at gmail.com Mon Mar 7 16:13:21 2016 From: belongtorobby at gmail.com (Lizzie Chandler) Date: Mon, 7 Mar 2016 18:13:21 -0600 Subject: [Bro] Port Scanning Detection advice In-Reply-To: References: <1234132513.2966070.1457391393031.JavaMail.yahoo.ref@mail.yahoo.com> <1234132513.2966070.1457391393031.JavaMail.yahoo@mail.yahoo.com> Message-ID: I saw the original question sent in, and I am / was interested in the same. The given response has left me more than a bit befuddled. Clarification? On Mar 7, 2016 6:12 PM, "Lizzie Chandler" wrote: > I saw the original question sent in, and I am / was interested in the same. > > The given response has left me more than a not befuddled. > > Clarification? > On Mar 7, 2016 5:04 PM, "Graham Bridgeland" > wrote: > >> Hello >> >> Wondering if anyone could shed some light on the best way to handle port >> scanning tasks within Bro. I'm particularly interested in creating a basic >> script to react when a threshold is met i.e. when X attacks are detected >> within a Y time window. Courting the attacks is fine but its how to relate >> to the time window I'm stuck on. With a start and end time I can create a >> duration but as time is continuous I don't know the best method to decide >> when to start and when to stop. >> >> I'm studying the scan.bro from the \misc folder but can't work out how it >> handles this time-window dilemma. Are there basic notes on these scripts >> other than the comments with them? Not sure if anyone can help but thought >> I'd ask. >> >> Thanks >> Graham >> >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160307/75403ff4/attachment.html From raj at bivio.net Mon Mar 7 16:36:39 2016 From: raj at bivio.net (Raj Srinivasan) Date: Tue, 8 Mar 2016 00:36:39 +0000 Subject: [Bro] Question on proxy configuration In-Reply-To: References: Message-ID: Figured it out, no need for any responses! Everything works as expected. Thanks, Raj From: Raj Srinivasan Sent: Thursday, March 03, 2016 3:51 PM To: bro at bro.org Subject: Question on proxy configuration The documentation does mention that workers are shared equally between proxies, but I wanted to make sure I wasn't missing anything before increasing the number of proxies (and workers) at a particular installation. Does the manager assign a worker to a proxy (and/or vice versa) automatically? Is there a way to explicitly define an association between a worker and a proxy? I am assuming that the order of the worker and proxy declarations in the node.cfg file, and the IP addresses of the nodes running the processes/threads don't matter. Thanks! Raj -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160308/5aeed1c7/attachment-0001.html From johanna at icir.org Mon Mar 7 17:08:15 2016 From: johanna at icir.org (Johanna Amann) Date: Mon, 7 Mar 2016 17:08:15 -0800 Subject: [Bro] SIP Analyzer and SDP payload In-Reply-To: <56DC7E86.4040406@gmx.com> References: <56DC7E86.4040406@gmx.com> Message-ID: <20160308010815.GA84243@wifi110.sys.ICSI.Berkeley.EDU> Hello Valerio, > Unfortunately, I am not able to see where the boolean variable > "sip_reply" accessed in sip-analyzer.pac:37 is actually defined and set. > The same holds for sip_request (sip-analyzer.pac:22), sip_header > (sip-analyzer.pac:53). > Can someone point me where and how these variables are populated? That is a tad tricky - the variables are populated in autogenerated code. The easiest way to figure out whate exactly happens might be to look into the code that is generated in the build/ directory after building bro is done. The in-a-nutshel variant is - since sip_reply is defined as an event, a EventHandlerPtr variable named sip_reply is created. This variable is populated automatically (by a call to the function internal_handler). Performing an if (sip_reply) call will return true if the sip_reply event is used somewhere in Bro scriptland and false otherwise. Johanna From johanna at icir.org Mon Mar 7 17:13:00 2016 From: johanna at icir.org (Johanna Amann) Date: Mon, 7 Mar 2016 17:13:00 -0800 Subject: [Bro] Port Scanning Detection advice In-Reply-To: <1234132513.2966070.1457391393031.JavaMail.yahoo@mail.yahoo.com> References: <1234132513.2966070.1457391393031.JavaMail.yahoo.ref@mail.yahoo.com> <1234132513.2966070.1457391393031.JavaMail.yahoo@mail.yahoo.com> Message-ID: <20160308011300.GB84243@wifi110.sys.ICSI.Berkeley.EDU> Hello Graham, On Mon, Mar 07, 2016 at 10:56:33PM +0000, Graham Bridgeland wrote: > Wondering if anyone could shed some light on the best way to handle port > scanning tasks within Bro. I'm particularly interested in creating a > basic script to react when a threshold is met i.e. when X attacks are > detected within a Y time window. Courting the attacks is fine but its > how to relate to the time window I'm stuck on. With a start and end time > I can create a duration but as time is continuous I don't know the best > method to decide when to start and when to stop. > I'm studying the scan.bro from the \misc folder but can't work out how > it handles this time-window dilemma. Are there basic notes on these > scripts other than the comments with them? Not sure if anyone can help > but thought I'd ask. The way that this is done in Bro currently is quite basic. The short answer is - we don't handle the time-window dilemma. The long answer is - scan.bro uses the Summary Statistics Framework (https://www.bro.org/sphinx/frameworks/sumstats.html). SumStats allows you to easily count things that are going on and set thresholds, etc. At the moment, these thresholds are epoch-based - you give SumStats a period of time during which you want to check the thresholds (e.g., one hour, a day, etc). If a threshold is reached during that time period, the callback function is called. After an epoch passed, all counters are reset to zero and counting starts from the beginning. There is currently no additional handling of time windows. So - currently the decision on where exactly these windows are is based on the startup time of Bro Johanna From umuta at sabanciuniv.edu Mon Mar 7 22:53:54 2016 From: umuta at sabanciuniv.edu (Umut Arus) Date: Tue, 8 Mar 2016 08:53:54 +0200 Subject: [Bro] Bad DNS Detection Message-ID: Hi, I'm setting up bro IDS recently. I will listen DNS traffic by span port but I wonder, how can I detect malwares and victim clients that is used bad DNS in network? thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160308/993dfa22/attachment.html From jazoff at illinois.edu Tue Mar 8 05:09:12 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Tue, 8 Mar 2016 13:09:12 +0000 Subject: [Bro] Bad DNS Detection In-Reply-To: References: Message-ID: <41018FFF-FE82-486E-B813-46DE93E4F0A7@illinois.edu> This script that I wrote a while ago may help: It creates an external_dns.log file (which is just dns.log that has been pre-filtered for you) as well as raising notices when it detects clients using external dns servers. -- - Justin Azoff > On Mar 8, 2016, at 12:53 AM, Umut Arus wrote: > > Hi, > > I'm setting up bro IDS recently. I will listen DNS traffic by span port but I wonder, how can I detect malwares and victim clients that is used bad DNS in network? > > thanks. > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160308/cfef4ebc/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: external-dns.bro Type: application/octet-stream Size: 2738 bytes Desc: external-dns.bro Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160308/cfef4ebc/attachment.obj From umuta at sabanciuniv.edu Tue Mar 8 05:18:49 2016 From: umuta at sabanciuniv.edu (Umut Arus) Date: Tue, 8 Mar 2016 15:18:49 +0200 Subject: [Bro] Bad DNS Detection In-Reply-To: <41018FFF-FE82-486E-B813-46DE93E4F0A7@illinois.edu> References: <41018FFF-FE82-486E-B813-46DE93E4F0A7@illinois.edu> Message-ID: Hi Justin, Thanks but I need a code or configuration that is query the malware dns/ip sources that is trying to connect and raising notices. Or how do you realise in your network malwared DDoS clients with the Bro? thanks.. On Tue, Mar 8, 2016 at 3:09 PM, Azoff, Justin S wrote: > This script that I wrote a while ago may help: > > > > > It creates an external_dns.log file (which is just dns.log that has been > pre-filtered for you) as well as raising notices when it detects clients > using external dns servers. > > > -- > - Justin Azoff > > > > On Mar 8, 2016, at 12:53 AM, Umut Arus wrote: > > > > Hi, > > > > I'm setting up bro IDS recently. I will listen DNS traffic by span port > but I wonder, how can I detect malwares and victim clients that is used bad > DNS in network? > > > > thanks. > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160308/6afbf6de/attachment.html From jlay at slave-tothe-box.net Tue Mar 8 05:57:20 2016 From: jlay at slave-tothe-box.net (James Lay) Date: Tue, 08 Mar 2016 06:57:20 -0700 Subject: [Bro] Bad DNS Detection In-Reply-To: References: <41018FFF-FE82-486E-B813-46DE93E4F0A7@illinois.edu> Message-ID: <1457445440.2847.1.camel@gamebox> This will get you there: https://intel.criticalstack.com/ also, not bro related, but graphically shows what you're looking for: https://github.com/stamparm/maltrail James On Tue, 2016-03-08 at 15:18 +0200, Umut Arus wrote: > Hi Justin, > > > > Thanks but I need a code or configuration that is query the malware > dns/ip sources that is trying to connect and raising notices. > > > Or how do you realise in your network malwared DDoS clients with the > Bro? > > > thanks.. > > > > > On Tue, Mar 8, 2016 at 3:09 PM, Azoff, Justin S > wrote: > > This script that I wrote a while ago may help: > > > > > > It creates an external_dns.log file (which is just dns.log > that has been pre-filtered for you) as well as raising notices > when it detects clients using external dns servers. > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160308/f0e0f0ea/attachment-0001.html From inetjunkmail at gmail.com Tue Mar 8 07:27:06 2016 From: inetjunkmail at gmail.com (inetjunkmail) Date: Tue, 8 Mar 2016 10:27:06 -0500 Subject: [Bro] Myricom help Message-ID: We're running Bro version 2.4.1 on CENTOS 7 with Myricom driver version 3.0.6. When we run broctl capstats, we get the error below. ------------------------------------------------------------ [bro at d2 ~]$ /usr/local/bro/bin/broctl capstats cannot get list of local IP addresses Interface kpps mbps (10s average) ---------------------------------------- d2-w-1: capstats failed (error: p4p1: snf_ring_open_id(ring=-1) failed: Device or resource busy) d3-w-1: capstats failed (error: p4p1: snf_ring_open_id(ring=-1) failed: Device or resource busy) d4-w-1: capstats failed (error: p4p1: snf_ring_open_id(ring=-1) failed: Device or resource busy) ------------------------------------------------------------ My worker config looks like this: ------------------------------------------------------------ [d2-w] type=worker host=d2 interface=p4p1 lb_method=myricom lb_procs=14 pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 [d3-w] type=worker host=d3 interface=p4p1 lb_method=myricom lb_procs=14 pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 [d4-w] type=worker host=d4 interface=p4p1 lb_method=myricom lb_procs=14 pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 ------------------------------------------------------------ Can anyone provide some direction on how to get capstats working? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160308/6a43c296/attachment.html From hhoffman at ip-solutions.net Tue Mar 8 07:42:18 2016 From: hhoffman at ip-solutions.net (Harry Hoffman) Date: Tue, 08 Mar 2016 10:42:18 -0500 Subject: [Bro] Myricom help In-Reply-To: Message-ID: <699c754a-c6cb-4bfc-972b-e5720ffe1937@email.android.com> An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160308/bd3a5c37/attachment.html From latt0050 at umn.edu Tue Mar 8 08:24:18 2016 From: latt0050 at umn.edu (Brandon Lattin) Date: Tue, 8 Mar 2016 10:24:18 -0600 Subject: [Bro] Myricom help In-Reply-To: References: Message-ID: Ensure you have the Sniffer10G license & drivers installed. Compile Bro with Myricom support (./configure --with-pcap=/opt/snf/). Use interfaces 'snf0' and 'snf1'. On Tue, Mar 8, 2016 at 9:27 AM, inetjunkmail wrote: > We're running Bro version 2.4.1 on CENTOS 7 with Myricom driver version > 3.0.6. When we run broctl capstats, we get the error below. > > ------------------------------------------------------------ > [bro at d2 ~]$ /usr/local/bro/bin/broctl capstats > > cannot get list of local IP addresses > > Interface kpps mbps (10s average) > ---------------------------------------- > d2-w-1: capstats failed (error: p4p1: snf_ring_open_id(ring=-1) failed: > Device or resource busy) > > d3-w-1: capstats failed (error: p4p1: snf_ring_open_id(ring=-1) failed: > Device or resource busy) > > d4-w-1: capstats failed (error: p4p1: snf_ring_open_id(ring=-1) failed: > Device or resource busy) > ------------------------------------------------------------ > > My worker config looks like this: > > ------------------------------------------------------------ > [d2-w] > type=worker > host=d2 > interface=p4p1 > lb_method=myricom > lb_procs=14 > pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 > > [d3-w] > type=worker > host=d3 > interface=p4p1 > lb_method=myricom > lb_procs=14 > pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 > > [d4-w] > type=worker > host=d4 > interface=p4p1 > lb_method=myricom > lb_procs=14 > pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 > ------------------------------------------------------------ > > Can anyone provide some direction on how to get capstats working? > > Thanks > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160308/7b4d5f6b/attachment.html From inetjunkmail at gmail.com Tue Mar 8 12:38:59 2016 From: inetjunkmail at gmail.com (inetjunkmail) Date: Tue, 8 Mar 2016 15:38:59 -0500 Subject: [Bro] Myricom help In-Reply-To: <699c754a-c6cb-4bfc-972b-e5720ffe1937@email.android.com> References: <699c754a-c6cb-4bfc-972b-e5720ffe1937@email.android.com> Message-ID: Thanks Harry, that was it. I made the following changes in case anyone else runs into it. I modified my workers config to add the SNF_APP_ID: ------------------------------------------------------------ [d2-w] type=worker host=d2 interface=p4p1 lb_method=myricom lb_procs=14 pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 env_vars=SNF_APP_ID=1 [d3-w] type=worker host=d3 interface=p4p1 lb_method=myricom lb_procs=14 pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 env_vars=SNF_APP_ID=1 [d4-w] type=worker host=d4 interface=p4p1 lb_method=myricom lb_procs=14 pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 env_vars=SNF_APP_ID=1 ------------------------------------------------------------ Fortunately, this only seems to be used when starting services but is ignored with capstats. To get capstats to work, I added and export SNF_APP_ID=2 to /etc/bashrc so all users inherit this by default (including the bro user). So now, when bro is started it uses the env_vars and runs bro with SNF_APP_ID=1 and when capstats runs, the "normal" environment variables are present and it runs with SNF_APP_ID=2. Another benefit is that if a user is logged in and runs tcpdump, it too inherits the SNF_APP_ID of 2. Thanks for your help! On Tue, Mar 8, 2016 at 10:42 AM, Harry Hoffman wrote: > Do you need to set the APP_ID (probably not the real var name) to allow > multiple apps to read from the cards? > > Cheers, > Harry > On Mar 8, 2016 10:27 AM, inetjunkmail wrote: > > We're running Bro version 2.4.1 on CENTOS 7 with Myricom driver version > 3.0.6. When we run broctl capstats, we get the error below. > > ------------------------------------------------------------ > [bro at d2 ~]$ /usr/local/bro/bin/broctl capstats > > cannot get list of local IP addresses > > Interface kpps mbps (10s average) > ---------------------------------------- > d2-w-1: capstats failed (error: p4p1: snf_ring_open_id(ring=-1) failed: > Device or resource busy) > > d3-w-1: capstats failed (error: p4p1: snf_ring_open_id(ring=-1) failed: > Device or resource busy) > > d4-w-1: capstats failed (error: p4p1: snf_ring_open_id(ring=-1) failed: > Device or resource busy) > ------------------------------------------------------------ > > My worker config looks like this: > > ------------------------------------------------------------ > [d2-w] > type=worker > host=d2 > interface=p4p1 > lb_method=myricom > lb_procs=14 > pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 > > [d3-w] > type=worker > host=d3 > interface=p4p1 > lb_method=myricom > lb_procs=14 > pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 > > [d4-w] > type=worker > host=d4 > interface=p4p1 > lb_method=myricom > lb_procs=14 > pin_cpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15 > ------------------------------------------------------------ > > Can anyone provide some direction on how to get capstats working? > > Thanks > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160308/99a99c09/attachment.html From pratikinamdar at gmail.com Tue Mar 8 23:17:12 2016 From: pratikinamdar at gmail.com (Pratik Inamdar) Date: Wed, 9 Mar 2016 07:17:12 +0000 (UTC) Subject: [Bro] Implementing new layer 2 Protocol References: Message-ID: unibw.de> writes: > > Hi all, > > My goal is to integrate a new protocol analyzer in Bro. This protocol > (PROFINET dyscovery and Basic Configuration Protocol) is working on layer > 2. My question is, are there special considerations to get at the data of > the layer 2? My colleague has tried creating an analyzer by following your > instructions for coding an analyzer by binpac. Before he went on vacation, > he told me, he could access data with binpac of layer 3 but not of layer > 2? Is that correct? If so does it work with the new binpac ++? Any pieces > of advice or suggestions how to get started would be greatly appreciated. > > Kind regards > > Marcel Odenwald > > _______________________________________________ > Bro mailing list > bro bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > Hi, I am facing a similar issue. I am trying to write analyzer for wifi protocol. I found some pointers, may be we can discuss them. Thanks, Pratik From umuta at sabanciuniv.edu Wed Mar 9 05:53:33 2016 From: umuta at sabanciuniv.edu (Umut Arus) Date: Wed, 9 Mar 2016 15:53:33 +0200 Subject: [Bro] Bad DNS Detection In-Reply-To: <1457445440.2847.1.camel@gamebox> References: <41018FFF-FE82-486E-B813-46DE93E4F0A7@illinois.edu> <1457445440.2847.1.camel@gamebox> Message-ID: Hi James, Maltrail is a wonderful tool that I'm looking for. thanks. On Tue, Mar 8, 2016 at 3:57 PM, James Lay wrote: > This will get you there: > > https://intel.criticalstack.com/ > > also, not bro related, but graphically shows what you're looking for: > > https://github.com/stamparm/maltrail > > James > > On Tue, 2016-03-08 at 15:18 +0200, Umut Arus wrote: > > Hi Justin, > > > > Thanks but I need a code or configuration that is query the malware dns/ip > sources that is trying to connect and raising notices. > > > > Or how do you realise in your network malwared DDoS clients with the Bro? > > > > thanks.. > > > > > On Tue, Mar 8, 2016 at 3:09 PM, Azoff, Justin S > wrote: > > This script that I wrote a while ago may help: > > > > > > It creates an external_dns.log file (which is just dns.log that has been > pre-filtered for you) as well as raising notices when it detects clients > using external dns servers. > > > _______________________________________________Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160309/1699b8d8/attachment.html From nmacia at cespi.unlp.edu.ar Wed Mar 9 16:44:00 2016 From: nmacia at cespi.unlp.edu.ar (Nicolas Macia CESPI) Date: Wed, 9 Mar 2016 21:44:00 -0300 Subject: [Bro] Scan UDP In-Reply-To: <56E04079.9030806@cert.unlp.edu.ar> References: <56E04079.9030806@cert.unlp.edu.ar> Message-ID: <56E0C350.2040700@cespi.unlp.edu.ar> Hi Seth, we where using [1] for some time and we found it trigger some false positive alerts. The problem was detected with NTP and DNS servers with a lot of activity. The script alerts that this servers were scanning UDP ports when in reality they were responding to requests to their services. Today we use an external bash script to determine whether or not it is a false positive (using knows udp ports).... not the best solution but it works pretty well [1] https://github.com/sethhall/bro-junk-drawer/blob/master/scan_udp.bro Cheers. Nico > El 12/02/16 a las 17:00, bro-request at bro.org escribi?: >> Today's Topics: >> >> 1. Re: Scan UDP (Seth Hall) >> 2. Re: Scan UDP (Forest Monsen) >> 3. Re: SHA256 Hash File Analyzer (Shawn Homan) >> >> >> ---------------------------------------------------------------------- >> >> Message: 1 >> Date: Thu, 11 Feb 2016 15:58:33 -0500 >> From: Seth Hall >> Subject: Re: [Bro] Scan UDP >> To: Cristian Daniel Barbaro >> Cc: bro at bro.org >> Message-ID: <82CCEB61-C63B-49C8-8CDA-35DDB1D05B01 at icir.org> >> Content-Type: text/plain; charset=us-ascii >> >> >>> On Feb 11, 2016, at 1:53 PM, Cristian Daniel Barbaro wrote: >>> >>> Bro implements this scan type detect? >> There is a prototype script that we put together a while ago that detects UDP scans. If you run it, I'd love to get any feedback that you have. >> https://github.com/sethhall/bro-junk-drawer/blob/master/scan_udp.bro >> >> .Seth >> ----- CeSPI Centro Superior para el Procesamiento de la Informaci?n Universidad Nacional de La Plata ------------------------------------------------------------------------------- Proteja el Medioambiente. No imprima este mail si no es absolutamente necesario From asharma at lbl.gov Wed Mar 9 17:12:09 2016 From: asharma at lbl.gov (Aashish Sharma) Date: Wed, 9 Mar 2016 17:12:09 -0800 Subject: [Bro] Scan UDP In-Reply-To: <56E0C350.2040700@cespi.unlp.edu.ar> References: <56E04079.9030806@cert.unlp.edu.ar> <56E0C350.2040700@cespi.unlp.edu.ar> Message-ID: <20160310011208.GN22092@yaksha.lbl.gov> Nicholas, If you don't mind sharing your bash script, May be we can look at that and incorporate those logic into this bro script itself. Aashish On Wed, Mar 09, 2016 at 09:44:00PM -0300, Nicolas Macia CESPI wrote: > > Hi Seth, we where using [1] for some time and we found it trigger some > false positive alerts. > > The problem was detected with NTP and DNS servers with a lot of > activity. The script alerts that this servers were scanning UDP ports > when in reality they were responding to requests to their services. > > Today we use an external bash script to determine whether or not it is a > false positive (using knows udp ports).... not the best solution but it > works pretty well > > > [1] https://github.com/sethhall/bro-junk-drawer/blob/master/scan_udp.bro > > > Cheers. > Nico > > > El 12/02/16 a las 17:00, bro-request at bro.org escribi?: > >> Today's Topics: > >> > >> 1. Re: Scan UDP (Seth Hall) > >> 2. Re: Scan UDP (Forest Monsen) > >> 3. Re: SHA256 Hash File Analyzer (Shawn Homan) > >> > >> > >> ---------------------------------------------------------------------- > >> > >> Message: 1 > >> Date: Thu, 11 Feb 2016 15:58:33 -0500 > >> From: Seth Hall > >> Subject: Re: [Bro] Scan UDP > >> To: Cristian Daniel Barbaro > >> Cc: bro at bro.org > >> Message-ID: <82CCEB61-C63B-49C8-8CDA-35DDB1D05B01 at icir.org> > >> Content-Type: text/plain; charset=us-ascii > >> > >> > >>> On Feb 11, 2016, at 1:53 PM, Cristian Daniel Barbaro wrote: > >>> > >>> Bro implements this scan type detect? > >> There is a prototype script that we put together a while ago that detects UDP scans. If you run it, I'd love to get any feedback that you have. > >> https://github.com/sethhall/bro-junk-drawer/blob/master/scan_udp.bro > >> > >> .Seth > >> > > ----- > CeSPI > Centro Superior para el Procesamiento de la Informaci?n > > Universidad Nacional de La Plata > ------------------------------------------------------------------------------- > Proteja el Medioambiente. No imprima este mail si no es absolutamente necesario > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From tgdesrochers at gmail.com Thu Mar 10 08:56:01 2016 From: tgdesrochers at gmail.com (Tim Desrochers) Date: Thu, 10 Mar 2016 11:56:01 -0500 Subject: [Bro] [bro] elasticsearch template Message-ID: <56e1a724.d0cc370a.5dca0.ffff986d@mx.google.com> Anyone using elasticsearch create a custom template for all bro logs and all fields. I?m using dynamic templates right now and it works fine but I?d like to have a bit more control over things and I?d rather not reinvent the wheel if its been done before. My google-fu has returned minimal results and none are for all possible bro logs with all possible fields -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160310/f1f988d9/attachment.html From valerio.click at gmx.com Thu Mar 10 10:23:09 2016 From: valerio.click at gmx.com (Valerio) Date: Thu, 10 Mar 2016 19:23:09 +0100 Subject: [Bro] SIP Analyzer and SDP payload In-Reply-To: <20160308010815.GA84243@wifi110.sys.ICSI.Berkeley.EDU> References: <56DC7E86.4040406@gmx.com> <20160308010815.GA84243@wifi110.sys.ICSI.Berkeley.EDU> Message-ID: <56E1BB8D.9070702@gmx.com> Hi Johanna, thanks for the feedback, I'll dig into the compiled source code to better understand what's going on. However, even at high-level, I don't understand why it is necessary to check for an event sip_reply whithin the sole function that is supposed to generate that very event. best regards, Valerio On 08/03/2016 02:08, Johanna Amann wrote: > Hello Valerio, > >> Unfortunately, I am not able to see where the boolean variable >> "sip_reply" accessed in sip-analyzer.pac:37 is actually defined and set. >> The same holds for sip_request (sip-analyzer.pac:22), sip_header >> (sip-analyzer.pac:53). >> Can someone point me where and how these variables are populated? > > That is a tad tricky - the variables are populated in autogenerated code. > The easiest way to figure out whate exactly happens might be to look into > the code that is generated in the build/ directory after building bro is > done. > > The in-a-nutshel variant is - since sip_reply is defined as an event, a > EventHandlerPtr variable named sip_reply is created. This variable is > populated automatically (by a call to the function internal_handler). > > Performing an if (sip_reply) call will return true if the sip_reply event > is used somewhere in Bro scriptland and false otherwise. > > Johanna > From johanna at icir.org Thu Mar 10 10:34:27 2016 From: johanna at icir.org (Johanna Amann) Date: Thu, 10 Mar 2016 10:34:27 -0800 Subject: [Bro] SIP Analyzer and SDP payload In-Reply-To: <56E1BB8D.9070702@gmx.com> References: <56DC7E86.4040406@gmx.com> <20160308010815.GA84243@wifi110.sys.ICSI.Berkeley.EDU> <56E1BB8D.9070702@gmx.com> Message-ID: <46315582-8CDF-4112-BCBE-6861201A56D2@icir.org> It is not strictly speaking necessary to do that. It reduces load a little bit - if there is no one listening for an event, there is no reason to create it. This is especially interesting if generating the event involves actual processing of data. Johanna On 10 Mar 2016, at 10:23, Valerio wrote: > Hi Johanna, > > thanks for the feedback, I'll dig into the compiled source code to > better understand what's going on. > > However, even at high-level, I don't understand why it is necessary to > check for an event sip_reply whithin the sole function that is > supposed > to generate that very event. > > best regards, > Valerio > > On 08/03/2016 02:08, Johanna Amann wrote: >> Hello Valerio, >> >>> Unfortunately, I am not able to see where the boolean variable >>> "sip_reply" accessed in sip-analyzer.pac:37 is actually defined and >>> set. >>> The same holds for sip_request (sip-analyzer.pac:22), sip_header >>> (sip-analyzer.pac:53). >>> Can someone point me where and how these variables are populated? >> >> That is a tad tricky - the variables are populated in autogenerated >> code. >> The easiest way to figure out whate exactly happens might be to look >> into >> the code that is generated in the build/ directory after building bro >> is >> done. >> >> The in-a-nutshel variant is - since sip_reply is defined as an event, >> a >> EventHandlerPtr variable named sip_reply is created. This variable is >> populated automatically (by a call to the function internal_handler). >> >> Performing an if (sip_reply) call will return true if the sip_reply >> event >> is used somewhere in Bro scriptland and false otherwise. >> >> Johanna >> From jonschipp at gmail.com Thu Mar 10 16:11:07 2016 From: jonschipp at gmail.com (Jon Schipp) Date: Thu, 10 Mar 2016 18:11:07 -0600 Subject: [Bro] Bro Cluster using Vagrant Issues In-Reply-To: <56D98BC0.4040607@cert.unlp.edu.ar> References: <56D780CF.1030408@cert.unlp.edu.ar> <56D98BC0.4040607@cert.unlp.edu.ar> Message-ID: The linked Vagrant configuration was used in the video and has been recently updated - https://github.com/jonschipp/vagrant/tree/master/bro-cluster It will provision a Bro cluster with 3 machines - 1x manager, 2x worker nodes. It automates the entire setup including the ssh configuration. On Fri, Mar 4, 2016 at 7:21 AM, Cristian Daniel Barbaro < cbarbaro at cert.unlp.edu.ar> wrote: > You could make cat .ssh / id_rsa.pub of master's file and then paste it > into .ssh / authorized_keys file of each worker (if not exists, create it). > Then you make sure if you can connect via ssh to each worker from master. > Everything I do with root (in /root folder). > > Saludos. > > El 03/03/16 a las 13:35, Christopher De Jesus escribi?: > > Well, that's the thing. I did it. not sure if I did it right, but I did > what i know of RSA. I made the sss-keygen, used that same key and copied it > into the shared folder and with the slave I took it and put it in the .ssh. > Is there another way of doing it? Besides, every time I tried to do 'scp' > it asked for the root password of the slave which I think it's a private > key. > > On Wed, Mar 2, 2016 at 8:09 PM, Cristian Daniel Barbaro < > cbarbaro at cert.unlp.edu.ar> wrote: > >> Do you add ssh key of master in authorized_keys file in workers into .ssh >> folder? >> Maybe here is the problem. >> >> Sorry for my english. >> >> El 02/03/16 a las 18:01, Christopher De Jesus escribi?: >> >> Hello, my name is Christopher and I?m doing undergrad research using Bro >> to make an IDS for one of the servers of my mentor. >> >> One of my first approaches was to start doing this video: >> >> https://www.youtube.com/watch?v=it7SZli61ZM >> >> in which it setups a cluster using Vagrant. I?ve been struggling for at >> least 5 days trying to make it work. I?ve changed permissions, ssh-keys, >> etc. >> >> The thing is that every time is use ?broctl? and do ?install? or ?deploy? >> on it.. it says this: >> >> >> Permission denied (publickey,password). >> >> Error: cannot create (some of the) directories >> /nsm/bro,/nsm/bro/logs,/nsm/bro/spool,/nsm/bro/spool/tmp on node worker-1 >> >> >> I used this site to install bro on Ubuntu of Vagrant: >> >> http://knowm.org/how-to-install-bro-network-security-monitor-on-ubuntu/ >> >> >> So yeah, apparently my host doesn?t have control over its slaves even >> tho? I could ping to it. I used 10.2.2.10 for the host and 10.2.2.11 for >> the slave in the Vagrantfile. >> >> >> >> Any other information that you guys need, let me know. This is pretty >> much the overview of what i?ve done. Thank you in advance. >> >> >> _______________________________________________ >> Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> >> -- >> Cristian Daniel Barbaro >> CERTUNLP >> -- >> >> > > -- > Cristian Daniel Barbaro > CERTUNLP > -- > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Jon Schipp, jonschipp.com, draconyx.net, open-nsm.net, -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160310/561e107c/attachment.html From tomboy64 at sina.cn Fri Mar 11 06:12:40 2016 From: tomboy64 at sina.cn (M.B.) Date: Fri, 11 Mar 2016 15:12:40 +0100 Subject: [Bro] Gentoo Package - Include Failure of CAF Message-ID: <56E2D258.9090006@sina.cn> Good afternoon, for Bro-2.4.1 I am currently creating a package for the Gentoo Distribution. Gentoo is a distro which compiles packages natively on the user's computer; for that our package manager hooks into the build system. I've created a package for the actor-framework and the its libs can be found in /usr/lib/libcaf_{io,core}.so.0.14.4; its headers can be found in /usr/include/caf. Attached you can find a build.log and from there you can see that during the configure phase, Libcaf is properly found. You can also see which options cmake is run with. However, while compiling broker, the file caf/detail/abstract_uniform_type_info.hpp cannot be found. Is there some magic involved regarding the passing on of include files to the actual compile of broker? Disabling broker, the whole compile works smoothly. Best regards, tomboy64 -------------- next part --------------  * Package: net-analyzer/bro-2.4.1  * Repository: gentoo  * Maintainer: tomboy64 at sina.cn proxy-maint at gentoo.org  * USE: abi_x86_64 amd64 broccoli broctl broker curl elibc_glibc geoip ipv6 kernel_linux python python_single_target_python3_4 python_targets_python2_7 python_targets_python3_4 python_targets_python3_5 tools userland_GNU  * FEATURES: preserve-libs sandbox userpriv usersandbox  * Package: net-analyzer/bro-2.4.1  * Repository: gentoo  * Maintainer: tomboy64 at sina.cn proxy-maint at gentoo.org  * USE: abi_x86_64 amd64 broccoli broctl broker curl elibc_glibc geoip ipv6 kernel_linux python python_single_target_python3_4 python_targets_python2_7 python_targets_python3_4 python_targets_python3_5 tools userland_GNU  * FEATURES: preserve-libs sandbox userpriv usersandbox >>> Unpacking source... >>> Unpacking bro-2.4.1.tar.gz to /tmp/portage/net-analyzer/bro-2.4.1/work >>> Source unpacked in /tmp/portage/net-analyzer/bro-2.4.1/work >>> Preparing source in /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1 ... >>> Source prepared. >>> Configuring source in /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1 ... >>> Working in BUILD_DIR: "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build" cmake --no-warn-unused-cli -C /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/gentoo_common_config.cmake -G Unix Makefiles -DCMAKE_INSTALL_PREFIX=/usr -DENABLE_DEBUG=false -DENABLE_JEMALLOC=false -DENABLE_PERFTOOLS=false -DENABLE_BROKER=true -DENABLE_CXX11=true -DBROKER_PYTHON_PREFIX=/usr -DLIBCAF_INCLUDE_DIR_CORE=/usr/include/ -DLIBCAF_INCLUDE_DIR_IO=/usr/include/ -DLIBCAF_ROOT_DIR=/usr -DENABLE_STATIC=false -DINSTALL_BROCCOLI=true -DINSTALL_BROCTL=true -DINSTALL_AUX_TOOLS=true -DENABLE_MOBILE_IPV6=true -DDISABLE_RUBY_BINDINGS=true -DDISABLE_PYTHON_BINDINGS=false -DBRO_LOG_DIR=/var/log/bro/ -DBRO_SPOOL_DIR=/var/spool/bro/ -DBRO_ETC_INSTALL_DIR=/etc/bro/ -DCMAKE_BUILD_TYPE=Gentoo -DCMAKE_INSTALL_DO_STRIP=OFF -DCMAKE_USER_MAKE_RULES_OVERRIDE=/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/gentoo_rules.cmake -DCMAKE_TOOLCHAIN_FILE=/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/gentoo_toolchain.cmake /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1 Not searching for unused variables given on the command line. loading initial cache file /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/gentoo_common_config.cmake -- The C compiler identification is GNU 5.3.0 -- The CXX compiler identification is GNU 5.3.0 -- Check for working C compiler: /usr/bin/x86_64-pc-linux-gnu-gcc -- Check for working C compiler: /usr/bin/x86_64-pc-linux-gnu-gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/x86_64-pc-linux-gnu-g++ -- Check for working CXX compiler: /usr/bin/x86_64-pc-linux-gnu-g++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found sed: /bin/sed -- Found Perl: /usr/bin/perl (found version "5.22.1") -- Found FLEX: 2.6.0 -- Found BISON: /usr/bin/bison -- Found PCAP: /usr/lib64/libpcap.so -- Performing Test PCAP_LINKS_SOLO -- Performing Test PCAP_LINKS_SOLO - Success -- Looking for pcap_get_pfring_id -- Looking for pcap_get_pfring_id - not found -- Found OpenSSL: /usr/lib64/libssl.so;/usr/lib64/libcrypto.so -- Performing Test ns_initparse_works_none -- Performing Test ns_initparse_works_none - Failed -- Performing Test res_mkquery_works_none -- Performing Test res_mkquery_works_none - Success -- Performing Test ns_initparse_works_libresolv.a -- Performing Test ns_initparse_works_libresolv.a - Success -- Performing Test res_mkquery_works_libresolv.a -- Performing Test res_mkquery_works_libresolv.a - Success -- Found BIND: /usr/lib64/libresolv.a -- Found ZLIB: /usr/lib64/libz.so (found version "1.2.8") -- Check if the system is big endian -- Searching 16 bit integer -- Looking for sys/types.h -- Looking for sys/types.h - found -- Looking for stdint.h -- Looking for stdint.h - found -- Looking for stddef.h -- Looking for stddef.h - found -- Check size of unsigned short -- Check size of unsigned short - done -- Using unsigned short -- Check if the system is big endian - little endian -- Check size of unsigned int -- Check size of unsigned int - done ==================| BinPAC Build Summary |==================== Install prefix: /usr Debug mode: false CC: /usr/bin/x86_64-pc-linux-gnu-gcc CFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused CXX: /usr/bin/x86_64-pc-linux-gnu-g++ CXXFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused CPP: /usr/bin/x86_64-pc-linux-gnu-g++ ================================================================ -- Found BinPAC: binpac -- Found LibGeoIP: /usr/lib64/libGeoIP.so -- Performing Test HAVE_GEOIP_COUNTRY_EDITION_V6 -- Performing Test HAVE_GEOIP_COUNTRY_EDITION_V6 - Success -- Performing Test HAVE_GEOIP_CITY_EDITION_REV0_V6 -- Performing Test HAVE_GEOIP_CITY_EDITION_REV0_V6 - Success -- Found GooglePerftools: /usr/lib64/libtcmalloc.so -- Found tcmalloc: /usr/lib64/libtcmalloc.so -- Check if the system is big endian -- Searching 16 bit integer -- Using unsigned short -- Check if the system is big endian - little endian -- Looking for htonll -- Looking for htonll - not found -- Check size of long int -- Check size of long int - done -- Check size of long long -- Check size of long long - done -- Check size of void * -- Check size of void * - done -- Check size of int32_t -- Check size of int32_t - done -- Check size of u_int32_t -- Check size of u_int32_t - done -- Check size of u_int16_t -- Check size of u_int16_t - done -- Check size of u_int8_t -- Check size of u_int8_t - done -- Check size of socklen_t -- Check size of socklen_t - done -- Check size of struct ip6_opt -- Check size of struct ip6_opt - done -- Check size of struct ip6_ext -- Check size of struct ip6_ext - done -- Looking for include file getopt.h -- Looking for include file getopt.h - found -- Looking for include file memory.h -- Looking for include file memory.h - found -- Looking for 4 include files sys/socket.h, ..., netinet/if_ether.h -- Looking for 4 include files sys/socket.h, ..., netinet/if_ether.h - found -- Looking for 4 include files sys/socket.h, ..., netinet/ip6.h -- Looking for 4 include files sys/socket.h, ..., netinet/ip6.h - found -- Looking for 3 include files sys/socket.h, ..., net/ethernet.h -- Looking for 3 include files sys/socket.h, ..., net/ethernet.h - found -- Looking for include file sys/ethernet.h -- Looking for include file sys/ethernet.h - not found -- Looking for include file net/ethertypes.h -- Looking for include file net/ethertypes.h - not found -- Looking for include file sys/time.h -- Looking for include file sys/time.h - found -- Looking for include files time.h, sys/time.h -- Looking for include files time.h, sys/time.h - found -- Looking for include file os-proto.h -- Looking for include file os-proto.h - not found -- Performing Test HAVE_READLINE_HISTORY_ENTRIES -- Performing Test HAVE_READLINE_HISTORY_ENTRIES - Failed -- Looking for include files stdio.h, readline/readline.h -- Looking for include files stdio.h, readline/readline.h - found -- Looking for include files stdio.h, readline/history.h -- Looking for include files stdio.h, readline/history.h - found -- Performing Test SIN_LEN -- Performing Test SIN_LEN - Failed -- Looking for IPPROTO_HOPOPTS -- Looking for IPPROTO_HOPOPTS - found -- Looking for IPPROTO_IPV6 -- Looking for IPPROTO_IPV6 - found -- Looking for IPPROTO_IPV4 -- Looking for IPPROTO_IPV4 - not found -- Looking for IPPROTO_ROUTING -- Looking for IPPROTO_ROUTING - found -- Looking for IPPROTO_FRAGMENT -- Looking for IPPROTO_FRAGMENT - found -- Looking for IPPROTO_ESP -- Looking for IPPROTO_ESP - found -- Looking for IPPROTO_AH -- Looking for IPPROTO_AH - found -- Looking for IPPROTO_ICMPV6 -- Looking for IPPROTO_ICMPV6 - found -- Looking for IPPROTO_NONE -- Looking for IPPROTO_NONE - found -- Looking for IPPROTO_DSTOPTS -- Looking for IPPROTO_DSTOPTS - found -- Looking for getopt_long -- Looking for getopt_long - found -- Looking for mallinfo -- Looking for mallinfo - found -- Looking for strcasestr -- Looking for strcasestr - found -- Looking for strerror -- Looking for strerror - found -- Looking for strsep -- Looking for strsep - found -- Looking for sigset -- Looking for sigset - found -- Performing Test DO_SOCK_DECL -- Performing Test DO_SOCK_DECL - Failed -- Performing Test SYSLOG_INT -- Performing Test SYSLOG_INT - Failed -- Looking for include file pcap-int.h -- Looking for include file pcap-int.h - not found -- Looking for pcap_freecode -- Looking for pcap_freecode - found -- Performing Test LIBPCAP_PCAP_COMPILE_NOPCAP_HAS_ERROR_PARAMETER -- Performing Test LIBPCAP_PCAP_COMPILE_NOPCAP_HAS_ERROR_PARAMETER - Failed -- Performing Test LIBPCAP_PCAP_COMPILE_NOPCAP_NO_ERROR_PARAMETER -- Performing Test LIBPCAP_PCAP_COMPILE_NOPCAP_NO_ERROR_PARAMETER - Success -- Looking for DLT_PPP_SERIAL -- Looking for DLT_PPP_SERIAL - found -- Performing Test including_ssl_h_works -- Performing Test including_ssl_h_works - Success -- Performing Test openssl_greater_than_0_9_7 -- Performing Test openssl_greater_than_0_9_7 - Success -- Performing Test OPENSSL_D2I_X509_USES_CONST_CHAR -- Performing Test OPENSSL_D2I_X509_USES_CONST_CHAR - Success -- Performing Test have_nameser_header -- Performing Test have_nameser_header - Success -- Found Libcaf: /usr/lib64/libcaf_core.so;/usr/lib64/libcaf_io.so -- Found CAF version: 0.14.4 -- Could NOT find RocksDB (missing: ROCKSDB_LIBRARIES ROCKSDB_INCLUDE_DIRS) -- Check if the system is big endian -- Searching 16 bit integer -- Using unsigned short -- Check if the system is big endian - little endian -- Found SWIG: /usr/bin/swig (found version "3.0.8") -- Found PythonInterp: /tmp/portage/net-analyzer/bro-2.4.1/temp/python3.4/bin/python (found version "3.4.3") -- Found PythonDev: /tmp/portage/net-analyzer/bro-2.4.1/temp/python3.4/bin/python-config -- Python bindings will be built and installed to: /usr/lib/python3.4/site-packages ==================| broker Config Summary |==================== Version: 0.4.0 SO version: 0 Build Type: Gentoo Install prefix: /usr Library prefix: /usr/lib Shared libs: true Static libs: false Enable RocksDB: FALSE Python bindings: true CC: /usr/bin/x86_64-pc-linux-gnu-gcc CFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused CXX: /usr/bin/x86_64-pc-linux-gnu-g++ CXXFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -std=c++11 ================================================================= -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE ===============| PySubnetTree Build Summary |================= Install dir: lib/python Debug mode: false CC: /usr/bin/x86_64-pc-linux-gnu-gcc CFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused CXX: /usr/bin/x86_64-pc-linux-gnu-g++ CXXFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -std=c++11 CPP: /usr/bin/x86_64-pc-linux-gnu-g++ ================================================================ =================| capstats Build Summary |=================== Install prefix: /usr Debug mode: false CXX: /usr/bin/x86_64-pc-linux-gnu-g++ CXXFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -std=c++11 CPP: /usr/bin/x86_64-pc-linux-gnu-g++ ================================================================ CMake Warning at aux/broctl/aux/trace-summary/CMakeLists.txt:30 (message): Could not find dependency: 'ipsumdump', trace-summary will not be able to read pcap traces =============| trace-summary Install Summary |================ Install destination: /usr/bin ================================================================ -- Found SubnetTree: build from source aux/pysubnettree =================| Broctl Install Summary |=================== Install prefix: /usr Bro root: /usr Scripts Dir: /usr/share/bro Spool Dir: /var/spool/bro/ Log Dir: /var/log/bro/ Config File Dir: /etc/bro/ ================================================================ ==================| Bro-Aux Build Summary |=================== Install prefix: /usr Debug mode: false CC: /usr/bin/x86_64-pc-linux-gnu-gcc CFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused CXX: /usr/bin/x86_64-pc-linux-gnu-g++ CXXFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -std=c++11 CPP: /usr/bin/x86_64-pc-linux-gnu-g++ ================================================================ -- Looking for geteuid -- Looking for geteuid - found -- Looking for getpwuid -- Looking for getpwuid - found -- Check size of uint -- Check size of uint - done -- Found Broccoli: broccoli ================| PyBroccoli Build Summary |================== Install dir: lib/python Debug mode: false CC: /usr/bin/x86_64-pc-linux-gnu-gcc CFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused CPP: /usr/bin/x86_64-pc-linux-gnu-gcc ================================================================ -- Not building broccoli-ruby bindings =================| Broccoli Build Summary |=================== Install prefix: /usr Library prefix: /usr/lib Debug mode: false Shared libs: true Static libs: false Config file: /etc/bro//broccoli.conf Packet support: true CC: /usr/bin/x86_64-pc-linux-gnu-gcc CFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused CPP: /usr/bin/x86_64-pc-linux-gnu-gcc ================================================================ ====================| Bro Build Summary |===================== Install prefix: /usr Bro Script Path: /usr/share/bro Debug mode: false CC: /usr/bin/x86_64-pc-linux-gnu-gcc CFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused CXX: /usr/bin/x86_64-pc-linux-gnu-g++ CXXFLAGS: -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -std=c++11 CPP: /usr/bin/x86_64-pc-linux-gnu-g++ Broker: true Broccoli: true Broctl: true Aux. Tools: true GeoIP: true gperftools found: true tcmalloc: true debugging: false jemalloc: false ================================================================ -- <<< Gentoo configuration >>> Build type Gentoo Install path /usr Compiler flags: C -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused C++ -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -std=c++11 Linker flags: Executable -Wl,-O1 -Wl,--as-needed Module -Wl,-O1 -Wl,--as-needed Shared -Wl,-O1 -Wl,--as-needed -- Configuring done -- Generating done -- Build files have been written to: /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build >>> Source configured. >>> Compiling source in /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1 ... >>> Working in BUILD_DIR: "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build" make -j1 VERBOSE=1 /usr/bin/cmake -H/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1 -B/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build --check-build-system CMakeFiles/Makefile.cmake 0 /usr/bin/cmake -E cmake_progress_start /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/CMakeFiles /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/CMakeFiles/progress.marks make -f CMakeFiles/Makefile2 all make[1]: Entering directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' make -f aux/binpac/lib/CMakeFiles/binpac_lib.dir/build.make aux/binpac/lib/CMakeFiles/binpac_lib.dir/depend make[2]: Entering directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1 /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/lib /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib/CMakeFiles/binpac_lib.dir/DependInfo.cmake --color= Dependee "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib/CMakeFiles/binpac_lib.dir/DependInfo.cmake" is newer than depender "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib/CMakeFiles/binpac_lib.dir/depend.internal". Dependee "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib/CMakeFiles/binpac_lib.dir/depend.internal". Scanning dependencies of target binpac_lib make[2]: Leaving directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' make -f aux/binpac/lib/CMakeFiles/binpac_lib.dir/build.make aux/binpac/lib/CMakeFiles/binpac_lib.dir/build make[2]: Entering directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' [ 0%] Building CXX object aux/binpac/lib/CMakeFiles/binpac_lib.dir/binpac_buffer.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/lib -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac_lib.dir/binpac_buffer.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/lib/binpac_buffer.cc [ 0%] Building CXX object aux/binpac/lib/CMakeFiles/binpac_lib.dir/binpac_bytestring.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/lib -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac_lib.dir/binpac_bytestring.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/lib/binpac_bytestring.cc [ 0%] Building CXX object aux/binpac/lib/CMakeFiles/binpac_lib.dir/binpac_regex.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/lib -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac_lib.dir/binpac_regex.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/lib/binpac_regex.cc [ 0%] Linking CXX static library libbinpac.a cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib && /usr/bin/cmake -P CMakeFiles/binpac_lib.dir/cmake_clean_target.cmake cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib && /usr/bin/cmake -E cmake_link_script CMakeFiles/binpac_lib.dir/link.txt --verbose=1 /usr/bin/x86_64-pc-linux-gnu-ar qc libbinpac.a CMakeFiles/binpac_lib.dir/binpac_buffer.cc.o CMakeFiles/binpac_lib.dir/binpac_bytestring.cc.o CMakeFiles/binpac_lib.dir/binpac_regex.cc.o /usr/bin/x86_64-pc-linux-gnu-ranlib libbinpac.a make[2]: Leaving directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' [ 0%] Built target binpac_lib make -f aux/binpac/src/CMakeFiles/binpac.dir/build.make aux/binpac/src/CMakeFiles/binpac.dir/depend make[2]: Entering directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' [ 0%] [FLEX][PACScanner] Building scanner with flex 2.6.0 cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src && /usr/bin/flex -o/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src/pac_scan.cc pac_scan.ll [ 0%] [BISON][PACParser] Building parser with bison 3.0.4 cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src && /usr/bin/bison --verbose --debug --defines=/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src/pac_parse.h -o /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src/pac_parse.cc pac_parse.yy cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1 /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src/CMakeFiles/binpac.dir/DependInfo.cmake --color= Dependee "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src/CMakeFiles/binpac.dir/DependInfo.cmake" is newer than depender "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src/CMakeFiles/binpac.dir/depend.internal". Dependee "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src/CMakeFiles/binpac.dir/depend.internal". Scanning dependencies of target binpac make[2]: Leaving directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' make -f aux/binpac/src/CMakeFiles/binpac.dir/build.make aux/binpac/src/CMakeFiles/binpac.dir/build make[2]: Entering directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' [ 0%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_parse.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_parse.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src/pac_parse.cc [ 0%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_scan.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_scan.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src/pac_scan.cc [ 0%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_action.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_action.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_action.cc [ 0%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_analyzer.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_analyzer.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_analyzer.cc [ 0%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_array.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_array.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_array.cc [ 1%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_attr.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_attr.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_attr.cc [ 1%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_btype.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_btype.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_btype.cc [ 1%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_case.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_case.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_case.cc [ 1%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_conn.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_conn.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_conn.cc [ 1%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_context.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_context.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_context.cc [ 1%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_cstr.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_cstr.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_cstr.cc [ 1%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_datadep.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_datadep.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_datadep.cc [ 1%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_dataptr.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_dataptr.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_dataptr.cc [ 1%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_dataunit.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_dataunit.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_dataunit.cc [ 1%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_decl.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_decl.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_decl.cc [ 2%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_embedded.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_embedded.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_embedded.cc [ 2%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_enum.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_enum.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_enum.cc [ 2%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_expr.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_expr.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_expr.cc [ 2%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_exttype.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_exttype.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_exttype.cc [ 2%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_field.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_field.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_field.cc [ 2%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_flow.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_flow.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_flow.cc [ 2%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_func.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_func.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_func.cc [ 2%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_id.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_id.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_id.cc [ 2%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_inputbuf.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_inputbuf.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_inputbuf.cc [ 2%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_let.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_let.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_let.cc [ 3%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_param.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_param.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_param.cc [ 3%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_paramtype.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_paramtype.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_paramtype.cc [ 3%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_primitive.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_primitive.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_primitive.cc [ 3%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_record.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_record.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_record.cc [ 3%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_redef.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_redef.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_redef.cc [ 3%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_regex.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_regex.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_regex.cc [ 3%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_state.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_state.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_state.cc [ 3%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_strtype.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_strtype.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_strtype.cc [ 3%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_type.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_type.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_type.cc [ 3%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_typedecl.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_typedecl.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_typedecl.cc [ 4%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_withinput.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_withinput.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_withinput.cc [ 4%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_output.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_output.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_output.cc [ 4%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_utils.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_utils.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_utils.cc [ 4%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_exception.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_exception.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_exception.cc [ 4%] Building CXX object aux/binpac/src/CMakeFiles/binpac.dir/pac_main.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/x86_64-pc-linux-gnu-g++ -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -o CMakeFiles/binpac.dir/pac_main.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/src/pac_main.cc [ 4%] Linking CXX executable binpac cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/binpac.dir/link.txt --verbose=1 /usr/bin/x86_64-pc-linux-gnu-g++ -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -Wl,-O1 -Wl,--as-needed CMakeFiles/binpac.dir/pac_parse.cc.o CMakeFiles/binpac.dir/pac_scan.cc.o CMakeFiles/binpac.dir/pac_action.cc.o CMakeFiles/binpac.dir/pac_analyzer.cc.o CMakeFiles/binpac.dir/pac_array.cc.o CMakeFiles/binpac.dir/pac_attr.cc.o CMakeFiles/binpac.dir/pac_btype.cc.o CMakeFiles/binpac.dir/pac_case.cc.o CMakeFiles/binpac.dir/pac_conn.cc.o CMakeFiles/binpac.dir/pac_context.cc.o CMakeFiles/binpac.dir/pac_cstr.cc.o CMakeFiles/binpac.dir/pac_datadep.cc.o CMakeFiles/binpac.dir/pac_dataptr.cc.o CMakeFiles/binpac.dir/pac_dataunit.cc.o CMakeFiles/binpac.dir/pac_decl.cc.o CMakeFiles/binpac.dir/pac_embedded.cc.o CMakeFiles/binpac.dir/pac_enum.cc.o CMakeFiles/binpac.dir/pac_expr.cc.o CMakeFiles/binpac.dir/pac_exttype.cc.o CMakeFiles/binpac.dir/pac_field.cc.o CMakeFiles/binpac.dir/pac_flow.cc.o CMakeFiles/binpac.dir/pac_func.cc.o CMakeFiles/binpac.dir/pac_id.cc.o CMakeFiles/binpac.dir/pac_inputbuf.cc.o CMakeFiles/binpac.dir/pac_let.cc.o CMakeFiles/binpac.dir/pac_param.cc.o CMakeFiles/binpac.dir/pac_paramtype.cc.o CMakeFiles/binpac.dir/pac_primitive.cc.o CMakeFiles/binpac.dir/pac_record.cc.o CMakeFiles/binpac.dir/pac_redef.cc.o CMakeFiles/binpac.dir/pac_regex.cc.o CMakeFiles/binpac.dir/pac_state.cc.o CMakeFiles/binpac.dir/pac_strtype.cc.o CMakeFiles/binpac.dir/pac_type.cc.o CMakeFiles/binpac.dir/pac_typedecl.cc.o CMakeFiles/binpac.dir/pac_withinput.cc.o CMakeFiles/binpac.dir/pac_output.cc.o CMakeFiles/binpac.dir/pac_utils.cc.o CMakeFiles/binpac.dir/pac_exception.cc.o CMakeFiles/binpac.dir/pac_main.cc.o -o binpac -rdynamic -Wl,-rpath,:::::::: make[2]: Leaving directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' [ 4%] Built target binpac make -f aux/broker/CMakeFiles/broker.dir/build.make aux/broker/CMakeFiles/broker.dir/depend make[2]: Entering directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1 /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/broker /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/broker /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/broker/CMakeFiles/broker.dir/DependInfo.cmake --color= Dependee "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/broker/CMakeFiles/broker.dir/DependInfo.cmake" is newer than depender "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/broker/CMakeFiles/broker.dir/depend.internal". Dependee "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/broker/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/broker/CMakeFiles/broker.dir/depend.internal". Scanning dependencies of target broker make[2]: Leaving directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' make -f aux/broker/CMakeFiles/broker.dir/build.make aux/broker/CMakeFiles/broker.dir/build make[2]: Entering directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' [ 4%] Building CXX object aux/broker/CMakeFiles/broker.dir/src/address.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/broker && /usr/bin/x86_64-pc-linux-gnu-g++ -Dbroker_EXPORTS -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/broker -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/lib -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -std=c++11 -fPIC -o CMakeFiles/broker.dir/src/address.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/broker/src/address.cc [ 4%] Building CXX object aux/broker/CMakeFiles/broker.dir/src/broker.cc.o cd /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/broker && /usr/bin/x86_64-pc-linux-gnu-g++ -Dbroker_EXPORTS -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/broker -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build/aux/binpac/lib -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/binpac/lib -I/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build -DNDEBUG -march=broadwell -O2 -pipe -mabm -Wall -Wno-unused -std=c++11 -fPIC -o CMakeFiles/broker.dir/src/broker.cc.o -c /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/broker/src/broker.cc In file included from /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/broker/src/broker.cc:9:0: /tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1/aux/broker/src/store/result_type_info.hh:5:53: fatal error: caf/detail/abstract_uniform_type_info.hpp: No such file or directory compilation terminated. aux/broker/CMakeFiles/broker.dir/build.make:86: recipe for target 'aux/broker/CMakeFiles/broker.dir/src/broker.cc.o' failed make[2]: *** [aux/broker/CMakeFiles/broker.dir/src/broker.cc.o] Error 1 make[2]: Leaving directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' CMakeFiles/Makefile2:253: recipe for target 'aux/broker/CMakeFiles/broker.dir/all' failed make[1]: *** [aux/broker/CMakeFiles/broker.dir/all] Error 2 make[1]: Leaving directory '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' Makefile:149: recipe for target 'all' failed make: *** [all] Error 2 * ERROR: net-analyzer/bro-2.4.1::gentoo failed (compile phase): * emake failed * * If you need support, post the output of `emerge --info '=net-analyzer/bro-2.4.1::gentoo'`, * the complete build log and the output of `emerge -pqv '=net-analyzer/bro-2.4.1::gentoo'`. * The complete build log is located at '/var/log/portage/net-analyzer:bro-2.4.1:20160311-134801.log'. * For convenience, a symlink to the build log is located at '/tmp/portage/net-analyzer/bro-2.4.1/temp/build.log'. * The ebuild environment file is located at '/tmp/portage/net-analyzer/bro-2.4.1/temp/environment'. * Working directory: '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1_build' * S: '/tmp/portage/net-analyzer/bro-2.4.1/work/bro-2.4.1' -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 951 bytes Desc: OpenPGP digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160311/ea1b52a0/attachment-0001.bin From tgdesrochers at gmail.com Fri Mar 11 06:24:44 2016 From: tgdesrochers at gmail.com (Tim Desrochers) Date: Fri, 11 Mar 2016 09:24:44 -0500 Subject: [Bro] [bro] Notices Message-ID: <56e2d531.56638c0a.9dce7.2faf@mx.google.com> I see that when some notice gets emailed (like SQL) it can contain extra data that is not in the notice.log. How does this get created? Is there a way to log it so I can send it to my SIEM? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160311/471e5943/attachment.html From jan.grashoefer at gmail.com Fri Mar 11 06:41:06 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Fri, 11 Mar 2016 15:41:06 +0100 Subject: [Bro] [bro] Notices In-Reply-To: <56e2d531.56638c0a.9dce7.2faf@mx.google.com> References: <56e2d531.56638c0a.9dce7.2faf@mx.google.com> Message-ID: <56E2D902.3070803@gmail.com> Hi, what is sent via email can be extended using email_body_sections (see https://www.bro.org/sphinx-git/frameworks/notice.html#extending-notice-emails). In detect-sqli.bro (https://github.com/bro/bro/blob/master/scripts/policy/protocols/http/detect-sqli.bro#L84) you can see how it is used to add additional information. Regards, Jan From seth at icir.org Fri Mar 11 08:49:00 2016 From: seth at icir.org (Seth Hall) Date: Fri, 11 Mar 2016 11:49:00 -0500 Subject: [Bro] Gentoo Package - Include Failure of CAF In-Reply-To: <56E2D258.9090006@sina.cn> References: <56E2D258.9090006@sina.cn> Message-ID: > On Mar 11, 2016, at 9:12 AM, M.B. wrote: > > Disabling broker, the whole compile works smoothly. For packaging 2.4.1, I would recommend only offering Broker disabled. It's not currently used in 2.4.1. It will be required in 2.5. I'll let someone else answer the question you actually asked. :) Thanks, .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160311/081ff1e1/attachment.bin From pawelec93 at googlemail.com Fri Mar 11 09:57:37 2016 From: pawelec93 at googlemail.com (=?UTF-8?Q?Pawe=C5=82_Piszczatowski?=) Date: Fri, 11 Mar 2016 17:57:37 +0000 Subject: [Bro] Can Bro Worker read packet capture files Message-ID: I've got a cluster setup with a manager and a worker. I have another VM that I want to do a packet capture. I would like the Worker to run "bro -r mypackets.trace local" that then would add the extra data so it would show up in the manager. However, there is no /logs/current folder in the Worker so where would I have the files in order to pass it to the Manager ? I don't want to go directly to the Manager, I would like the packet capture to go through the worker first. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160311/26aad676/attachment.html From nmacia at cespi.unlp.edu.ar Fri Mar 11 12:09:35 2016 From: nmacia at cespi.unlp.edu.ar (Nicolas Macia CESPI) Date: Fri, 11 Mar 2016 17:09:35 -0300 Subject: [Bro] Scan UDP In-Reply-To: <20160310011208.GN22092@yaksha.lbl.gov> References: <56E04079.9030806@cert.unlp.edu.ar> <56E0C350.2040700@cespi.unlp.edu.ar> <20160310011208.GN22092@yaksha.lbl.gov> Message-ID: <56E325FF.4070900@cespi.unlp.edu.ar> Aashish, you misunderstood me. What we did was not to consider communications from those ports (NTP & DNS). I think the problem is that in con.log there are a lot of UDP conections marked with is_local flag in T when they are not. I guessed that this is done because of some packets dropped at nids installation, but netstat -ni does not show any drop or error on the capture interface. nico El 09/03/16 a las 22:12, Aashish Sharma escribi?: > Nicholas, > > If you don't mind sharing your bash script, May be we can look at that and incorporate those logic into this bro script > itself. > > Aashish > > On Wed, Mar 09, 2016 at 09:44:00PM -0300, Nicolas Macia CESPI wrote: >> Hi Seth, we where using [1] for some time and we found it trigger some >> false positive alerts. >> >> The problem was detected with NTP and DNS servers with a lot of >> activity. The script alerts that this servers were scanning UDP ports >> when in reality they were responding to requests to their services. >> >> Today we use an external bash script to determine whether or not it is a >> false positive (using knows udp ports).... not the best solution but it >> works pretty well >> >> >> [1] https://github.com/sethhall/bro-junk-drawer/blob/master/scan_udp.bro >> >> >> Cheers. >> Nico >> >>> El 12/02/16 a las 17:00, bro-request at bro.org escribi?: >>>> Today's Topics: >>>> >>>> 1. Re: Scan UDP (Seth Hall) >>>> 2. Re: Scan UDP (Forest Monsen) >>>> 3. Re: SHA256 Hash File Analyzer (Shawn Homan) >>>> >>>> >>>> ---------------------------------------------------------------------- >>>> >>>> Message: 1 >>>> Date: Thu, 11 Feb 2016 15:58:33 -0500 >>>> From: Seth Hall >>>> Subject: Re: [Bro] Scan UDP >>>> To: Cristian Daniel Barbaro >>>> Cc: bro at bro.org >>>> Message-ID: <82CCEB61-C63B-49C8-8CDA-35DDB1D05B01 at icir.org> >>>> Content-Type: text/plain; charset=us-ascii >>>> >>>> >>>>> On Feb 11, 2016, at 1:53 PM, Cristian Daniel Barbaro wrote: >>>>> >>>>> Bro implements this scan type detect? >>>> There is a prototype script that we put together a while ago that detects UDP scans. If you run it, I'd love to get any feedback that you have. >>>> https://github.com/sethhall/bro-junk-drawer/blob/master/scan_udp.bro >>>> >>>> .Seth >>>> >> ----- >> CeSPI >> Centro Superior para el Procesamiento de la Informaci?n >> >> Universidad Nacional de La Plata >> ------------------------------------------------------------------------------- >> Proteja el Medioambiente. No imprima este mail si no es absolutamente necesario >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro ----- CeSPI Centro Superior para el Procesamiento de la Informaci??n Universidad Nacional de La Plata ------------------------------------------------------------------------------- Proteja el Medioambiente. No imprima este mail si no es absolutamente necesario From fmonsen at ucsc.edu Fri Mar 11 14:53:10 2016 From: fmonsen at ucsc.edu (Forest Monsen) Date: Fri, 11 Mar 2016 14:53:10 -0800 Subject: [Bro] Scan UDP In-Reply-To: <56E0C350.2040700@cespi.unlp.edu.ar> References: <56E04079.9030806@cert.unlp.edu.ar> <56E0C350.2040700@cespi.unlp.edu.ar> Message-ID: <56E34C56.8090004@ucsc.edu> On 03/09/2016 04:44 PM, Nicolas Macia CESPI wrote: > The problem was detected with NTP and DNS servers with a lot of > activity. The script alerts that this servers were scanning UDP ports > when in reality they were responding to requests to their services. Ah yes. We saw this behavior with Bluehost recursive DNS. I don't have a pcap, I'm sorry. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160311/44c707af/attachment.bin From vallentin at icir.org Sat Mar 12 11:47:41 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Sat, 12 Mar 2016 11:47:41 -0800 Subject: [Bro] Gentoo Package - Include Failure of CAF In-Reply-To: <56E2D258.9090006@sina.cn> References: <56E2D258.9090006@sina.cn> Message-ID: <20160312194740.GB1883@ninja> > I've created a package for the actor-framework and the its libs can be > found in /usr/lib/libcaf_{io,core}.so.0.14.4; its headers can be found > in /usr/include/caf. That sounds right. Would you mind sharing the CAF ebuild with us? > However, while compiling broker, the file > caf/detail/abstract_uniform_type_info.hpp cannot be found. Is there > some magic involved regarding the passing on of include files to the > actual compile of broker? The reason why this fails is because caf/detail/abstract_uniform_type_info.hpp has been removed with the 0.14 releases. The current Broker release 0.4 does not support CAF 0.14. Broker 0.4 requires CAF 0.13.2. However, the master branch of Broker supports 0.14. We're currently in the process of porting Broker to 0.15, which Bro 2.5 will depend on. Sorry about the inconvenience. Matthias From vallentin at icir.org Sat Mar 12 16:53:41 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Sat, 12 Mar 2016 16:53:41 -0800 Subject: [Bro] Gentoo Package - Include Failure of CAF In-Reply-To: <56E48AFF.2070808@sina.cn> References: <56E2D258.9090006@sina.cn> <20160312194740.GB1883@ninja> <56E48AFF.2070808@sina.cn> Message-ID: <20160313005341.GE717@shogun> The CAF ebuild looks good to me. The deprecation warnings arise because of the combination of Broker 0.4 with CAF >= 0.14. If you want a stable release, you would have to use the following combination: Bro 2.4.1 + Broker 0.4 + CAF 0.13.2 And as mentioned in an earlier email, for the next release Bro 2.5, it would be Broker 0.5 and CAF 0.15. I don't have a gentoo installation handy so that I could further debug the issue with the symlinked .sh file. Would you mind giving us a bit more insight in to that error? Matthias On Sat, Mar 12, 2016 at 10:32:47PM +0100, M.B. wrote: > Hello Matthias, > > >> I've created a package for the actor-framework and the its libs can be > >> found in /usr/lib/libcaf_{io,core}.so.0.14.4; its headers can be found > >> in /usr/include/caf. > > > > That sounds right. Would you mind sharing the CAF ebuild with us? > > Sure thing; it's preliminary and can be found in my fork of the Gentoo > tree along with the bro ebuild: > https://github.com/gentoo/gentoo/compare/master...tomboy-64:bro > It supports multilib and works (relatively) fine; lots of deprecation > warnings, though. > It doesn't support next, riaf and cash yet - I still need to create > ebuilds for those. > > >> However, while compiling broker, the file > >> caf/detail/abstract_uniform_type_info.hpp cannot be found. Is there > >> some magic involved regarding the passing on of include files to the > >> actual compile of broker? > > > > The reason why this fails is because > > caf/detail/abstract_uniform_type_info.hpp has been removed with the 0.14 > > releases. The current Broker release 0.4 does not support CAF 0.14. > > Broker 0.4 requires CAF 0.13.2. However, the master branch of Broker > > supports 0.14. > > > > We're currently in the process of porting Broker to 0.15, which Bro 2.5 > > will depend on. Sorry about the inconvenience. > > No worries. As long as you eventually present me with a version that > builds against a specific version of CAF, I'm a happy camper. > You might, however, try the Bro ebuild; there's a minor warning about a > symlinked .sh in the end. Not sure whether that's because of the > explicitely passed spool directory? > However, I will only push it into the tree once I'm convinced that it's > ready for end users. > > With kind regards, > tomboy64 > From tomboy64 at sina.cn Sun Mar 13 15:14:20 2016 From: tomboy64 at sina.cn (M.B.) Date: Sun, 13 Mar 2016 23:14:20 +0100 Subject: [Bro] Fwd: Re: Gentoo Package - Include Failure of CAF In-Reply-To: <56E57DAE.5060103@sina.cn> References: <56E57DAE.5060103@sina.cn> Message-ID: <56E5E63C.1060503@sina.cn> Good morning, Am 13.03.2016 um 01:53 schrieb Matthias Vallentin: > > The CAF ebuild looks good to me. The deprecation warnings arise because > of the combination of Broker 0.4 with CAF >= 0.14. This was a misunderstanding; I meant the internal deprecation warnings that arise when compiling CAF 0.14.4. CAF's 0.13.2 build.log is completely devoid of that. > If you want a stable > release, you would have to use the following combination: > > Bro 2.4.1 + Broker 0.4 + CAF 0.13.2 I've created a 0.13.2 CAF ebuild and something's going wrong with Swig/Python. See attached bro-swig-python-failure.log. > And as mentioned in an earlier email, for the next release Bro 2.5, it > would be Broker 0.5 and CAF 0.15. > > I don't have a gentoo installation handy so that I could further debug > the issue with the symlinked .sh file. Would you mind giving us a bit > more insight in to that error? Sure. It's an automated QA warning by portage: Symbolic link /usr/share/broctl/scripts/broctl-config.sh points to /var/spool/bro/broctl-config.sh which does not exist. The install phase is listed in attached bro-warning.log. The QA warning is at the very end. Please ignore the warnings in the middle; they are coming from a bug in portage. With kind regards, Markus > On Sat, Mar 12, 2016 at 10:32:47PM +0100, M.B. wrote: >> Hello Matthias, >> >>>> I've created a package for the actor-framework and the its libs can be >>>> found in /usr/lib/libcaf_{io,core}.so.0.14.4; its headers can be found >>>> in /usr/include/caf. >>> >>> That sounds right. Would you mind sharing the CAF ebuild with us? >> >> Sure thing; it's preliminary and can be found in my fork of the Gentoo >> tree along with the bro ebuild: >> https://github.com/gentoo/gentoo/compare/master...tomboy-64:bro >> It supports multilib and works (relatively) fine; lots of deprecation >> warnings, though. >> It doesn't support next, riaf and cash yet - I still need to create >> ebuilds for those. >> >>>> However, while compiling broker, the file >>>> caf/detail/abstract_uniform_type_info.hpp cannot be found. Is there >>>> some magic involved regarding the passing on of include files to the >>>> actual compile of broker? >>> >>> The reason why this fails is because >>> caf/detail/abstract_uniform_type_info.hpp has been removed with the 0.14 >>> releases. The current Broker release 0.4 does not support CAF 0.14. >>> Broker 0.4 requires CAF 0.13.2. However, the master branch of Broker >>> supports 0.14. >>> >>> We're currently in the process of porting Broker to 0.15, which Bro 2.5 >>> will depend on. Sorry about the inconvenience. >> >> No worries. As long as you eventually present me with a version that >> builds against a specific version of CAF, I'm a happy camper. >> You might, however, try the Bro ebuild; there's a minor warning about a >> symlinked .sh in the end. Not sure whether that's because of the >> explicitely passed spool directory? >> However, I will only push it into the tree once I'm convinced that it's >> ready for end users. >> >> With kind regards, >> tomboy64 >> > > -------------- next part -------------- A non-text attachment was scrubbed... Name: bro-swig-python-failure.log.xz Type: application/x-xz Size: 6560 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160313/aa12815c/attachment.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: bro-warning.log.xz Type: application/x-xz Size: 8364 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160313/aa12815c/attachment-0001.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 951 bytes Desc: OpenPGP digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160313/aa12815c/attachment-0002.bin From mgill6 at student.concordia.ab.ca Sun Mar 13 18:08:06 2016 From: mgill6 at student.concordia.ab.ca (Manmeet Gill) Date: Sun, 13 Mar 2016 19:08:06 -0600 Subject: [Bro] Is it possible to generate a alarm based on threshold value. In-Reply-To: References: Message-ID: Example average number of connection per second for port 21 Falls between one to three connections per second. If Port 21 Falls above the average of 3 connections. Then there is abnormal behavior. Above scenario is possible ? If it is then how? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160313/c60b6ffb/attachment-0001.html From vallentin at icir.org Mon Mar 14 10:43:42 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Mon, 14 Mar 2016 10:43:42 -0700 Subject: [Bro] Fwd: Re: Gentoo Package - Include Failure of CAF In-Reply-To: <56E5E63C.1060503@sina.cn> References: <56E57DAE.5060103@sina.cn> <56E5E63C.1060503@sina.cn> Message-ID: <20160314174342.GD43666@samurai.ICIR.org> > This was a misunderstanding; I meant the internal deprecation warnings > that arise when compiling CAF 0.14.4. CAF's 0.13.2 build.log is > completely devoid of that. Ah, okay. Thanks for clarifying. > See attached bro-swig-python-failure.log. > [..] > The install phase is listed in attached bro-warning.log. I cannot uncompress what you have attached. Various combinations of tar switches seem to fail [1]. Feel free to send the relevant part of the log file inline. As long as we're not talking multiple MBs, you can also send plain text attachments to the list. Matthias [1] https://xkcd.com/1168/ From jazoff at illinois.edu Mon Mar 14 11:03:26 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Mon, 14 Mar 2016 18:03:26 +0000 Subject: [Bro] Gentoo Package - Include Failure of CAF In-Reply-To: <56E5E63C.1060503@sina.cn> References: <56E57DAE.5060103@sina.cn> <56E5E63C.1060503@sina.cn> Message-ID: <355BB6ED-8ACD-4FD4-B416-0B1AB1BD08BB@illinois.edu> > On Mar 13, 2016, at 6:14 PM, M.B. wrote: > > Sure. It's an automated QA warning by portage: > Symbolic link /usr/share/broctl/scripts/broctl-config.sh points to > /var/spool/bro/broctl-config.sh which does not exist. > > The install phase is listed in attached bro-warning.log. The QA warning > is at the very end. Please ignore the warnings in the middle; they are > coming from a bug in portage. > Ah.. this has come up before. The target of the symlink is auto created the first time bro runs. https://bro-tracker.atlassian.net/browse/BIT-1437 is somewhat related.. Homebrew ran into this as well, they fixed it by moving spool and logs under var/ which actually makes more sense. https://github.com/Homebrew/homebrew/commit/3591e099860b00a57b7f9f67ef69d5e6eac0fa47 https://github.com/Homebrew/homebrew/pull/42179 -- - Justin Azoff From hacecky at jlab.org Mon Mar 14 13:48:14 2016 From: hacecky at jlab.org (Eric Hacecky) Date: Mon, 14 Mar 2016 16:48:14 -0400 (EDT) Subject: [Bro] How should I be calling an external script from Bro? Message-ID: <1442241653.10099820.1457988494716.JavaMail.zimbra@jlab.org> New to Bro. Trying to make sure I follow best practice here configuring it for my environment. Currently Bro generates an email alert for HTTP::SQL_Injection_Attacker from detect-sqli.bro. I wrote a python script to accept some parameters, including the attacker's IP that will put in a block at my firewall. I was just going to tail Bro's notice.log and pull out the IP to feed my script anytime a SQL attack was logged there, but I figured it would be better to get Bro to do some of that lifting for me instead. Being new to bro, I don't know how to do this. I've googled around a bit and this is my best guess. (definitely a guess) - Exec module is the best way to go about this? - If so, I'm going to do what...make a something.bro file that basically says @load base/utils/exec when ( happens = Exec::run($cmd="myScript.py 55.66.77.88 -time 720") ) - Then I would @load something.bro in my local.bro file ----------- Assuming that's the gist of it, how am I supposed to figure out what event to look for? when ( HTTP::SQL_Injection_Attacker )? Every example I look at has uses 'local result' instead. Ex. https://github.com/sooshie/bro-scripts/blob/master/misc/vt_check.bro when (local result = Exec::run). Why? I don't see result defined anywhere previously? I don't understand how that condition is ever met. How do I make bro pass the IP to my script? Exec::run($cmd="myScript.py [$host=c$id$orig_h]")? Thanks, Eric From jazoff at illinois.edu Mon Mar 14 14:03:42 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Mon, 14 Mar 2016 21:03:42 +0000 Subject: [Bro] How should I be calling an external script from Bro? In-Reply-To: <1442241653.10099820.1457988494716.JavaMail.zimbra@jlab.org> References: <1442241653.10099820.1457988494716.JavaMail.zimbra@jlab.org> Message-ID: <61D4AFB1-D106-41DF-A59D-DE68218893C2@illinois.edu> Hi, This repo has code in it that does everything you are trying to do: https://github.com/ncsa/bhr-bro You should be able to see how to modify it for your environment. This video details how the Exec works: https://www.youtube.com/watch?v=oo4zDC24xHU -- - Justin Azoff > On Mar 14, 2016, at 4:48 PM, Eric Hacecky wrote: > > New to Bro. Trying to make sure I follow best practice here configuring it for my environment. > > Currently Bro generates an email alert for HTTP::SQL_Injection_Attacker from detect-sqli.bro. > > I wrote a python script to accept some parameters, including the attacker's IP that will put in a block at my firewall. > > I was just going to tail Bro's notice.log and pull out the IP to feed my script anytime a SQL attack was logged there, but I figured it would be better to get Bro to do some of that lifting for me instead. > > Being new to bro, I don't know how to do this. > > I've googled around a bit and this is my best guess. (definitely a guess) > > - Exec module is the best way to go about this? > > - If so, I'm going to do what...make a something.bro file that basically says > > @load base/utils/exec > > when ( happens = Exec::run($cmd="myScript.py 55.66.77.88 -time 720") ) > > - Then I would @load something.bro in my local.bro file > > ----------- > > Assuming that's the gist of it, how am I supposed to figure out what event to look for? > > when ( HTTP::SQL_Injection_Attacker )? > > Every example I look at has uses 'local result' instead. Ex. https://github.com/sooshie/bro-scripts/blob/master/misc/vt_check.bro > > when (local result = Exec::run). Why? I don't see result defined anywhere previously? I don't understand how that condition is ever met. > > How do I make bro pass the IP to my script? > > Exec::run($cmd="myScript.py [$host=c$id$orig_h]")? > > Thanks, > Eric > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From troyj at maine.edu Mon Mar 14 14:06:11 2016 From: troyj at maine.edu (Troy Jordan) Date: Mon, 14 Mar 2016 17:06:11 -0400 Subject: [Bro] Spicy and meta data Message-ID: <56E727C3.6090104@maine.edu> Spicy devs, Is this a hypothetical example, or is there currently a mechanism in Spicy to insert layer-4 meta data into an application layer stream, as suggested in the Spicy tech report, p5: " To implement that, the TCP dissector would insert marks into the input stream corresponding to packet boundaries for the HTTP dissector to skip ahead to. " - Troy -- Troy Jordan t r o y j @ m a i n e . e d u GIAC GCIH,GCIA ------------------------------------------------------------ Network Systems Security Analyst Information Technology Security Office University of Maine System ------------------------------------------------------------ 233 Science Building | voice: 207.561.3590 Portland, ME 04103 | fax: 509.351.3650 "As you all know, Security Is Mortals chiefest Enemy" William Shakespeare, Macbeth From valerio.click at gmx.com Mon Mar 14 14:42:06 2016 From: valerio.click at gmx.com (Valerio) Date: Mon, 14 Mar 2016 22:42:06 +0100 Subject: [Bro] Access pcap filename in script land Message-ID: <56E7302E.6070609@gmx.com> Hi all, in case bro is executed offline on a pcap with: bro -r file1.pcap script.bro is there a directive I can insert into script.bro to access the pcap filename? many thanks in advance, Valerio From robin at icir.org Mon Mar 14 14:50:04 2016 From: robin at icir.org (Robin Sommer) Date: Mon, 14 Mar 2016 14:50:04 -0700 Subject: [Bro] Spicy and meta data In-Reply-To: <56E727C3.6090104@maine.edu> References: <56E727C3.6090104@maine.edu> Message-ID: <20160314215004.GC96848@icir.org> On Mon, Mar 14, 2016 at 17:06 -0400, Troy Jordan wrote: > Is this a hypothetical example, or is there currently a mechanism in > Spicy to insert layer-4 meta data into an application layer stream, as > suggested in the Spicy tech report, p5: Yes, that mechanism exists, see the tests/binpac/synchronize/sync-at-mark.pac2 for an example: the pac-driver command line in there specifies positions to mark, where the second unit then re-synchronizes when encountering errors. > " To implement that, the TCP dissector would insert marks into the input > stream corresponding to packet boundaries for the HTTP dissector to skip > ahead to. " What's hypothetical here is the TCP dissector using the mechanism, that's not implemented. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From anthony.kasza at gmail.com Mon Mar 14 17:23:30 2016 From: anthony.kasza at gmail.com (anthony kasza) Date: Mon, 14 Mar 2016 17:23:30 -0700 Subject: [Bro] Access pcap filename in script land In-Reply-To: <56E7302E.6070609@gmx.com> References: <56E7302E.6070609@gmx.com> Message-ID: Nope. -AK On Mar 14, 2016 2:51 PM, "Valerio" wrote: > Hi all, > > in case bro is executed offline on a pcap with: > > bro -r file1.pcap script.bro > > is there a directive I can insert into script.bro to access the pcap > filename? > > many thanks in advance, > Valerio > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160314/d5da6003/attachment.html From tomboy64 at sina.cn Tue Mar 15 05:36:14 2016 From: tomboy64 at sina.cn (M.B.) Date: Tue, 15 Mar 2016 13:36:14 +0100 Subject: [Bro] Fwd: Re: Gentoo Package - Include Failure of CAF In-Reply-To: <20160314174342.GD43666@samurai.ICIR.org> References: <56E57DAE.5060103@sina.cn> <56E5E63C.1060503@sina.cn> <20160314174342.GD43666@samurai.ICIR.org> Message-ID: <56E801BE.4070303@sina.cn> Hello Matthias, Am 14.03.2016 um 18:43 schrieb Matthias Vallentin: >> See attached bro-swig-python-failure.log. >> [..] >> The install phase is listed in attached bro-warning.log. > > I cannot uncompress what you have attached. Various combinations of tar > switches seem to fail [1]. Feel free to send the relevant part of the > log file inline. As long as we're not talking multiple MBs, you can also > send plain text attachments to the list. > > Matthias > > [1] https://xkcd.com/1168/ The attached files are compressed with xz; `xz -d` should do the trick if installed. Tar can only handle files which are additionally tar'd. Mails to the list are capped at 100KBytes (including the e-mail itself), which is easily exceeded with MIME encoded stuff. That's why I compressed them. With kind regards, tomboy64 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 951 bytes Desc: OpenPGP digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160315/cb6e9f60/attachment.bin From valerio.click at gmx.com Tue Mar 15 12:47:35 2016 From: valerio.click at gmx.com (Valerio) Date: Tue, 15 Mar 2016 20:47:35 +0100 Subject: [Bro] Access pcap filename in script land In-Reply-To: References: <56E7302E.6070609@gmx.com> Message-ID: <56E866D7.3070404@gmx.com> Hi Anthony, Thanks for your answer. Let me just generalize my requirement: Is it possible when I start bro to pass external arguments to a bro script? I think this feature would be useful in case you want to add some external info (not strictly present in the pcap or flow that bro i analyzing) into .log files produced by a bro script. thanks, Valerio On 15/03/2016 01:23, anthony kasza wrote: > Nope. > > -AK > On Mar 14, 2016 2:51 PM, "Valerio" wrote: > >> Hi all, >> >> in case bro is executed offline on a pcap with: >> >> bro -r file1.pcap script.bro >> >> is there a directive I can insert into script.bro to access the pcap >> filename? >> >> many thanks in advance, >> Valerio >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > From hacecky at jlab.org Tue Mar 15 14:15:44 2016 From: hacecky at jlab.org (Eric Hacecky) Date: Tue, 15 Mar 2016 17:15:44 -0400 (EDT) Subject: [Bro] How should I be calling an external script from Bro? In-Reply-To: <443660264.10433638.1458076492241.JavaMail.zimbra@jlab.org> References: <1442241653.10099820.1457988494716.JavaMail.zimbra@jlab.org> <61D4AFB1-D106-41DF-A59D-DE68218893C2@illinois.edu> Message-ID: <54420445.10433727.1458076544388.JavaMail.zimbra@jlab.org> Justin, Thanks for the guidance, that got me on the right path. Here's where I am: IPBlock.bro // module IPBLOCK; export { redef enum Notice::Action += { ACTION_IPBLOCK, }; const block_types: set[Notice::Type] = {} &redef; } hook Notice::policy(n: Notice::Info) { add n$actions[ACTION_IPBLOCK]; local cmd = string_cat("/usr/bin/python /usr/local/bro/share/bro/site/scripts/blockIP.py -a Bro -c 'SQL Injection' -t 72", n$src); local res = Exec::run([$cmd=cmd]); } // local.bro // @load IPBlocker.bro redef IPBLOCK::block_types += { HTTP::SQL_Injection_Attacker, }; // ----------------- broctl takes it fine with no errors (not verified as working). I still don't understand what line 63 from your module is doing: // when (local res = Exec::run([$cmd=cmd, $stdin=stdin]) // What is local res? I don't understand how that is executing the command. Regards, Eric ----- Original Message ----- From: "Justin S Azoff" To: "Eric Hacecky" Cc: bro at bro.org Sent: Monday, March 14, 2016 5:03:42 PM Subject: Re: [Bro] How should I be calling an external script from Bro? Hi, This repo has code in it that does everything you are trying to do: https://github.com/ncsa/bhr-bro You should be able to see how to modify it for your environment. This video details how the Exec works: https://www.youtube.com/watch?v=oo4zDC24xHU -- - Justin Azoff > On Mar 14, 2016, at 4:48 PM, Eric Hacecky wrote: > > New to Bro. Trying to make sure I follow best practice here configuring it for my environment. > > Currently Bro generates an email alert for HTTP::SQL_Injection_Attacker from detect-sqli.bro. > > I wrote a python script to accept some parameters, including the attacker's IP that will put in a block at my firewall. > > I was just going to tail Bro's notice.log and pull out the IP to feed my script anytime a SQL attack was logged there, but I figured it would be better to get Bro to do some of that lifting for me instead. > > Being new to bro, I don't know how to do this. > > I've googled around a bit and this is my best guess. (definitely a guess) > > - Exec module is the best way to go about this? > > - If so, I'm going to do what...make a something.bro file that basically says > > @load base/utils/exec > > when ( happens = Exec::run($cmd="myScript.py 55.66.77.88 -time 720") ) > > - Then I would @load something.bro in my local.bro file > > ----------- > > Assuming that's the gist of it, how am I supposed to figure out what event to look for? > > when ( HTTP::SQL_Injection_Attacker )? > > Every example I look at has uses 'local result' instead. Ex. https://github.com/sooshie/bro-scripts/blob/master/misc/vt_check.bro > > when (local result = Exec::run). Why? I don't see result defined anywhere previously? I don't understand how that condition is ever met. > > How do I make bro pass the IP to my script? > > Exec::run($cmd="myScript.py [$host=c$id$orig_h]")? > > Thanks, > Eric > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From jazoff at illinois.edu Tue Mar 15 14:26:00 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Tue, 15 Mar 2016 21:26:00 +0000 Subject: [Bro] How should I be calling an external script from Bro? In-Reply-To: <54420445.10433727.1458076544388.JavaMail.zimbra@jlab.org> References: <1442241653.10099820.1457988494716.JavaMail.zimbra@jlab.org> <61D4AFB1-D106-41DF-A59D-DE68218893C2@illinois.edu> <54420445.10433727.1458076544388.JavaMail.zimbra@jlab.org> Message-ID: <0DD1ECE6-8B25-4A75-A1BF-EC0AD295D843@illinois.edu> > On Mar 15, 2016, at 5:15 PM, Eric Hacecky wrote: > > Justin, > > Thanks for the guidance, that got me on the right path. > > Here's where I am: > > IPBlock.bro > // > module IPBLOCK; > > export > { > redef enum Notice::Action += > { > ACTION_IPBLOCK, > }; > > const block_types: set[Notice::Type] = {} &redef; > > } > > hook Notice::policy(n: Notice::Info) > { > add n$actions[ACTION_IPBLOCK]; > > local cmd = string_cat("/usr/bin/python /usr/local/bro/share/bro/site/scripts/blockIP.py -a Bro -c 'SQL Injection' -t 72", n$src); > > local res = Exec::run([$cmd=cmd]); > } > // > > local.bro > // > @load IPBlocker.bro > > redef IPBLOCK::block_types += > { > HTTP::SQL_Injection_Attacker, > }; > // > > ----------------- > > broctl takes it fine with no errors (not verified as working). > > I still don't understand what line 63 from your module is doing: > // > when (local res = Exec::run([$cmd=cmd, $stdin=stdin]) > // > > What is local res? I don't understand how that is executing the command. > > Regards, > Eric That's not quite right.. it may run, but it won't do what you want. You're not looking at block_types inside the notice policy, so that is going to try to block every single host that sets off any notice. See how in my notice policy the first thing I do is hook Notice::policy(n: Notice::Info) { if ( n$note !in block_types ) return; that prevents it from running for notice types that are not in block_types. You also shouldn't hardcode SQL Injection, you should grab what is in n$note and use that for the message. I think you are over thinking things with the when block. That line is just doing local res = Exec::run([$cmd=cmd, $stdin=stdin]) Just run is an asynchronous operation so it needs to be wrapped in a when (). -- - Justin Azoff From mz89924 at 126.com Tue Mar 15 20:49:01 2016 From: mz89924 at 126.com (mz) Date: Wed, 16 Mar 2016 11:49:01 +0800 Subject: [Bro] Bro Tutorial Message-ID: <000101d17f36$c79afd80$56d0f880$@126.com> Dear all Where can I find the Bro Tutorial. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160316/d7fb5897/attachment.html From daniel.guerra69 at gmail.com Wed Mar 16 01:14:35 2016 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Wed, 16 Mar 2016 09:14:35 +0100 Subject: [Bro] Bro Tutorial In-Reply-To: <000101d17f36$c79afd80$56d0f880$@126.com> References: <000101d17f36$c79afd80$56d0f880$@126.com> Message-ID: <4E53537C-10F8-4F4D-85EE-D7B1321003EC@gmail.com> In google ;) > On 16 Mar 2016, at 04:49, mz wrote: > > Dear all > Where can I find the Bro Tutorial. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160316/ce4276b6/attachment.html From bkellogg at dresser-rand.com Wed Mar 16 06:11:50 2016 From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN)) Date: Wed, 16 Mar 2016 13:11:50 +0000 Subject: [Bro] Bro Tutorial In-Reply-To: <000101d17f36$c79afd80$56d0f880$@126.com> References: <000101d17f36$c79afd80$56d0f880$@126.com> Message-ID: https://www.bro.org/documentation/tutorials/index.html From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of mz Sent: Tuesday, March 15, 2016 11:49 PM To: bro at bro.org Subject: [Bro] Bro Tutorial Dear all Where can I find the Bro Tutorial. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160316/1bb856c2/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5073 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160316/1bb856c2/attachment-0001.bin From robin at icir.org Wed Mar 16 08:28:49 2016 From: robin at icir.org (Robin Sommer) Date: Wed, 16 Mar 2016 08:28:49 -0700 Subject: [Bro] Access pcap filename in script land In-Reply-To: <56E866D7.3070404@gmx.com> References: <56E7302E.6070609@gmx.com> <56E866D7.3070404@gmx.com> Message-ID: <20160316152849.GD40337@icir.org> On Tue, Mar 15, 2016 at 20:47 +0100, Valerio wrote: > Is it possible when I start bro to pass external arguments to a bro > script? Yes, you can override script variables, like this: # cat args.bro const pcap_file = "" &redef; event bro_init() { print(pcap_file); } # bro ./args.bro 'pcap_file=\"Foo\"' "Foo" Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From valerio.click at gmx.com Wed Mar 16 09:10:31 2016 From: valerio.click at gmx.com (Valerio) Date: Wed, 16 Mar 2016 17:10:31 +0100 Subject: [Bro] Access pcap filename in script land In-Reply-To: <20160316152849.GD40337@icir.org> References: <56E7302E.6070609@gmx.com> <56E866D7.3070404@gmx.com> <20160316152849.GD40337@icir.org> Message-ID: <82A8DAAC-3862-484F-BD1E-6A335A86024B@gmx.com> Thanks a lot!!! Valerio > Il giorno 16 mar 2016, alle ore 16:28, Robin Sommer ha scritto: > > > >> On Tue, Mar 15, 2016 at 20:47 +0100, Valerio wrote: >> >> Is it possible when I start bro to pass external arguments to a bro >> script? > > Yes, you can override script variables, like this: > > # cat args.bro > > const pcap_file = "" &redef; > > event bro_init() > { > print(pcap_file); > } > > # bro ./args.bro 'pcap_file=\"Foo\"' > "Foo" > > > Robin > > -- > Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From tomboy64 at sina.cn Wed Mar 16 12:06:35 2016 From: tomboy64 at sina.cn (M.B.) Date: Wed, 16 Mar 2016 20:06:35 +0100 Subject: [Bro] Bro packaged for Gentoo In-Reply-To: <355BB6ED-8ACD-4FD4-B416-0B1AB1BD08BB@illinois.edu> References: <56E57DAE.5060103@sina.cn> <56E5E63C.1060503@sina.cn> <355BB6ED-8ACD-4FD4-B416-0B1AB1BD08BB@illinois.edu> Message-ID: <56E9AEBB.2080506@sina.cn> Hello again, I managed to resolve most issues with Bro and submitted the .ebuild via PR. If someone would like to review and comment on it, feel free to do so. https://github.com/gentoo/gentoo/pull/1069/ Otherwise, I'm positive it will hit the tree within a week or so. With kind regards, tomboy64 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 951 bytes Desc: OpenPGP digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160316/77e2d3dd/attachment.bin From dnj0496 at gmail.com Wed Mar 16 18:21:16 2016 From: dnj0496 at gmail.com (Dk Jack) Date: Wed, 16 Mar 2016 18:21:16 -0700 Subject: [Bro] bro http/ssl question Message-ID: Hi, I have a unique situation where I am receiving traffic traffic from two interfaces eth0 and eth1. I've modified the node.cfg file to distribute the traffic to multiple workers i.e. two workers for eth0 and two workers for eth1. The interface eth0 receives HTTP traffic and the interface eth1 is receiving HTTPS traffic. The tricky parties, both interfaces are actually receiving the same traffic i.e. same 5-tuple (src.ip/port, dst.ip/port, protocol). The port number for the plain HTTP traffic is also rcvd on port 443. The diagram below shows the details: +-----------+ ------+------>| decryptor |-----------+ HTTP | +-----------+ | dst.port=443 | V | +-----------+ | HTTPS | eth0 | | dst.port=443 | | +------------------------>|eth1 | | | | Bro | +-----------+ For a second, if we forget the question of "Why are you doing this crazy stuff?", would such a setup cause problems for Bro? What I've noticed is that (although the traffic volume is relatively the same on both interfaces) the connections are not showing up in the http.log. Although, some of them do show up (less than 1% of the traffic). The ssl.log shows a record for each connections. I am suspecting that un-encrypted http traffic received on port 443 is being parsed as ssl traffic by Bro. Is my observation correct? Is there a way to force the Bro to interpret the plain http data correctly in this sort of configuration? Thanks. Dk. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160316/4ce5fae3/attachment.html From jazoff at illinois.edu Thu Mar 17 06:29:13 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Thu, 17 Mar 2016 13:29:13 +0000 Subject: [Bro] bro http/ssl question In-Reply-To: References: Message-ID: <9A3785F9-B9D2-4454-A3FA-6B8DBDA79FF7@illinois.edu> > On Mar 16, 2016, at 9:21 PM, Dk Jack wrote: > > What I've noticed is that (although the traffic volume is relatively the same on both interfaces) the > connections are not showing up in the http.log. Although, some of them do show up (less than 1% > of the traffic). The ssl.log shows a record for each connections. I am suspecting that un-encrypted > http traffic received on port 443 is being parsed as ssl traffic by Bro. > That port doesn't matter... Does that decryption device send correct tcp checksums? The lack of proper checksums would explain why most of the traffic is missing. See https://www.bro.org/documentation/faq.html#why-isn-t-bro-producing-the-logs-i-expect-a-note-about-checksums If the traffic was being received on the same interface I'd say that this probably wouldn't work at all since the tcp reassembler would get horribly confused, but since separate processes are receiving the different streams I think it should work. you say that the unencrypted connections are not showing up in http.log, are they showing up in the conn.log? -- - Justin Azoff From puntogtg at tiscali.it Thu Mar 17 08:33:25 2016 From: puntogtg at tiscali.it (puntogtg at tiscali.it) Date: Thu, 17 Mar 2016 16:33:25 +0100 Subject: [Bro] =?utf-8?q?x509=2Elog=3A_certificate=2Enot=5Fvalid=5Fbefore_?= =?utf-8?q?=26_certificate=2Enot=5Fvalid=5Fafter?= Message-ID: <93646c34c6cf64734009c66d2c47929f@tiscali.it> Hello, in the x509.log normally the values regarding certificate.not_valid_before & certificate.not_valid_after look like: 1444082400.000000 1475791199.000000 I found some value like this: -3600.000000 2.153226e+09 Is it possible to modify something in order to have 2153226000 instead 2.153226e+09 ? Thanks Connetti gratis il mondo con la nuova indoona: hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo. E chiami gratis anche i numeri fissi e mobili nel mondo! Scarica subito l?app Vai su https://www.indoona.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160317/c66e7a9e/attachment.html From tgdesrochers at gmail.com Thu Mar 17 09:27:35 2016 From: tgdesrochers at gmail.com (Tim Desrochers) Date: Thu, 17 Mar 2016 12:27:35 -0400 Subject: [Bro] [bro] ssh connetions. Message-ID: <56eadaf7.4551370a.d0a4c.2ca2@mx.google.com> Is it possible for someone to establish an SSH session but the bro log not to show ?auth_success? as true. Thanks Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160317/2514e98c/attachment.html From vladg at illinois.edu Thu Mar 17 10:40:22 2016 From: vladg at illinois.edu (Vlad Grigorescu) Date: Thu, 17 Mar 2016 12:40:22 -0500 Subject: [Bro] [bro] ssh connetions. In-Reply-To: <56eadaf7.4551370a.d0a4c.2ca2@mx.google.com> References: <56eadaf7.4551370a.d0a4c.2ca2@mx.google.com> Message-ID: Yes. A good example of this is if SSH compression is enabled. I would hope that auth_success is set to "-" and not set to the incorrect T or F state, but it's possible that there's some server/client combination out there that's throwing off the detection. If you are seeing such cases, please send a PCAP and I can look at improving the detection. --Vlad Tim Desrochers writes: > [ text/plain ] > Is it possible for someone to establish an SSH session but the bro log not to show ?auth_success? as true. > > Thanks > Tim > > > > [ text/plain ] > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160317/a5b6dab3/attachment-0001.bin From tgdesrochers at gmail.com Thu Mar 17 10:43:28 2016 From: tgdesrochers at gmail.com (Tim Desrochers) Date: Thu, 17 Mar 2016 13:43:28 -0400 Subject: [Bro] [bro] ssh connetions. In-Reply-To: References: <56eadaf7.4551370a.d0a4c.2ca2@mx.google.com> Message-ID: <56eaecc0.d7fd8c0a.260bf.35cf@mx.google.com> What I am seeing is ?-? but on successful connects from internal host to internal host I am seeing ?success? I am in the process of examining pcap and auth logs on the server at this moment to determine success or failure From: Vlad Grigorescu Sent: Thursday, March 17, 2016 1:40 PM To: Tim Desrochers; bro at bro.org Subject: Re: [Bro] [bro] ssh connetions. Yes. A good example of this is if SSH compression is enabled. I would hope that auth_success is set to "-" and not set to the incorrect T or F state, but it's possible that there's some server/client combination out there that's throwing off the detection. If you are seeing such cases, please send a PCAP and I can look at improving the detection. --Vlad Tim Desrochers writes: > [ text/plain ] > Is it possible for someone to establish an SSH session but the bro log not to show ?auth_success? as true. > > Thanks > Tim > > > > [ text/plain ] > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160317/3db1c23d/attachment.html From bkellogg at dresser-rand.com Thu Mar 17 10:55:40 2016 From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN)) Date: Thu, 17 Mar 2016 17:55:40 +0000 Subject: [Bro] [bro] ssh connetions. In-Reply-To: References: <56eadaf7.4551370a.d0a4c.2ca2@mx.google.com> Message-ID: Similarly I?ve seen SSH sessions not identified when SSH is multiplexed with other protocols on the same port; e.g. SSH and HTTP on port 80. Wish I had more time to help with detecting cases like this. https://github.com/stealth/sshttp -----Original Message----- From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Vlad Grigorescu Sent: Thursday, March 17, 2016 1:40 PM To: Tim Desrochers; bro at bro.org Subject: Re: [Bro] [bro] ssh connetions. Yes. A good example of this is if SSH compression is enabled. I would hope that auth_success is set to "-" and not set to the incorrect T or F state, but it's possible that there's some server/client combination out there that's throwing off the detection. If you are seeing such cases, please send a PCAP and I can look at improving the detection. --Vlad Tim Desrochers writes: > [ text/plain ] > Is it possible for someone to establish an SSH session but the bro log not to show ?auth_success? as true. > > Thanks > Tim > > > > [ text/plain ] > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5073 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160317/16cb05b8/attachment.bin From jazoff at illinois.edu Thu Mar 17 11:03:22 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Thu, 17 Mar 2016 18:03:22 +0000 Subject: [Bro] [bro] ssh connetions. In-Reply-To: References: <56eadaf7.4551370a.d0a4c.2ca2@mx.google.com> Message-ID: > On Mar 17, 2016, at 1:55 PM, Kellogg, Brian D (OLN) wrote: > > Similarly I?ve seen SSH sessions not identified when SSH is multiplexed with other protocols on the same port; e.g. SSH and HTTP on port 80. Wish I had more time to help with detecting cases like this. > > https://github.com/stealth/sshttp I've been working on that as part of https://bro-tracker.atlassian.net/browse/BIT-1521 There's a bug in the current known services policy that causes multiple protocols on the same port to not be logged to known_services.log, but they should still show up in conn.log as the proper service. There is a slightly different but related issue in that if you send an http request to an ssh server or an ssh client banner to an http server, bro won't attach both analyzers to the connection. So, you'll get either an http log or an ssh log, but not both. -- - Justin Azoff From bkellogg at dresser-rand.com Thu Mar 17 11:14:18 2016 From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN)) Date: Thu, 17 Mar 2016 18:14:18 +0000 Subject: [Bro] [bro] ssh connetions. In-Reply-To: References: <56eadaf7.4551370a.d0a4c.2ca2@mx.google.com> Message-ID: Nice, thanks for the explanation. I sometimes see this when working cases in ELSA as I always look for BRO_SSH entries. It?s the second case that I believe I'm seeing and I think Seth explained it to me a couple months back as well. Suricata picks them up, so it hasn't been a high priority for me to delve into the Bro analyzers. That and I haven't done any real C++ programming in a very long time though I wish I could. -----Original Message----- From: Azoff, Justin S [mailto:jazoff at illinois.edu] Sent: Thursday, March 17, 2016 2:03 PM To: Kellogg, Brian D (OLN) Cc: Grigorescu, Vlad; Tim Desrochers; bro at bro.org Subject: Re: [Bro] [bro] ssh connetions. > On Mar 17, 2016, at 1:55 PM, Kellogg, Brian D (OLN) wrote: > > Similarly I?ve seen SSH sessions not identified when SSH is multiplexed with other protocols on the same port; e.g. SSH and HTTP on port 80. Wish I had more time to help with detecting cases like this. > > https://github.com/stealth/sshttp I've been working on that as part of https://bro-tracker.atlassian.net/browse/BIT-1521 There's a bug in the current known services policy that causes multiple protocols on the same port to not be logged to known_services.log, but they should still show up in conn.log as the proper service. There is a slightly different but related issue in that if you send an http request to an ssh server or an ssh client banner to an http server, bro won't attach both analyzers to the connection. So, you'll get either an http log or an ssh log, but not both. -- - Justin Azoff -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5073 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160317/e37660ba/attachment-0001.bin From mz89924 at 126.com Thu Mar 17 19:13:08 2016 From: mz89924 at 126.com (mz) Date: Fri, 18 Mar 2016 10:13:08 +0800 Subject: [Bro] how to filter http traffic Message-ID: <000001d180bb$b7b88d20$2729a760$@126.com> Dear I only want to monitor and capture http traffic, how to configure? In addition, I found that when reading official manual picture below: It will figure event where the configuration script engine -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160318/4972fda1/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/jpeg Size: 9508 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160318/4972fda1/attachment.jpe From tgdesrochers at gmail.com Fri Mar 18 03:25:23 2016 From: tgdesrochers at gmail.com (Tim Desrochers) Date: Fri, 18 Mar 2016 06:25:23 -0400 Subject: [Bro] [bro] Scanning IP's In-Reply-To: References: Message-ID: Sorry to beat a dead horse here but I am having a few issues with setting the alert_email_types. I set the following in my local.bro: redef Notice::emailed_types += { Weird::Activity, Signatures::Sensitive_Signature, Signatures::Multiple_Signatures, Signatures::Multiple_Sig_Responders, Signatures::Count_Signature, Intel::Notice, TeamCymruMalwareHashRegistry::Match, Traceroute::Detected, FTP::Bruteforcing, FTP::Site_Exec_Success, SMTP::Blocklist_Error_Message, SMTP::Blocklist_Blocked_Host, SMTP::Suspicious_Origination, SSH::Login_By_Password_Guesser, SSH::Interesting_Hostname_Login, }; Now here I would expect to only get emails from the notice framework for the defined types. But in actuality I get email from other things as well such as SQL_Injection, Weird_Activity, etc. I want the notice framework to log all these action but I don't want emails sent to me for them. I am using the emailed types to send emails to a alert dashboard for analysts to looka t. I only want things to go there that require immediate action by the analyst, all other notices I want logged and they can view them when they do their hourly checks of the net. Did I configure the email_types incorrectly. The end of my local.bro files contains the following email types modifications I have made: redef Notice::emailed_types += { Weird::Activity, Signatures::Sensitive_Signature, Signatures::Multiple_Signatures, Signatures::Multiple_Sig_Responders, Signatures::Count_Signature, Intel::Notice, TeamCymruMalwareHashRegistry::Match, Traceroute::Detected, FTP::Bruteforcing, FTP::Site_Exec_Success, SMTP::Blocklist_Error_Message, SMTP::Blocklist_Blocked_Host, SMTP::Suspicious_Origination, SSH::Login_By_Password_Guesser, SSH::Interesting_Hostname_Login, }; # Only receive Scan Notices if they are from local network. const local_emailed_types: set[Notice::Type] = { SSH::Password_Guessing, } &redef; hook Notice::policy(n: Notice::Info) { if (n$note in local_emailed_types && Site::is_local_addr(n$src)) add n$actions[Notice::ACTION_EMAIL]; } Any help would be appreciated. Thanks On Sun, Feb 14, 2016 at 8:42 AM, Tim Desrochers wrote: > Followup question: > > If I set this will I still get the other notices emailed to me such as > items from the intel framework that I have set meta.do_notice and > meta.if_in. Or will I have to make another notice hook to still allow for > those to send emails when observed. > > Obviously I have some bro scripting classes to attend, but in the > meanwhile I am just trying to hack this together. > > Tim > > On Sun, Feb 14, 2016 at 7:35 AM, Azoff, Justin S > wrote: > >> The thing to understand is that the ignored_types and emailed_types are >> just tables defined to make tweaking the base notice policy easier. >> >> That default notice policy is: >> >> hook Notice::policy(n: Notice::Info) &priority=10 >> { >> if ( n$note in Notice::ignored_types ) >> break; >> >> if ( n$note in Notice::not_suppressed_types ) >> n$suppress_for=0secs; >> if ( n$note in Notice::alarmed_types ) >> add n$actions[ACTION_ALARM]; >> if ( n$note in Notice::emailed_types ) >> add n$actions[ACTION_EMAIL]; >> >> if ( n$note in Notice::type_suppression_intervals ) >> n$suppress_for=Notice::type_suppression_intervals[n$note]; >> >> # Logging is a default action. It can be removed in a later hook >> if desired. >> add n$actions[ACTION_LOG]; >> } >> >> As you can see, adding notice types to those tables just tweaks the >> behavior of the default Notice::policy hook. To do some of the things you >> want to do, you just need a hook like >> >> hook Notice::policy(n: Notice::Info) >> { >> if (n$note == Scan::Port_Scan && Site::is_local_addr(n$src)) >> add n$actions[Notice::ACTION_EMAIL]; >> } >> >> If that would get repetitive, you can create your own table like >> >> const local_emailed_types: set[Notice::Type] = {} &redef; >> >> and have the policy be >> >> hook Notice::policy(n: Notice::Info) >> { >> if (n$note in local_emailed_types && Site::is_local_addr(n$src)) >> add n$actions[Notice::ACTION_EMAIL]; >> } >> >> -- >> - Justin Azoff >> >> > On Feb 14, 2016, at 6:14 AM, Tim Desrochers >> wrote: >> > >> > As with every infrastructure I am plagued with people scanning my >> external edge. I see little value in getting notices for scanning attempts >> and password guessing attempts but I do see value in running monthly >> reports and generating blocklists based on repeat offenders. >> > >> > Is there a way to tell the notice framework to only create alarms >> (emails) if it sees scans of any kind (address, port, password guessing, >> etc) if they are from the IP's in my $HOME_NET defined in network.cfg? >> > >> > Justification, If I >> > >> > redef Notice::ignored_types += { >> > SSH::Password_Guessing, >> > Scan::Address_Scan, >> > Scan::Port_Scan, >> > HTTP::SQL_Injection_Attacker, >> > ShellShock::Scanner, >> > ScanUDP::Address_Scan, >> > ScanUDP::Port_Scan, >> > }; >> > >> > Then I get no logging of the events anywhere. Therefore I can't run >> reports of offenders and build active blocklists or other intel gathering >> activities. >> > >> > If I: >> > >> > # Set rule to only email specific notice types: >> > redef Notice::emailed_types += { >> > Weird::Activity, >> > Signatures::Sensitive_Signature, >> > Signatures::Multiple_Signatures, >> > Signatures::Multiple_Sig_Responders, >> > Signatures::Count_Signature, >> > Intel::Notice, >> > TeamCymruMalwareHashRegistry::Match, >> > Traceroute::Detected, >> > FTP::Bruteforcing, >> > FTP::Site_Exec_Success, >> > HTTP::SQL_Injection_Victim, >> > SMTP::Blocklist_Error_Message, >> > SMTP::Blocklist_Blocked_Host, >> > SMTP::Suspicious_Origination, >> > SSH::Login_By_Password_Guesser, >> > SSH::Interesting_Hostname_Login, >> > }; >> > >> > Then I get flooded with email from any of the guessing activity (Side >> note: I find that the above logic doesn't restrict email notices to just >> those listed in the defined email types above. I still get plenty of >> notices about events not listed in the list above). If the redef >> Notice::emailed_types worked it would be a start but I'd still like to get >> emails about IP addresses in my internal net getting scanned by other IP's >> in my internal net, that definitely an indicator of unwanted behavior. >> > >> > Any assistance would be greatly appreciated. Just trying to tune >> things to a manageable level. >> > >> > Thanks >> > Tim >> > _______________________________________________ >> > Bro mailing list >> > bro at bro-ids.org >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160318/f34baf18/attachment-0001.html From jan.grashoefer at gmail.com Fri Mar 18 04:36:06 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Fri, 18 Mar 2016 12:36:06 +0100 Subject: [Bro] [bro] Scanning IP's In-Reply-To: References: Message-ID: <56EBE826.7000401@gmail.com> Hi, > redef Notice::emailed_types += { Blind guess, try: redef Notice::emailed_types = { Regards, Jan From jlay at slave-tothe-box.net Fri Mar 18 14:47:49 2016 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 18 Mar 2016 15:47:49 -0600 Subject: [Bro] Notice on duration Message-ID: <74889ccbe1083f200ac899e86387800d@localhost> Hey all, I've been tasked with seeing about getting an alert of some kind when a session (tcp/udp/icmp) lasts longer then a certain time. Is this something well suited for bro, or should I go looking at something like ntop-ng instead? Thank you. James From aniketpsavanand at gmail.com Sat Mar 19 17:43:08 2016 From: aniketpsavanand at gmail.com (Aniket Savanand) Date: Sat, 19 Mar 2016 17:43:08 -0700 Subject: [Bro] Requesting some pointers- Adding a new protocol to BRO- Facing problems Message-ID: Hi I am trying to write a new protocol AMQP to the BRO. So I wrote analyzer files for AMQP by referring to the existing protocols files written in src/analyzer/protocol. I build and installed it correctly. and even tried to detect AMQP traffic using BRO. But this case BRO does not. Where would be wrong? is it the correct way to add new protocol/analyzer to the BRO? Could you point me to right direction. Thanks Aniket Savanand SJSU, CA 669-226-8162 -- *Regards, * *Aniket Savanand,* *MS Software Engineering 2016,* *San Jose State University, CA* *Email **Cellphone- +1-669-226-8162* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160319/b3107868/attachment.html From pratikinamdar at gmail.com Sun Mar 20 18:50:41 2016 From: pratikinamdar at gmail.com (pratik inamdar) Date: Sun, 20 Mar 2016 18:50:41 -0700 Subject: [Bro] Integrating WiFi Analyzer within Bro Message-ID: Hi, In my project, I am integrating a WiFi protocol analyzer with bro to parse and monitor WiFi packets header information. I am using BinPac to generate template for the WiFi protocol analyzer in the src/analyzer/protocol directory. As per my knowledge WiFi(802.11) is not a TCP type of protocol. So I wish to know what should I use instead of the option "--tcp" while using the command: python start.py WiFi "WiFi Protocol" ../bro --tcp Any help will be really appreciated! -- Thanks & Regards. Pratik Inamdar -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160320/3a272c66/attachment.html From js688886 at gmail.com Mon Mar 21 08:01:50 2016 From: js688886 at gmail.com (john smith) Date: Mon, 21 Mar 2016 11:01:50 -0400 Subject: [Bro] BRO 2.4.1. extracted file handling Message-ID: I?ve two questions about file extraction handling with BRO 2.4.1. 1. Right now, all the extracted files are in ASCII format. Is there any easy way to save them in JSON? 2. Would it be possible to add an extracted file itself to file.log? If not, is there any way to copy the extracted file to a new log stream? Thank you very much in advance. John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160321/6b281c6c/attachment.html From vladg at illinois.edu Mon Mar 21 08:28:18 2016 From: vladg at illinois.edu (Vlad Grigorescu) Date: Mon, 21 Mar 2016 10:28:18 -0500 Subject: [Bro] Integrating WiFi Analyzer within Bro In-Reply-To: References: Message-ID: Unfortunately, there is no way to implement lower level protocols with BinPAC quickstart right now. Similary, we don't have any examples of a BinPAC lower-level analyzer if you were to do it manually. If you are able to get it working, I'd certainly be interested in how you did it, and would look at adding it to binpac_quickstart. --Vlad pratik inamdar writes: > [ text/plain ] > Hi, > > In my project, I am integrating a WiFi protocol analyzer with bro to parse > and monitor WiFi packets header information. > > I am using BinPac to generate template for the WiFi protocol analyzer in > the src/analyzer/protocol directory. > > As per my knowledge WiFi(802.11) is not a TCP type of protocol. So I wish > to know what should I use instead of the option "--tcp" while using the > command: > > python start.py WiFi "WiFi Protocol" ../bro --tcp > > Any help will be really appreciated! > > -- > > Thanks & Regards. > > Pratik Inamdar > [ text/plain ] > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160321/665533db/attachment.bin From vladg at illinois.edu Mon Mar 21 08:31:13 2016 From: vladg at illinois.edu (Vlad Grigorescu) Date: Mon, 21 Mar 2016 10:31:13 -0500 Subject: [Bro] Requesting some pointers- Adding a new protocol to BRO- Facing problems In-Reply-To: References: Message-ID: Hello, Our relevant documentation is available at: https://www.bro.org/development/howtos/dpd.html https://www.bro.org/development/howtos/binpac-sample-analyzer.html My guess is that there's an issue with how the analyzer is registered in the Bro scripts and it's not being attached to the correct traffic. The DPD write-up should go into detail about that. --Vlad Aniket Savanand writes: > [ text/plain ] > Hi > > I am trying to write a new protocol AMQP to the BRO. > So I wrote analyzer files for AMQP by referring to the existing protocols > files written in src/analyzer/protocol. > I build and installed it correctly. and even tried to detect AMQP traffic > using BRO. > But this case BRO does not. > > Where would be wrong? is it the correct way to add new protocol/analyzer to > the BRO? > > Could you point me to right direction. > > Thanks > Aniket Savanand > SJSU, CA > 669-226-8162 > > -- > *Regards, * > *Aniket Savanand,* > *MS Software Engineering 2016,* > *San Jose State University, CA* > *Email **Cellphone- +1-669-226-8162* > [ text/plain ] > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160321/b4197f7b/attachment.bin From pratikinamdar at gmail.com Mon Mar 21 08:43:53 2016 From: pratikinamdar at gmail.com (pratik inamdar) Date: Mon, 21 Mar 2016 08:43:53 -0700 Subject: [Bro] Integrating WiFi Analyzer within Bro In-Reply-To: References: Message-ID: Hi, Thank you for your reply! Is BinPac parser the only way to generate an analyzer for a protocol? What is other way I can do it? Because as far as I know, BinPac also generated 3 files automatically in base/scripts folder which cannot be done manually. Please let me know another way I could do it without using BinPac. Please shed some light on this. Thanks, Pratik inamdar On Mar 21, 2016 08:28, "Vlad Grigorescu" wrote: > Unfortunately, there is no way to implement lower level protocols with > BinPAC quickstart right now. Similary, we don't have any examples of a > BinPAC lower-level analyzer if you were to do it manually. > > If you are able to get it working, I'd certainly be interested in how > you did it, and would look at adding it to binpac_quickstart. > > --Vlad > > pratik inamdar writes: > > > [ text/plain ] > > Hi, > > > > In my project, I am integrating a WiFi protocol analyzer with bro to > parse > > and monitor WiFi packets header information. > > > > I am using BinPac to generate template for the WiFi protocol analyzer in > > the src/analyzer/protocol directory. > > > > As per my knowledge WiFi(802.11) is not a TCP type of protocol. So I wish > > to know what should I use instead of the option "--tcp" while using the > > command: > > > > python start.py WiFi "WiFi Protocol" ../bro --tcp > > > > Any help will be really appreciated! > > > > -- > > > > Thanks & Regards. > > > > Pratik Inamdar > > [ text/plain ] > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160321/59ba3cff/attachment.html From vladg at illinois.edu Mon Mar 21 09:06:02 2016 From: vladg at illinois.edu (Vlad Grigorescu) Date: Mon, 21 Mar 2016 11:06:02 -0500 Subject: [Bro] Notice on duration In-Reply-To: <74889ccbe1083f200ac899e86387800d@localhost> References: <74889ccbe1083f200ac899e86387800d@localhost> Message-ID: Hi James, James Lay writes: > I've been tasked with seeing about getting an alert of some kind when a > session (tcp/udp/icmp) lasts longer then a certain time. Is this > something well suited for bro...? It should be. Check out ConnPolling: https://www.bro.org/sphinx/scripts/base/protocols/conn/polling.bro.html This is a little-known feature that hasn't seen much use, but I'd be very interested if you get this working for your use-case. So far, it's been used to look for large (or fast) connections, such as: https://github.com/JustinAzoff/bro-react/blob/master/conn-bulk.bro --Vlad -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160321/6d78f6e3/attachment.bin From jlay at slave-tothe-box.net Mon Mar 21 11:12:34 2016 From: jlay at slave-tothe-box.net (James Lay) Date: Mon, 21 Mar 2016 12:12:34 -0600 Subject: [Bro] Notice on duration In-Reply-To: References: <74889ccbe1083f200ac899e86387800d@localhost> Message-ID: <90ab812762fca8bcc1d7306ead955ecf@localhost> On 2016-03-21 10:06, Vlad Grigorescu wrote: > Hi James, > > James Lay writes: > >> I've been tasked with seeing about getting an alert of some kind when >> a >> session (tcp/udp/icmp) lasts longer then a certain time. Is this >> something well suited for bro...? > > It should be. Check out ConnPolling: > > https://www.bro.org/sphinx/scripts/base/protocols/conn/polling.bro.html > > This is a little-known feature that hasn't seen much use, but I'd be > very interested if you get this working for your use-case. So far, it's > been used to look for large (or fast) connections, such as: > > https://github.com/JustinAzoff/bro-react/blob/master/conn-bulk.bro > > --Vlad Thanks Vlad...I'll give this a go. James From aniketpsavanand at gmail.com Mon Mar 21 13:47:47 2016 From: aniketpsavanand at gmail.com (Aniket Savanand) Date: Mon, 21 Mar 2016 13:47:47 -0700 Subject: [Bro] Requesting some pointers- Adding a new protocol to BRO- Facing problems In-Reply-To: References: Message-ID: Thank a lot. I will start integrating AMQP analyzer with step mentioned on binpac page. Thanks Aniket On Mon, Mar 21, 2016 at 8:31 AM, Vlad Grigorescu wrote: > Hello, > > Our relevant documentation is available at: > > https://www.bro.org/development/howtos/dpd.html > https://www.bro.org/development/howtos/binpac-sample-analyzer.html > > My guess is that there's an issue with how the analyzer is registered in > the Bro scripts and it's not being attached to the correct traffic. The > DPD write-up should go into detail about that. > > --Vlad > > Aniket Savanand writes: > > > [ text/plain ] > > Hi > > > > I am trying to write a new protocol AMQP to the BRO. > > So I wrote analyzer files for AMQP by referring to the existing protocols > > files written in src/analyzer/protocol. > > I build and installed it correctly. and even tried to detect AMQP traffic > > using BRO. > > But this case BRO does not. > > > > Where would be wrong? is it the correct way to add new protocol/analyzer > to > > the BRO? > > > > Could you point me to right direction. > > > > Thanks > > Aniket Savanand > > SJSU, CA > > 669-226-8162 > > > > -- > > *Regards, * > > *Aniket Savanand,* > > *MS Software Engineering 2016,* > > *San Jose State University, CA* > > *Email **Cellphone- +1-669-226-8162* > > [ text/plain ] > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- *Regards, * *Aniket Savanand,* *MS Software Engineering 2016,* *San Jose State University, CA* *Email **Cellphone- +1-669-226-8162* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160321/3c04d4bd/attachment.html From mz89924 at 126.com Tue Mar 22 02:52:51 2016 From: mz89924 at 126.com (mz) Date: Tue, 22 Mar 2016 17:52:51 +0800 Subject: [Bro] how to send log with kafka Message-ID: <000c01d18420$99cec5a0$cd6c50e0$@126.com> Dear all How to make use librdkafka bro logs to send by kafka? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160322/bdc00f7d/attachment.html From tgdesrochers at gmail.com Tue Mar 22 03:14:35 2016 From: tgdesrochers at gmail.com (Tim Desrochers) Date: Tue, 22 Mar 2016 06:14:35 -0400 Subject: [Bro] how to send log with kafka In-Reply-To: <000c01d18420$99cec5a0$cd6c50e0$@126.com> References: <000c01d18420$99cec5a0$cd6c50e0$@126.com> Message-ID: <56f11b17.8dcb370a.5db1b.fffff1dd@mx.google.com> I?m not sure I understand your question completely but you can look here for detailed instructions on how to enable bro logging to kafka. https://github.com/g-clef/KafkaLogger From: mz Sent: Tuesday, March 22, 2016 6:04 AM To: bro at bro.org Subject: [Bro] how to send log with kafka Dear all ???????? How to make use librdkafka bro logs to send by kafka? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160322/30ed0048/attachment.html From nick at nickallen.org Tue Mar 22 04:25:28 2016 From: nick at nickallen.org (Nick Allen) Date: Tue, 22 Mar 2016 07:25:28 -0400 Subject: [Bro] how to send log with kafka In-Reply-To: <000c01d18420$99cec5a0$cd6c50e0$@126.com> References: <000c01d18420$99cec5a0$cd6c50e0$@126.com> Message-ID: Bro-plugins now includes a Kafka writer. It uses librdkafka as part of the implementation. https://github.com/bro/bro-plugins On Mar 22, 2016 6:04 AM, "mz" wrote: > Dear all > > How to make use librdkafka bro logs to send by kafka? > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160322/1627a58c/attachment.html From franky.meier.1 at gmx.de Tue Mar 22 08:57:52 2016 From: franky.meier.1 at gmx.de (Frank Meier) Date: Tue, 22 Mar 2016 16:57:52 +0100 Subject: [Bro] redef script variable from bro command line Message-ID: <20160322165752.2b1be331@NB181106> Hi! This is just a minor problem, but I would still like to understand, why it does not work. I am trying to set the log rotation from the command line. It works when I use the -e flag: # bro -r 2009-m57.00.pcap -e "redef Log::default_rotation_interval = 20min" It does not work, if I set the variable at the command prompt: # bro test.bro -r 2009-m57.00.pcap rotation=20min 20.0 min 10.0 min test.bro: global rotation = 10min &redef; redef Log::default_rotation_interval = rotation; event bro_init() { print(rotation); print(Log::default_rotation_interval); } I suppose the reason for this is the order in which variables are initialized. Can someone please explain? Thanks! Franky From robin at icir.org Tue Mar 22 10:26:13 2016 From: robin at icir.org (Robin Sommer) Date: Tue, 22 Mar 2016 10:26:13 -0700 Subject: [Bro] redef script variable from bro command line In-Reply-To: <20160322165752.2b1be331@NB181106> References: <20160322165752.2b1be331@NB181106> Message-ID: <20160322172613.GA83560@icir.org> On Tue, Mar 22, 2016 at 16:57 +0100, Frank Meier wrote: > # bro -r 2009-m57.00.pcap -e "redef Log::default_rotation_interval = 20min" > # bro test.bro -r 2009-m57.00.pcap rotation=20min Well, you're setting different things. If I change the 2nd case to this: bro test.bro "Log::default_rotation_interval=20min" ... then it works as well. In your version, when you set "rotation" it changes that global value, but at a time when it's old value has already been assigned to Log::default_rotation_interval. The change will not carry over, as the assignment doesn't take place again. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From cosmotraumatika at gmail.com Tue Mar 22 11:14:20 2016 From: cosmotraumatika at gmail.com (Jamie Saker) Date: Tue, 22 Mar 2016 13:14:20 -0500 Subject: [Bro] Cluster minimal logs on manager Message-ID: <6CDF0917-EE4C-4D3A-B2AA-E091AA23304C@gmail.com> After upgrading/reinstalling the OS on my Bro manager, with a network of a dozen workers, I?ve managed to end up where I?m only seeing minimal logs at the manager (the manager is also the sole proxy): communication.log loaded_scripts.logreporter.log stderr.log stdout.log When I run Bro standalone on one of the sensors, all is well again. I?ve exchanged the keys so that Bro can manage the workers just fine but apparently the logging isn?t being communicated correctly. Any recommendations other than rebuilding sensors from the OS up? I also know the sensors are seeing good traffic - Snort runs just fine on a tested sensor along with tcpdump, etc. Thanks - Jamie From jazoff at illinois.edu Tue Mar 22 11:44:24 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Tue, 22 Mar 2016 18:44:24 +0000 Subject: [Bro] Cluster minimal logs on manager In-Reply-To: <6CDF0917-EE4C-4D3A-B2AA-E091AA23304C@gmail.com> References: <6CDF0917-EE4C-4D3A-B2AA-E091AA23304C@gmail.com> Message-ID: Can your workers connect to your manager on tcp port 47761-47763 or so? You probably want an iptables rule similar to iptables -A INPUT -s your.subnet.here/24 -p tcp -m multiport --dports 47000:48000 -m comment --comment "200 accept bro cluster connections" -j ACCEPT on the different machines so everything can communicate properly. -- - Justin Azoff > On Mar 22, 2016, at 2:14 PM, Jamie Saker wrote: > > After upgrading/reinstalling the OS on my Bro manager, with a network of a dozen workers, I?ve managed to end up where I?m only seeing minimal logs at the manager (the manager is also the sole proxy): > > communication.log > loaded_scripts.logreporter.log > stderr.log > stdout.log > > When I run Bro standalone on one of the sensors, all is well again. I?ve exchanged the keys so that Bro can manage the workers just fine but apparently the logging isn?t being communicated correctly. Any recommendations other than rebuilding sensors from the OS up? I also know the sensors are seeing good traffic - Snort runs just fine on a tested sensor along with tcpdump, etc. > > Thanks - > > Jamie > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From cosmotraumatika at gmail.com Tue Mar 22 12:02:34 2016 From: cosmotraumatika at gmail.com (Jamie Saker) Date: Tue, 22 Mar 2016 14:02:34 -0500 Subject: [Bro] Cluster minimal logs on manager In-Reply-To: References: <6CDF0917-EE4C-4D3A-B2AA-E091AA23304C@gmail.com> Message-ID: <0D4961AA-A928-4643-91A5-24A56BE2F0F8@gmail.com> Justin - That was it! Sigh? a little over-eager UFW implementation. I added the range and that did the trick. Now to lock it down to only sensor IPs? :) Thank you so much. Cheers - Jamie > On Mar 22, 2016, at 1:44 PM, Azoff, Justin S wrote: > > Can your workers connect to your manager on tcp port 47761-47763 or so? > > You probably want an iptables rule similar to > > iptables -A INPUT -s your.subnet.here/24 -p tcp -m multiport --dports 47000:48000 -m comment --comment "200 accept bro cluster connections" -j ACCEPT > > on the different machines so everything can communicate properly. > > -- > - Justin Azoff > >> On Mar 22, 2016, at 2:14 PM, Jamie Saker wrote: >> >> After upgrading/reinstalling the OS on my Bro manager, with a network of a dozen workers, I?ve managed to end up where I?m only seeing minimal logs at the manager (the manager is also the sole proxy): >> >> communication.log >> loaded_scripts.logreporter.log >> stderr.log >> stdout.log >> >> When I run Bro standalone on one of the sensors, all is well again. I?ve exchanged the keys so that Bro can manage the workers just fine but apparently the logging isn?t being communicated correctly. Any recommendations other than rebuilding sensors from the OS up? I also know the sensors are seeing good traffic - Snort runs just fine on a tested sensor along with tcpdump, etc. >> >> Thanks - >> >> Jamie >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > From seth at icir.org Tue Mar 22 12:42:24 2016 From: seth at icir.org (Seth Hall) Date: Tue, 22 Mar 2016 15:42:24 -0400 Subject: [Bro] x509.log: certificate.not_valid_before & certificate.not_valid_after In-Reply-To: <93646c34c6cf64734009c66d2c47929f@tiscali.it> References: <93646c34c6cf64734009c66d2c47929f@tiscali.it> Message-ID: > On Mar 17, 2016, at 11:33 AM, puntogtg at tiscali.it wrote: > > Is it possible to modify something in order to have 2153226000 instead 2.153226e+09 ? Yep, agreed. Scientific notation in the logs like that is not the desired output. Here's a link to the ticket I filed: https://bro-tracker.atlassian.net/browse/BIT-1558 .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From johanna at icir.org Tue Mar 22 13:29:35 2016 From: johanna at icir.org (Johanna Amann) Date: Tue, 22 Mar 2016 13:29:35 -0700 Subject: [Bro] x509.log: certificate.not_valid_before & certificate.not_valid_after In-Reply-To: References: <93646c34c6cf64734009c66d2c47929f@tiscali.it> Message-ID: <20160322202935.GB34567@wifi101.sys.ICSI.Berkeley.EDU> Somewhat related - do you happen to have the certificate (or the name of the site from which the certificate was served)? I am kind of curious how a negative timestamp managed to happen there... Johanna On Tue, Mar 22, 2016 at 03:42:24PM -0400, Seth Hall wrote: > > > On Mar 17, 2016, at 11:33 AM, puntogtg at tiscali.it wrote: > > > > Is it possible to modify something in order to have 2153226000 instead 2.153226e+09 ? > > Yep, agreed. Scientific notation in the logs like that is not the desired output. Here's a link to the ticket I filed: > https://bro-tracker.atlassian.net/browse/BIT-1558 > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From navraj42 at gmail.com Tue Mar 22 14:00:25 2016 From: navraj42 at gmail.com (Navraj Singh) Date: Tue, 22 Mar 2016 15:00:25 -0600 Subject: [Bro] Definition of a connection in conn.log Message-ID: Hi, I'm trying to understand how connections are formed by Bro before reporting them to conn.log - in particular, the following questions: 1. Is it safe to assume that any given packet will be assigned to at most one connection, and thus to at most one row in conn.log? 2. Why is it that some rows in conn.log do not have the duration field set? I see see several row with a '-' in the duration field. 3. The bro documentation states that "For UDP and ICMP, ?connections? are to be interpreted using flow semantics (sequence of packets from a source host/port to a destination host/port)." However, what is the exact definition for a TCP flow? How does Bro decide which packets to include in a connection? 4. For an ongoing 'connection', does Bro wait until the connection is over before logging it? What if the connection is quite long in duration...won't that cause a lag? Or does Bro automatically chop up long flows based on some configurable limit parameter? Basically, I'm trying to understand how Bro defines a 'connection', for the purposes of interpreting conn.log. I've looked at the online documentation but didn't find what I was trying to understand. Thanks to anyone who can shed some light on this, or point me in the right direction! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160322/30e784c3/attachment.html From franky.meier.1 at gmx.de Tue Mar 22 23:38:40 2016 From: franky.meier.1 at gmx.de (Frank Meier) Date: Wed, 23 Mar 2016 07:38:40 +0100 Subject: [Bro] redef script variable from bro command line In-Reply-To: <20160322172613.GA83560@icir.org> References: <20160322165752.2b1be331@NB181106> <20160322172613.GA83560@icir.org> Message-ID: <20160323073840.3d7427f9@NB181106> On Tue, 22 Mar 2016 10:26:13 -0700 Robin Sommer wrote: > > On Tue, Mar 22, 2016 at 16:57 +0100, Frank Meier wrote: > > > # bro -r 2009-m57.00.pcap -e "redef Log::default_rotation_interval > > = 20min" # bro test.bro -r 2009-m57.00.pcap rotation=20min > > Well, you're setting different things. If I change the 2nd case to > this: > > bro test.bro "Log::default_rotation_interval=20min" > > ... then it works as well. > That's an even easier solution. I thought I had to use "redef". > In your version, when you set "rotation" it changes that global value, > but at a time when it's old value has already been assigned to > Log::default_rotation_interval. The change will not carry over, as the > assignment doesn't take place again. > Thank you very much! This makes things clear. Franky From robin at icir.org Wed Mar 23 07:56:48 2016 From: robin at icir.org (Robin Sommer) Date: Wed, 23 Mar 2016 07:56:48 -0700 Subject: [Bro] redef script variable from bro command line In-Reply-To: <20160323073840.3d7427f9@NB181106> References: <20160322165752.2b1be331@NB181106> <20160322172613.GA83560@icir.org> <20160323073840.3d7427f9@NB181106> Message-ID: <20160323145648.GE13137@icir.org> On Wed, Mar 23, 2016 at 07:38 +0100, Frank Meier wrote: > That's an even easier solution. I thought I had to use "redef". The command line assignment is essentially a "redef", just shorter for convienience. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From scotty.b.brown at gmail.com Wed Mar 23 21:54:06 2016 From: scotty.b.brown at gmail.com (Scotty Brown) Date: Thu, 24 Mar 2016 14:54:06 +1000 Subject: [Bro] Bro email notice question Message-ID: <56F372EE.9060909@gmail.com> Hi all, I'm using Bro in Security Onion with Critical stack for intel feeds, we've alsoenabled email notices for Bro which are working well (as per https://github.com/Security-Onion-Solutions/security-onion/wiki/Email). The email notices generated though just contain something like: Message: Intel hit on 'some.domain' at 'DNS::IN_REQUEST' Sub-Message: some.domain Connection: x.x.x.x -> x.x.x.x Connection uid: aaaaa Email Extensions ----- orig/src hostname: box.internal resp/dst hostname: some.domain I then have to go grep the critical stack intel file for the description related to the particular hit to see whats up. I've tried, but can't figure out how I add $sources from the Intel log into say $sub in /opt/bro/share/bro/policy/intel/do_notice.bro I'm missing something small - can anyone help me out? Cheers, Scotty From mz89924 at 126.com Wed Mar 23 23:32:02 2016 From: mz89924 at 126.com (ine) Date: Thu, 24 Mar 2016 14:32:02 +0800 (CST) Subject: [Bro] about logs-to-elasticsearch.bro script Message-ID: <6145eb3b.7700.153a7529b88.Coremail.mz89924@126.com> Dear all how to set index when use logs-to-elasticsearch.bro. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160324/7a68111a/attachment.html From daniel.guerra69 at gmail.com Thu Mar 24 02:26:33 2016 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Thu, 24 Mar 2016 10:26:33 +0100 Subject: [Bro] about logs-to-elasticsearch.bro script In-Reply-To: <6145eb3b.7700.153a7529b88.Coremail.mz89924@126.com> References: <6145eb3b.7700.153a7529b88.Coremail.mz89924@126.com> Message-ID: What do you mean with index ? > On 24 Mar 2016, at 07:32, ine wrote: > > Dear all > how to set index when use logs-to-elasticsearch.bro. > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160324/e98a2f9f/attachment.html From jan.grashoefer at gmail.com Thu Mar 24 08:14:54 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Thu, 24 Mar 2016 16:14:54 +0100 Subject: [Bro] Bro email notice question In-Reply-To: <56F372EE.9060909@gmail.com> References: <56F372EE.9060909@gmail.com> Message-ID: <56F4046E.5070301@gmail.com> Hi Scotty, > I've tried, but can't figure out how I add $sources from the Intel log into say $sub in /opt/bro/share/bro/policy/intel/do_notice.bro Some time ago, I adapted the do_notice.bro script to add an identifier (for notice suppression) and also added some information (e.g. intel source) to the mails (see https://gist.github.com/J-Gras/c2e0853c93c0bdc74522). I hope this will help you :) Regards, Jan From grant at grantstavely.com Thu Mar 24 08:48:24 2016 From: grant at grantstavely.com (Grant Stavely) Date: Thu, 24 Mar 2016 08:48:24 -0700 Subject: [Bro] about logs-to-elasticsearch.bro script In-Reply-To: <6145eb3b.7700.153a7529b88.Coremail.mz89924@126.com> References: <6145eb3b.7700.153a7529b88.Coremail.mz89924@126.com> Message-ID: <4E345DD0-DB3C-455B-8885-E8DDD5F57E59@grantstavely.com> Hi ine, In local.bro, redef the consts defined in https://github.com/bro/bro-plugins/blob/9b7943e1a61062005f01b48eaad11bbb3b7ae757/elasticsearch/scripts/init.bro , e.g.: # Configure Elasticsearch redef LogElasticSearch::server_host = "x.x.x.x"; redef LogElasticSearch::server_port= 9200; redef LogElasticSearch::cluster_name = "security"; redef LogElasticSearch::index_prefix = "bro"; redef LogElasticSearch::excluded_log_ids += { Known::HOSTS_LOG, }; Grant > On Mar 23, 2016, at 23:32, ine wrote: > > Dear all > how to set index when use logs-to-elasticsearch.bro. > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160324/98578ac6/attachment.html From mz89924 at 126.com Thu Mar 24 18:30:22 2016 From: mz89924 at 126.com (mz) Date: Fri, 25 Mar 2016 09:30:22 +0800 Subject: [Bro] =?gb2312?b?tPC4tDogIGFib3V0IGxvZ3MtdG8tZWxhc3RpY3NlYXJjaC5i?= =?gb2312?b?cm8gc2NyaXB0?= In-Reply-To: References: <6145eb3b.7700.153a7529b88.Coremail.mz89924@126.com> Message-ID: <000401d18635$e7351760$b59f4620$@126.com> Dear Index for Kibana. ???: Daniel Guerra [mailto:daniel.guerra69 at gmail.com] ????: 2016?3?24? 17:27 ???: ine ??: bro at bro.org ??: Re: [Bro] about logs-to-elasticsearch.bro script What do you mean with index ? On 24 Mar 2016, at 07:32, ine > wrote: Dear all how to set index when use logs-to-elasticsearch.bro. _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/5ab1db18/attachment.html From mz89924 at 126.com Thu Mar 24 18:32:30 2016 From: mz89924 at 126.com (mz) Date: Fri, 25 Mar 2016 09:32:30 +0800 Subject: [Bro] =?gb2312?b?tPC4tDogIGFib3V0IGxvZ3MtdG8tZWxhc3RpY3NlYXJjaC5i?= =?gb2312?b?cm8gc2NyaXB0?= In-Reply-To: <4E345DD0-DB3C-455B-8885-E8DDD5F57E59@grantstavely.com> References: <6145eb3b.7700.153a7529b88.Coremail.mz89924@126.com> <4E345DD0-DB3C-455B-8885-E8DDD5F57E59@grantstavely.com> Message-ID: <001301d18636$32d9de30$988d9a90$@126.com> Thanks very much ???: Grant Stavely [mailto:grant at grantstavely.com] ????: 2016?3?24? 23:48 ???: ine ??: bro at bro.org ??: Re: [Bro] about logs-to-elasticsearch.bro script Hi ine, In local.bro, redef the consts defined in https://github.com/bro/bro-plugins/blob/9b7943e1a61062005f01b48eaad11bbb3b7a e757/elasticsearch/scripts/init.bro, e.g.: # Configure Elasticsearch redef LogElasticSearch::server_host = "x.x.x.x"; redef LogElasticSearch::server_port= 9200; redef LogElasticSearch::cluster_name = "security"; redef LogElasticSearch::index_prefix = "bro"; redef LogElasticSearch::excluded_log_ids += { Known::HOSTS_LOG, }; Grant On Mar 23, 2016, at 23:32, ine > wrote: Dear all how to set index when use logs-to-elasticsearch.bro. _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/6302bebd/attachment.html From cyrille.piatte at telecom-bretagne.eu Fri Mar 25 02:31:38 2016 From: cyrille.piatte at telecom-bretagne.eu (Cyrille PIATTE) Date: Fri, 25 Mar 2016 10:31:38 +0100 (CET) Subject: [Bro] issue with bro size Message-ID: <1404499821.12343528.1458898298757.JavaMail.zimbra@telecom-bretagne.eu> Hello, I am a new user of Bro and I have some questions: I have read that in the next version of Bro, the BroKer communication system will be included in the installation. -When is the next release of Bro ? Will Broccoli be deleted ? When I install Bro with the option ./configure --enable-broker, my Bro directory is about 1.5Gb. Otherwise, it is about 180Mb. At the same time, the binary is about the same size (145Mb). -Why such a difference ? Moreover, when I use all the other --disabled options, the difference is not that important (about 200Mb). -Is it normal ? Are there any other options that would allow us to reduce the size of the program ? We plan to use Bro on constrained devices with the BroKer library. I have found some documentation about Broccoli security options. But I have not found any documentation regarding BroKer security options. -Does it use also SSL ? In which communications ? Are there some differences with Broccoli concerning security ? Could we find documentation about security protocols used by BroKer communications ? I thank you in advance for your support. Best regards, Cyrille Piatte From mz89924 at 126.com Fri Mar 25 02:42:02 2016 From: mz89924 at 126.com (mz) Date: Fri, 25 Mar 2016 17:42:02 +0800 Subject: [Bro] logs-to-elasticsearch.bro error Message-ID: <004c01d1867a$968de9f0$c3a9bdd0$@126.com> Dear Use logs-to-elasticsearch.bro send logs to ES. Is now work. ES error logs: [2016-03-25 17:30:52,957][DEBUG][action.bulk ] [node-1] [whbro-201603251500][1] failed to execute bulk item (index) index {[whbro-201603251500][dns][AVOtHLQHooGOx5uLgLSQ], source[{"_timestamp":1458898236411,"ts":1458898206267,"uid":"ClbNI74bIcRQ8Gs 6Wc","id.orig_h":"10.100.78.88","id.orig_p":137,"id.resp_h":"10.100.79.255", "id.resp_p":137,"proto":"udp","trans_id":47282,"query":"ISATAP","qclass":1," qclass_name":"C_INTERNET","qtype":32,"qtype_name":"NB","AA":false,"TC":false ,"RD":true,"RA":false,"Z":1,"rejected":false}]} MapperParsingException[Field [_timestamp] is a metadata field and cannot be added inside a document. Use the index API request parameters.] at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.jav a:213) at org.elasticsearch.index.mapper.DocumentParser.innerParseDocument(DocumentPar ser.java:131) at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.j ava:79) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:304) at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:500) at org.elasticsearch.index.shard.IndexShard.prepareCreateOnPrimary(IndexShard.j ava:481) at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnP rimary(TransportIndexAction.java:214) at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPri mary(TransportIndexAction.java:223) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(T ransportShardBulkAction.java:326) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrima ry(TransportShardBulkAction.java:119) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrima ry(TransportShardBulkAction.java:68) at org.elasticsearch.action.support.replication.TransportReplicationAction$Prim aryPhase.doRun(TransportReplicationAction.java:595) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnab le.java:37) at org.elasticsearch.action.support.replication.TransportReplicationAction$Prim aryOperationTransportHandler.messageReceived(TransportReplicationAction.java :263) at org.elasticsearch.action.support.replication.TransportReplicationAction$Prim aryOperationTransportHandler.messageReceived(TransportReplicationAction.java :260) at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:3 50) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnab le.java:37) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 42) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 17) at java.lang.Thread.run(Thread.java:745) Bro config file: /usr/local/bro/lib/bro/plugins/Bro_ElasticSearch/scripts/init.bro module LogElasticSearch; export { ## Destination for the ES logs. Valid options are ## "direct" to directly connect to ES and "nsq" to ## transfer the logs into an nsqd instance. const destination = "direct" &redef; ## Name of the ES cluster. const cluster_name = "my-application" &redef; ## ES server. const server_host = "10.100.79.10" &redef; ## ES port. const server_port = 9200 &redef; ## Name of the ES index. const index_prefix = "testooo" &redef; ## Should the index names be in UTC or in local time? ## Setting this to true would be more compatible with Kibana and other tools. const index_name_in_utc = F &redef; ## Format for the index names. ## Setting this to "%Y.%m.%d-%H" would be more compatible Kibana and other tools. #const index_name_fmt = "%Y%m%d" &redef; const index_name_fmt = "%Y%m%d%H%M" &redef; ## The ES type prefix comes before the name of the related log. ## e.g. prefix = "bro\_" would create types of bro_dns, bro_software, etc. const type_prefix = "" &redef; ## The time before an ElasticSearch transfer will timeout. Note that ## the fractional part of the timeout will be ignored. In particular, ## time specifications less than a second result in a timeout value of ## 0, which means "no timeout." const transfer_timeout = 2secs; ## The batch size is the number of messages that will be queued up before ## they are sent to be bulk indexed. const max_batch_size = 1000 &redef; ## The maximum amount of wall-clock time that is allowed to pass without ## finishing a bulk log send. This represents the maximum delay you ## would like to have with your logs before they are sent to ElasticSearch. const max_batch_interval = 1min &redef; ## The maximum byte size for a buffered JSON string to send to the bulk ## insert API. const max_byte_size = 1024 * 1024 &redef; ## If the "nsq" destination is given, this is the topic ## that Bro will push logs into. const nsq_topic = "bro_logs" &redef; } -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/440d72ec/attachment-0001.html From daniel.guerra69 at gmail.com Fri Mar 25 02:50:56 2016 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Fri, 25 Mar 2016 10:50:56 +0100 Subject: [Bro] logs-to-elasticsearch.bro error In-Reply-To: <004c01d1867a$968de9f0$c3a9bdd0$@126.com> References: <004c01d1867a$968de9f0$c3a9bdd0$@126.com> Message-ID: <7A63317E-EAC2-4612-867F-E01FB67736DD@gmail.com> Hi, To make this work you need some patches or use an elasticsearch version lower than 2 (1.7) I made a docker image for this https://hub.docker.com/r/danielguerra/bro-debian-elasticsearch/ In the git there is a map bro-patch https://github.com/danielguerra69/bro-debian-elasticsearch.git Regards, Daniel > On 25 Mar 2016, at 10:42, mz wrote: > > Dear > Use logs-to-elasticsearch.bro send logs to ES. Is now work. > > ES error logs: > [2016-03-25 17:30:52,957][DEBUG][action.bulk ] [node-1] [whbro-201603251500][1] failed to execute bulk item (index) index {[whbro-201603251500][dns][AVOtHLQHooGOx5uLgLSQ], source[{"_timestamp":1458898236411,"ts":1458898206267,"uid":"ClbNI74bIcRQ8Gs6Wc","id.orig_h":"10.100.78.88","id.orig_p":137,"id.resp_h":"10.100.79.255","id.resp_p":137,"proto":"udp","trans_id":47282,"query":"ISATAP","qclass":1,"qclass_name":"C_INTERNET","qtype":32,"qtype_name":"NB","AA":false,"TC":false,"RD":true,"RA":false,"Z":1,"rejected":false}]} > MapperParsingException[Field [_timestamp] is a metadata field and cannot be added inside a document. Use the index API request parameters.] > at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:213) > at org.elasticsearch.index.mapper.DocumentParser.innerParseDocument(DocumentParser.java:131) > at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:79) > at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:304) > at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:500) > at org.elasticsearch.index.shard.IndexShard.prepareCreateOnPrimary(IndexShard.java:481) > at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:214) > at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:223) > at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326) > at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119) > at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68) > at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:595) > at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) > at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:263) > at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:260) > at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:350) > at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > > Bro config file: > /usr/local/bro/lib/bro/plugins/Bro_ElasticSearch/scripts/init.bro > module LogElasticSearch; > > export { > ## Destination for the ES logs. Valid options are > ## "direct" to directly connect to ES and "nsq" to > ## transfer the logs into an nsqd instance. > const destination = "direct" &redef; > > ## Name of the ES cluster. > const cluster_name = "my-application" &redef; > > ## ES server. > const server_host = "10.100.79.10" &redef; > > ## ES port. > const server_port = 9200 &redef; > > ## Name of the ES index. > const index_prefix = "testooo" &redef; > > ## Should the index names be in UTC or in local time? > ## Setting this to true would be more compatible with Kibana and other tools. > const index_name_in_utc = F &redef; > > ## Format for the index names. > ## Setting this to "%Y.%m.%d-%H" would be more compatible Kibana and other tools. > #const index_name_fmt = "%Y%m%d" &redef; > const index_name_fmt = "%Y%m%d%H%M" &redef; > ## The ES type prefix comes before the name of the related log. > ## e.g. prefix = "bro\_" would create types of bro_dns, bro_software, etc. > const type_prefix = "" &redef; > > ## The time before an ElasticSearch transfer will timeout. Note that > ## the fractional part of the timeout will be ignored. In particular, > ## time specifications less than a second result in a timeout value of > ## 0, which means "no timeout." > const transfer_timeout = 2secs; > > ## The batch size is the number of messages that will be queued up before > ## they are sent to be bulk indexed. > const max_batch_size = 1000 &redef; > > ## The maximum amount of wall-clock time that is allowed to pass without > ## finishing a bulk log send. This represents the maximum delay you > ## would like to have with your logs before they are sent to ElasticSearch. > const max_batch_interval = 1min &redef; > > ## The maximum byte size for a buffered JSON string to send to the bulk > ## insert API. > const max_byte_size = 1024 * 1024 &redef; > > ## If the "nsq" destination is given, this is the topic > ## that Bro will push logs into. > const nsq_topic = "bro_logs" &redef; > } > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/8ba5db00/attachment-0001.html From tgdesrochers at gmail.com Fri Mar 25 05:21:19 2016 From: tgdesrochers at gmail.com (Tim Desrochers) Date: Fri, 25 Mar 2016 08:21:19 -0400 Subject: [Bro] [bro] smtp log strangeness Message-ID: <56f52d54.8953810a.43002.ffffca33@mx.google.com> While parsing smpt logs I notice a bunch of strange data contained in my from/to/subject fields Example: "subject":"=?utf-8?q?CBO_drops_the_March_base=E2=80=A6line?=" "subject":"=?Windows-1252?Q?Automatic_reply:_CBO_drops_the_March_base=85line?=", "from":"\u0022NAMEOFPERSON\u0022 " Why am I getting all of this extra info in these fields? I am printing logs as JSON not CSV. Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/2d349ea3/attachment.html From raj at bivio.net Fri Mar 25 06:50:57 2016 From: raj at bivio.net (Raj Srinivasan) Date: Fri, 25 Mar 2016 13:50:57 +0000 Subject: [Bro] Bro cluster synchronization Message-ID: We are running around 20 to 30 Bro worker threads and 2 to 3 proxies, and having problems with performance. From what we can see, the bottleneck seems to be proxy communication. Cpus don't seem to be too busy, but spend time waiting for IO. I would like to understand what types of data the proxy is synchronizing in addition to active IP addresses. If we use load balancing based on IP addresses only, so all sessions between two IP addresses are processed by the same worker, will we be missing any functionality by running Bro in standalone mode on each of our processors/cores? If we do this, I believe that related sessions should (almost always) be processed by the same worker, except in a few cases which I hope we can do without! Thanks, Raj -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/70aa609b/attachment.html From vallentin at icir.org Fri Mar 25 07:48:18 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Fri, 25 Mar 2016 07:48:18 -0700 Subject: [Bro] issue with bro size In-Reply-To: <1404499821.12343528.1458898298757.JavaMail.zimbra@telecom-bretagne.eu> References: <1404499821.12343528.1458898298757.JavaMail.zimbra@telecom-bretagne.eu> Message-ID: <20160325144818.GD52470@shogun> > I am a new user of Bro and I have some questions: Welcome aboard! > I have read that in the next version of Bro, the BroKer communication > system will be included in the installation. > -When is the next release of Bro ? Will Broccoli be deleted ? The next release of Bro, version 2.5, is contingent on a stable running integration with Broker. We will also deprecate Broccoli as soon as we make Broker the default communication component. > When I install Bro with the option ./configure --enable-broker, my Bro > directory is about 1.5Gb. Otherwise, it is about 180Mb. At the same > time, the binary is about the same size (145Mb). > -Why such a difference ? Can you give us more detail what files contribute to the increase in size? What happens when you remove debugging symbols (i.e., remove the -g switch from the compilation process)? Note that debugging symbols do not have any runtime overhead during execution. > Moreover, when I use all the other --disabled options, the difference > is not that important (about 200Mb). -Is it normal ? Are there any > other options that would allow us to reduce the size of the program ? > We plan to use Bro on constrained devices with the BroKer library. My hunch is that Broker (and in particular CAF) generates huge symbol tables, due to the excessive amount of C++ template code and insane name mangling. > I have found some documentation about Broccoli security options. But I > have not found any documentation regarding BroKer security options. > -Does it use also SSL ? In which communications ? Are there some > differences with Broccoli concerning security ? Could we find > documentation about security protocols used by BroKer communications ? At this point Broker does not support encryption, but it's on our TODO list. Once the underlying communication library CAF supports encryption, it will be rather simple to simple to lift it into Broker. We hope that TLS support will make it into the next release, but it's not clear at this point. Matthias From blackhole.em at gmail.com Fri Mar 25 08:39:36 2016 From: blackhole.em at gmail.com (Joe Blow) Date: Fri, 25 Mar 2016 11:39:36 -0400 Subject: [Bro] Renaming carved files In-Reply-To: <229AF808-3A67-40B6-88FE-D305CEC1437C@gmail.com> References: <4F710A0A-5714-4816-AB41-AE1D20BA24D4@icir.org> <5383627E-D740-44FE-8C7F-80FB5C08DA72@icir.org> <229AF808-3A67-40B6-88FE-D305CEC1437C@gmail.com> Message-ID: This code works well. Any nice way to remove the spaces from the DOCs? Cheers, JB On Thu, Mar 3, 2016 at 5:28 PM, Micha? Purzy?ski wrote: > > > I'd be curious to see how many files don't match their declared mime > types, I bet a lot. I thought about writing a script to do this once, but > then stopped myself because at the very least, there are lots of favicon > files that are jpegs and gifs, but the remote server even declares in the > header that it's actually an icon file (since servers typically just base > on the file extension). I would still be interested to see what people's > experiences are if anyone ever takes it on though (i.e., does it catch > anything worth following). > > > > That's exactly the kind of script I've just written. Will send an update > how it behaves in a week or so. It's going to be deployed in several busy > offices. > > > > Thanks, > > .Seth > > > > -- > > Seth Hall > > International Computer Science Institute > > (Bro) because everyone has a network > > http://www.bro.org/ > > > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/dce2a085/attachment.html From jlay at slave-tothe-box.net Fri Mar 25 09:35:26 2016 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 25 Mar 2016 10:35:26 -0600 Subject: [Bro] Tracing email process Message-ID: Hey all, So I four hosts using ssmtp (sendmail replacement-ish) using bro. Two work just fine with ssmtp, the other two do not. Is there a way I can trace the emailing that bro does? Thank you. James From pratikinamdar at gmail.com Fri Mar 25 09:40:23 2016 From: pratikinamdar at gmail.com (pratik inamdar) Date: Fri, 25 Mar 2016 09:40:23 -0700 Subject: [Bro] Integrating WiFi Analyzer within Bro In-Reply-To: References: Message-ID: Hi Vlad, Hope you are doing good! I chose to switch the protocol. So now I am writing an analyzer for 6LoWPAN instead of WiFi. Quick question: Will I be able to successfully use BinPac to write an analyzer for 6LoWPAN? Also, if possible, please guide me with some key points to remember while writing analyzer for 6LoWPAN. Your help will be greatly appreciated! Thanks, Pratik Inamdar On Mon, Mar 21, 2016 at 8:28 AM, Vlad Grigorescu wrote: > Unfortunately, there is no way to implement lower level protocols with > BinPAC quickstart right now. Similary, we don't have any examples of a > BinPAC lower-level analyzer if you were to do it manually. > > If you are able to get it working, I'd certainly be interested in how > you did it, and would look at adding it to binpac_quickstart. > > --Vlad > > pratik inamdar writes: > > > [ text/plain ] > > Hi, > > > > In my project, I am integrating a WiFi protocol analyzer with bro to > parse > > and monitor WiFi packets header information. > > > > I am using BinPac to generate template for the WiFi protocol analyzer in > > the src/analyzer/protocol directory. > > > > As per my knowledge WiFi(802.11) is not a TCP type of protocol. So I wish > > to know what should I use instead of the option "--tcp" while using the > > command: > > > > python start.py WiFi "WiFi Protocol" ../bro --tcp > > > > Any help will be really appreciated! > > > > -- > > > > Thanks & Regards. > > > > Pratik Inamdar > > [ text/plain ] > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Thanks & Regards. Pratik Inamdar -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/26950552/attachment.html From johanna at icir.org Fri Mar 25 09:48:45 2016 From: johanna at icir.org (Johanna Amann) Date: Fri, 25 Mar 2016 09:48:45 -0700 Subject: [Bro] Integrating WiFi Analyzer within Bro In-Reply-To: References: Message-ID: <59AF02C0-6ABE-4021-9F6C-840F4CB50CD7@icir.org> Hello Pratik, I think the answer stays the same - if I understand things correctly, you have to implement IEEE 802.15.4, which is a lower level protocol, which currently cannot be implemented with just BinPAC and needs core changes (probably in src/iosource/Packet.cc and others). There are currently no examples for that, besides the existing code. Johanna On 25 Mar 2016, at 9:40, pratik inamdar wrote: > Hi Vlad, > > Hope you are doing good! > > I chose to switch the protocol. So now I am writing an analyzer for > 6LoWPAN > instead of WiFi. > > Quick question: > > Will I be able to successfully use BinPac to write an analyzer for > 6LoWPAN? > > Also, if possible, please guide me with some key points to remember > while > writing analyzer for 6LoWPAN. > > Your help will be greatly appreciated! > > Thanks, > Pratik Inamdar > > On Mon, Mar 21, 2016 at 8:28 AM, Vlad Grigorescu > wrote: > >> Unfortunately, there is no way to implement lower level protocols >> with >> BinPAC quickstart right now. Similary, we don't have any examples of >> a >> BinPAC lower-level analyzer if you were to do it manually. >> >> If you are able to get it working, I'd certainly be interested in how >> you did it, and would look at adding it to binpac_quickstart. >> >> --Vlad >> >> pratik inamdar writes: >> >>> [ text/plain ] >>> Hi, >>> >>> In my project, I am integrating a WiFi protocol analyzer with bro to >> parse >>> and monitor WiFi packets header information. >>> >>> I am using BinPac to generate template for the WiFi protocol >>> analyzer in >>> the src/analyzer/protocol directory. >>> >>> As per my knowledge WiFi(802.11) is not a TCP type of protocol. So I >>> wish >>> to know what should I use instead of the option "--tcp" while using >>> the >>> command: >>> >>> python start.py WiFi "WiFi Protocol" ../bro --tcp >>> >>> Any help will be really appreciated! >>> >>> -- >>> >>> Thanks & Regards. >>> >>> Pratik Inamdar >>> [ text/plain ] >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > > > > -- > > Thanks & Regards. > > Pratik Inamdar > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From jan.grashoefer at gmail.com Fri Mar 25 09:49:36 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Fri, 25 Mar 2016 17:49:36 +0100 Subject: [Bro] [bro] smtp log strangeness In-Reply-To: <56f52d54.8953810a.43002.ffffca33@mx.google.com> References: <56f52d54.8953810a.43002.ffffca33@mx.google.com> Message-ID: <56F56C20.4050902@gmail.com> Hi, > Why am I getting all of this extra info in these fields? The subject headers seem to look that strange to support other encodings than ASCII (see https://en.wikipedia.org/wiki/Unicode_and_email#Unicode_support_in_message_header). The from header seems to include the display-name (see https://tools.ietf.org/html/rfc5322#section-3.4). As Bro logs the content of the headers without further processing, you are getting this extra info. Regards, Jan From pratikinamdar at gmail.com Fri Mar 25 09:55:25 2016 From: pratikinamdar at gmail.com (pratik inamdar) Date: Fri, 25 Mar 2016 09:55:25 -0700 Subject: [Bro] Integrating WiFi Analyzer within Bro In-Reply-To: <59AF02C0-6ABE-4021-9F6C-840F4CB50CD7@icir.org> References: <59AF02C0-6ABE-4021-9F6C-840F4CB50CD7@icir.org> Message-ID: Hi, Thanks you for the prompt response! My task is to write an analyzer in bro using BinPac for an IoT protocol. I have already written analyzers for application layer protocols namely MQTT and AMQP. Now I wish to write an analyzer for an IoT protocol which does not fall in the application layer. The IoT protocol should be able to use Bro BinPac language. Could you please suggest me one? Thanks, Pratik Inamdar On Mar 25, 2016 09:48, "Johanna Amann" wrote: > Hello Pratik, > > I think the answer stays the same - if I understand things correctly, you > have to implement IEEE 802.15.4, which is a lower level protocol, which > currently cannot be implemented with just BinPAC and needs core changes > (probably in src/iosource/Packet.cc and others). There are currently no > examples for that, besides the existing code. > > Johanna > > On 25 Mar 2016, at 9:40, pratik inamdar wrote: > > Hi Vlad, >> >> Hope you are doing good! >> >> I chose to switch the protocol. So now I am writing an analyzer for >> 6LoWPAN >> instead of WiFi. >> >> Quick question: >> >> Will I be able to successfully use BinPac to write an analyzer for >> 6LoWPAN? >> >> Also, if possible, please guide me with some key points to remember while >> writing analyzer for 6LoWPAN. >> >> Your help will be greatly appreciated! >> >> Thanks, >> Pratik Inamdar >> >> On Mon, Mar 21, 2016 at 8:28 AM, Vlad Grigorescu >> wrote: >> >> Unfortunately, there is no way to implement lower level protocols with >>> BinPAC quickstart right now. Similary, we don't have any examples of a >>> BinPAC lower-level analyzer if you were to do it manually. >>> >>> If you are able to get it working, I'd certainly be interested in how >>> you did it, and would look at adding it to binpac_quickstart. >>> >>> --Vlad >>> >>> pratik inamdar writes: >>> >>> [ text/plain ] >>>> Hi, >>>> >>>> In my project, I am integrating a WiFi protocol analyzer with bro to >>>> >>> parse >>> >>>> and monitor WiFi packets header information. >>>> >>>> I am using BinPac to generate template for the WiFi protocol analyzer in >>>> the src/analyzer/protocol directory. >>>> >>>> As per my knowledge WiFi(802.11) is not a TCP type of protocol. So I >>>> wish >>>> to know what should I use instead of the option "--tcp" while using the >>>> command: >>>> >>>> python start.py WiFi "WiFi Protocol" ../bro --tcp >>>> >>>> Any help will be really appreciated! >>>> >>>> -- >>>> >>>> Thanks & Regards. >>>> >>>> Pratik Inamdar >>>> [ text/plain ] >>>> _______________________________________________ >>>> Bro mailing list >>>> bro at bro-ids.org >>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>> >>> >>> >> >> >> -- >> >> Thanks & Regards. >> >> Pratik Inamdar >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/aef042cd/attachment.html From johanna at icir.org Fri Mar 25 09:58:26 2016 From: johanna at icir.org (Johanna Amann) Date: Fri, 25 Mar 2016 09:58:26 -0700 Subject: [Bro] Definition of a connection in conn.log In-Reply-To: References: Message-ID: <20160325165826.GA47770@wifi101.sys.ICSI.Berkeley.EDU> Hello, let me try to give a few quick answers to your question. On Tue, Mar 22, 2016 at 03:00:25PM -0600, Navraj Singh wrote: > 1. Is it safe to assume that any given packet will be assigned to at most > one connection, and thus to at most one row in conn.log? No - there is a special case for tunnels, where a connection can be the parent of another connection (the child shows its parent in the tunnel_parents field of conn.log). In that case, the packet is assigned to both the child and the parent. > 2. Why is it that some rows in conn.log do not have the duration field set? > I see see several row with a '-' in the duration field. That should mean that the duration was "0" (e.g. single packet), if I am not mistaken. > 3. The bro documentation states that "For UDP and ICMP, ?connections? are > to be interpreted using flow semantics (sequence of packets from a source > host/port to a destination host/port)." However, what is the exact > definition for a TCP flow? How does Bro decide which packets to include in > a connection? That s a not quite straightforward to answer question. Generally Bro counts connections a 5-tuples; however, there are several timeouts at work (after no packets were arrived for a certain amount of time, a connection is seen as finished; if theree are new packets, a new connection will begin). For TCP, a connection also can be ended by fin and rst packets, and a new connection will begin afterwards. The timeouts are set using configuration variables - e.g. the default tcp_inactivity_timeout, after which a TCP connection is considered closed when no more packets are seen is 5 minutes. For UDP, it is 1 minute. For ICMP it also is one minute. > 4. For an ongoing 'connection', does Bro wait until the connection is over > before logging it? What if the connection is quite long in duration...won't > that cause a lag? Or does Bro automatically chop up long flows based on > some configurable limit parameter? Yup. And you are right, it will cause a lag in logging. I hope this helps, Johanna From johanna at icir.org Fri Mar 25 10:02:20 2016 From: johanna at icir.org (Johanna Amann) Date: Fri, 25 Mar 2016 10:02:20 -0700 Subject: [Bro] BRO 2.4.1. extracted file handling In-Reply-To: References: Message-ID: <20160325170220.GB47770@wifi101.sys.ICSI.Berkeley.EDU> Hello, On Mon, Mar 21, 2016 at 11:01:50AM -0400, john smith wrote: > 1. Right now, all the extracted files are in ASCII format. Is there any > easy way to save them in JSON? The files are extracted in the way that they are encountered on the wire. Bro does not do any processing on them. So - if they are ASCII, they are written as ASCII. > 2. Would it be possible to add an extracted file itself to file.log? If > not, is there any way to copy the extracted file to a new log stream? File extractions happens outside of the normal logging framework; there currently is no easy way to copy extracted files to other log streams. I hope this helps, Johanna From johanna at icir.org Fri Mar 25 10:03:56 2016 From: johanna at icir.org (Johanna Amann) Date: Fri, 25 Mar 2016 10:03:56 -0700 Subject: [Bro] how to filter http traffic In-Reply-To: <000001d180bb$b7b88d20$2729a760$@126.com> References: <000001d180bb$b7b88d20$2729a760$@126.com> Message-ID: <20160325170356.GC47770@wifi101.sys.ICSI.Berkeley.EDU> Hi, If you only want to monitor port 80, e.g., you can set a custom capture filter. See, e.g., https://www.bro.org/documentation/faq.html#how-can-i-set-a-custom-capture-filter Johanna On Fri, Mar 18, 2016 at 10:13:08AM +0800, mz wrote: > Dear > > I only want to monitor and capture http traffic, how to configure? > > In addition, I found that when reading official manual picture below: > > > > > > It will figure event where the configuration script engine > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From johanna at icir.org Fri Mar 25 10:06:06 2016 From: johanna at icir.org (Johanna Amann) Date: Fri, 25 Mar 2016 10:06:06 -0700 Subject: [Bro] Integrating WiFi Analyzer within Bro In-Reply-To: References: <59AF02C0-6ABE-4021-9F6C-840F4CB50CD7@icir.org> Message-ID: Hi, the answer is the same for all protocols that are underneath tcp/udp. You will have to change the core for any of them. If you have an IoT protocol that works over UDP/TCP, you should be able to use BinPac alone. Johanna On 25 Mar 2016, at 9:55, pratik inamdar wrote: > Hi, > > Thanks you for the prompt response! > > My task is to write an analyzer in bro using BinPac for an IoT > protocol. I > have already written analyzers for application layer protocols namely > MQTT > and AMQP. > > Now I wish to write an analyzer for an IoT protocol which does not > fall in > the application layer. > > The IoT protocol should be able to use Bro BinPac language. Could you > please suggest me one? > > Thanks, > Pratik Inamdar > On Mar 25, 2016 09:48, "Johanna Amann" wrote: > >> Hello Pratik, >> >> I think the answer stays the same - if I understand things correctly, >> you >> have to implement IEEE 802.15.4, which is a lower level protocol, >> which >> currently cannot be implemented with just BinPAC and needs core >> changes >> (probably in src/iosource/Packet.cc and others). There are currently >> no >> examples for that, besides the existing code. >> >> Johanna >> >> On 25 Mar 2016, at 9:40, pratik inamdar wrote: >> >> Hi Vlad, >>> >>> Hope you are doing good! >>> >>> I chose to switch the protocol. So now I am writing an analyzer for >>> 6LoWPAN >>> instead of WiFi. >>> >>> Quick question: >>> >>> Will I be able to successfully use BinPac to write an analyzer for >>> 6LoWPAN? >>> >>> Also, if possible, please guide me with some key points to remember >>> while >>> writing analyzer for 6LoWPAN. >>> >>> Your help will be greatly appreciated! >>> >>> Thanks, >>> Pratik Inamdar >>> >>> On Mon, Mar 21, 2016 at 8:28 AM, Vlad Grigorescu >>> >>> wrote: >>> >>> Unfortunately, there is no way to implement lower level protocols >>> with >>>> BinPAC quickstart right now. Similary, we don't have any examples >>>> of a >>>> BinPAC lower-level analyzer if you were to do it manually. >>>> >>>> If you are able to get it working, I'd certainly be interested in >>>> how >>>> you did it, and would look at adding it to binpac_quickstart. >>>> >>>> --Vlad >>>> >>>> pratik inamdar writes: >>>> >>>> [ text/plain ] >>>>> Hi, >>>>> >>>>> In my project, I am integrating a WiFi protocol analyzer with bro >>>>> to >>>>> >>>> parse >>>> >>>>> and monitor WiFi packets header information. >>>>> >>>>> I am using BinPac to generate template for the WiFi protocol >>>>> analyzer in >>>>> the src/analyzer/protocol directory. >>>>> >>>>> As per my knowledge WiFi(802.11) is not a TCP type of protocol. So >>>>> I >>>>> wish >>>>> to know what should I use instead of the option "--tcp" while >>>>> using the >>>>> command: >>>>> >>>>> python start.py WiFi "WiFi Protocol" ../bro --tcp >>>>> >>>>> Any help will be really appreciated! >>>>> >>>>> -- >>>>> >>>>> Thanks & Regards. >>>>> >>>>> Pratik Inamdar >>>>> [ text/plain ] >>>>> _______________________________________________ >>>>> Bro mailing list >>>>> bro at bro-ids.org >>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>> >>>> >>>> >>> >>> >>> -- >>> >>> Thanks & Regards. >>> >>> Pratik Inamdar >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> >> From pratikinamdar at gmail.com Fri Mar 25 10:10:13 2016 From: pratikinamdar at gmail.com (pratik inamdar) Date: Fri, 25 Mar 2016 10:10:13 -0700 Subject: [Bro] Integrating WiFi Analyzer within Bro In-Reply-To: References: <59AF02C0-6ABE-4021-9F6C-840F4CB50CD7@icir.org> Message-ID: So just to verify if I understood it correctly. You mean to say that all the protocols on or above the Transport Layer ONLY should be able to use Bro BinPac? Thanks, Pratik Inamdar On Mar 25, 2016 10:06, "Johanna Amann" wrote: > Hi, > > the answer is the same for all protocols that are underneath tcp/udp. You > will have to change the core for any of them. If you have an IoT protocol > that works over UDP/TCP, you should be able to use BinPac alone. > > Johanna > > On 25 Mar 2016, at 9:55, pratik inamdar wrote: > > Hi, >> >> Thanks you for the prompt response! >> >> My task is to write an analyzer in bro using BinPac for an IoT protocol. I >> have already written analyzers for application layer protocols namely MQTT >> and AMQP. >> >> Now I wish to write an analyzer for an IoT protocol which does not fall in >> the application layer. >> >> The IoT protocol should be able to use Bro BinPac language. Could you >> please suggest me one? >> >> Thanks, >> Pratik Inamdar >> On Mar 25, 2016 09:48, "Johanna Amann" wrote: >> >> Hello Pratik, >>> >>> I think the answer stays the same - if I understand things correctly, you >>> have to implement IEEE 802.15.4, which is a lower level protocol, which >>> currently cannot be implemented with just BinPAC and needs core changes >>> (probably in src/iosource/Packet.cc and others). There are currently no >>> examples for that, besides the existing code. >>> >>> Johanna >>> >>> On 25 Mar 2016, at 9:40, pratik inamdar wrote: >>> >>> Hi Vlad, >>> >>>> >>>> Hope you are doing good! >>>> >>>> I chose to switch the protocol. So now I am writing an analyzer for >>>> 6LoWPAN >>>> instead of WiFi. >>>> >>>> Quick question: >>>> >>>> Will I be able to successfully use BinPac to write an analyzer for >>>> 6LoWPAN? >>>> >>>> Also, if possible, please guide me with some key points to remember >>>> while >>>> writing analyzer for 6LoWPAN. >>>> >>>> Your help will be greatly appreciated! >>>> >>>> Thanks, >>>> Pratik Inamdar >>>> >>>> On Mon, Mar 21, 2016 at 8:28 AM, Vlad Grigorescu >>>> wrote: >>>> >>>> Unfortunately, there is no way to implement lower level protocols with >>>> >>>>> BinPAC quickstart right now. Similary, we don't have any examples of a >>>>> BinPAC lower-level analyzer if you were to do it manually. >>>>> >>>>> If you are able to get it working, I'd certainly be interested in how >>>>> you did it, and would look at adding it to binpac_quickstart. >>>>> >>>>> --Vlad >>>>> >>>>> pratik inamdar writes: >>>>> >>>>> [ text/plain ] >>>>> >>>>>> Hi, >>>>>> >>>>>> In my project, I am integrating a WiFi protocol analyzer with bro to >>>>>> >>>>>> parse >>>>> >>>>> and monitor WiFi packets header information. >>>>>> >>>>>> I am using BinPac to generate template for the WiFi protocol analyzer >>>>>> in >>>>>> the src/analyzer/protocol directory. >>>>>> >>>>>> As per my knowledge WiFi(802.11) is not a TCP type of protocol. So I >>>>>> wish >>>>>> to know what should I use instead of the option "--tcp" while using >>>>>> the >>>>>> command: >>>>>> >>>>>> python start.py WiFi "WiFi Protocol" ../bro --tcp >>>>>> >>>>>> Any help will be really appreciated! >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & Regards. >>>>>> >>>>>> Pratik Inamdar >>>>>> [ text/plain ] >>>>>> _______________________________________________ >>>>>> Bro mailing list >>>>>> bro at bro-ids.org >>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> -- >>>> >>>> Thanks & Regards. >>>> >>>> Pratik Inamdar >>>> _______________________________________________ >>>> Bro mailing list >>>> bro at bro-ids.org >>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>> >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/819edd2f/attachment.html From pratikinamdar at gmail.com Fri Mar 25 10:32:02 2016 From: pratikinamdar at gmail.com (pratik inamdar) Date: Fri, 25 Mar 2016 10:32:02 -0700 Subject: [Bro] Integrating WiFi Analyzer within Bro In-Reply-To: References: <59AF02C0-6ABE-4021-9F6C-840F4CB50CD7@icir.org> Message-ID: If what I said in my previous email is correct then why was I able to integrate RIP(Routing Information protocol) analyzer with bro? RIP is a network layer protocol which lied below Transport Layer. Similarly, 6LoWPAN is another name for IPV6 and is used for Low powered devices. If I was able to integrate RIP, do you think I will be integrate 6LoWPAN? Thanks, Pratik Inamdar On Mar 25, 2016 10:10, "pratik inamdar" wrote: > So just to verify if I understood it correctly. > > You mean to say that all the protocols on or above the Transport Layer > ONLY should be able to use Bro BinPac? > > Thanks, > Pratik Inamdar > On Mar 25, 2016 10:06, "Johanna Amann" wrote: > >> Hi, >> >> the answer is the same for all protocols that are underneath tcp/udp. You >> will have to change the core for any of them. If you have an IoT protocol >> that works over UDP/TCP, you should be able to use BinPac alone. >> >> Johanna >> >> On 25 Mar 2016, at 9:55, pratik inamdar wrote: >> >> Hi, >>> >>> Thanks you for the prompt response! >>> >>> My task is to write an analyzer in bro using BinPac for an IoT protocol. >>> I >>> have already written analyzers for application layer protocols namely >>> MQTT >>> and AMQP. >>> >>> Now I wish to write an analyzer for an IoT protocol which does not fall >>> in >>> the application layer. >>> >>> The IoT protocol should be able to use Bro BinPac language. Could you >>> please suggest me one? >>> >>> Thanks, >>> Pratik Inamdar >>> On Mar 25, 2016 09:48, "Johanna Amann" wrote: >>> >>> Hello Pratik, >>>> >>>> I think the answer stays the same - if I understand things correctly, >>>> you >>>> have to implement IEEE 802.15.4, which is a lower level protocol, which >>>> currently cannot be implemented with just BinPAC and needs core changes >>>> (probably in src/iosource/Packet.cc and others). There are currently no >>>> examples for that, besides the existing code. >>>> >>>> Johanna >>>> >>>> On 25 Mar 2016, at 9:40, pratik inamdar wrote: >>>> >>>> Hi Vlad, >>>> >>>>> >>>>> Hope you are doing good! >>>>> >>>>> I chose to switch the protocol. So now I am writing an analyzer for >>>>> 6LoWPAN >>>>> instead of WiFi. >>>>> >>>>> Quick question: >>>>> >>>>> Will I be able to successfully use BinPac to write an analyzer for >>>>> 6LoWPAN? >>>>> >>>>> Also, if possible, please guide me with some key points to remember >>>>> while >>>>> writing analyzer for 6LoWPAN. >>>>> >>>>> Your help will be greatly appreciated! >>>>> >>>>> Thanks, >>>>> Pratik Inamdar >>>>> >>>>> On Mon, Mar 21, 2016 at 8:28 AM, Vlad Grigorescu >>>>> wrote: >>>>> >>>>> Unfortunately, there is no way to implement lower level protocols with >>>>> >>>>>> BinPAC quickstart right now. Similary, we don't have any examples of a >>>>>> BinPAC lower-level analyzer if you were to do it manually. >>>>>> >>>>>> If you are able to get it working, I'd certainly be interested in how >>>>>> you did it, and would look at adding it to binpac_quickstart. >>>>>> >>>>>> --Vlad >>>>>> >>>>>> pratik inamdar writes: >>>>>> >>>>>> [ text/plain ] >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> In my project, I am integrating a WiFi protocol analyzer with bro to >>>>>>> >>>>>>> parse >>>>>> >>>>>> and monitor WiFi packets header information. >>>>>>> >>>>>>> I am using BinPac to generate template for the WiFi protocol >>>>>>> analyzer in >>>>>>> the src/analyzer/protocol directory. >>>>>>> >>>>>>> As per my knowledge WiFi(802.11) is not a TCP type of protocol. So I >>>>>>> wish >>>>>>> to know what should I use instead of the option "--tcp" while using >>>>>>> the >>>>>>> command: >>>>>>> >>>>>>> python start.py WiFi "WiFi Protocol" ../bro --tcp >>>>>>> >>>>>>> Any help will be really appreciated! >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & Regards. >>>>>>> >>>>>>> Pratik Inamdar >>>>>>> [ text/plain ] >>>>>>> _______________________________________________ >>>>>>> Bro mailing list >>>>>>> bro at bro-ids.org >>>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks & Regards. >>>>> >>>>> Pratik Inamdar >>>>> _______________________________________________ >>>>> Bro mailing list >>>>> bro at bro-ids.org >>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>> >>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/d5eed165/attachment-0001.html From johanna at icir.org Fri Mar 25 10:37:10 2016 From: johanna at icir.org (Johanna Amann) Date: Fri, 25 Mar 2016 10:37:10 -0700 Subject: [Bro] Integrating WiFi Analyzer within Bro In-Reply-To: References: <59AF02C0-6ABE-4021-9F6C-840F4CB50CD7@icir.org> Message-ID: <0B0D37F8-D7E1-4807-B22E-E90995AE937A@icir.org> It is as I said - there is a straightforward way to implement everything above TCP/UDP with BinPac. RIP is not below the transport layer - it uses UDP as its transport protocol. Johanna On 25 Mar 2016, at 10:32, pratik inamdar wrote: > If what I said in my previous email is correct then why was I able to > integrate RIP(Routing Information protocol) analyzer with bro? RIP is > a > network layer protocol which lied below Transport Layer. > > Similarly, 6LoWPAN is another name for IPV6 and is used for Low > powered > devices. If I was able to integrate RIP, do you think I will be > integrate > 6LoWPAN? > > Thanks, > Pratik Inamdar > On Mar 25, 2016 10:10, "pratik inamdar" > wrote: > >> So just to verify if I understood it correctly. >> >> You mean to say that all the protocols on or above the Transport >> Layer >> ONLY should be able to use Bro BinPac? >> >> Thanks, >> Pratik Inamdar >> On Mar 25, 2016 10:06, "Johanna Amann" wrote: >> >>> Hi, >>> >>> the answer is the same for all protocols that are underneath >>> tcp/udp. You >>> will have to change the core for any of them. If you have an IoT >>> protocol >>> that works over UDP/TCP, you should be able to use BinPac alone. >>> >>> Johanna >>> >>> On 25 Mar 2016, at 9:55, pratik inamdar wrote: >>> >>> Hi, >>>> >>>> Thanks you for the prompt response! >>>> >>>> My task is to write an analyzer in bro using BinPac for an IoT >>>> protocol. >>>> I >>>> have already written analyzers for application layer protocols >>>> namely >>>> MQTT >>>> and AMQP. >>>> >>>> Now I wish to write an analyzer for an IoT protocol which does not >>>> fall >>>> in >>>> the application layer. >>>> >>>> The IoT protocol should be able to use Bro BinPac language. Could >>>> you >>>> please suggest me one? >>>> >>>> Thanks, >>>> Pratik Inamdar >>>> On Mar 25, 2016 09:48, "Johanna Amann" wrote: >>>> >>>> Hello Pratik, >>>>> >>>>> I think the answer stays the same - if I understand things >>>>> correctly, >>>>> you >>>>> have to implement IEEE 802.15.4, which is a lower level protocol, >>>>> which >>>>> currently cannot be implemented with just BinPAC and needs core >>>>> changes >>>>> (probably in src/iosource/Packet.cc and others). There are >>>>> currently no >>>>> examples for that, besides the existing code. >>>>> >>>>> Johanna >>>>> >>>>> On 25 Mar 2016, at 9:40, pratik inamdar wrote: >>>>> >>>>> Hi Vlad, >>>>> >>>>>> >>>>>> Hope you are doing good! >>>>>> >>>>>> I chose to switch the protocol. So now I am writing an analyzer >>>>>> for >>>>>> 6LoWPAN >>>>>> instead of WiFi. >>>>>> >>>>>> Quick question: >>>>>> >>>>>> Will I be able to successfully use BinPac to write an analyzer >>>>>> for >>>>>> 6LoWPAN? >>>>>> >>>>>> Also, if possible, please guide me with some key points to >>>>>> remember >>>>>> while >>>>>> writing analyzer for 6LoWPAN. >>>>>> >>>>>> Your help will be greatly appreciated! >>>>>> >>>>>> Thanks, >>>>>> Pratik Inamdar >>>>>> >>>>>> On Mon, Mar 21, 2016 at 8:28 AM, Vlad Grigorescu >>>>>> >>>>>> wrote: >>>>>> >>>>>> Unfortunately, there is no way to implement lower level protocols >>>>>> with >>>>>> >>>>>>> BinPAC quickstart right now. Similary, we don't have any >>>>>>> examples of a >>>>>>> BinPAC lower-level analyzer if you were to do it manually. >>>>>>> >>>>>>> If you are able to get it working, I'd certainly be interested >>>>>>> in how >>>>>>> you did it, and would look at adding it to binpac_quickstart. >>>>>>> >>>>>>> --Vlad >>>>>>> >>>>>>> pratik inamdar writes: >>>>>>> >>>>>>> [ text/plain ] >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> In my project, I am integrating a WiFi protocol analyzer with >>>>>>>> bro to >>>>>>>> >>>>>>>> parse >>>>>>> >>>>>>> and monitor WiFi packets header information. >>>>>>>> >>>>>>>> I am using BinPac to generate template for the WiFi protocol >>>>>>>> analyzer in >>>>>>>> the src/analyzer/protocol directory. >>>>>>>> >>>>>>>> As per my knowledge WiFi(802.11) is not a TCP type of protocol. >>>>>>>> So I >>>>>>>> wish >>>>>>>> to know what should I use instead of the option "--tcp" while >>>>>>>> using >>>>>>>> the >>>>>>>> command: >>>>>>>> >>>>>>>> python start.py WiFi "WiFi Protocol" ../bro --tcp >>>>>>>> >>>>>>>> Any help will be really appreciated! >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks & Regards. >>>>>>>> >>>>>>>> Pratik Inamdar >>>>>>>> [ text/plain ] >>>>>>>> _______________________________________________ >>>>>>>> Bro mailing list >>>>>>>> bro at bro-ids.org >>>>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & Regards. >>>>>> >>>>>> Pratik Inamdar >>>>>> _______________________________________________ >>>>>> Bro mailing list >>>>>> bro at bro-ids.org >>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>>> >>>>>> >>>>> From pratikinamdar at gmail.com Fri Mar 25 11:15:43 2016 From: pratikinamdar at gmail.com (pratik inamdar) Date: Fri, 25 Mar 2016 11:15:43 -0700 Subject: [Bro] Integrating WiFi Analyzer within Bro In-Reply-To: <0B0D37F8-D7E1-4807-B22E-E90995AE937A@icir.org> References: <59AF02C0-6ABE-4021-9F6C-840F4CB50CD7@icir.org> <0B0D37F8-D7E1-4807-B22E-E90995AE937A@icir.org> Message-ID: I see! So you mean to say that no matter which layer does the protocol itself lies. If that protocol uses a port number (TCP/UDP) for transportation of its packets then ONLY I can use BinPac to write an analyzer for it. Thanks, Pratik Inamdar On Mar 25, 2016 10:37, "Johanna Amann" wrote: > It is as I said - there is a straightforward way to implement everything > above TCP/UDP with BinPac. RIP is not below the transport layer - it uses > UDP as its transport protocol. > > Johanna > > On 25 Mar 2016, at 10:32, pratik inamdar wrote: > > If what I said in my previous email is correct then why was I able to >> integrate RIP(Routing Information protocol) analyzer with bro? RIP is a >> network layer protocol which lied below Transport Layer. >> >> Similarly, 6LoWPAN is another name for IPV6 and is used for Low powered >> devices. If I was able to integrate RIP, do you think I will be integrate >> 6LoWPAN? >> >> Thanks, >> Pratik Inamdar >> On Mar 25, 2016 10:10, "pratik inamdar" wrote: >> >> So just to verify if I understood it correctly. >>> >>> You mean to say that all the protocols on or above the Transport Layer >>> ONLY should be able to use Bro BinPac? >>> >>> Thanks, >>> Pratik Inamdar >>> On Mar 25, 2016 10:06, "Johanna Amann" wrote: >>> >>> Hi, >>>> >>>> the answer is the same for all protocols that are underneath tcp/udp. >>>> You >>>> will have to change the core for any of them. If you have an IoT >>>> protocol >>>> that works over UDP/TCP, you should be able to use BinPac alone. >>>> >>>> Johanna >>>> >>>> On 25 Mar 2016, at 9:55, pratik inamdar wrote: >>>> >>>> Hi, >>>> >>>>> >>>>> Thanks you for the prompt response! >>>>> >>>>> My task is to write an analyzer in bro using BinPac for an IoT >>>>> protocol. >>>>> I >>>>> have already written analyzers for application layer protocols namely >>>>> MQTT >>>>> and AMQP. >>>>> >>>>> Now I wish to write an analyzer for an IoT protocol which does not fall >>>>> in >>>>> the application layer. >>>>> >>>>> The IoT protocol should be able to use Bro BinPac language. Could you >>>>> please suggest me one? >>>>> >>>>> Thanks, >>>>> Pratik Inamdar >>>>> On Mar 25, 2016 09:48, "Johanna Amann" wrote: >>>>> >>>>> Hello Pratik, >>>>> >>>>>> >>>>>> I think the answer stays the same - if I understand things correctly, >>>>>> you >>>>>> have to implement IEEE 802.15.4, which is a lower level protocol, >>>>>> which >>>>>> currently cannot be implemented with just BinPAC and needs core >>>>>> changes >>>>>> (probably in src/iosource/Packet.cc and others). There are currently >>>>>> no >>>>>> examples for that, besides the existing code. >>>>>> >>>>>> Johanna >>>>>> >>>>>> On 25 Mar 2016, at 9:40, pratik inamdar wrote: >>>>>> >>>>>> Hi Vlad, >>>>>> >>>>>> >>>>>>> Hope you are doing good! >>>>>>> >>>>>>> I chose to switch the protocol. So now I am writing an analyzer for >>>>>>> 6LoWPAN >>>>>>> instead of WiFi. >>>>>>> >>>>>>> Quick question: >>>>>>> >>>>>>> Will I be able to successfully use BinPac to write an analyzer for >>>>>>> 6LoWPAN? >>>>>>> >>>>>>> Also, if possible, please guide me with some key points to remember >>>>>>> while >>>>>>> writing analyzer for 6LoWPAN. >>>>>>> >>>>>>> Your help will be greatly appreciated! >>>>>>> >>>>>>> Thanks, >>>>>>> Pratik Inamdar >>>>>>> >>>>>>> On Mon, Mar 21, 2016 at 8:28 AM, Vlad Grigorescu >>>>>> > >>>>>>> wrote: >>>>>>> >>>>>>> Unfortunately, there is no way to implement lower level protocols >>>>>>> with >>>>>>> >>>>>>> BinPAC quickstart right now. Similary, we don't have any examples of >>>>>>>> a >>>>>>>> BinPAC lower-level analyzer if you were to do it manually. >>>>>>>> >>>>>>>> If you are able to get it working, I'd certainly be interested in >>>>>>>> how >>>>>>>> you did it, and would look at adding it to binpac_quickstart. >>>>>>>> >>>>>>>> --Vlad >>>>>>>> >>>>>>>> pratik inamdar writes: >>>>>>>> >>>>>>>> [ text/plain ] >>>>>>>> >>>>>>>> Hi, >>>>>>>>> >>>>>>>>> In my project, I am integrating a WiFi protocol analyzer with bro >>>>>>>>> to >>>>>>>>> >>>>>>>>> parse >>>>>>>>> >>>>>>>> >>>>>>>> and monitor WiFi packets header information. >>>>>>>> >>>>>>>>> >>>>>>>>> I am using BinPac to generate template for the WiFi protocol >>>>>>>>> analyzer in >>>>>>>>> the src/analyzer/protocol directory. >>>>>>>>> >>>>>>>>> As per my knowledge WiFi(802.11) is not a TCP type of protocol. So >>>>>>>>> I >>>>>>>>> wish >>>>>>>>> to know what should I use instead of the option "--tcp" while using >>>>>>>>> the >>>>>>>>> command: >>>>>>>>> >>>>>>>>> python start.py WiFi "WiFi Protocol" ../bro --tcp >>>>>>>>> >>>>>>>>> Any help will be really appreciated! >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks & Regards. >>>>>>>>> >>>>>>>>> Pratik Inamdar >>>>>>>>> [ text/plain ] >>>>>>>>> _______________________________________________ >>>>>>>>> Bro mailing list >>>>>>>>> bro at bro-ids.org >>>>>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks & Regards. >>>>>>> >>>>>>> Pratik Inamdar >>>>>>> _______________________________________________ >>>>>>> Bro mailing list >>>>>>> bro at bro-ids.org >>>>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>>>> >>>>>>> >>>>>>> >>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/4cc100e9/attachment.html From navraj42 at gmail.com Fri Mar 25 19:48:04 2016 From: navraj42 at gmail.com (Navraj Singh) Date: Fri, 25 Mar 2016 20:48:04 -0600 Subject: [Bro] Definition of a connection in conn.log In-Reply-To: <20160325165826.GA47770@wifi101.sys.ICSI.Berkeley.EDU> References: <20160325165826.GA47770@wifi101.sys.ICSI.Berkeley.EDU> Message-ID: Thanks Johanna! This helps a lot. On Fri, Mar 25, 2016 at 10:58 AM, Johanna Amann wrote: > Hello, > > let me try to give a few quick answers to your question. > > On Tue, Mar 22, 2016 at 03:00:25PM -0600, Navraj Singh wrote: > > 1. Is it safe to assume that any given packet will be assigned to at most > > one connection, and thus to at most one row in conn.log? > > No - there is a special case for tunnels, where a connection can be the > parent of another connection (the child shows its parent in the > tunnel_parents field of conn.log). In that case, the packet is assigned to > both the child and the parent. > > > 2. Why is it that some rows in conn.log do not have the duration field > set? > > I see see several row with a '-' in the duration field. > > That should mean that the duration was "0" (e.g. single packet), if I am > not mistaken. > > > 3. The bro documentation states that "For UDP and ICMP, ?connections? are > > to be interpreted using flow semantics (sequence of packets from a source > > host/port to a destination host/port)." However, what is the exact > > definition for a TCP flow? How does Bro decide which packets to include > in > > a connection? > > That s a not quite straightforward to answer question. Generally Bro > counts connections a 5-tuples; however, there are several timeouts at work > (after no packets were arrived for a certain amount of time, a connection > is seen as finished; if theree are new packets, a new connection will > begin). For TCP, a connection also can be ended by fin and rst packets, > and a new connection will begin afterwards. > > The timeouts are set using configuration variables - e.g. the default > tcp_inactivity_timeout, after which a TCP connection is considered closed > when no more packets are seen is 5 minutes. For UDP, it is 1 minute. For > ICMP it also is one minute. > > > 4. For an ongoing 'connection', does Bro wait until the connection is > over > > before logging it? What if the connection is quite long in > duration...won't > > that cause a lag? Or does Bro automatically chop up long flows based on > > some configurable limit parameter? > > Yup. And you are right, it will cause a lag in logging. > > I hope this helps, > Johanna > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160325/ebd27dfd/attachment.html From mz89924 at 126.com Sat Mar 26 19:18:31 2016 From: mz89924 at 126.com (mz) Date: Sun, 27 Mar 2016 10:18:31 +0800 Subject: [Bro] =?gb2312?b?tPC4tDogIGxvZ3MtdG8tZWxhc3RpY3NlYXJjaC5icm8gIGVy?= =?gb2312?b?cm9y?= In-Reply-To: <7A63317E-EAC2-4612-867F-E01FB67736DD@gmail.com> References: <004c01d1867a$968de9f0$c3a9bdd0$@126.com> <7A63317E-EAC2-4612-867F-E01FB67736DD@gmail.com> Message-ID: <000801d187ce$f62eccd0$e28c6670$@126.com> HI I installed the patch you provide, and the emergence of new error.By the way: I did not find this file krb-main bro source and elasticsearch plugins source code, so krb-main.patch I did not install the patch [2016-03-27 10:06:31,295][DEBUG][action.bulk ] [node-1] [mzh-201603190900][0] failed to execute bulk item (index) index {[mzh-201603190900][http][AVO10phhcNJqDxEYDvYi], source[{"ts":"2016-03-19T09:48:21.250090Z","uid":"CU0uvD4pYTE2YeoKh","id.ori g_h":"222.246.191.234","id.orig_p":11325,"id.resp_h":"119.143.122.225","id.r esp_p":80,"trans_depth":1,"method":"GET","host":"xxxxxx.com.cn","uri":"/img/ xxxxxx/Uploads/2015-08-24/55daef788341b.jpg","user_agent":"WeChat/6.3.13.17 CFNetwork/758.2.8 Darwin/15.0.0","request_body_len":0,"response_body_len":15201,"status_code": 200,"status_msg":"OK","tags":[],"resp_fuids":["F3xb6m3Ffqs0QW1AI4"],"resp_mi me_types":["image/jpeg"]}]} MapperParsingException[Field name [id.orig_h] cannot contain '.'] at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parsePropertie s(ObjectMapper.java:276) at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseObjectOrD ocumentTypeProperties(ObjectMapper.java:221) at org.elasticsearch.index.mapper.object.RootObjectMapper$TypeParser.parse(Root ObjectMapper.java:138) at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperPars er.java:119) at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperPars er.java:100) at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:435) at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor .applyRequest(MetaDataMappingService.java:257) at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor .execute(MetaDataMappingService.java:230) at org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor (InternalClusterService.java:458) at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(Inte rnalClusterService.java:762) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$Tie BreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java :231) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$Tie BreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 42) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 17) at java.lang.Thread.run(Thread.java:745) ???: Daniel Guerra [mailto:daniel.guerra69 at gmail.com] ????: 2016?3?25? 17:51 ???: mz ??: bro at bro.org ??: Re: [Bro] logs-to-elasticsearch.bro error Hi, To make this work you need some patches or use an elasticsearch version lower than 2 (1.7) I made a docker image for this https://hub.docker.com/r/danielguerra/bro-debian-elasticsearch/ In the git there is a map bro-patch https://github.com/danielguerra69/bro-debian-elasticsearch.git Regards, Daniel On 25 Mar 2016, at 10:42, mz > wrote: Dear Use logs-to-elasticsearch.bro send logs to ES. Is now work. ES error logs: [2016-03-25 17:30:52,957][DEBUG][action.bulk ] [node-1] [whbro-201603251500][1] failed to execute bulk item (index) index {[whbro-201603251500][dns][AVOtHLQHooGOx5uLgLSQ], source[{"_timestamp":1458898236411,"ts":1458898206267,"uid":"ClbNI74bIcRQ8Gs 6Wc","id.orig_h":"10.100.78.88","id.orig_p":137,"id.resp_h":"10.100.79.255", "id.resp_p":137,"proto":"udp","trans_id":47282,"query":"ISATAP","qclass":1," qclass_name":"C_INTERNET","qtype":32,"qtype_name":"NB","AA":false,"TC":false ,"RD":true,"RA":false,"Z":1,"rejected":false}]} MapperParsingException[Field [_timestamp] is a metadata field and cannot be added inside a document. Use the index API request parameters.] at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.jav a:213) at org.elasticsearch.index.mapper.DocumentParser.innerParseDocument(DocumentPar ser.java:131) at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.j ava:79) at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:304) at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:500) at org.elasticsearch.index.shard.IndexShard.prepareCreateOnPrimary(IndexShard.j ava:481) at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnP rimary(TransportIndexAction.java:214) at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPri mary(TransportIndexAction.java:223) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(T ransportShardBulkAction.java:326) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrima ry(TransportShardBulkAction.java:119) at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrima ry(TransportShardBulkAction.java:68) at org.elasticsearch.action.support.replication.TransportReplicationAction$Prim aryPhase.doRun(TransportReplicationAction.java:595) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnab le.java:37) at org.elasticsearch.action.support.replication.TransportReplicationAction$Prim aryOperationTransportHandler.messageReceived(TransportReplicationAction.java :263) at org.elasticsearch.action.support.replication.TransportReplicationAction$Prim aryOperationTransportHandler.messageReceived(TransportReplicationAction.java :260) at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:3 50) at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnab le.java:37) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 42) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 17) at java.lang.Thread.run(Thread.java:745) Bro config file: /usr/local/bro/lib/bro/plugins/Bro_ElasticSearch/scripts/init.bro module LogElasticSearch; export { ## Destination for the ES logs. Valid options are ## "direct" to directly connect to ES and "nsq" to ## transfer the logs into an nsqd instance. const destination = "direct" &redef; ## Name of the ES cluster. const cluster_name = "my-application" &redef; ## ES server. const server_host = "10.100.79.10" &redef; ## ES port. const server_port = 9200 &redef; ## Name of the ES index. const index_prefix = "testooo" &redef; ## Should the index names be in UTC or in local time? ## Setting this to true would be more compatible with Kibana and other tools. const index_name_in_utc = F &redef; ## Format for the index names. ## Setting this to "%Y.%m.%d-%H" would be more compatible Kibana and other tools. #const index_name_fmt = "%Y%m%d" &redef; const index_name_fmt = "%Y%m%d%H%M" &redef; ## The ES type prefix comes before the name of the related log. ## e.g. prefix = "bro\_" would create types of bro_dns, bro_software, etc. const type_prefix = "" &redef; ## The time before an ElasticSearch transfer will timeout. Note that ## the fractional part of the timeout will be ignored. In particular, ## time specifications less than a second result in a timeout value of ## 0, which means "no timeout." const transfer_timeout = 2secs; ## The batch size is the number of messages that will be queued up before ## they are sent to be bulk indexed. const max_batch_size = 1000 &redef; ## The maximum amount of wall-clock time that is allowed to pass without ## finishing a bulk log send. This represents the maximum delay you ## would like to have with your logs before they are sent to ElasticSearch. const max_batch_interval = 1min &redef; ## The maximum byte size for a buffered JSON string to send to the bulk ## insert API. const max_byte_size = 1024 * 1024 &redef; ## If the "nsq" destination is given, this is the topic ## that Bro will push logs into. const nsq_topic = "bro_logs" &redef; } _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160327/853883e0/attachment-0001.html From ansaf_130 at yahoo.com Sun Mar 27 01:56:12 2016 From: ansaf_130 at yahoo.com (Aneela Safdar) Date: Sun, 27 Mar 2016 08:56:12 +0000 (UTC) Subject: [Bro] Question : How can I change a particular log file format? References: <266584401.1260510.1459068973002.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <266584401.1260510.1459068973002.JavaMail.yahoo@mail.yahoo.com> Hi, I am a newbie at bro and wanted to change log format of http.log file to json.?Currently I have made changes in ascii.bro and now I am getting all logs format in json but what I have to do if I only want http.log to have that format and others keep default? Also my log files have got still .log extension, how can I change it o .json? Thanks,??Regards,?Aneela Safdar -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160327/aaf89f33/attachment.html From valerio.click at gmx.com Sun Mar 27 05:07:10 2016 From: valerio.click at gmx.com (Valerio) Date: Sun, 27 Mar 2016 14:07:10 +0200 Subject: [Bro] Access pcap filename in script land In-Reply-To: <82A8DAAC-3862-484F-BD1E-6A335A86024B@gmx.com> References: <56E7302E.6070609@gmx.com> <56E866D7.3070404@gmx.com> <20160316152849.GD40337@icir.org> <82A8DAAC-3862-484F-BD1E-6A335A86024B@gmx.com> Message-ID: <56F7CCEE.7010009@gmx.com> Hi, I am trying to adapt the suggested methodology to the following scenario: I have a custom main.bro script in scripts/base/protocols/proto1/main.bro to which I would like to pass arguments from command line each and every time I run bro with the following command bro -r pcap_file_name.pcap I set const arg1 =""&redef into main.bro but if I run bro -r pcap_file_name.pcap 'arg1=\"test\"' I get the following error error in , line 1: unrecognized character - \ error in , line 1: unrecognized character - " error in , line 1: unknown identifier test, at or near "test" many thanks in advance, Valerio On 16/03/2016 17:10, Valerio wrote: > Thanks a lot!!! > > Valerio > >> Il giorno 16 mar 2016, alle ore 16:28, Robin Sommer ha scritto: >> >> >> >>> On Tue, Mar 15, 2016 at 20:47 +0100, Valerio wrote: >>> >>> Is it possible when I start bro to pass external arguments to a bro >>> script? >> >> Yes, you can override script variables, like this: >> >> # cat args.bro >> >> const pcap_file = "" &redef; >> >> event bro_init() >> { >> print(pcap_file); >> } >> >> # bro ./args.bro 'pcap_file=\"Foo\"' >> "Foo" >> >> >> Robin >> >> -- >> Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > From vladg at illinois.edu Sun Mar 27 06:32:16 2016 From: vladg at illinois.edu (Vlad Grigorescu) Date: Sun, 27 Mar 2016 08:32:16 -0500 Subject: [Bro] Access pcap filename in script land In-Reply-To: <56F7CCEE.7010009@gmx.com> References: <56E7302E.6070609@gmx.com> <56E866D7.3070404@gmx.com> <20160316152849.GD40337@icir.org> <82A8DAAC-3862-484F-BD1E-6A335A86024B@gmx.com> <56F7CCEE.7010009@gmx.com> Message-ID: Try: > bro -r pcap_file_name.pcap -e 'arg1="test"' > -e|--exec | augment loaded policies by given code --Vlad Valerio writes: > Hi, > > I am trying to adapt the suggested methodology to the following > scenario: I have a custom main.bro script in > scripts/base/protocols/proto1/main.bro to which I would like to pass > arguments from command line each and every time I run bro with the > following command > > bro -r pcap_file_name.pcap > > > I set const arg1 =""&redef into main.bro but if I run > > bro -r pcap_file_name.pcap 'arg1=\"test\"' I get the following error > > > error in , line 1: unrecognized character - \ > error in , line 1: unrecognized character - " > error in , line 1: unknown identifier test, at or near "test" > > many thanks in advance, > Valerio > > On 16/03/2016 17:10, Valerio wrote: >> Thanks a lot!!! >> >> Valerio >> >>> Il giorno 16 mar 2016, alle ore 16:28, Robin Sommer ha scritto: >>> >>> >>> >>>> On Tue, Mar 15, 2016 at 20:47 +0100, Valerio wrote: >>>> >>>> Is it possible when I start bro to pass external arguments to a bro >>>> script? >>> >>> Yes, you can override script variables, like this: >>> >>> # cat args.bro >>> >>> const pcap_file = "" &redef; >>> >>> event bro_init() >>> { >>> print(pcap_file); >>> } >>> >>> # bro ./args.bro 'pcap_file=\"Foo\"' >>> "Foo" >>> >>> >>> Robin >>> >>> -- >>> Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160327/f65829ee/attachment.bin From valerio.click at gmx.com Sun Mar 27 07:06:26 2016 From: valerio.click at gmx.com (Valerio) Date: Sun, 27 Mar 2016 16:06:26 +0200 Subject: [Bro] Access pcap filename in script land In-Reply-To: References: <56E7302E.6070609@gmx.com> <56E866D7.3070404@gmx.com> <20160316152849.GD40337@icir.org> <82A8DAAC-3862-484F-BD1E-6A335A86024B@gmx.com> <56F7CCEE.7010009@gmx.com> Message-ID: <56F7E8E2.4030208@gmx.com> Thanks!! now it works running following command: bro -r pcap_file_name.pcap -e 'redef Prot1::arg1="test"' AND including 'const arg1=""&redef' within the export block of main.bro where module Prot1 was defined. best regards, Valerio On 27/03/2016 15:32, Vlad Grigorescu wrote: > Try: > >> bro -r pcap_file_name.pcap -e 'arg1="test"' > >> -e|--exec | augment loaded policies by given code > > --Vlad > > Valerio writes: > >> Hi, >> >> I am trying to adapt the suggested methodology to the following >> scenario: I have a custom main.bro script in >> scripts/base/protocols/proto1/main.bro to which I would like to pass >> arguments from command line each and every time I run bro with the >> following command >> >> bro -r pcap_file_name.pcap >> >> >> I set const arg1 =""&redef into main.bro but if I run >> >> bro -r pcap_file_name.pcap 'arg1=\"test\"' I get the following error >> >> >> error in , line 1: unrecognized character - \ >> error in , line 1: unrecognized character - " >> error in , line 1: unknown identifier test, at or near "test" >> >> many thanks in advance, >> Valerio >> >> On 16/03/2016 17:10, Valerio wrote: >>> Thanks a lot!!! >>> >>> Valerio >>> >>>> Il giorno 16 mar 2016, alle ore 16:28, Robin Sommer ha scritto: >>>> >>>> >>>> >>>>> On Tue, Mar 15, 2016 at 20:47 +0100, Valerio wrote: >>>>> >>>>> Is it possible when I start bro to pass external arguments to a bro >>>>> script? >>>> >>>> Yes, you can override script variables, like this: >>>> >>>> # cat args.bro >>>> >>>> const pcap_file = "" &redef; >>>> >>>> event bro_init() >>>> { >>>> print(pcap_file); >>>> } >>>> >>>> # bro ./args.bro 'pcap_file=\"Foo\"' >>>> "Foo" >>>> >>>> >>>> Robin >>>> >>>> -- >>>> Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin >>> >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From mz89924 at 126.com Mon Mar 28 00:07:10 2016 From: mz89924 at 126.com (mz) Date: Mon, 28 Mar 2016 15:07:10 +0800 Subject: [Bro] how to Modify the field name Message-ID: <000001d188c0$736ac310$5a404930$@126.com> Dear I use logs-to-elastisearch script to send logs to ES when being given field names can not contain ".", So how to modify the log field name -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160328/71c4a239/attachment.html From jazoff at illinois.edu Mon Mar 28 06:30:17 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Mon, 28 Mar 2016 13:30:17 +0000 Subject: [Bro] Access pcap filename in script land In-Reply-To: <56F7CCEE.7010009@gmx.com> References: <56E7302E.6070609@gmx.com> <56E866D7.3070404@gmx.com> <20160316152849.GD40337@icir.org> <82A8DAAC-3862-484F-BD1E-6A335A86024B@gmx.com> <56F7CCEE.7010009@gmx.com> Message-ID: <3D5FD476-2D1F-42C5-9B1E-817BBF3B27AB@illinois.edu> > On Mar 27, 2016, at 8:07 AM, Valerio wrote: > > Hi, > > I am trying to adapt the suggested methodology to the following > scenario: I have a custom main.bro script in > scripts/base/protocols/proto1/main.bro to which I would like to pass > arguments from command line each and every time I run bro with the > following command > > bro -r pcap_file_name.pcap > > > I set const arg1 =""&redef into main.bro but if I run > > bro -r pcap_file_name.pcap 'arg1=\"test\"' I get the following error > bro -r pcap_file_name.pcap arg1=test would have worked. -- - Justin Azoff From valerio.click at gmx.com Mon Mar 28 07:01:44 2016 From: valerio.click at gmx.com (Valerio) Date: Mon, 28 Mar 2016 16:01:44 +0200 Subject: [Bro] Access pcap filename in script land In-Reply-To: <3D5FD476-2D1F-42C5-9B1E-817BBF3B27AB@illinois.edu> References: <56E7302E.6070609@gmx.com> <56E866D7.3070404@gmx.com> <20160316152849.GD40337@icir.org> <82A8DAAC-3862-484F-BD1E-6A335A86024B@gmx.com> <56F7CCEE.7010009@gmx.com> <3D5FD476-2D1F-42C5-9B1E-817BBF3B27AB@illinois.edu> Message-ID: <56F93948.7050303@gmx.com> If I run bro -r pcap_file_name.pcap 'arg1="test"' I get the following error: error in , line 1: "redef" used but not previously defined (arg1) regards, Valerio On 28/03/2016 15:30, Azoff, Justin S wrote: > >> On Mar 27, 2016, at 8:07 AM, Valerio wrote: >> >> Hi, >> >> I am trying to adapt the suggested methodology to the following >> scenario: I have a custom main.bro script in >> scripts/base/protocols/proto1/main.bro to which I would like to pass >> arguments from command line each and every time I run bro with the >> following command >> >> bro -r pcap_file_name.pcap >> >> >> I set const arg1 =""&redef into main.bro but if I run >> >> bro -r pcap_file_name.pcap 'arg1=\"test\"' I get the following error >> > > bro -r pcap_file_name.pcap arg1=test > > would have worked. > From jazoff at illinois.edu Mon Mar 28 07:07:14 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Mon, 28 Mar 2016 14:07:14 +0000 Subject: [Bro] Access pcap filename in script land In-Reply-To: <56F93948.7050303@gmx.com> References: <56E7302E.6070609@gmx.com> <56E866D7.3070404@gmx.com> <20160316152849.GD40337@icir.org> <82A8DAAC-3862-484F-BD1E-6A335A86024B@gmx.com> <56F7CCEE.7010009@gmx.com> <3D5FD476-2D1F-42C5-9B1E-817BBF3B27AB@illinois.edu> <56F93948.7050303@gmx.com> Message-ID: <7A942398-1776-4787-AF61-11F6100B5F5E@illinois.edu> > On Mar 28, 2016, at 10:01 AM, Valerio wrote: > > If I run bro -r pcap_file_name.pcap 'arg1="test"' I get the following error: > > error in , line 1: "redef" used but not previously defined (arg1) > > regards, > Valerio That's because you did not define arg1... Also, 'arg1="test"' does not work. arg1=test works. 'arg1=test' would work. "arg1=test" would work, 'arg1="test"' does not work. $ cat t.bro const arg1 ="x" &redef ; event bro_init() { print arg1; } $ bro t.bro x $ bro t.bro arg1=test test $ bro t.bro 'arg1=test two' test two -- - Justin Azoff From valerio.click at gmx.com Mon Mar 28 09:20:52 2016 From: valerio.click at gmx.com (Valerio) Date: Mon, 28 Mar 2016 18:20:52 +0200 Subject: [Bro] Access pcap filename in script land In-Reply-To: <7A942398-1776-4787-AF61-11F6100B5F5E@illinois.edu> References: <56E7302E.6070609@gmx.com> <56E866D7.3070404@gmx.com> <20160316152849.GD40337@icir.org> <82A8DAAC-3862-484F-BD1E-6A335A86024B@gmx.com> <56F7CCEE.7010009@gmx.com> <3D5FD476-2D1F-42C5-9B1E-817BBF3B27AB@illinois.edu> <56F93948.7050303@gmx.com> <7A942398-1776-4787-AF61-11F6100B5F5E@illinois.edu> Message-ID: <56F959E4.6090205@gmx.com> Hi, thanks for your feedback. However I think I am missing something. In fact, if I run: bro -r pcap_file_name.pcap arg1=test I get the following error error in , line 1: unknown identifier test, at or near "test" Please notice that, as I described in my previous mail: "I have a custom main.bro script in scripts/base/protocols/proto1/main.bro to which I would like to pass arguments from command line" in the main.bro I defined arg1 as const arg1=""&redef within the export{} block and having define module Prot1; Even if I run bro -r pcap_file_name.pcap Prot1::arg1=test I get the following error error in , line 1: unknown identifier test, at or near "test" the same with: bro -r 28122015-whatsapp_iphone_traffic.pcap 'Wa::arg1=test' best regards, Valerio On 28/03/2016 16:07, Azoff, Justin S wrote: > >> On Mar 28, 2016, at 10:01 AM, Valerio wrote: >> >> If I run bro -r pcap_file_name.pcap 'arg1="test"' I get the following error: >> >> error in , line 1: "redef" used but not previously defined (arg1) >> >> regards, >> Valerio > > That's because you did not define arg1... Also, 'arg1="test"' does not work. arg1=test works. 'arg1=test' would work. "arg1=test" would work, 'arg1="test"' does not work. > > $ cat t.bro > const arg1 ="x" &redef ; > > event bro_init() { > print arg1; > } > $ bro t.bro > x > $ bro t.bro arg1=test > test > $ bro t.bro 'arg1=test two' > test two > From jazoff at illinois.edu Mon Mar 28 09:29:42 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Mon, 28 Mar 2016 16:29:42 +0000 Subject: [Bro] Access pcap filename in script land In-Reply-To: <56F959E4.6090205@gmx.com> References: <56E7302E.6070609@gmx.com> <56E866D7.3070404@gmx.com> <20160316152849.GD40337@icir.org> <82A8DAAC-3862-484F-BD1E-6A335A86024B@gmx.com> <56F7CCEE.7010009@gmx.com> <3D5FD476-2D1F-42C5-9B1E-817BBF3B27AB@illinois.edu> <56F93948.7050303@gmx.com> <7A942398-1776-4787-AF61-11F6100B5F5E@illinois.edu> <56F959E4.6090205@gmx.com> Message-ID: <21956566-F334-4F3D-8D9A-B9664E586A44@illinois.edu> > On Mar 28, 2016, at 12:20 PM, Valerio wrote: > > Hi, > > thanks for your feedback. However I think I am missing something. > In fact, if I run: > > bro -r pcap_file_name.pcap arg1=test > > I get the following error > > error in , line 1: unknown identifier test, at or near "test" > > Please notice that, as I described in my previous mail: > > "I have a custom main.bro script in > scripts/base/protocols/proto1/main.bro to which I would like to pass > arguments from command line" > > in the main.bro I defined arg1 as const arg1=""&redef within the > export{} block and having define module Prot1; > > Even if I run > > bro -r pcap_file_name.pcap Prot1::arg1=test > > I get the following error > error in , line 1: unknown identifier test, at or near "test" > > the same with: > bro -r 28122015-whatsapp_iphone_traffic.pcap 'Wa::arg1=test' > > best regards, > Valerio The different methods do work: $ cat t.bro module Foo; export { const arg1 ="x" &redef ; } event bro_init() { print arg1; } $ bro t.bro x $ bro t.bro Foo::arg1=test test $ bro t.bro 'Foo::arg1=test two' test two $ bro t.bro -e 'redef Foo::arg1="test three"' test three Are you actually loading your proto1 script anywhere? local scripts really belong under share/bro/site, not under the base/ directory. You likely want to move your proto1 directory to share/bro/site and add @load ./proto1 to share/bro/site/local.bro -- - Justin Azoff From valerio.click at gmx.com Mon Mar 28 09:32:04 2016 From: valerio.click at gmx.com (Valerio) Date: Mon, 28 Mar 2016 18:32:04 +0200 Subject: [Bro] Access pcap filename in script land In-Reply-To: <21956566-F334-4F3D-8D9A-B9664E586A44@illinois.edu> References: <56E7302E.6070609@gmx.com> <56E866D7.3070404@gmx.com> <20160316152849.GD40337@icir.org> <82A8DAAC-3862-484F-BD1E-6A335A86024B@gmx.com> <56F7CCEE.7010009@gmx.com> <3D5FD476-2D1F-42C5-9B1E-817BBF3B27AB@illinois.edu> <56F93948.7050303@gmx.com> <7A942398-1776-4787-AF61-11F6100B5F5E@illinois.edu> <56F959E4.6090205@gmx.com> <21956566-F334-4F3D-8D9A-B9664E586A44@illinois.edu> Message-ID: <56F95C84.8080700@gmx.com> Thanks!! by moving the script from base to share/bro/site it works!! best, Valerio On 28/03/2016 18:29, Azoff, Justin S wrote: > >> On Mar 28, 2016, at 12:20 PM, Valerio wrote: >> >> Hi, >> >> thanks for your feedback. However I think I am missing something. >> In fact, if I run: >> >> bro -r pcap_file_name.pcap arg1=test >> >> I get the following error >> >> error in , line 1: unknown identifier test, at or near "test" >> >> Please notice that, as I described in my previous mail: >> >> "I have a custom main.bro script in >> scripts/base/protocols/proto1/main.bro to which I would like to pass >> arguments from command line" >> >> in the main.bro I defined arg1 as const arg1=""&redef within the >> export{} block and having define module Prot1; >> >> Even if I run >> >> bro -r pcap_file_name.pcap Prot1::arg1=test >> >> I get the following error >> error in , line 1: unknown identifier test, at or near "test" >> >> the same with: >> bro -r 28122015-whatsapp_iphone_traffic.pcap 'Wa::arg1=test' >> >> best regards, >> Valerio > > The different methods do work: > > $ cat t.bro > module Foo; > export { > const arg1 ="x" &redef ; > } > > event bro_init() { > print arg1; > } > $ bro t.bro > x > $ bro t.bro Foo::arg1=test > test > $ bro t.bro 'Foo::arg1=test two' > test two > $ bro t.bro -e 'redef Foo::arg1="test three"' > test three > > Are you actually loading your proto1 script anywhere? local scripts really belong under share/bro/site, not under the base/ directory. > > You likely want to move your proto1 directory to share/bro/site and add > > @load ./proto1 > > to share/bro/site/local.bro > From rmkml at ligfy.org Mon Mar 28 12:05:36 2016 From: rmkml at ligfy.org (rmkml) Date: Mon, 28 Mar 2016 21:05:36 +0200 (CEST) Subject: [Bro] Bro v241 locked with pcap file ? Message-ID: Hi, I have replayed many pcap files on Bro v2.4.1, but discovered one pcap "lock" bro process (bro never quit). Simply replay: bro241 -r bro241lock.pcap Added -C or -b or 'export BRO_DNS_FAKE=1' have same pb. Please find a joigned a pcap file link: http://etplc.org/bro241lock.pcap.gz (This partial file sended to bro list around 2009 not by me...) Open ticket BIT-1562 on Bro Tracker. Tested on Ubuntu v14.04.4 LTS. Regards @Rmkml From john.b.althouse at gmail.com Mon Mar 28 13:24:18 2016 From: john.b.althouse at gmail.com (John Althouse) Date: Mon, 28 Mar 2016 16:24:18 -0400 Subject: [Bro] Detection Scripts Message-ID: What are some good repos to visit to find detection scripts? A lot of us are writing detection scripts because our jobs require us to detect all the things but we should avoid building the same thing twice in different silos if we can ;) Here's the ones I talked about at Bro4Pros: https://github.com/darkphyber/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160328/8b128ecf/attachment.html From npratley at redhat.com Mon Mar 28 18:26:58 2016 From: npratley at redhat.com (Nick Pratley) Date: Tue, 29 Mar 2016 11:26:58 +1000 Subject: [Bro] Detection Scripts In-Reply-To: References: Message-ID: <1459214818.4132.2.camel@redhat.com> On Mon, 2016-03-28 at 16:24 -0400, John Althouse wrote: > What are some good repos to visit to find detection scripts? > > A lot of us are writing detection scripts because our jobs require us > to detect all the things but we should avoid building the same thing > twice in different silos if we can ;)? > > Here's the ones I talked about at Bro4Pros: > https://github.com/darkphyber/bro John, I saw the slides, looks like it was a good talk. Thanks for sharing the scripts. This is probably too obvious but?https://github.com/trending/bro?has a good list of repos with Bro detection scripts. I think there is some work under way for a centralised repository: ?http://blog.bro.org/2015/12/bro-receives-200k-grant-from-mozilla.html Cheers, Nick From scotty.b.brown at gmail.com Mon Mar 28 21:41:21 2016 From: scotty.b.brown at gmail.com (Scotty Brown) Date: Tue, 29 Mar 2016 14:41:21 +1000 Subject: [Bro] Bro email notice question In-Reply-To: <56F4046E.5070301@gmail.com> References: <56F372EE.9060909@gmail.com> <56F4046E.5070301@gmail.com> Message-ID: <56FA0771.6020909@gmail.com> Hi Jan, Thank you! This is exactly what I was after. I did have to add a missing closing bracket ) to line 39. Did you ever have any discussion on getting this added/changed to the default do_notice that is distributed with bro? Cheers, Scotty On 25/03/16 01:14, Jan Grash?fer wrote: > Hi Scotty, > >> I've tried, but can't figure out how I add $sources from the Intel log into say $sub in /opt/bro/share/bro/policy/intel/do_notice.bro > Some time ago, I adapted the do_notice.bro script to add an identifier > (for notice suppression) and also added some information (e.g. intel > source) to the mails (see > https://gist.github.com/J-Gras/c2e0853c93c0bdc74522). I hope this will > help you :) > > Regards, > Jan > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From jan.grashoefer at gmail.com Tue Mar 29 09:15:41 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Tue, 29 Mar 2016 18:15:41 +0200 Subject: [Bro] Bro email notice question In-Reply-To: <56FA0771.6020909@gmail.com> References: <56F372EE.9060909@gmail.com> <56F4046E.5070301@gmail.com> <56FA0771.6020909@gmail.com> Message-ID: <56FAAA2D.9010705@gmail.com> Hi Scotty, > Thank you! This is exactly what I was after. I did have to add a > missing closing bracket ) to line 39. You are welcome! I fixed the bracket as well as the misleading indentation of the script. > Did you ever have any discussion on getting this added/changed to the > default do_notice that is distributed with bro? If I remember correctly, the intention of do_notice.bro was to provide an example how the intel-framework could be used in this context. I think the example somehow became the default. Therefore I was not sure whether these changes would be suited for the do_notice.bro shipped with Bro. Regards, Jan From mz89924 at 126.com Tue Mar 29 23:03:48 2016 From: mz89924 at 126.com (mz) Date: Wed, 30 Mar 2016 14:03:48 +0800 Subject: [Bro] where is pcap file Message-ID: <000201d18a49$ee162750$ca4275f0$@126.com> Hi everyone Where is bro captured packets? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160330/98adcccc/attachment.html From mz89924 at 126.com Wed Mar 30 03:14:51 2016 From: mz89924 at 126.com (mz) Date: Wed, 30 Mar 2016 18:14:51 +0800 Subject: [Bro] Bro Intrusion Detection Features Message-ID: <002501d18a6d$01c60cc0$05522640$@126.com> Hi Everyone About IDS has several questions: 1. Intrusion detection rules official Bro is provided only these? https://www.bro.org/sphinx/bro-noticeindex.html 2. Where can I find more intrusion detection rules? 3. Bro whether now also supports the import snort rules? If support how to import? Regards, Siemon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160330/14898c36/attachment.html From castle1126 at yahoo.com Wed Mar 30 08:04:29 2016 From: castle1126 at yahoo.com (Stephen Castellarin) Date: Wed, 30 Mar 2016 15:04:29 +0000 (UTC) Subject: [Bro] Bro 2.4.1 and issue with smtp-embedded-url-bloom.bro References: <585487798.2626246.1459350269647.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <585487798.2626246.1459350269647.JavaMail.yahoo@mail.yahoo.com> Hi all, I've set up a Bro instance to test out URL extraction from SMTP, using the smtp-embedded-url-bloom.bro scripts. ?For the most part the extract/logging is working, but many times I'll find that the host and url logged will be truncated. ?As an example I'd see one email listed that has 20 links extracted, but one log entry would have host name as "award" with the url as "http://award". ?The remaining URLs for that email look to be extracted correctly. Has anyone else noticed this issue? Thanks,Steve -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160330/6722fbc9/attachment.html From filus at psc.edu Wed Mar 30 15:37:33 2016 From: filus at psc.edu (Shane Filus) Date: Wed, 30 Mar 2016 18:37:33 -0400 Subject: [Bro] bro script/code sharing Message-ID: <56FC552D.4020908@psc.edu> All, Hoping not to reinvent the wheel or duplicate work, but didn't find much via google or in git. Any info/pointers/code to the following areas would be greatly appreciated: 1. dDoS detection script This was a list topic back in 9/14. The thread hinted that a script was sent, but I didn't see one in email or the list archive. 2. bro log reporting Back in the 2.0/BroLite days, there was 'site-report.pl' that could generate daily summaries from bro logs. Anyone using anything similar for recent releases? 3. bro 'configurator' The dream is for a web interface you could use to configure and inspect a bro deployment. Something that scans all the code(base/policy/site) for modules/vars/notices and generates HTML. thanks in advance! Shane From jlay at slave-tothe-box.net Wed Mar 30 16:54:37 2016 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 30 Mar 2016 17:54:37 -0600 Subject: [Bro] Bro 2.4.1 and issue with smtp-embedded-url-bloom.bro In-Reply-To: <585487798.2626246.1459350269647.JavaMail.yahoo@mail.yahoo.com> References: <585487798.2626246.1459350269647.JavaMail.yahoo.ref@mail.yahoo.com> <585487798.2626246.1459350269647.JavaMail.yahoo@mail.yahoo.com> Message-ID: <1459382077.2808.4.camel@gamebox> On Wed, 2016-03-30 at 15:04 +0000, Stephen Castellarin wrote: > Hi all, > > > I've set up a Bro instance to test out URL extraction from SMTP, using > the smtp-embedded-url-bloom.bro scripts. For the most part the > extract/logging is working, but many times I'll find that the host and > url logged will be truncated. As an example I'd see one email listed > that has 20 links extracted, but one log entry would have host name as > "award" with the url as "http://award". The remaining URLs for that > email look to be extracted correctly. > > > Has anyone else noticed this issue? > > Thanks, > Steve > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro Yep...I suspect emails that are quoted-printable emails fall victim to this: https://en.wikipedia.org/wiki/Quoted-printable James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160330/d283c4d6/attachment.html From asharma at lbl.gov Wed Mar 30 17:05:40 2016 From: asharma at lbl.gov (Aashish Sharma) Date: Wed, 30 Mar 2016 17:05:40 -0700 Subject: [Bro] Bro 2.4.1 and issue with smtp-embedded-url-bloom.bro In-Reply-To: <1459382077.2808.4.camel@gamebox> References: <585487798.2626246.1459350269647.JavaMail.yahoo.ref@mail.yahoo.com> <585487798.2626246.1459350269647.JavaMail.yahoo@mail.yahoo.com> <1459382077.2808.4.camel@gamebox> Message-ID: <20160331000538.GM22895@yaksha.lbl.gov> Hello James, Yes, that was caused in a very early version of the script because of using You should try this: - event mime_segment_data(c: connection, length: count, data: string) &priority=-5 + event mime_all_data(c: connection, length: count, data: string) &priority=-5 Or try this policy: https://github.com/initconf/smtp-analysis/blob/master/smtp-embedded-url-bloom.bro Aashish On Wed, Mar 30, 2016 at 05:54:37PM -0600, James Lay wrote: > > On Wed, 2016-03-30 at 15:04 +0000, Stephen Castellarin wrote: > > Hi all, > > I've set up a Bro instance to test out URL extraction from SMTP, using the > smtp-embedded-url-bloom.bro scripts. For the most part the > extract/logging is working, but many times I'll find that the host and url > logged will be truncated. As an example I'd see one email listed that has > 20 links extracted, but one log entry would have host name as "award" with > the url as "http://award". The remaining URLs for that email look to be > extracted correctly. > > Has anyone else noticed this issue? > Thanks, > > Steve > > _______________________________________________ > Bro mailing list > [1]bro at bro-ids.org > [2]http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > Yep...I suspect emails that are quoted-printable emails fall victim to this: > [3]https://en.wikipedia.org/wiki/Quoted-printable > James > > References > > 1. mailto:bro at bro-ids.org > 2. http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > 3. https://en.wikipedia.org/wiki/Quoted-printable > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From hckim at narusec.com Wed Mar 30 18:44:41 2016 From: hckim at narusec.com (=?UTF-8?B?6rmA7Z2s7LKg?=) Date: Thu, 31 Mar 2016 10:44:41 +0900 Subject: [Bro] bridge interface(br0) does not show SYN in bro Message-ID: Hi I have setup bro 2.4.1 to monitor bridge interface(br0) with pf-ring in the conn.log history field, there are lot of them missing 'S' I did not have this problem with bro2.3 with same setup. in bro 2.4.1 if I change br0 to physical NIC eth4, this problem goes away is there a way to work around this? has anyone encounter this kind of problem? my setup is cpu: AMD Opteron 6376 32core ram: 64G ubuntu 12.04.5 bro 2.4.1 pf-ring version-5.6.1, mode 0, RX+TX intel NIC 4port (igb dirver) *conn.log history count TOP 25 :* 15265 Dd 8796 D 7267 hadfF 6558 hadf 2629 FRa 2294 Fr 1938 hadFf 1883 Fa 1298 S 1245 hadfFR 1134 hf 1067 d 1043 - 1001 F 984 R 858 hdf 700 hdaFf 667 FRr 643 hdfFa 608 ShADadFr 568 ShADfFa 517 r 474 hadR 416 hdafF 393 hFf 363 hdaf 360 hadFR *bro node.cfg* [manager] type=manager host=localhost [proxy-1] type=proxy host=localhost [proxy-2] type=proxy host=localhost [br0] type=worker host=localhost interface=br0 lb_method=pf_ring lb_procs=8 pin_cpus=2,3,4,5,6,7,8,9 *bro network.cfg* 192.168.0.0/16 *network NIC and bridge setup:* rmmod igb && modprobe igb modprobe pf_ring transparent_mode=0 enable_tx_capture=1 ifconfig eth4 down ethtool -K eth4 rx off ethtool -K eth4 tx off ethtool -K eth4 sg off ethtool -K eth4 tso off ethtool -K eth4 gso off ethtool -K eth4 gro off ifconfig eth4 mtu 1514 ifconfig eth5 down ethtool -K eth5 rx off ethtool -K eth5 tx off ethtool -K eth5 sg off ethtool -K eth5 tso off ethtool -K eth5 gso off ethtool -K eth5 gro off ifconfig eth5 mtu 1514 brctl addbr br0 brctl addif br0 eth4 ifconfig eth4 promisc up -multicast brctl addif br0 eth5 ifconfig eth5 promisc up -multicast ethtool stp br0 on ethtool -K br0 sg off ethtool -K br0 tso off ethtool -K br0 gso off ethtool -K br0 gro off ethtool -K br0 lro off ethtool -K br0 rxvlan off ethtool -K br0 txvlan off ifconfig br0 mtu 1514 ifconfig br0 promisc up -multicast -- ------------------------------------------------------ Hichul Kim ??? ?? ??? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160331/88be652a/attachment.html From jlay at slave-tothe-box.net Thu Mar 31 05:11:29 2016 From: jlay at slave-tothe-box.net (James Lay) Date: Thu, 31 Mar 2016 06:11:29 -0600 Subject: [Bro] Bro 2.4.1 and issue with smtp-embedded-url-bloom.bro In-Reply-To: <20160331000538.GM22895@yaksha.lbl.gov> References: <585487798.2626246.1459350269647.JavaMail.yahoo.ref@mail.yahoo.com> <585487798.2626246.1459350269647.JavaMail.yahoo@mail.yahoo.com> <1459382077.2808.4.camel@gamebox> <20160331000538.GM22895@yaksha.lbl.gov> Message-ID: <1459426289.2795.1.camel@gamebox> Thank you Aashish...that's awesome! James On Wed, 2016-03-30 at 17:05 -0700, Aashish Sharma wrote: > Hello James, > > Yes, that was caused in a very early version of the script because of using > > You should try this: > > - event mime_segment_data(c: connection, length: count, data: string) &priority=-5 > + event mime_all_data(c: connection, length: count, data: string) &priority=-5 > > > Or try this policy: > > https://github.com/initconf/smtp-analysis/blob/master/smtp-embedded-url-bloom.bro > > Aashish > > > > > On Wed, Mar 30, 2016 at 05:54:37PM -0600, James Lay wrote: > > > > On Wed, 2016-03-30 at 15:04 +0000, Stephen Castellarin wrote: > > > > Hi all, > > > > I've set up a Bro instance to test out URL extraction from SMTP, using the > > smtp-embedded-url-bloom.bro scripts. For the most part the > > extract/logging is working, but many times I'll find that the host and url > > logged will be truncated. As an example I'd see one email listed that has > > 20 links extracted, but one log entry would have host name as "award" with > > the url as "http://award". The remaining URLs for that email look to be > > extracted correctly. > > > > Has anyone else noticed this issue? > > Thanks, > > > > Steve > > > > _______________________________________________ > > Bro mailing list > > [1]bro at bro-ids.org > > [2]http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > Yep...I suspect emails that are quoted-printable emails fall victim to this: > > [3]https://en.wikipedia.org/wiki/Quoted-printable > > James > > > > References > > > > 1. mailto:bro at bro-ids.org > > 2. http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > 3. https://en.wikipedia.org/wiki/Quoted-printable > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160331/37b00846/attachment-0001.html From js688886 at gmail.com Thu Mar 31 09:25:42 2016 From: js688886 at gmail.com (john smith) Date: Thu, 31 Mar 2016 09:25:42 -0700 Subject: [Bro] SFTP analysis Message-ID: Hello, Does anyone know if Bro supports SFTP? Thanks in advance. John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160331/7b75f7ff/attachment.html From jlay at slave-tothe-box.net Thu Mar 31 13:43:32 2016 From: jlay at slave-tothe-box.net (James Lay) Date: Thu, 31 Mar 2016 14:43:32 -0600 Subject: [Bro] Bro 2.4.1 and issue with smtp-embedded-url-bloom.bro In-Reply-To: <20160331000538.GM22895@yaksha.lbl.gov> References: <585487798.2626246.1459350269647.JavaMail.yahoo.ref@mail.yahoo.com> <585487798.2626246.1459350269647.JavaMail.yahoo@mail.yahoo.com> <1459382077.2808.4.camel@gamebox> <20160331000538.GM22895@yaksha.lbl.gov> Message-ID: <71329479382e3d4a7f856cbe721df3b5@localhost> Unfortunately I get this when running the latest version: 1459456959.248537 expression error in /usr/local/bro/share/bro/site/smtp-embedded-url-bloom.bro, line 156: field value missing [SMTPurl::c$smtp$from] Thank you. James On 2016-03-30 18:05, Aashish Sharma wrote: > Hello James, > > Yes, that was caused in a very early version of the script because of > using > > You should try this: > > - event mime_segment_data(c: connection, length: count, data: string) > &priority=-5 > + event mime_all_data(c: connection, length: count, data: string) > &priority=-5 > > > Or try this policy: > > https://github.com/initconf/smtp-analysis/blob/master/smtp-embedded-url-bloom.bro > > Aashish > > > > > On Wed, Mar 30, 2016 at 05:54:37PM -0600, James Lay wrote: >> >> On Wed, 2016-03-30 at 15:04 +0000, Stephen Castellarin wrote: >> >> Hi all, >> >> I've set up a Bro instance to test out URL extraction from SMTP, >> using the >> smtp-embedded-url-bloom.bro scripts. For the most part >> the >> extract/logging is working, but many times I'll find that the >> host and url >> logged will be truncated. As an example I'd see one email listed >> that has >> 20 links extracted, but one log entry would have host name as >> "award" with >> the url as "http://award". The remaining URLs for that email >> look to be >> extracted correctly. >> >> Has anyone else noticed this issue? >> Thanks, >> >> Steve >> >> _______________________________________________ >> Bro mailing list >> [1]bro at bro-ids.org >> [2]http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> Yep...I suspect emails that are quoted-printable emails fall victim >> to this: >> [3]https://en.wikipedia.org/wiki/Quoted-printable >> James >> >> References >> >> 1. mailto:bro at bro-ids.org >> 2. http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> 3. https://en.wikipedia.org/wiki/Quoted-printable > >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From asharma at lbl.gov Thu Mar 31 14:10:26 2016 From: asharma at lbl.gov (Aashish Sharma) Date: Thu, 31 Mar 2016 14:10:26 -0700 Subject: [Bro] Bro 2.4.1 and issue with smtp-embedded-url-bloom.bro In-Reply-To: <71329479382e3d4a7f856cbe721df3b5@localhost> References: <585487798.2626246.1459350269647.JavaMail.yahoo.ref@mail.yahoo.com> <585487798.2626246.1459350269647.JavaMail.yahoo@mail.yahoo.com> <1459382077.2808.4.camel@gamebox> <20160331000538.GM22895@yaksha.lbl.gov> <71329479382e3d4a7f856cbe721df3b5@localhost> Message-ID: <20160331211024.GN28715@yaksha.lbl.gov> Ah! I see the entires in reporter.log I have uploaded a revised version. This should fix the issue. Please try this https://github.com/initconf/smtp-analysis/blob/master/smtp-embedded-url-cluster.bro Also note: SMTP_Link_in_EMAIL_Clicked will only partially work in the cluster setup with this policy. I have a clusterized version of this policy but I am not entirely satisfied with it. It syncs extracted URLs across the nodes so check against all HTTP traffic ranter than just the node which saw the smtp connection. However, there are a few corner cases I need to address. Aashish On Thu, Mar 31, 2016 at 02:43:32PM -0600, James Lay wrote: > Unfortunately I get this when running the latest version: > > 1459456959.248537 expression error in > /usr/local/bro/share/bro/site/smtp-embedded-url-bloom.bro, line 156: > field value missing [SMTPurl::c$smtp$from] > > Thank you. > > James > > On 2016-03-30 18:05, Aashish Sharma wrote: > >Hello James, > > > >Yes, that was caused in a very early version of the script because > >of using > > > >You should try this: > > > >- event mime_segment_data(c: connection, length: count, data: string) > >&priority=-5 > >+ event mime_all_data(c: connection, length: count, data: string) > >&priority=-5 > > > > > >Or try this policy: > > > >https://github.com/initconf/smtp-analysis/blob/master/smtp-embedded-url-bloom.bro > > > >Aashish > > > > > > > > > >On Wed, Mar 30, 2016 at 05:54:37PM -0600, James Lay wrote: > >> > >> On Wed, 2016-03-30 at 15:04 +0000, Stephen Castellarin wrote: > >> > >> Hi all, > >> > >> I've set up a Bro instance to test out URL extraction from > >>SMTP, using the > >> smtp-embedded-url-bloom.bro scripts. For the most > >>part the > >> extract/logging is working, but many times I'll find that > >>the host and url > >> logged will be truncated. As an example I'd see one email > >>listed that has > >> 20 links extracted, but one log entry would have host name > >>as "award" with > >> the url as "http://award". The remaining URLs for that > >>email look to be > >> extracted correctly. > >> > >> Has anyone else noticed this issue? > >> Thanks, > >> > >> Steve > >> > >>_______________________________________________ > >>Bro mailing list > >>[1]bro at bro-ids.org > >>[2]http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > >> > >> Yep...I suspect emails that are quoted-printable emails fall > >>victim to this: > >> [3]https://en.wikipedia.org/wiki/Quoted-printable > >> James > >> > >>References > >> > >> 1. mailto:bro at bro-ids.org > >> 2. http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > >> 3. https://en.wikipedia.org/wiki/Quoted-printable > > > >>_______________________________________________ > >>Bro mailing list > >>bro at bro-ids.org > >>http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro