From seth at corelight.com Thu Feb 1 08:10:23 2018 From: seth at corelight.com (Seth Hall) Date: Thu, 01 Feb 2018 11:10:23 -0500 Subject: [Bro] A little more confusion with Intel In-Reply-To: References: <60a34e5669db124ce69305bc1232ab3c@localhost> <309A727C-26D8-4D26-975D-8FDD5158A82E@corelight.com> <3ccd76709d5eafabe321885b5acec101@localhost> <3A565BA7-E885-420B-BB4A-3689FFE9150F@corelight.com> Message-ID: Ahh! I understand your use case better now. We could use the effective-tld package to create a new "seen" injector for the intel framework that pokes effective TLDs into the intel framework. I don't know what the overhead effects of this would be, but it might not be too bad. .Seth On 31 Jan 2018, at 18:21, James Lay wrote: > Thanks Seth, > > I basically modified this for bro use: > > https://isc.sans.edu/forums/diary/Tracking+Newly+Registered+Domains/23127/ > > It's basically a list of domain names that have been newly registered. > Does that help? > > James > > On 2018-01-29 09:14, Seth Hall wrote: >> On 22 Jan 2018, at 11:30, James Lay wrote: >> >>> It's actually the inverse of what I'm seeing.? In my tests if I >>> have Intel::DOMAIN yahoo.com and I did a >>> "dig?[www.yahoo.com",]()?the domain intel >>> would not match because the dns request was for "www.yahoo.com", not >>> yahoo.com.? Does that make sense?? Thank you. >> >> Yeah, if we had a more comprehensive matcher for the intel framework >> then you'd have a lot of options open for you. I suppose that my >> main >> point was that at the moment you will have to just include the exact >> domain that you want to match on. >> >> Do you have a large list where you'd like to watch for any hits on >> the >> effective second level domain like you're describing here? >> >> .Seth >> >> -- >> Seth Hall * Corelight, Inc * www.corelight.com -- Seth Hall * Corelight, Inc * www.corelight.com From jan.grashoefer at gmail.com Thu Feb 1 08:50:50 2018 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Thu, 1 Feb 2018 17:50:50 +0100 Subject: [Bro] A little more confusion with Intel In-Reply-To: References: <60a34e5669db124ce69305bc1232ab3c@localhost> <309A727C-26D8-4D26-975D-8FDD5158A82E@corelight.com> <3ccd76709d5eafabe321885b5acec101@localhost> <3A565BA7-E885-420B-BB4A-3689FFE9150F@corelight.com> Message-ID: <974524ba-b30b-ec48-0f7e-d408835494e2@gmail.com> On 01/02/18 17:10, Seth Hall wrote: > We could use the > effective-tld package to create a new "seen" injector for the intel > framework that pokes effective TLDs into the intel framework. I don't > know what the overhead effects of this would be, but it might not be too > bad. Friendly reminder: https://github.com/J-Gras/intel-seen-more ;) Jan From seth at corelight.com Thu Feb 1 10:09:26 2018 From: seth at corelight.com (Seth Hall) Date: Thu, 01 Feb 2018 13:09:26 -0500 Subject: [Bro] A little more confusion with Intel In-Reply-To: <974524ba-b30b-ec48-0f7e-d408835494e2@gmail.com> References: <60a34e5669db124ce69305bc1232ab3c@localhost> <309A727C-26D8-4D26-975D-8FDD5158A82E@corelight.com> <3ccd76709d5eafabe321885b5acec101@localhost> <3A565BA7-E885-420B-BB4A-3689FFE9150F@corelight.com> <974524ba-b30b-ec48-0f7e-d408835494e2@gmail.com> Message-ID: On 1 Feb 2018, at 11:50, Jan Grash?fer wrote: > On 01/02/18 17:10, Seth Hall wrote: >> We could use the >> effective-tld package to create a new "seen" injector for the intel >> framework that pokes effective TLDs into the intel framework. I >> don't >> know what the overhead effects of this would be, but it might not be >> too >> bad. > > Friendly reminder: https://github.com/J-Gras/intel-seen-more ;) Hahaha! Sorry, I forgot about that already! I had a thought about it too, what do you think about changing Intel::EFFECTIVE_DOMAIN to Intel::EFFECTIVE_TLD? Seems like it makes sense since the TLD is what you end up matching with this and it fits James' use case correctly. .Seth -- Seth Hall * Corelight, Inc * www.corelight.com From jan.grashoefer at gmail.com Thu Feb 1 11:34:51 2018 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Thu, 1 Feb 2018 20:34:51 +0100 Subject: [Bro] A little more confusion with Intel In-Reply-To: References: <60a34e5669db124ce69305bc1232ab3c@localhost> <309A727C-26D8-4D26-975D-8FDD5158A82E@corelight.com> <3ccd76709d5eafabe321885b5acec101@localhost> <3A565BA7-E885-420B-BB4A-3689FFE9150F@corelight.com> <974524ba-b30b-ec48-0f7e-d408835494e2@gmail.com> Message-ID: On 01/02/18 19:09, Seth Hall wrote: >> Friendly reminder: https://github.com/J-Gras/intel-seen-more ;) > > Hahaha!? Sorry, I forgot about that already! > > I had a thought about it too, what do you think about changing > Intel::EFFECTIVE_DOMAIN to Intel::EFFECTIVE_TLD??? Seems like it makes > sense since the TLD is what you end up matching with this and it fits > James' use case correctly. To be honest, I just wrote the POC down without giving a thought to naming. Quick recap: In the example of "www.yahoo.com", the TLD is ".com". For "www.yahoo.co.uk" the TLD is ".uk". Now you introduced the package to obtain the 'effective TLD', which would be ".co.uk" - technically a 2nd-level domain. But actually, we want to match against "yahoo.co.uk". Maybe one could call that the 'effective SLD/2LD'. So in case of changing, I would tend to use Intel::EFFECTIVE_SLD. However, to me this seems a bit counter intuitive. Jan From liburdi.joshua at gmail.com Thu Feb 1 16:13:33 2018 From: liburdi.joshua at gmail.com (Josh Liburdi) Date: Thu, 1 Feb 2018 16:13:33 -0800 Subject: [Bro] Inconsistent file size during extraction Message-ID: Hi all, I'm seeing instances where files are being extracted inconsistently with what is reported in files.log. Here is a redacted example: files.log: #fields ts fuid tx_hosts rx_hosts conn_uids source depth analyzers mime_type filename duration local_orig is_orig *seen_bytes* *total_bytes* missing_bytes overflow_bytes timedout parent_fuid md5 sha1 sha256 extracted extracted_cutoff extracted_size #types time string set[addr] set[addr] set[string] string count set[string] string string interval bool bool count count count count bool string string string string string bool count 1517528771.042220 Fz2Z2m3zwQcc3gqDS3 x.x.x.x x.x.x.x CpaGD227W0Cy2BA1Tf HTTP 0 EXTRACT application/vnd.openxmlformats-officedocument.spreadsheetml.sheet 0.258350 - F *219414* *12977556* 0 0 F - - - - extract-1517528771.04222-HTTP-Fz2Z2m3zwQcc3gqDS3 F - File on disk: *219414* Feb 1 16:04 extract-1517528771.04222-HTTP-Fz2Z2m3zwQcc3gqDS3 The file on disk is the same size as the amount of bytes sent to the file analyzer (seen_bytes field) -- it should be the same size as the total_bytes field. I've seen this happen many times (though, relatively speaking, it is rare). Any thoughts on this behavior? I'm seeing this on Bro 2.5.1. Thanks, Josh -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180201/59866a8a/attachment.html From liburdi.joshua at gmail.com Thu Feb 1 19:07:23 2018 From: liburdi.joshua at gmail.com (Josh Liburdi) Date: Thu, 1 Feb 2018 19:07:23 -0800 Subject: [Bro] Inconsistent file size during extraction In-Reply-To: References: Message-ID: Seems that this particular connection may be affected by tapping issues. On Thu, Feb 1, 2018 at 4:13 PM, Josh Liburdi wrote: > Hi all, > > I'm seeing instances where files are being extracted inconsistently with > what is reported in files.log. Here is a redacted example: > > files.log: > #fields ts fuid tx_hosts rx_hosts conn_uids source depth analyzers > mime_type filename duration local_orig is_orig *seen_bytes* *total_bytes* > missing_bytes overflow_bytes timedout parent_fuid md5 sha1 sha256 > extracted extracted_cutoff extracted_size > #types time string set[addr] set[addr] set[string] string count > set[string] string string interval bool bool count count count count bool > string string string string string bool count > 1517528771.042220 Fz2Z2m3zwQcc3gqDS3 x.x.x.x x.x.x.x CpaGD227W0Cy2BA1Tf > HTTP 0 EXTRACT application/vnd.openxmlformats-officedocument. > spreadsheetml.sheet 0.258350 - F *219414* *12977556* 0 0 F - - - - > extract-1517528771.04222-HTTP-Fz2Z2m3zwQcc3gqDS3 F - > > File on disk: > *219414* Feb 1 16:04 extract-1517528771.04222-HTTP-Fz2Z2m3zwQcc3gqDS3 > > The file on disk is the same size as the amount of bytes sent to the file > analyzer (seen_bytes field) -- it should be the same size as the > total_bytes field. I've seen this happen many times (though, relatively > speaking, it is rare). > > Any thoughts on this behavior? I'm seeing this on Bro 2.5.1. > > Thanks, > Josh > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180201/1557e971/attachment.html From seth at corelight.com Thu Feb 1 19:49:28 2018 From: seth at corelight.com (Seth Hall) Date: Thu, 01 Feb 2018 22:49:28 -0500 Subject: [Bro] Inconsistent file size during extraction In-Reply-To: References: Message-ID: <33407C74-F909-44F6-A188-7A4A8162C328@corelight.com> Yep, I was going to comment that that's probably the issue, but I'll give some more details on why things may end up that way. "total_bytes" - is for when the size of the file is known by some secondary mechanism, like the file size being transmitted as part of a protocol or a file being read off disk. "seen_bytes" - represents the number of actual bytes of data that were passed into the file analysis framework. This is another case where small packet loss issues can have outsized effects because the following bytes can't be reassembled into the file correctly and you don't get anymore data. Also, nice to see on the mailing list again Josh! .Seth On 1 Feb 2018, at 22:07, Josh Liburdi wrote: > Seems that this particular connection may be affected by tapping > issues. > > On Thu, Feb 1, 2018 at 4:13 PM, Josh Liburdi > > wrote: > >> Hi all, >> >> I'm seeing instances where files are being extracted inconsistently >> with >> what is reported in files.log. Here is a redacted example: >> >> files.log: >> #fields ts fuid tx_hosts rx_hosts conn_uids source depth analyzers >> mime_type filename duration local_orig is_orig *seen_bytes* >> *total_bytes* >> missing_bytes overflow_bytes timedout parent_fuid md5 sha1 sha256 >> extracted extracted_cutoff extracted_size >> #types time string set[addr] set[addr] set[string] string count >> set[string] string string interval bool bool count count count count >> bool >> string string string string string bool count >> 1517528771.042220 Fz2Z2m3zwQcc3gqDS3 x.x.x.x x.x.x.x >> CpaGD227W0Cy2BA1Tf >> HTTP 0 EXTRACT application/vnd.openxmlformats-officedocument. >> spreadsheetml.sheet 0.258350 - F *219414* *12977556* 0 0 F - - - - >> extract-1517528771.04222-HTTP-Fz2Z2m3zwQcc3gqDS3 F - >> >> File on disk: >> *219414* Feb 1 16:04 >> extract-1517528771.04222-HTTP-Fz2Z2m3zwQcc3gqDS3 >> >> The file on disk is the same size as the amount of bytes sent to the >> file >> analyzer (seen_bytes field) -- it should be the same size as the >> total_bytes field. I've seen this happen many times (though, >> relatively >> speaking, it is rare). >> >> Any thoughts on this behavior? I'm seeing this on Bro 2.5.1. >> >> Thanks, >> Josh >> > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Seth Hall * Corelight, Inc * www.corelight.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180201/98b496a3/attachment-0001.html From seth at corelight.com Thu Feb 1 19:52:57 2018 From: seth at corelight.com (Seth Hall) Date: Thu, 01 Feb 2018 22:52:57 -0500 Subject: [Bro] A little more confusion with Intel In-Reply-To: References: <60a34e5669db124ce69305bc1232ab3c@localhost> <309A727C-26D8-4D26-975D-8FDD5158A82E@corelight.com> <3ccd76709d5eafabe321885b5acec101@localhost> <3A565BA7-E885-420B-BB4A-3689FFE9150F@corelight.com> <974524ba-b30b-ec48-0f7e-d408835494e2@gmail.com> Message-ID: <4B1211FD-60B7-4E8D-85CF-8DDCE34E4F0E@corelight.com> On 1 Feb 2018, at 14:34, Jan Grash?fer wrote: > But actually, we want to match against "yahoo.co.uk". Maybe one could > call that the 'effective SLD/2LD'. So in case of changing, I would > tend to use Intel::EFFECTIVE_SLD. However, to me this seems a bit > counter intuitive. Hah! You're exactly right and apparently I didn't think deeply enough when I was writing my email too. I think EFFECTIVE_DOMAIN is better and it matches the function being called in the domain-tld package. It's been quite a while since I looked at that. Thanks, .Seth -- Seth Hall * Corelight, Inc * www.corelight.com From makarchuk at group-ib.com Fri Feb 2 05:12:16 2018 From: makarchuk at group-ib.com (Timur Makarchuk) Date: Fri, 02 Feb 2018 13:12:16 +0000 Subject: [Bro] Fwd: Certificate extraction issue In-Reply-To: References: Message-ID: ---------- Forwarded message --------- From: Timur Makarchuk Date: ??, 2 ????. 2018 ?. ? 16:09 Subject: Certificate extraction issue To: Hello, everybody I have a trouble I can't wrap my head around. I'm trying to extract SSL certificates from traffic and I have and event handler like this: ``` event x509_certificate (f: fa_file, cert_ref: opaque of x509, cert: X509::Certificate) { local fileName = fmt("%s", current_time()); print fileName; local fname = fmt("%s%s.%s", path, fileName, "pem"); local args: Files::AnalyzerArgs = record($extract_filename=fname); Files::add_analyzer(f, Files::ANALYZER_EXTRACT, args); } ``` For some reason I don't understand Bro can't add Analyzer to my files and I'm not getting any files extracted ``` 1517409279.894576 warning in /opt/bro/share/bro/base/frameworks/files/./main.bro, line 394: Analyzer Files::ANALYZER_EXTRACT not added successfully to file Fp4AgEzEtME36Nfl2. ``` Any help will be much appreciated Thanks, Timur -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180202/e93ba5ab/attachment.html From johanna at icir.org Fri Feb 2 09:21:07 2018 From: johanna at icir.org (Johanna Amann) Date: Fri, 2 Feb 2018 09:21:07 -0800 Subject: [Bro] Fwd: Certificate extraction issue In-Reply-To: References: Message-ID: <20180202172107.vwwrs6c7li7ct3if@user237.sys.ICSI.Berkeley.EDU> Hi Timur, > Hello, everybody > > > I have a trouble I can't wrap my head around. > I'm trying to extract SSL certificates from traffic and I have and event > handler like this: > > ``` > event x509_certificate (f: fa_file, cert_ref: opaque of x509, cert: > X509::Certificate) { > local fileName = fmt("%s", current_time()); > print fileName; > local fname = fmt("%s%s.%s", path, fileName, "pem"); > local args: Files::AnalyzerArgs = record($extract_filename=fname); > Files::add_analyzer(f, Files::ANALYZER_EXTRACT, args); > } > ``` > > For some reason I don't understand Bro can't add Analyzer to my files and > I'm not getting any files extracted > ``` > 1517409279.894576 warning in > /opt/bro/share/bro/base/frameworks/files/./main.bro, line 394: Analyzer > Files::ANALYZER_EXTRACT not added successfully to file Fp4AgEzEtME36Nfl2. > ``` Files::add_analyzer can only be called quite early - when all of the bytes of the file are still buffered in the core; I think the last time this is possible is the file_sniff event. That being said, with certificates you have a few other options for saving them to disk later. I would recommend looking at the policy/protocols/ssl/extract-certs-pem.bro script that ships with Bro. Johanna From vern at corelight.com Fri Feb 2 10:57:09 2018 From: vern at corelight.com (Vern Paxson) Date: Fri, 02 Feb 2018 10:57:09 -0800 Subject: [Bro] A little more confusion with Intel In-Reply-To: <4B1211FD-60B7-4E8D-85CF-8DDCE34E4F0E@corelight.com> (Thu, 01 Feb 2018 22:52:57 EST). Message-ID: <20180202185709.442C62C439E@rock.ICSI.Berkeley.EDU> > I think EFFECTIVE_DOMAIN is better and > it matches the function being called in the domain-tld package. FWIW, sometimes the term "registered domain" is used for this. Vern From jlay at slave-tothe-box.net Fri Feb 2 14:08:35 2018 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 02 Feb 2018 15:08:35 -0700 Subject: [Bro] A little more confusion with Intel In-Reply-To: <20180202185709.442C62C439E@rock.ICSI.Berkeley.EDU> References: <20180202185709.442C62C439E@rock.ICSI.Berkeley.EDU> Message-ID: <339f20e8d1c404efc8313ebe806e21dd@localhost> Thanks Gents, Looks like the intel-seen-more may fix the problem..I'll test and report my findings. James On 2018-02-02 11:57, Vern Paxson wrote: >> I think EFFECTIVE_DOMAIN is better and >> it matches the function being called in the domain-tld package. > > FWIW, sometimes the term "registered domain" is used for this. > > Vern > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From seth at corelight.com Sat Feb 3 19:53:06 2018 From: seth at corelight.com (Seth Hall) Date: Sat, 03 Feb 2018 22:53:06 -0500 Subject: [Bro] A little more confusion with Intel In-Reply-To: <20180202185709.442C62C439E@rock.ICSI.Berkeley.EDU> References: <20180202185709.442C62C439E@rock.ICSI.Berkeley.EDU> Message-ID: <03658312-6E48-4C47-AB24-FA2793EB3EB7@corelight.com> On 2 Feb 2018, at 13:57, Vern Paxson wrote: >> I think EFFECTIVE_DOMAIN is better and >> it matches the function being called in the domain-tld package. > > FWIW, sometimes the term "registered domain" is used for this. Ah, I hadn't heard of that before but that does seem like a much more obvious name. The "effective tld" thing was derived from it's usage at Mozilla for some Firefox stuff and I repurposed it for effective domain, but I do like registered domain better. .Seth -- Seth Hall * Corelight, Inc * www.corelight.com From liburdi.joshua at gmail.com Sun Feb 4 09:56:13 2018 From: liburdi.joshua at gmail.com (Josh Liburdi) Date: Sun, 4 Feb 2018 09:56:13 -0800 Subject: [Bro] Inconsistent file size during extraction In-Reply-To: <33407C74-F909-44F6-A188-7A4A8162C328@corelight.com> References: <33407C74-F909-44F6-A188-7A4A8162C328@corelight.com> Message-ID: Yup, that clears up some things I forgot. And thanks, happy to be active again! On Thu, Feb 1, 2018 at 7:49 PM, Seth Hall wrote: > Yep, I was going to comment that that's probably the issue, but I'll give > some more details on why things may end up that way. > > "total_bytes" - is for when the size of the file is known by some > secondary mechanism, like the file size being transmitted as part of a > protocol or a file being read off disk. > "seen_bytes" - represents the number of actual bytes of data that were > passed into the file analysis framework. > > This is another case where small packet loss issues can have outsized > effects because the following bytes can't be reassembled into the file > correctly and you don't get anymore data. > > Also, nice to see on the mailing list again Josh! > > .Seth > > On 1 Feb 2018, at 22:07, Josh Liburdi wrote: > > Seems that this particular connection may be affected by tapping issues. > > On Thu, Feb 1, 2018 at 4:13 PM, Josh Liburdi > wrote: > >> Hi all, >> >> I'm seeing instances where files are being extracted inconsistently with >> what is reported in files.log. Here is a redacted example: >> >> files.log: >> #fields ts fuid tx_hosts rx_hosts conn_uids source depth analyzers >> mime_type filename duration local_orig is_orig *seen_bytes* *total_bytes* >> missing_bytes overflow_bytes timedout parent_fuid md5 sha1 sha256 >> extracted extracted_cutoff extracted_size >> #types time string set[addr] set[addr] set[string] string count >> set[string] string string interval bool bool count count count count bool >> string string string string string bool count >> 1517528771.042220 Fz2Z2m3zwQcc3gqDS3 x.x.x.x x.x.x.x CpaGD227W0Cy2BA1Tf >> HTTP 0 EXTRACT application/vnd.openxmlformats >> -officedocument.spreadsheetml.sheet 0.258350 - F *219414* *12977556* 0 0 >> F - - - - extract-1517528771.04222-HTTP-Fz2Z2m3zwQcc3gqDS3 F - >> >> File on disk: >> *219414* Feb 1 16:04 extract-1517528771.04222-HTTP-Fz2Z2m3zwQcc3gqDS3 >> >> The file on disk is the same size as the amount of bytes sent to the file >> analyzer (seen_bytes field) -- it should be the same size as the >> total_bytes field. I've seen this happen many times (though, relatively >> speaking, it is rare). >> >> Any thoughts on this behavior? I'm seeing this on Bro 2.5.1. >> >> Thanks, >> Josh >> > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -- > Seth Hall * Corelight, Inc * www.corelight.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180204/2a94f087/attachment.html From ambros.novak.89 at gmail.com Mon Feb 5 14:03:42 2018 From: ambros.novak.89 at gmail.com (Ambros Novak) Date: Mon, 5 Feb 2018 17:03:42 -0500 Subject: [Bro] using YARA signatures within Bro Message-ID: Hello, I'm currently using YARA rules (yararules.yar) to inspect files from bro (extract-all-files.bro). Besides using bro to inspect files with YARA, how can I get bro to use YARA rules to inspect traffic and also certificates? Thank you for your help. I'm still learning bro and YARA. -Am -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180205/c3ada24d/attachment.html From rahulbroids at gmail.com Mon Feb 5 20:59:58 2018 From: rahulbroids at gmail.com (rahul rakesh) Date: Tue, 6 Feb 2018 10:29:58 +0530 Subject: [Bro] Help Message-ID: Dear Team, i am a noob to working with broids need some help with the signature framework i have created a .sig file as shown in the document- signature my-first-sig { ip-proto == tcp dst-port == 80 payload /.*root/ event "Found root!" } and loading this signature using /base/init-bare.bro using the @load-sig directive also included /frameworks/signature/main.bro in local.bro script then running bro using broctl and command deploy after that sending any packet matching that signature is not creating any signature.log or notice.log Please guide me Regards, Rahul -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180206/d3754ea6/attachment.html From jlamps at sandia.gov Tue Feb 6 07:55:24 2018 From: jlamps at sandia.gov (Lamps, Jereme) Date: Tue, 6 Feb 2018 15:55:24 +0000 Subject: [Bro] [EXTERNAL] Re: Bro-2.5.2 and PF_RING 6.7 not load balancing properly In-Reply-To: <3A7060C0-DB4B-47FD-A39E-C2C3EF49970C@illinois.edu> References: <3A7060C0-DB4B-47FD-A39E-C2C3EF49970C@illinois.edu> Message-ID: Justin, Your solution seems to have fixed it. Thanks! Jereme On 1/31/18, 11:27 AM, "Azoff, Justin S" wrote: > On Jan 30, 2018, at 3:07 PM, Lamps, Jereme wrote: > > It appears PF_RING is not properly load balancing between Bro instances. For example, I have a single Bro node with 5 bro procs. Every entry in http.log is duplicated 5 times (exact timestamp and all fields are identical except the UID). My conclusion is pf_ring is not splitting the traffic and that all procs are seeing all the traffic. You may be running into an issue that was recently fixed in broctl and will be resolved in the next release. Depending on the order you install things in, pf_ring load balancing can end up disabled. What does the following output for your host? [root at bro-dev ~]# broctl config | grep pfring pfringclusterid = 21 pfringclustertype = 4-tuple ringfirstappinstance = 0 if you have pfringclusterid set to 0, that's the problem that was just fixed. You can easily workaround that by adding PFRINGClusterID = 21 to your /usr/local/bro/etc/broctl.cfg Once that is there, a broctl deploy should get everything working. ? Justin Azoff From fatema.bannatwala at gmail.com Tue Feb 6 13:49:49 2018 From: fatema.bannatwala at gmail.com (fatema bannatwala) Date: Tue, 6 Feb 2018 16:49:49 -0500 Subject: [Bro] Help Message-ID: Did you include your signature file in local.bro file, with *@load-sigs *? Thanks, Fatema. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180206/b1abe5cf/attachment.html From christian at corelight.com Wed Feb 7 14:54:15 2018 From: christian at corelight.com (Christian Kreibich) Date: Wed, 7 Feb 2018 14:54:15 -0800 Subject: [Bro] using YARA signatures within Bro In-Reply-To: References: Message-ID: Hey Am, On 02/05/2018 02:03 PM, Ambros Novak wrote: > Hello, > > I'm currently using YARA rules (yararules.yar) to inspect files from bro > (extract-all-files.bro). > > Besides using bro to inspect files with YARA, how can I get bro to use YARA > rules to inspect traffic and also certificates? Bro doesn't currently integrate YARA, but there's at least this plugin that pulls YARA file analysis more directly into Bro: https://github.com/hempnall/broyara We're considering expanding Bro's YARA support for file analysis and potentially beyond, though much of that will need work on the YARA side to make it operate in a more streaming-oriented fashion. We'd definitely like to hear of Bro use cases for YARA that you guys can think of. Best, -C. From andrew.ratcliffe at nswcsystems.co.uk Thu Feb 8 02:33:04 2018 From: andrew.ratcliffe at nswcsystems.co.uk (Andrew Ratcliffe) Date: Thu, 8 Feb 2018 10:33:04 +0000 Subject: [Bro] X.509 extensions can be used for covert channel data transfer and C2 Message-ID: <19F3F8ED-4396-453F-98F2-1667D439C27B@nswcsystems.co.uk> Hi Everyone, Has anyone looked at this research https://www.fidelissecurity.com/threatgeek/2018/02/exposing-x509-vulnerabilities with a view to creating a Bro detection? Looks as simple as checking a value in the TLS extension to see if it falls on an expected length to be a hash value. Kind regards, Andy Andrew.Ratcliffe at NSWCSystems.co.uk CISSP, CSTA, CSTP, CWSA GIAC: GCIA, GCIH, GPEN, GWAPT, GCFE, GREM, GPYC, GNFA Computer Forensic & Security Specialist Blog.InfoSecMatters.net -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180208/6455e009/attachment.html From johanna at icir.org Thu Feb 8 07:12:49 2018 From: johanna at icir.org (Johanna Amann) Date: Thu, 8 Feb 2018 07:12:49 -0800 Subject: [Bro] X.509 extensions can be used for covert channel data transfer and C2 In-Reply-To: <19F3F8ED-4396-453F-98F2-1667D439C27B@nswcsystems.co.uk> References: <19F3F8ED-4396-453F-98F2-1667D439C27B@nswcsystems.co.uk> Message-ID: <20180208151244.23gq5djhcnmwr2o4@Beezling.local> Hi Andy, > Has anyone looked at this research > https://www.fidelissecurity.com/threatgeek/2018/02/exposing-x509-vulnerabilities > with a view to creating a Bro detection? Doing what they recommend in Bro is not a problem at all; Bro raises an event for all X.509 extensions (https://www.bro.org/sphinx/scripts/base/bif/plugins/Bro_X509.events.bif.bro.html#id-x509_extension) and you can just check the length there. Bro also can perform validation of all certificates, which is the second remedy that they proposed. However, please note that just thinking about this for 30 seconds, I can think of at least 3 other ways to hide data in TLS handshakes that this would not catch (and that they did not talk about), some of them easier to implement for an attacker than this. Plus - if you can establish a TLS connection without there being a DLP device in the middle you could always just send the data after encryption kicks in. Johanna From pierre at droids-corp.org Thu Feb 8 11:06:21 2018 From: pierre at droids-corp.org (Pierre LALET) Date: Thu, 8 Feb 2018 20:06:21 +0100 Subject: [Bro] Logging TCP server banners Message-ID: <20180208190621.kj7typgywbcsrouz@droids-corp.org> Hi everyone, For a network recon framework I am working on, I would like to log each "TCP server banner" Bro sees. I call "TCP server banner" the first chunk of data a server sends, before the client has sent data (if the client sends data before the server, I don't want to log anything). Here is what I have done so far (`PassiveRecon` is my module's name): ``` export { redef tcp_content_deliver_all_resp = T; [...] } [...] event tcp_contents(c: connection, is_orig: bool, seq: count, contents: string) { if (! is_orig && seq == 1 && c$orig$num_pkts == 2) { Log::write(PassiveRecon::LOG, [$ts=c$start_time, $host=c$id$resp_h, $srvport=c$id$resp_p, $recon_type=TCP_SERVER_BANNER, $value=contents]); } } ``` Basically, I consider we have a "TCP server banner" when `is_orig` is false, when `seq` equals 1 and when we have seen exactly two packets from the client (which should be a SYN and the first ACK). This solution generally works **but** I sometimes log a data chunk when I should not, particularly if I have missed part of the traffic. As an example, the following Scapy script creates a PCAP file that would trick my script into logging a "TCP server banner" while the client has actually sent some data (and we have missed an ACK packet, left as a comment in the script): ``` wrpcap("test.cap", [ Ether() / IP(dst="1.2.3.4", src="5.6.7.8") / TCP(dport=80, sport=5678, flags="S", ack=0, seq=555678), Ether() / IP(src="1.2.3.4", dst="5.6.7.8") / TCP(sport=80, dport=5678, flags="SA", seq=111234, ack=555679), # Ether() / IP(dst="1.2.3.4", src="5.6.7.8") / # TCP(dport=80, sport=5678, flags="A", ack=111235, seq=555679), Ether() / IP(dst="1.2.3.4", src="5.6.7.8") / TCP(dport=80, sport=5678, flags="PA", ack=111235, seq=555679) / "DATA", Ether() / IP(src="1.2.3.4", dst="5.6.7.8") / TCP(sport=80, dport=5678, flags="PA", seq=111235, ack=555683) / "DATA" ]) ``` Is there a way to know that I have not missed any packet from the client and/or a way to know that the client has not sent any data on the connection (like an equivalent of the `seq` parameter, but for the `ack`)? Also, when `seq` equals 1, am I certain that I have not missed any packet from the server? One more question: is there a better, cleaner, etc. way to do what I'm trying to do? Thanks a lot, Pierre -- Pierre http://pierre.droids-corp.org/ From lagoon7 at gmail.com Thu Feb 8 11:30:12 2018 From: lagoon7 at gmail.com (Ludwig Goon) Date: Thu, 8 Feb 2018 14:30:12 -0500 Subject: [Bro] Any information on BROCON 2018 In-Reply-To: References: Message-ID: Can anyone in the bro community comment or provide information on BRO con for 2018? Trying to plan my 2018 training -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180208/5971754c/attachment.html From slagell at illinois.edu Thu Feb 8 11:35:45 2018 From: slagell at illinois.edu (Slagell, Adam J) Date: Thu, 8 Feb 2018 19:35:45 +0000 Subject: [Bro] Any information on BROCON 2018 In-Reply-To: References: Message-ID: <7A532118-0254-42B2-A538-84ED35F2F5B6@illinois.edu> No. We are trying to secure a location in September still. On Feb 8, 2018, at 1:30 PM, Ludwig Goon > wrote: Can anyone in the bro community comment or provide information on BRO con for 2018? Trying to plan my 2018 training _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro ------ Adam J. Slagell Director, Cybersecurity & Networking Division Chief Information Security Officer National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180208/839019a3/attachment-0001.html From perry29 at llnl.gov Fri Feb 9 10:30:42 2018 From: perry29 at llnl.gov (Perry, David) Date: Fri, 9 Feb 2018 18:30:42 +0000 Subject: [Bro] schema question(s) Message-ID: I am new to Bro and this list. I have a project that involves storing Bro http data in Solr (or Elasticsearch). I think I have the Bro end of things pretty well covered (or will), as I have expert help, but the other end not so much. I am currently looking at Solr but I am not yet committed to it (over Elasticsearch). I am hoping someone has a Solr schema for the Bro http logs that they are willing to share. My plan is to generate and ingest http logs in JSON format. David Perry From schaiba at gmail.com Mon Feb 12 05:58:03 2018 From: schaiba at gmail.com (Rares Aioanei) Date: Mon, 12 Feb 2018 15:58:03 +0200 Subject: [Bro] Reading pcap files from Python wrapper? Message-ID: Hello, I know that I can easily run bro from the CLI with a .pcap file and then analyze the logs it generates. However, what I need is to use the Bro API (preferrably Python) to _open_ the pcap file and generate the logs. Is this possible? Thanks a lot in advance. -- Rares Aioanei From roberixion at gmail.com Mon Feb 12 08:04:55 2018 From: roberixion at gmail.com (=?UTF-8?Q?Rober_Fern=C3=A1ndez?=) Date: Mon, 12 Feb 2018 17:04:55 +0100 Subject: [Bro] Write logs Message-ID: I want to write several logs in the same connexion. The problem with logging framework is that only can filter a log; For example, If i had three filters with log_metadata= T, i would like to write in three differents logs filter-1.log filter-3.log filter-10.log function filter_conn(id: Log::ID, path: string, rec: Conn::Info) : string { local filters = Filter::filters; local file_filter = ""; for (i in filters) { if (i?$log_metadata && i$log_metadata == T) { file_filter = "filter-" + cat(i$id_fil); break; } } return file_filter; } event bro_init() { local metadata: Log::Filter = [ $name="metadata", $path="metadata", $include=set("ts","id.orig_h", "id.orig_p", "id.resp_h", "id.resp_p", "proto", "service", "duration", "orig_bytes", "resp_bytes", "missed_bytes", "orig_pkts", "resp_pkts", "orig_file", "resp_file"), $path_func=filter_conn ]; Log::add_filter(Conn::LOG, metadata); } With this code, I only return the first match, filter-1.log How could I do this? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180212/fcd0f7c8/attachment.html From jazoff at illinois.edu Mon Feb 12 08:11:35 2018 From: jazoff at illinois.edu (Azoff, Justin S) Date: Mon, 12 Feb 2018 16:11:35 +0000 Subject: [Bro] Write logs In-Reply-To: References: Message-ID: <99E74F0C-1F64-413D-BE4F-052C708B427F@illinois.edu> > On Feb 12, 2018, at 11:04 AM, Rober Fern?ndez wrote: > > I want to write several logs in the same connexion. The problem with logging framework is that only can filter a log; The problem is that path_func is not for filtering, it is for splitting a single log stream into one or more different output files. If you want to filter a log file, you need to use $pred, not $path_func. You simply need to call Log::add_filter 3 times, once for each pred function. ? Justin Azoff > For example, If i had three filters with log_metadata= T, i would like to write in three differents logs > > filter-1.log > filter-3.log > filter-10.log > > > > function filter_conn(id: Log::ID, path: string, rec: Conn::Info) : string { > > local filters = Filter::filters; > local file_filter = ""; > > for (i in filters) { > > if (i?$log_metadata && i$log_metadata == T) { > > file_filter = "filter-" + cat(i$id_fil); > break; > } > } > > return file_filter; > } > > > event bro_init() { > > local metadata: Log::Filter = [ > $name="metadata", > $path="metadata", > $include=set("ts","id.orig_h", "id.orig_p", "id.resp_h", "id.resp_p", "proto", "service", > "duration", "orig_bytes", "resp_bytes", "missed_bytes", "orig_pkts", "resp_pkts", "orig_file", > "resp_file"), > $path_func=filter_conn > ]; > > Log::add_filter(Conn::LOG, metadata); > > } > > With this code, I only return the first match, filter-1.log > > How could I do this? > > Thanks > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From dopheide at gmail.com Mon Feb 12 08:37:05 2018 From: dopheide at gmail.com (Mike Dopheide) Date: Mon, 12 Feb 2018 10:37:05 -0600 Subject: [Bro] Reading pcap files from Python wrapper? In-Reply-To: References: Message-ID: I'm not a core developer, but I can pretty much guarantee the API doesn't support that. Reading pcaps is generally regarded as a testing mechanism, not part of a production architecture. For those folks where reading pcaps is a hard requirement, they usually end up building something around tcpreplay. -Dop On Mon, Feb 12, 2018 at 7:58 AM, Rares Aioanei wrote: > Hello, > > I know that I can easily run bro from the CLI with a .pcap file and > then analyze the logs it generates. However, what I need is to use the > Bro API (preferrably Python) to _open_ the pcap file and generate the > logs. Is this possible? > > Thanks a lot in advance. > > -- > Rares Aioanei > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180212/2768d592/attachment.html From roberixion at gmail.com Mon Feb 12 09:35:08 2018 From: roberixion at gmail.com (=?UTF-8?Q?Rober_Fern=C3=A1ndez?=) Date: Mon, 12 Feb 2018 18:35:08 +0100 Subject: [Bro] Change directory Message-ID: Hi, If I execute bro in command line $ bro -i interface script.bro All logs are written in the currently directory. It's possible change the directory if I execute in command line? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180212/b6287e9a/attachment.html From seth at corelight.com Mon Feb 12 14:05:45 2018 From: seth at corelight.com (Seth Hall) Date: Mon, 12 Feb 2018 17:05:45 -0500 Subject: [Bro] Change directory In-Reply-To: References: Message-ID: <07100A18-DDC9-44E1-8B5E-51ACBD03DE5B@corelight.com> On 12 Feb 2018, at 12:35, Rober Fern?ndez wrote: > If I execute bro in command line > $ bro -i interface script.bro > All logs are written in the currently directory. It's possible change > the > directory if I execute in command line? Funny enough, that has never been a feature in Bro because it's not necessary with how people typically deploy Bro. I would recommend changing to the directory where you want your logs to land before starting Bro. .Seth -- Seth Hall * Corelight, Inc * www.corelight.com From jlay at slave-tothe-box.net Wed Feb 14 05:03:57 2018 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 14 Feb 2018 06:03:57 -0700 Subject: [Bro] Detecting remote powershell Message-ID: <1518613437.2390.1.camel@slave-tothe-box.net> Hey All, Topic really...has anyone put some work/sigs into detecting remote powershell? ?Figured I'd start here first...thank you. James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180214/f51371af/attachment.html From johanna at icir.org Wed Feb 14 09:46:37 2018 From: johanna at icir.org (Johanna Amann) Date: Wed, 14 Feb 2018 09:46:37 -0800 Subject: [Bro] Bro 2.5.3 release (security update) Message-ID: <20180214174637.5qlcscvwezjhixt3@Beezling.local> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 We announce the release of Bro v2.5.3. The new version is now available for download at: https://bro.org/download/index.html or directly at: https://www.bro.org/downloads/bro-2.5.3.tar.gz Binary packages for the new version are currently building and will be available in the next hours at: https://bro.org/download/packages.html This is a security release that fixes an integer overflow in code generated by binpac. This issue can be used by remote attackers to crash Bro (i.e. a DoS attack). There also is a possibility this can be exploited in other ways. This bug was found by Philippe Antoine of Catena cyber. A CVE will be assigned to this bug. Bro 2.5.3 does not contain any other changes. We urge everyone to update their installation as quickly as possible. Johanna -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJahHMjAAoJECOZ8Wl8E8ZdgC0QAKLtWFynqWO7GyHitGCCnw60 AiBPBYZMcLeO7QRwkba2JvuFwYDWZGKkkiUdVWIfaVGCiYw0ZJ9WueHz6kVSU6zW OMQrtunO74iizdIgWqbvM0MnKMosc6im7wISDXW/q3DwcP4UfCajwiKiQqciK+x0 i3kTAm5jjqhD5BIHAMr05zHetF/gBOqRd1+2+xFqeLuUkxK9TlqMnhjORNMlSCRB d56fV00vZMIQNgpsMiDA9ICWBz8fsbyCkme1tbver6AytM1IvhAcXr89Wsfe9z4T VhrsUdf1klnCaiOmMUg2xGkJLxaosfUiQCyCs+G2JvH7DDPuf2CDFDK4nQohpppN T3PYcQa0w6T6YXnfz+lil/INN4g0l7PscWSaexv9fof8gwgljn1LWhYJ+rsEOzwa sM5fYdUHfRUg9n9F3lsTi7Qo34nh5HK3NXyYpwB4GH/yoCDglRKyGOVsrjc7FPeg NUjRchFHgfMCpcD9OGXcn0a/jNXiEtuRRsR4hec1IU7fVe40Y6CUyK/ka4QCA+E4 xzgJUOaVNT7NaTILoMfM+fjiVXFImm2e1kJXQizVzrkQUesUTY4eebDQXcFVIrpW ZoTZ3TtG5LqJy4/8g0mq5h7Bz48GdtvQce7XmMhYHt3Yp0AjIB9tSKfhxXxO/kaP TNhR36N1vDiFCo1z1QIi =2I6P -----END PGP SIGNATURE----- From christian at corelight.com Wed Feb 14 14:20:08 2018 From: christian at corelight.com (Christian Kreibich) Date: Wed, 14 Feb 2018 14:20:08 -0800 Subject: [Bro] Logging TCP server banners In-Reply-To: <20180208190621.kj7typgywbcsrouz@droids-corp.org> References: <20180208190621.kj7typgywbcsrouz@droids-corp.org> Message-ID: <1c1f2fc7-38f3-5579-26bb-24f7f3e117df@corelight.com> Hi Pierre, On 02/08/2018 11:06 AM, Pierre LALET wrote: > event tcp_contents(c: connection, is_orig: bool, seq: count, contents: string) > { > if (! is_orig && seq == 1 && c$orig$num_pkts == 2) > { > Log::write(PassiveRecon::LOG, [$ts=c$start_time, > $host=c$id$resp_h, > $srvport=c$id$resp_p, > $recon_type=TCP_SERVER_BANNER, > $value=contents]); > } > } I'd recommend a different approach, namely to keep additional state for each connection to indicate whether you've seen contents from the respective endpoints. That way, when you see contents arrive from the responder you can check directly whether you've previously seen anything from the originator, and if so, ignore the responder's content. > wrpcap("test.cap", [ > Ether() / IP(dst="1.2.3.4", src="5.6.7.8") / > TCP(dport=80, sport=5678, flags="S", ack=0, seq=555678), > Ether() / IP(src="1.2.3.4", dst="5.6.7.8") / > TCP(sport=80, dport=5678, flags="SA", seq=111234, ack=555679), > # Ether() / IP(dst="1.2.3.4", src="5.6.7.8") / > # TCP(dport=80, sport=5678, flags="A", ack=111235, seq=555679), > Ether() / IP(dst="1.2.3.4", src="5.6.7.8") / > TCP(dport=80, sport=5678, flags="PA", ack=111235, seq=555679) / "DATA", > Ether() / IP(src="1.2.3.4", dst="5.6.7.8") / > TCP(sport=80, dport=5678, flags="PA", seq=111235, ack=555683) / "DATA" > ]) (With the commented-out line this is missing the pure TCP ACK packet that completes the TCP handshake, so may not be a good test case.) Best, Christian From ambros.novak.89 at gmail.com Wed Feb 14 15:23:51 2018 From: ambros.novak.89 at gmail.com (Ambros Novak) Date: Wed, 14 Feb 2018 18:23:51 -0500 Subject: [Bro] Knowing when a worker crashes Message-ID: <838C9A80-49A0-4847-B5D4-F7DD759D62E2@gmail.com> Hello, What is the easiest way to monitor if a worker crashes? And if a worker crashes, is there a way to automatically bring it back up? Ambros Sent from my iPhone From asharma at lbl.gov Wed Feb 14 16:22:07 2018 From: asharma at lbl.gov (Aashish Sharma) Date: Wed, 14 Feb 2018 16:22:07 -0800 Subject: [Bro] Knowing when a worker crashes In-Reply-To: <838C9A80-49A0-4847-B5D4-F7DD759D62E2@gmail.com> References: <838C9A80-49A0-4847-B5D4-F7DD759D62E2@gmail.com> Message-ID: <20180215002205.GO32630@MacPro-2331.local> > What is the easiest way to monitor if a worker crashes? > And if a worker crashes, is there a way to automatically bring it back up? 1) broctl cron helps. Running it every N (5?) mins will check if any worker has crashed and will restart those: ### broctl cron: process and disk maintenance */5 * * * * /usr/local/bin/randsleep 59 && broctl cron > What is the easiest way to monitor if a worker crashes? Additional checks (nagios plugins) that help: 2) Bro process counts : each bro worker is two bro process + 1 run-bro process - so a nagios monitor (or a simple bro process count helps too). 3) conn log line counts : A while ago, I experienced a issue where bro process count checks out but bro won't process the packets on the interfaces. So there is another check which counts how many conn logs each worker is generating and if there is a discripency (or a worker missing), it generates an alert. for this you'd have to load conn-peer.bro : https://gist.github.com/JustinAzoff/446d0abba2c6dd8ff242 Hope this helps, Aashish On Wed, Feb 14, 2018 at 06:23:51PM -0500, Ambros Novak wrote: > Hello, > > What is the easiest way to monitor if a worker crashes? > And if a worker crashes, is there a way to automatically bring it back up? > > Ambros > > Sent from my iPhone > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From koshybibin3 at gmail.com Thu Feb 15 05:18:33 2018 From: koshybibin3 at gmail.com (Bibin Koshy) Date: Thu, 15 Feb 2018 13:18:33 +0000 Subject: [Bro] Bro Signatures Message-ID: Hi, I am trying to compare Snort and Bro IDS on the basis of signatures/rules.Is there any repository for Bro rules/signatures? I haven't got any signatures examples online. It would be a great help if you could suggest some signatures to find basic attacks. Thank you Bibin Koshy -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180215/a72cdceb/attachment.html From zeolla at gmail.com Thu Feb 15 07:09:15 2018 From: zeolla at gmail.com (Zeolla@GMail.com) Date: Thu, 15 Feb 2018 15:09:15 +0000 Subject: [Bro] Bro Signatures In-Reply-To: References: Message-ID: Bro doesn't really work that way, so it would be hard to make that comparison. https://www.bro.org/sphinx/frameworks/signatures.html#so-how-about-using-snort-signatures-with-bro Bro does have the concept of signatures, it's just used in a way that is very different than Snort would. It may make sense to read more of https://www.bro.org/sphinx/frameworks/signatures.html There is also this - https://github.com/corelight/bro-protosigs - for using signatures in bro to do simple detection of some protocols, but it definitely isn't meant to work in the way Snort signatures would. Jon On Thu, Feb 15, 2018 at 8:36 AM Bibin Koshy wrote: > Hi, > > I am trying to compare Snort and Bro IDS on the basis of > signatures/rules.Is there any repository for Bro rules/signatures? I > haven't got any signatures examples online. It would be a great help if you > could suggest some signatures to find basic attacks. > > Thank you > Bibin Koshy > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180215/34d4b6c0/attachment.html From zeolla at gmail.com Thu Feb 15 07:11:16 2018 From: zeolla at gmail.com (Zeolla@GMail.com) Date: Thu, 15 Feb 2018 15:11:16 +0000 Subject: [Bro] Bro Signatures In-Reply-To: References: Message-ID: You may also want to check out what bro ships with here - https://github.com/bro/bro/tree/master/scripts And what is available as bro packages (a new-ish platform for sharing bro 'things') - https://github.com/bro/packages Jon On Thu, Feb 15, 2018 at 10:08 AM Zeolla at GMail.com wrote: > Bro doesn't really work that way, so it would be hard to make that > comparison. > https://www.bro.org/sphinx/frameworks/signatures.html#so-how-about-using-snort-signatures-with-bro > > Bro does have the concept of signatures, it's just used in a way that is > very different than Snort would. It may make sense to read more of > https://www.bro.org/sphinx/frameworks/signatures.html > > There is also this - https://github.com/corelight/bro-protosigs - for > using signatures in bro to do simple detection of some protocols, but it > definitely isn't meant to work in the way Snort signatures would. > > Jon > > On Thu, Feb 15, 2018 at 8:36 AM Bibin Koshy wrote: > >> Hi, >> >> I am trying to compare Snort and Bro IDS on the basis of >> signatures/rules.Is there any repository for Bro rules/signatures? I >> haven't got any signatures examples online. It would be a great help if you >> could suggest some signatures to find basic attacks. >> >> Thank you >> Bibin Koshy >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -- > > Jon > -- Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180215/c1543fe6/attachment-0001.html From ambros.novak.89 at gmail.com Fri Feb 16 03:08:10 2018 From: ambros.novak.89 at gmail.com (Ambros Novak) Date: Fri, 16 Feb 2018 06:08:10 -0500 Subject: [Bro] Extract only certain files types Message-ID: Hello What should the extract-all-files.bro look like in order to only extract pdf, exe, doc and docx? My attempts are extracting so many files. Dankon, Ambros ?? From mfernandez at mitre.org Fri Feb 16 03:47:57 2018 From: mfernandez at mitre.org (Fernandez, Mark I) Date: Fri, 16 Feb 2018 11:47:57 +0000 Subject: [Bro] Extract only certain files types In-Reply-To: References: Message-ID: Ambros, >> What should the extract-all-files.bro look like in order to >> only extract pdf, exe, doc and docx? The fa_metadata record contains the MIME type. Using the MIME type, you can make a condition on whether or not to extract the file. Mark From koshybibin3 at gmail.com Fri Feb 16 09:52:20 2018 From: koshybibin3 at gmail.com (Bibin Koshy) Date: Fri, 16 Feb 2018 17:52:20 +0000 Subject: [Bro] Bro Signatures In-Reply-To: References: Message-ID: Hi, I tried running bro using the signature file i created. It outputs conn.log, notice.log, weird.log and other files. I just didnt get the signature.log file which is what i am looking for. I am sure i have a fair amount of signatures defined at least to find one alert/alarm on the pcap file i am using. Is there anything that can be done. Like should there be any scripts i need to @load to my local.bro file to output the signature.log file? Thank you On 15 February 2018 at 15:11, Zeolla at GMail.com wrote: > You may also want to check out what bro ships with here - > https://github.com/bro/bro/tree/master/scripts > > And what is available as bro packages (a new-ish platform for sharing bro > 'things') - https://github.com/bro/packages > > Jon > > On Thu, Feb 15, 2018 at 10:08 AM Zeolla at GMail.com > wrote: > >> Bro doesn't really work that way, so it would be hard to make that >> comparison. https://www.bro.org/sphinx/frameworks/ >> signatures.html#so-how-about-using-snort-signatures-with-bro >> >> Bro does have the concept of signatures, it's just used in a way that is >> very different than Snort would. It may make sense to read more of >> https://www.bro.org/sphinx/frameworks/signatures.html >> >> There is also this - https://github.com/corelight/bro-protosigs - for >> using signatures in bro to do simple detection of some protocols, but it >> definitely isn't meant to work in the way Snort signatures would. >> >> Jon >> >> On Thu, Feb 15, 2018 at 8:36 AM Bibin Koshy >> wrote: >> >>> Hi, >>> >>> I am trying to compare Snort and Bro IDS on the basis of >>> signatures/rules.Is there any repository for Bro rules/signatures? I >>> haven't got any signatures examples online. It would be a great help if you >>> could suggest some signatures to find basic attacks. >>> >>> Thank you >>> Bibin Koshy >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> -- >> >> Jon >> > -- > > Jon > -- *Bibin* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180216/4e2376de/attachment.html From jdickenson at gmail.com Fri Feb 16 10:32:23 2018 From: jdickenson at gmail.com (James Dickenson) Date: Fri, 16 Feb 2018 10:32:23 -0800 Subject: [Bro] Detecting remote powershell In-Reply-To: <1518613437.2390.1.camel@slave-tothe-box.net> References: <1518613437.2390.1.camel@slave-tothe-box.net> Message-ID: I don't believe I've seen any work in this regard for Bro, it would be great if someone invested the time to build something. I do know that there is the Attack Detection team that have been submitting a lot of powershell,empire,etc based rules to the ET ruleset for Snort/Suricata. -James D. On Wed, Feb 14, 2018 at 5:03 AM, James Lay wrote: > Hey All, > > Topic really...has anyone put some work/sigs into detecting remote > powershell? Figured I'd start here first...thank you. > > James > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180216/0918c1f0/attachment.html From ambros.novak.89 at gmail.com Mon Feb 19 02:09:45 2018 From: ambros.novak.89 at gmail.com (Ambros Novak) Date: Mon, 19 Feb 2018 10:09:45 +0000 Subject: [Bro] Bro training/ brocon Message-ID: <4B980F04-0083-4334-8F8C-6EDEAB8FD26E@gmail.com> Hello, Is there a bro training I could attended? When is brocon this year and how much training is offered? Ambros ?? From dwdixon at umich.edu Tue Feb 20 09:46:39 2018 From: dwdixon at umich.edu (Drew Dixon) Date: Tue, 20 Feb 2018 12:46:39 -0500 Subject: [Bro] Bro Traffic analysis flow shunting questions Message-ID: Hello, I have some questions regarding implementing shunting of traffic that has no analysis value and what the recommended approach is for doing so with a large bro cluster that will be analyzing very high volumes of traffic. I am aware of the Bro net-control framework and am wondering if this is where I should start and what should be leveraged to begin implement shunting with the latest version of bro? - In addition to "ordinary" elephant flows, I'm also interested in shunting out large video streaming services (netflix, hulu, prime video, hbogo, vimeo, etc.) which do not have analysis value, first I'm wondering would any of this get picked up in the bulk transfer/flow detection already built into the existing conn-bulk.bro script detection? - Is the use of dumbno with bro-react/conn-bulk.bro for detection and shunting of bulk transfers/elephant flows relevant still considering bro-netcontrol exists? Are these viewed as plugins/backends for bro-netcontrol? I believe bro-netcontrol also has some shunting (built in?) is any of dumbno/conn-bulk.bro built into bro-netcontrol already, or is further configuration required to setup dumbno/conn-bulk.bro with net-control? - If the answer to first bullet point the above is no, is anyone already shunting this type of video streaming traffic and is there a bro script out there anyone could share so we aren't reinventing the wheel? Assuming nothing exists, would it be plausible to leverage the ssl.log domain name metadata to do the detection piece for this, then pulling out the IP/Port info to insert shunting ACL's via the packet broker API? Would extending dumbno/bro-react/conn-bulk.bro to support this (assuming it doesn't catch this traffic already) be something desirable? Also, would this be a good approach for detecting large video streaming traffic flows, or would there be a better approach? Wondering if someone is already doing this and would be willing to share or link to something on GitHub perhaps? Many thanks, -Drew -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180220/ac7a9fb1/attachment.html From seth at corelight.com Tue Feb 20 13:58:03 2018 From: seth at corelight.com (Seth Hall) Date: Tue, 20 Feb 2018 16:58:03 -0500 Subject: [Bro] Bro Traffic analysis flow shunting questions In-Reply-To: References: Message-ID: <63136E75-C3BF-43C1-A996-93834C89AD98@corelight.com> On 20 Feb 2018, at 12:46, Drew Dixon wrote: > - In addition to "ordinary" elephant flows, I'm also interested in > shunting > out large video streaming services (netflix, hulu, prime video, hbogo, > vimeo, etc.) which do not have analysis value, first I'm wondering > would > any of this get picked up in the bulk transfer/flow detection already > built > into the existing conn-bulk.bro script detection? You may just want to do some traffic filtering by address space if you're looking to cut out those sort of known high volume hosts. > - Is the use of dumbno with > bro-react/conn-bulk.bro for detection and shunting of bulk > transfers/elephant flows relevant still considering bro-netcontrol > exists? I wrote a script a while ago for detecting "bursty connections": https://github.com/corelight/conn-burst This script has very low overhead on Bro and has been running at a few sites for a while and appears to be doing a good job of detecting and logging bursting connections. You can run the script to get a conn_burst.log, the script doesn't change any traffic monitoring policy as it is now so it should be safe to load in any environment. It should be installed through the Bro package manager too. > Are these viewed as plugins/backends for bro-netcontrol? I believe > bro-netcontrol also has some shunting (built in?) is any of > dumbno/conn-bulk.bro built into bro-netcontrol already, or is further > configuration required to setup dumbno/conn-bulk.bro with net-control? I wouldn't view these sort of scripts as plugins to netcontrol. They are really scripts that *use* netcontrol. Plugins to netcontrol are only to integrate with the network equipment for implementing whatever control change you want to take effect on the network. We do need to add some more scripts to netcontrol for things like connection shunting. I think there is one for flow shunting right now, but not for connection shunting. > Assuming nothing exists, would it be plausible to leverage the ssl.log > domain name > metadata to do the detection piece for this, then pulling out the > IP/Port > info to insert shunting ACL's via the packet broker API? The better option might be to shunt connections after the "ssl_established" event has fired. That way you get the benefit across all ssl/tls traffic. I would still look more toward static packet filters if you want to cut out the majority of traffic to these sort of known high volume sites though. I think we're still suffering from a bit of an adoption lag with netcontrol so there is a lack of people to provide first hand experiences right now. This should be getting rectified soon as I have some work small integration tasks to take care of with netcontrol which should clear up some of this. .Seth -- Seth Hall * Corelight, Inc * www.corelight.com From jazoff at illinois.edu Tue Feb 20 15:53:14 2018 From: jazoff at illinois.edu (Azoff, Justin S) Date: Tue, 20 Feb 2018 23:53:14 +0000 Subject: [Bro] Bro Traffic analysis flow shunting questions In-Reply-To: References: Message-ID: <6A1D62D2-AFD5-4E80-96AD-D9F40BB2B288@illinois.edu> > On Feb 20, 2018, at 12:46 PM, Drew Dixon wrote: > > Hello, > > I have some questions regarding implementing shunting of traffic that has no analysis value and what the recommended approach is for doing so with a large bro cluster that will be analyzing very high volumes of traffic. > > I am aware of the Bro net-control framework and am wondering if this is where I should start and what should be leveraged to begin implement shunting with the latest version of bro? > > - In addition to "ordinary" elephant flows, I'm also interested in shunting out large video streaming services (netflix, hulu, prime video, hbogo, vimeo, etc.) which do not have analysis value, first I'm wondering would any of this get picked up in the bulk transfer/flow detection already built into the existing conn-bulk.bro script detection? No, since those are not really bulk flows. According to netflix a stream is "up to 3 GB per hour for HD and 7 GB per hour for Ultra HD" which works out to 7-15mbit. Dumbno is really intended to filter flows that are 1000mbit+. The kind of flows that would completely overwhelm the NICs in your sensors. Video streaming is more of a "death by a thousand paper cuts" kind of thing. > - Is the use of dumbno with bro-react/conn-bulk.bro for detection and shunting of bulk transfers/elephant flows relevant still considering bro-netcontrol exists? Are these viewed as plugins/backends for bro-netcontrol? I believe bro-netcontrol also has some shunting (built in?) is any of dumbno/conn-bulk.bro built into bro-netcontrol already, or is further configuration required to setup dumbno/conn-bulk.bro with net-control? bro-react predates netcontrol and hasn't been updated to work with the netcontrol framework. Well, updated is really the wrong word, since there's not really anything to update.. The whole thing is one function and one hook, shorter than this email. If I get a chance I should be able to build a bro package that implements a native dumbno client in C++ and hook it into the netcontrol api. You'd still need something like conn-bulk to flag the elephant flows. > - If the answer to first bullet point the above is no, is anyone already shunting this type of video streaming traffic and is there a bro script out there anyone could share so we aren't reinventing the wheel? Assuming nothing exists, would it be plausible to leverage the ssl.log domain name metadata to do the detection piece for this, then pulling out the IP/Port info to insert shunting ACL's via the packet broker API? Would extending dumbno/bro-react/conn-bulk.bro to support this (assuming it doesn't catch this traffic already) be something desirable? Also, would this be a good approach for detecting large video streaming traffic flows, or would there be a better approach? The best way to detect video services would be using ssl events and match on the SNI hostname. The problem you'll run into is that depending on which Arista you have, you can only have a certain number of ACL entries. Shunting every single netflix video flow individually will likely max out the TCAM on the switch. And because of things like https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP I don't think netflix streams you a 6GB video file in one request, I think the client pulls down multiple smaller chunks across multiple connections. It would likely be a much better idea to just identity what the upstream netflix CDN hosts are for your network[1] and filter them manually. You could possibly use sumstats to identify them automatically, but that would filter things individually and you could end up filtering every address in a /24 using 255 rules which wouldn't be very efficient. There's also the skip_further_processing function that tells bro to not do anything with packets that are part of a particular connection, which can also help if you are low on resources. [1]: $ grep nflxvideo.net ssl.log |cut -f 5|sort|uniq -c|sort -nr|head 8332 216.171.0.106 189 198.38.109.241 162 198.38.108.206 161 198.38.109.202 130 198.38.109.200 113 198.38.109.220 102 198.38.108.208 98 198.38.108.205 93 198.38.108.162 $ grep nflxvideo.net ssl.log |cut -f 5|cut -d . -f 1-3|sort|uniq -c|sort -nr 8388 216.171.0 2346 198.38.108 2107 198.38.109 44 ... So, 1 host and 2 /24s account for most of the netflix cdn, 3 acls would catch almost all of it. Filtering individual flows, dumbno would have tried to shunt 10,000+ connections. If things were tweaked to ignore the ports and just filter on src+dst, that's still hundreds of individual acls. ? Justin Azoff From ambros.novak.89 at gmail.com Tue Feb 20 22:32:54 2018 From: ambros.novak.89 at gmail.com (Ambros Novak) Date: Wed, 21 Feb 2018 06:32:54 +0000 Subject: [Bro] Extract files not authentic copy of file Message-ID: Hello, The configuration is extracting certain file types but the files that are extracted are not authentic replications of the files in the stream. The hashes do no match the real files at the user?s endpoint. Upon inspecting the extracted files there seems to be mismatched and duplicated streams. How can this be corrected? I would like the extracted files to be exactly what the user would download. Thank you kindly for your help. Ambros ?? From krasinski at cines.fr Thu Feb 22 01:30:18 2018 From: krasinski at cines.fr (Nicolas KRASINSKI) Date: Thu, 22 Feb 2018 10:30:18 +0100 (CET) Subject: [Bro] Arp script : Bro doesn't log all traffic In-Reply-To: References: Message-ID: <375304840.1527687.1519291818735.JavaMail.zimbra@cines.fr> Hello, When I load arp_main script ( https://gist.github.com/grigorescu/a28b814a8fb626e2a7b4715d278198aa ) in local.bro, Bro log only arp traffic and not more. I just have this logs : stdout stderr stats notice arp When I don't load this arp script, bro log normaly all traffic... Do you know why ? Thanks in advance Nicolas. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180222/74862c3d/attachment.html From jazoff at illinois.edu Thu Feb 22 05:44:21 2018 From: jazoff at illinois.edu (Azoff, Justin S) Date: Thu, 22 Feb 2018 13:44:21 +0000 Subject: [Bro] Arp script : Bro doesn't log all traffic In-Reply-To: <375304840.1527687.1519291818735.JavaMail.zimbra@cines.fr> References: <375304840.1527687.1519291818735.JavaMail.zimbra@cines.fr> Message-ID: > On Feb 22, 2018, at 4:30 AM, Nicolas KRASINSKI wrote: > > Hello, > > When I load arp_main script (https://gist.github.com/grigorescu/a28b814a8fb626e2a7b4715d278198aa) in local.bro, Bro log only arp traffic and not more. > I just have this logs : > stdout > stderr > stats > notice > arp > > When I don't load this arp script, bro log normaly all traffic... > Do you know why ? Removing this line should fix things: redef capture_filters += { ["arp"] = "arp" }; ? Justin Azoff From vladg at illinois.edu Thu Feb 22 07:00:34 2018 From: vladg at illinois.edu (Vlad Grigorescu) Date: Thu, 22 Feb 2018 09:00:34 -0600 Subject: [Bro] Arp script : Bro doesn't log all traffic In-Reply-To: <201802221346.w1MDkg8L007200@vladg.net> References: <375304840.1527687.1519291818735.JavaMail.zimbra@cines.fr> <201802221346.w1MDkg8L007200@vladg.net> Message-ID: Thanks, Justin. I updated the gist (which is just hosting a copy of the script found in the mailing list) to remove that line. It's been on my todo list to turn that into a Bro package. --Vlad "Azoff, Justin S" writes: >> On Feb 22, 2018, at 4:30 AM, Nicolas KRASINSKI wrote: >> >> Hello, >> >> When I load arp_main script (https://gist.github.com/grigorescu/a28b814a8fb626e2a7b4715d278198aa) in local.bro, Bro log only arp traffic and not more. >> I just have this logs : >> stdout >> stderr >> stats >> notice >> arp >> >> When I don't load this arp script, bro log normaly all traffic... >> Do you know why ? > > > Removing this line should fix things: > > redef capture_filters += { ["arp"] = "arp" }; > > > ? > Justin Azoff > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 861 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180222/fed0daa9/attachment.bin From krasinski at cines.fr Thu Feb 22 07:30:09 2018 From: krasinski at cines.fr (Nicolas KRASINSKI) Date: Thu, 22 Feb 2018 16:30:09 +0100 (CET) Subject: [Bro] Arp script : Bro doesn't log all traffic In-Reply-To: References: <375304840.1527687.1519291818735.JavaMail.zimbra@cines.fr> Message-ID: <679714043.1602336.1519313409553.JavaMail.zimbra@cines.fr> Great! Thank you very much, it works. Nicolas. De: "Azoff, Justin S" ?: "krasinski" Cc: bro at bro.org Envoy?: Jeudi 22 F?vrier 2018 14:44:21 Objet: Re: [Bro] Arp script : Bro doesn't log all traffic > On Feb 22, 2018, at 4:30 AM, Nicolas KRASINSKI wrote: > > Hello, > > When I load arp_main script (https://gist.github.com/grigorescu/a28b814a8fb626e2a7b4715d278198aa) in local.bro, Bro log only arp traffic and not more. > I just have this logs : > stdout > stderr > stats > notice > arp > > When I don't load this arp script, bro log normaly all traffic... > Do you know why ? Removing this line should fix things: redef capture_filters += { ["arp"] = "arp" }; ? Justin Azoff -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180222/de451a07/attachment.html From seth at corelight.com Thu Feb 22 10:07:29 2018 From: seth at corelight.com (Seth Hall) Date: Thu, 22 Feb 2018 13:07:29 -0500 Subject: [Bro] Extract files not authentic copy of file In-Reply-To: References: Message-ID: Are you having any trouble with dropped packets? If you are dropping a lot of packets, it's possible for your extracted files to be problematic. .Seth On 21 Feb 2018, at 1:32, Ambros Novak wrote: > Hello, > > The configuration is extracting certain file types but the files that > are extracted are not authentic replications of the files in the > stream. The hashes do no match the real files at the user?s > endpoint. Upon inspecting the extracted files there seems to be > mismatched and duplicated streams. > > How can this be corrected? I would like the extracted files to be > exactly what the user would download. > > Thank you kindly for your help. > > Ambros > > ?? > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Seth Hall * Corelight, Inc * www.corelight.com From vern at corelight.com Thu Feb 22 17:14:20 2018 From: vern at corelight.com (Vern Paxson) Date: Thu, 22 Feb 2018 17:14:20 -0800 Subject: [Bro] Extract files not authentic copy of file In-Reply-To: (Thu, 22 Feb 2018 13:07:29 EST). Message-ID: <20180223011420.3BD8B2C4030@rock.ICSI.Berkeley.EDU> > Are you having any trouble with dropped packets? If you are dropping a > lot of packets, it's possible for your extracted files to be > problematic. Along with that, another possibility is that the host does some transformation before storing the file. What types of files are these? Vern From ambros.novak.89 at gmail.com Thu Feb 22 18:32:31 2018 From: ambros.novak.89 at gmail.com (Ambros Novak) Date: Thu, 22 Feb 2018 21:32:31 -0500 Subject: [Bro] Extract files not authentic copy of file In-Reply-To: <20180223011420.3BD8B2C4030@rock.ICSI.Berkeley.EDU> References: <20180223011420.3BD8B2C4030@rock.ICSI.Berkeley.EDU> Message-ID: <5F698E39-EA05-4CB8-968F-D00915BA41A1@gmail.com> Thank you Seth and Vern. Im unsure any packets are being dropped. How would I check if packets are being dropped? Would dropped packets also have duplicated streams? I?m seeing the same text repeated anywhere from 2-4 times in extracted files. I?m looking at PDF, EXE, and various MS Office files. ?? On Feb 22, 2018, at 8:14 PM, Vern Paxson wrote: >> Are you having any trouble with dropped packets? If you are dropping a >> lot of packets, it's possible for your extracted files to be >> problematic. > > Along with that, another possibility is that the host does some transformation > before storing the file. What types of files are these? > > Vern From seth at corelight.com Fri Feb 23 16:38:38 2018 From: seth at corelight.com (Seth Hall) Date: Fri, 23 Feb 2018 19:38:38 -0500 Subject: [Bro] Extract files not authentic copy of file In-Reply-To: <5F698E39-EA05-4CB8-968F-D00915BA41A1@gmail.com> References: <20180223011420.3BD8B2C4030@rock.ICSI.Berkeley.EDU> <5F698E39-EA05-4CB8-968F-D00915BA41A1@gmail.com> Message-ID: <1E511203-67C9-432D-A44F-DFAE35389D65@corelight.com> On 22 Feb 2018, at 21:32, Ambros Novak wrote: > Im unsure any packets are being dropped. How would I check if packets > are being dropped? One heuristic you can use is the capture_loss.log. It will give an estimated percentage of dropped packets based on TCP analysis. > Would dropped packets also have duplicated streams? I?m seeing the > same text repeated anywhere from 2-4 times in extracted files. That seems unlikely to me. The way that the file extraction analyzer and the files framework works should prevent this sort of behavior. .Seth -- Seth Hall * Corelight, Inc * www.corelight.com From bill.de.ping at gmail.com Mon Feb 26 01:10:54 2018 From: bill.de.ping at gmail.com (william de ping) Date: Mon, 26 Feb 2018 11:10:54 +0200 Subject: [Bro] - try-bro package Message-ID: Hi there, I was just wondering how could I have a try-bro site on an internal network ? How can I install https://github.com/bro/try-bro on a local pc ? thank you very much B -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180226/67b771b1/attachment.html From jazoff at illinois.edu Mon Feb 26 06:07:44 2018 From: jazoff at illinois.edu (Azoff, Justin S) Date: Mon, 26 Feb 2018 14:07:44 +0000 Subject: [Bro] - try-bro package In-Reply-To: References: Message-ID: Add https://github.com/bro/try-bro/blob/master/trybro.service to /etc/systemd/system and start it, that's it. ? Justin Azoff > On Feb 26, 2018, at 4:10 AM, william de ping wrote: > > Hi there, > > I was just wondering how could I have a try-bro site on an internal network ? > How can I install https://github.com/bro/try-bro on a local pc ? > > thank you very much > B > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From brolist at vt.edu Tue Feb 27 11:49:04 2018 From: brolist at vt.edu (brolist at vt.edu) Date: Tue, 27 Feb 2018 14:49:04 -0500 Subject: [Bro] Finding Golden Tickets in Kerberos Logs Message-ID: Hey all, Does anyone have a reliable method to find Active Directory Golden or Silver Tickets in the Bro Kerberos logs? I was planning to look into doing this (maybe based partially on expiration) but wanted to ask the list first. I appreciate any advice. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180227/cc57bc60/attachment.html From koshybibin3 at gmail.com Wed Feb 28 05:14:01 2018 From: koshybibin3 at gmail.com (Bibin Koshy) Date: Wed, 28 Feb 2018 13:14:01 +0000 Subject: [Bro] Bro IDS Message-ID: Hi there, When you run Bro. Are all the protocols such as http, ftp etc enabled on local.bro as default? Because i am not sure if i need to add all the protocols manually or if they are already running as default? Thank you, *Bibin* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180228/684509cb/attachment.html From zeolla at gmail.com Wed Feb 28 07:19:08 2018 From: zeolla at gmail.com (Zeolla@GMail.com) Date: Wed, 28 Feb 2018 15:19:08 +0000 Subject: [Bro] Bro IDS In-Reply-To: References: Message-ID: Not all protocols are enabled by default, but http and ftp should be. Check out: *https://github.com/bro/bro/blob/master/scripts/base/init-default.bro#L46-L74 * Jon On Wed, Feb 28, 2018 at 8:14 AM Bibin Koshy wrote: > Hi there, > > When you run Bro. Are all the protocols such as http, ftp etc enabled on > local.bro as default? Because i am not sure if i need to add all the > protocols manually or if they are already running as default? > > Thank you, > *Bibin* > -- Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180228/dd6b5546/attachment.html From dopheide at gmail.com Wed Feb 28 12:27:44 2018 From: dopheide at gmail.com (Mike Dopheide) Date: Wed, 28 Feb 2018 14:27:44 -0600 Subject: [Bro] Bro updates force some bro-pkg rebuilds.. sometimes. Message-ID: I'm sure others have experienced this, when updating Bro, packages with C++ components will need to be re-installed/rebuilt (sometimes, not always). There may be some crazy C++ way of avoiding that, but outside of that I've been trying to come up with options to suggest so our upgrade process can be more easily automated. What do folks think about a couple options to bro-pkg such as: 1) a new package.meta file field: rebuild=yes 2) And a corresponding bro-pkg command: bro-pkg rebuild [nodeps] Separately, but kinda related, I wouldn't mind a '--yes' flag or something similar when packages aren't being installed interactively. Alternatively, the idea has been tossed around that we build the bro packages we use into RPMs, but then that sorta defeats the bro-pkg command entirely. -Dop -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180228/8a1bdb79/attachment.html From zeolla at gmail.com Wed Feb 28 12:38:36 2018 From: zeolla at gmail.com (Zeolla@GMail.com) Date: Wed, 28 Feb 2018 20:38:36 +0000 Subject: [Bro] Bro updates force some bro-pkg rebuilds.. sometimes. In-Reply-To: References: Message-ID: Sounds appealing to me, I would use it if it existed. Jon On Wed, Feb 28, 2018 at 3:36 PM Mike Dopheide wrote: > I'm sure others have experienced this, when updating Bro, packages with > C++ components will need to be re-installed/rebuilt (sometimes, not > always). There may be some crazy C++ way of avoiding that, but outside of > that I've been trying to come up with options to suggest so our upgrade > process can be more easily automated. > > What do folks think about a couple options to bro-pkg such as: > 1) a new package.meta file field: > rebuild=yes > 2) And a corresponding bro-pkg command: bro-pkg rebuild [nodeps] > > Separately, but kinda related, I wouldn't mind a '--yes' flag or something > similar when packages aren't being installed interactively. > > Alternatively, the idea has been tossed around that we build the bro > packages we use into RPMs, but then that sorta defeats the bro-pkg command > entirely. > > -Dop > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180228/d7a2e885/attachment.html From mkhan at mitre.org Wed Feb 28 14:21:13 2018 From: mkhan at mitre.org (Khan, Murad A.) Date: Wed, 28 Feb 2018 22:21:13 +0000 Subject: [Bro] Bro HTTP/2 Decoder/Analyzer Plugin Released by MITRE Message-ID: <251D52ED-C043-427C-9D6C-1D9ACC26B142@mitre.org> All, MITRE has created a plugin for Bro that adds an analyzer for the HTTP/2 protocol (RFC 7540) and released it open source at https://github.com/MITRECND/bro-http2 A little background on HTTP/2 ? after using HTTP 1.x for the longest time companies wanted to come up with a successor protocol that took into consideration the changes that had occurred with the web since the creation of the HTTP 1.x specifications (HTTP 1.1, RFC 2616 came out in 1999!). This led to the creation of SPDY, realizing that SPDY was useful, the IETF took its concepts and formalized a new protocol and called it ?HTTP/2?. HTTP/2 introduces a number of changes and improvements on top of HTTP 1.x including providing native multiplexed communication channels. HTTP/2 also completely changes the transport mechanism, now being a binary protocol (for those not intimately familiar with HTTP 1.x, it is a text-oriented protocol). The analyzer has two dependencies ? libnghttp2, available via apt (Ubuntu) and yum (CentOS EPEL) and brotli (not available via repos, only via github at https://github.com/google/brotli). It also currently doesn?t support the Bro Package Manager (this is on the todo list). After installing the plugin, it needs to be loaded, which can be done by putting ?@load http2? into your bro policy/script file. This analyzer will create an http2.log file where http2 transactions will go to. For more information, reference the README on github. There are a couple of very important caveats with using this analyzer. First, you will likely not see any HTTP2 traffic in the clear, pretty much ever, since the major browsers, afaik, have decided on only using HTTP/2 with TLS ALPN (RFC 7301). So, this means, to use this analyzer you will need to have some SSL/TLS interception capability in place to decrypt traffic and provide it to bro which will in turn allow this analyzer to analyze the traffic. If someone finds this to be untrue and sees a significant amount of http2 traffic in the clear, I?d like to hear about. Secondly, this analyzer doesn?t have a dpd.sig file so you?ll need to specify, explicitly, which ports to analyze ? by default ports 80 and 443 are configured which should be good enough for most people. Lastly, this analyzer, although stable, still has some things on the to-do list and could probably use some more testing and feedback so if you decide to install/run it and run into an issue please contact us about it. P.S: For those who have ssl logs and are interested in seeing how much HTTP/2 traffic their organization is seeing, take a look at the ?next_protocol? column in ssl.log which indicates the ALPN negotiated protocol. -Murad -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180228/2624d426/attachment-0001.html From jsiwek at corelight.com Wed Feb 28 18:59:29 2018 From: jsiwek at corelight.com (Jon Siwek) Date: Thu, 1 Mar 2018 02:59:29 +0000 Subject: [Bro] Bro updates force some bro-pkg rebuilds.. sometimes. In-Reply-To: References: Message-ID: On Wed, Feb 28, 2018 at 8:27 PM, Mike Dopheide wrote: > What do folks think about a couple options to bro-pkg such as: > 1) a new package.meta file field: > rebuild=yes I wonder if that's needed since bro-pkg can already see what packages have specified a 'build_command' ? i.e. bro-pkg can simply rebuild any package that has supplied a build command. I'd think that even for packages which don't strictly require a rebuild, it's not harmful to rebuild/reinstall anyway. > 2) And a corresponding bro-pkg command: bro-pkg rebuild [nodeps] Makes sense to me. Unless there's further feedback from this thread, I'd say go ahead and create an issue on on github for adding a 'rebuild' command or take a crack at a PR in case I don't get to it immediately. Possibly a workaround to do in the meantime would be to do: $ bro-pkg bundle everything.bundle $ bro-pkg unbundle everything.bundle $ rm everything.bundle Which should result in a rebuild/reinstall of all installed packages. > Separately, but kinda related, I wouldn't mind a '--yes' flag or something > similar when packages aren't being installed interactively. The --force flag will suppress all interactive prompts. Does that work like you need? - Jon From anthony.kasza at gmail.com Wed Feb 28 19:15:31 2018 From: anthony.kasza at gmail.com (anthony kasza) Date: Wed, 28 Feb 2018 20:15:31 -0700 Subject: [Bro] Bro HTTP/2 Decoder/Analyzer Plugin Released by MITRE In-Reply-To: <251D52ED-C043-427C-9D6C-1D9ACC26B142@mitre.org> References: <251D52ED-C043-427C-9D6C-1D9ACC26B142@mitre.org> Message-ID: This is very awesome. Thanks for sharing, Murad! Tangentially related, nghttp2's client and server both are able to communicate over HTTP/2 in the clear and are perfect for generating test traffic. -AK On Feb 28, 2018 15:30, "Khan, Murad A." wrote: All, MITRE has created a plugin for Bro that adds an analyzer for the HTTP/2 protocol (RFC 7540) and released it open source at https://github.com/MITRECND/bro-http2 A little background on HTTP/2 ? after using HTTP 1.x for the longest time companies wanted to come up with a successor protocol that took into consideration the changes that had occurred with the web since the creation of the HTTP 1.x specifications (HTTP 1.1, RFC 2616 came out in 1999!). This led to the creation of SPDY, realizing that SPDY was useful, the IETF took its concepts and formalized a new protocol and called it ?HTTP/2?. HTTP/2 introduces a number of changes and improvements on top of HTTP 1.x including providing native multiplexed communication channels. HTTP/2 also completely changes the transport mechanism, now being a binary protocol (for those not intimately familiar with HTTP 1.x, it is a text-oriented protocol). The analyzer has two dependencies ? libnghttp2, available via apt (Ubuntu) and yum (CentOS EPEL) and brotli (not available via repos, only via github at https://github.com/google/brotli). It also currently doesn?t support the Bro Package Manager (this is on the todo list). After installing the plugin, it needs to be loaded, which can be done by putting ?@load http2? into your bro policy/script file. This analyzer will create an http2.log file where http2 transactions will go to. For more information, reference the README on github. There are a couple of very important caveats with using this analyzer. First, you will likely not see any HTTP2 traffic in the clear, pretty much ever, since the major browsers, afaik, have decided on only using HTTP/2 with TLS ALPN (RFC 7301). So, this means, to use this analyzer you will need to have some SSL/TLS interception capability in place to decrypt traffic and provide it to bro which will in turn allow this analyzer to analyze the traffic. If someone finds this to be untrue and sees a significant amount of http2 traffic in the clear, I?d like to hear about. Secondly, this analyzer doesn?t have a dpd.sig file so you?ll need to specify, explicitly, which ports to analyze ? by default ports 80 and 443 are configured which should be good enough for most people. Lastly, this analyzer, although stable, still has some things on the to-do list and could probably use some more testing and feedback so if you decide to install/run it and run into an issue please contact us about it. P.S: For those who have ssl logs and are interested in seeing how much HTTP/2 traffic their organization is seeing, take a look at the ?next_protocol? column in ssl.log which indicates the ALPN negotiated protocol. -Murad _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180228/f39272ca/attachment.html From dopheide at gmail.com Wed Feb 28 19:25:12 2018 From: dopheide at gmail.com (Mike Dopheide) Date: Wed, 28 Feb 2018 21:25:12 -0600 Subject: [Bro] Bro updates force some bro-pkg rebuilds.. sometimes. In-Reply-To: References: Message-ID: > I wonder if that's needed since bro-pkg can already see what packages > have specified a 'build_command' ? i.e. bro-pkg can simply rebuild any > package that has supplied a build command. I'd think that even for > packages which don't strictly require a rebuild, it's not harmful to > rebuild/reinstall anyway. > Great point, build_command serves the same purpose. > > 2) And a corresponding bro-pkg command: bro-pkg rebuild [nodeps] > > Makes sense to me. Unless there's further feedback from this thread, > I'd say go ahead and create an issue on on github for adding a > 'rebuild' command or take a crack at a PR in case I don't get to it > immediately. > > Possibly a workaround to do in the meantime would be to do: > > $ bro-pkg bundle everything.bundle > $ bro-pkg unbundle everything.bundle > $ rm everything.bundle > > Which should result in a rebuild/reinstall of all installed packages Good to know. I may take a stab at a PR for 'rebuild', but I can't promise anything soon either. Will wait for additional community feedback. > > Separately, but kinda related, I wouldn't mind a '--yes' flag or > something > > similar when packages aren't being installed interactively. > > The --force flag will suppress all interactive prompts. Does that > work like you need ? Not sure I realized --force was an option, but that should work. Thanks, Jon. -Dop -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180228/d0c1025a/attachment.html