From zied.turki at outlook.com Tue Dec 1 04:43:43 2015 From: zied.turki at outlook.com (Zied Turki) Date: Tue, 1 Dec 2015 12:43:43 +0000 Subject: [Bro] SMB connections In-Reply-To: References: , Message-ID: Hello, I have already set this variable to False. I have also tried some others scripts to log the SMB connections. I've got random log outputs : only few SMB connections were logged but not all of them.. Many thanks, BR, Zied > Date: Mon, 30 Nov 2015 11:44:13 -0800 > Subject: Re: [Bro] SMB connections > From: rrotsted at gmail.com > To: zied.turki at outlook.com > CC: bro at bro.org > > Hi Zied, > > By default, the Exfil framework will only attach to flows originated > by addresses in 10.0.0.0/8 that have a non-local responder. > > Try setting "ignore_local_dest_conn" to F in app-exfil-conn.bro. > > --bob > > > On Mon, Nov 30, 2015 at 2:48 AM, Zied Turki wrote: > > Hello Bro Community, > > > > I am working on the data exfiltration and I have just tested the Exfil > > Framework. > > I have noticed, that the script failed to detect file uploads from the file > > server using SMB protocol. Looking to the connections logs (conn.log), the > > SMB connections are unfortunately not logged. > > Would it be a known issue ? or should I tune some params ? > > Please note that the trafic arrives to Bro machine (I have checked using > > tcpdump). > > > > Many thanks, > > > > BR, > > Zied > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151201/c3d39f1b/attachment.html From rrotsted at gmail.com Tue Dec 1 07:47:14 2015 From: rrotsted at gmail.com (Robert Rotsted) Date: Tue, 01 Dec 2015 15:47:14 +0000 Subject: [Bro] SMB connections In-Reply-To: References: Message-ID: How big are the files that you are transferring? What percentage loss are you seeing in you capture_loss log? On Tue, Dec 1, 2015 at 4:43 AM Zied Turki wrote: > Hello, > > I have already set this variable to False. > I have also tried some others scripts to log the SMB connections. I've got > random log outputs : only few SMB connections were logged but not all of > them.. > > Many thanks, > > BR, > Zied > > > Date: Mon, 30 Nov 2015 11:44:13 -0800 > > Subject: Re: [Bro] SMB connections > > From: rrotsted at gmail.com > > To: zied.turki at outlook.com > > CC: bro at bro.org > > > > > Hi Zied, > > > > By default, the Exfil framework will only attach to flows originated > > by addresses in 10.0.0.0/8 that have a non-local responder. > > > > Try setting "ignore_local_dest_conn" to F in app-exfil-conn.bro. > > > > --bob > > > > > > On Mon, Nov 30, 2015 at 2:48 AM, Zied Turki > wrote: > > > Hello Bro Community, > > > > > > I am working on the data exfiltration and I have just tested the Exfil > > > Framework. > > > I have noticed, that the script failed to detect file uploads from the > file > > > server using SMB protocol. Looking to the connections logs (conn.log), > the > > > SMB connections are unfortunately not logged. > > > Would it be a known issue ? or should I tune some params ? > > > Please note that the trafic arrives to Bro machine (I have checked > using > > > tcpdump). > > > > > > Many thanks, > > > > > > BR, > > > Zied > > > > > > _______________________________________________ > > > Bro mailing list > > > bro at bro-ids.org > > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151201/51afa676/attachment-0001.html From zied.turki at outlook.com Tue Dec 1 08:33:12 2015 From: zied.turki at outlook.com (Zied Turki) Date: Tue, 1 Dec 2015 16:33:12 +0000 Subject: [Bro] SMB connections In-Reply-To: References: , , Message-ID: Hi, I have tried with ~10MB and ~100 MB files. Yes, I'm seeing some packet drop in the notice.log. I'll activate the packet_loss module to get the exact percentage. ps : I'm running 4 workers and everything seems to be ok so far : low cpu and memory usage.(the packet loss still exists...) Regards, Zied From: rrotsted at gmail.com Date: Tue, 1 Dec 2015 15:47:14 +0000 Subject: Re: [Bro] SMB connections To: zied.turki at outlook.com CC: bro at bro.org How big are the files that you are transferring? What percentage loss are you seeing in you capture_loss log? On Tue, Dec 1, 2015 at 4:43 AM Zied Turki wrote: Hello, I have already set this variable to False. I have also tried some others scripts to log the SMB connections. I've got random log outputs : only few SMB connections were logged but not all of them.. Many thanks, BR, Zied > Date: Mon, 30 Nov 2015 11:44:13 -0800 > Subject: Re: [Bro] SMB connections > From: rrotsted at gmail.com > To: zied.turki at outlook.com > CC: bro at bro.org > > Hi Zied, > > By default, the Exfil framework will only attach to flows originated > by addresses in 10.0.0.0/8 that have a non-local responder. > > Try setting "ignore_local_dest_conn" to F in app-exfil-conn.bro. > > --bob > > > On Mon, Nov 30, 2015 at 2:48 AM, Zied Turki wrote: > > Hello Bro Community, > > > > I am working on the data exfiltration and I have just tested the Exfil > > Framework. > > I have noticed, that the script failed to detect file uploads from the file > > server using SMB protocol. Looking to the connections logs (conn.log), the > > SMB connections are unfortunately not logged. > > Would it be a known issue ? or should I tune some params ? > > Please note that the trafic arrives to Bro machine (I have checked using > > tcpdump). > > > > Many thanks, > > > > BR, > > Zied > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151201/9fd9557d/attachment.html From liburdi.joshua at gmail.com Tue Dec 1 12:38:28 2015 From: liburdi.joshua at gmail.com (Josh Liburdi) Date: Tue, 1 Dec 2015 15:38:28 -0500 Subject: [Bro] Plugin regex issue Message-ID: Hi all, I have a quick question regarding Bro?s plugin structure. I?m working on a protocol analyzer plugin that works well, except for one thing? I can?t use Binpac's RE primitive type in my protocol.pac file. My suspicion is that regex may need to be explicitly loaded for the plugin. I?ve tested the same code by compiling it into Bro and it works there, just not as a plugin. Is anyone (perhaps the Bro team, Robin?) aware of an issue using RE primitives in plugins? Every other primitive type I?ve tested works. The analyzer is a very early work in progress, but you can find and test the issue if you want: https://github.com/jshlbrd/bro-analyzers/tree/master/stun-protocol-plugin Further details below ? type STUN_UDP_MAGIC_PDU(is_orig: bool) = record { message_type: uint16; message_len: uint16; magic_cookie: RE/\x21\x12\xa4\x42/; trans_id: bytestring &length=12; #attributes: STUN_ATTRIBUTE[] &until($input.length() == 0); } &byteorder=bigendian &length=message_len+20; In the record above, when used as a plugin, the magic_cookie is skipped. When compiled, it works as expected. Thanks! Josh -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151201/5a2f804f/attachment.html From troyj at maine.edu Tue Dec 1 19:29:23 2015 From: troyj at maine.edu (Troy Jordan) Date: Tue, 1 Dec 2015 22:29:23 -0500 Subject: [Bro] Spicy & logging framework Message-ID: <565E6593.4020405@maine.edu> Hello, Is there a special method for invoking the Bro logging framework when writing protocol analyzers in Spicy? In my case, I've disabled the legacy parser (modbus) to not clash with the Spicy parser I'm writing. Where should the code for invoking the logging framework reside in this case? Thanks. - Troy -- Troy Jordan t r o y j @ m a i n e . e d u GIAC GCIH,GCIA ------------------------------------------------------------ Network Systems Security Analyst Information Technology Security Office University of Maine System ------------------------------------------------------------ 233 Science Building | voice: 207.561.3590 Portland, ME 04103 | fax: 509.351.3650 "As you all know, Security Is Mortals chiefest Enemy" William Shakespeare, Macbeth From robin at icir.org Tue Dec 1 20:28:14 2015 From: robin at icir.org (Robin Sommer) Date: Tue, 1 Dec 2015 20:28:14 -0800 Subject: [Bro] Spicy & logging framework In-Reply-To: <565E6593.4020405@maine.edu> References: <565E6593.4020405@maine.edu> Message-ID: <20151202042814.GD54820@icir.org> On Tue, Dec 01, 2015 at 22:29 -0500, Troy Jordan wrote: > Is there a special method for invoking the Bro logging framework when > writing protocol analyzers in Spicy? The model is to keep doing that from Bro script-land, just as with the standard analyzers as well. So you'd trigger the events from Spicy, via the *.evt files, and then write Bro script code to create your log file. If your new Spicy-based Modbus parser generated exactly the same events as the legacy one, you'd automatically get the same log file as well. If not (which I deem more likely :-), you'll have to write new scripts replacing the current ones. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From seth at icir.org Tue Dec 1 21:27:12 2015 From: seth at icir.org (Seth Hall) Date: Wed, 2 Dec 2015 00:27:12 -0500 Subject: [Bro] conditional loading In-Reply-To: References: Message-ID: <285BE09C-ED49-4D4D-9658-1FA9C26C6543@icir.org> You already got the correct advice in this thread, but just to close it out I?ll go into a bit more detail on why it couldn?t work. Since ?@if? is a parser directive, that code is executed while the code is being parsed but the code within a ?when? block is executed asynchronously. There is higher-order problem that I?ll get to next, but conceptually that @if wouldn?t work anymore since it wouldn?t be executed until later when the when statement?s body executes. The high level problem is that it looks like you have a lot of code outside of event handlers. There isn?t much support for code outside of event handlers in Bro since that means the code would only be executed at parse time which is frequently not a useful time to accomplish things. If you want to do something at startup you would work in a bro_init event handler (which still wouldn?t work for what you want to do unfortunately). I would follow Anthony?s advice and do a condition in a script that just changes behavior based on the result of the command that gets executed. On the upside, this give you the flexibility to re-run the command later and have behavior change dynamically. .Seth > On Nov 30, 2015, at 9:43 PM, Dk Jack wrote: > > the directives don't seem to have file test operators. I tried to do the following... > > local c = "test -f filters.bro"; > local cmd = Exec::Command($cmd=c); > > when (local res = Exec::run(cmd)) > { > @if (res$exit_code == 0) > { > @load filters.bro; > } > } > > > However, I am getting parsing errors... > > --------------------------------------------------------------- > referencing a local name in @if (res) > invalid expression in @if (res$exit_code == 0) > --------------------------------------------------------------- > > Dnj > > > On Mon, Nov 30, 2015 at 4:41 PM, anthony kasza wrote: > See here: > https://www.bro.org/sphinx/script-reference/directives.html > > -AK > > On Nov 30, 2015 4:36 PM, "Dk Jack" wrote: > Hi, > Is it possible to perform conditional loading of bro script files ? > In my local.bro file, I'd like to do something like this: > > if file 'filters.bro' exists > { > @load filters.bro > } > > The file filters.bro may or may not exist because it's getting generated from an external program. > Thanks for your help in advance. > > Dnj > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From seth at icir.org Tue Dec 1 21:28:30 2015 From: seth at icir.org (Seth Hall) Date: Wed, 2 Dec 2015 00:28:30 -0500 Subject: [Bro] surgical file extraction In-Reply-To: References: <51BCA40C-1E34-4FF0-876A-F8BBDFFB8374@gmail.com> Message-ID: > On Nov 30, 2015, at 7:14 PM, Brandon Glaze wrote: > > I validated that if I comment out my @load line for the new file extraction script this error goes away in a "broctl check" check. You may be missing a semi-colon somewhere near the end of your script. Bro?s error reporting will accidentally report the problem on the first line of the next script in this case on accident. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From bglaze at gmail.com Wed Dec 2 07:06:42 2015 From: bglaze at gmail.com (Brandon Glaze) Date: Wed, 2 Dec 2015 07:06:42 -0800 Subject: [Bro] surgical file extraction In-Reply-To: References: <51BCA40C-1E34-4FF0-876A-F8BBDFFB8374@gmail.com> Message-ID: Seth, Thanks, but I have pulled up quite a few other examples and dont see where I am missing a semi-colon. I am wondering if its because I am running 32+ worker nodes and the script isnt built correctly to utilize the clustering (shot in the dark). Here is my script as I have it now. I was thinking I may want to move it to use a URL instead of the IP, especially if I start finding CnC comms to load balancer sites: global mime_to_ext: table[string] of string = { ["text/plain"] = "txt", ["text/html"] = "html", }; event file_sniff(f: fa_file, meta: fa_metadata) { for ( cid in f$conns ) { if ( f$conns[cid]$id$resp_h != 123.123.123.123 ) return; if ( f$source != "HTTP" ) return; if ( ! meta?$mime_type ) return; if ( meta$mime_type !in mime_to_ext ) return; local fname = fmt("%s-%s.%s", f$source, f$id, mime_to_ext[meta$mime_type]); print fmt("Extracting file %s", fname); Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]); } ===================== Brandon Glaze bglaze at gmail.com "Lead me, follow me, or get the hell out of my way." - General George Patton Jr On Tue, Dec 1, 2015 at 9:28 PM, Seth Hall wrote: > > > On Nov 30, 2015, at 7:14 PM, Brandon Glaze wrote: > > > > I validated that if I comment out my @load line for the new file > extraction script this error goes away in a "broctl check" check. > > You may be missing a semi-colon somewhere near the end of your script. > Bro?s error reporting will accidentally report the problem on the first > line of the next script in this case on accident. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151202/8970930d/attachment-0001.html From kmcmahon at mitre.org Wed Dec 2 07:29:34 2015 From: kmcmahon at mitre.org (McMahon, Kevin J) Date: Wed, 2 Dec 2015 15:29:34 +0000 Subject: [Bro] surgical file extraction In-Reply-To: References: <51BCA40C-1E34-4FF0-876A-F8BBDFFB8374@gmail.com> , Message-ID: Brandon, It may be the extraneous "," at the end of your mime_to_ext table. Kevin ________________________________ From: bro-bounces at bro.org on behalf of Brandon Glaze Sent: Wednesday, December 2, 2015 10:06 AM To: bro at bro.org Subject: Re: [Bro] surgical file extraction Seth, Thanks, but I have pulled up quite a few other examples and dont see where I am missing a semi-colon. I am wondering if its because I am running 32+ worker nodes and the script isnt built correctly to utilize the clustering (shot in the dark). Here is my script as I have it now. I was thinking I may want to move it to use a URL instead of the IP, especially if I start finding CnC comms to load balancer sites: global mime_to_ext: table[string] of string = { ["text/plain"] = "txt", ["text/html"] = "html", }; event file_sniff(f: fa_file, meta: fa_metadata) { for ( cid in f$conns ) { if ( f$conns[cid]$id$resp_h != 123.123.123.123 ) return; if ( f$source != "HTTP" ) return; if ( ! meta?$mime_type ) return; if ( meta$mime_type !in mime_to_ext ) return; local fname = fmt("%s-%s.%s", f$source, f$id, mime_to_ext[meta$mime_type]); print fmt("Extracting file %s", fname); Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]); } ===================== Brandon Glaze bglaze at gmail.com "Lead me, follow me, or get the hell out of my way." - General George Patton Jr On Tue, Dec 1, 2015 at 9:28 PM, Seth Hall > wrote: > On Nov 30, 2015, at 7:14 PM, Brandon Glaze > wrote: > > I validated that if I comment out my @load line for the new file extraction script this error goes away in a "broctl check" check. You may be missing a semi-colon somewhere near the end of your script. Bro's error reporting will accidentally report the problem on the first line of the next script in this case on accident. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151202/5b80cdd2/attachment.html From seth at icir.org Wed Dec 2 07:31:59 2015 From: seth at icir.org (Seth Hall) Date: Wed, 2 Dec 2015 10:31:59 -0500 Subject: [Bro] surgical file extraction In-Reply-To: References: <51BCA40C-1E34-4FF0-876A-F8BBDFFB8374@gmail.com> Message-ID: <4D2DAAA8-63D5-48C0-A9B5-80EFF91CE08B@icir.org> You are missing a right curly brace at the end. If you go through and clean up your indentation it should quickly become clear what?s missing. :) .Seth > On Dec 2, 2015, at 10:06 AM, Brandon Glaze wrote: > > Seth, > Thanks, but I have pulled up quite a few other examples and dont see where I am missing a semi-colon. I am wondering if its because I am running 32+ worker nodes and the script isnt built correctly to utilize the clustering (shot in the dark). Here is my script as I have it now. I was thinking I may want to move it to use a URL instead of the IP, especially if I start finding CnC comms to load balancer sites: > > global mime_to_ext: table[string] of string = { > ["text/plain"] = "txt", > ["text/html"] = "html", > }; > event file_sniff(f: fa_file, meta: fa_metadata) > { > for ( cid in f$conns ) > { > if ( f$conns[cid]$id$resp_h != 123.123.123.123 ) > return; > > if ( f$source != "HTTP" ) > return; > > if ( ! meta?$mime_type ) > return; > > if ( meta$mime_type !in mime_to_ext ) > return; > > local fname = fmt("%s-%s.%s", f$source, f$id, mime_to_ext[meta$mime_type]); > print fmt("Extracting file %s", fname); > Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]); > } > > > ===================== > Brandon Glaze > bglaze at gmail.com > > "Lead me, follow me, or get the hell out of my way." > - General George Patton Jr > > > > On Tue, Dec 1, 2015 at 9:28 PM, Seth Hall wrote: > > > On Nov 30, 2015, at 7:14 PM, Brandon Glaze wrote: > > > > I validated that if I comment out my @load line for the new file extraction script this error goes away in a "broctl check" check. > > You may be missing a semi-colon somewhere near the end of your script. Bro?s error reporting will accidentally report the problem on the first line of the next script in this case on accident. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From thomastan81 at gmail.com Wed Dec 2 07:41:54 2015 From: thomastan81 at gmail.com (Thomas Tan) Date: Wed, 2 Dec 2015 16:41:54 +0100 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: Dear All, I have checked out the TCPRS-plugin ( https://github.com/bro/bro-plugins/tree/master/tcprs/scripts/Bro/TCPRS). Unfortunately, it does not do the job. It cannot get TCP options and the order of the options down from a SYN packet. The TCP options of a SYN packet I am concerning are described below. # NOP option # EOL option # window scaling option, value nnn (or * or %nnn) # maximum segment size option, value nnn (or * or %nnn) # selective ACK OK # timestamp # timestamp with zero value # unrecognized option number n. Your kind help will be very much appreciated. Best regards, Thomas On 26 November 2015 at 12:29, Thomas Tan wrote: > Dear Jan, > > Many thanks for you reply. I am using tcp_option event. However, it seems > to me that the event can't tell which TCP options are from the SYN packet > of a connection and which ones are from other packets of the connection. I > think I will look into the TCPRS-plugin. > > Best regards, > > Thomas > > On 26 November 2015 at 12:16, Jan Grashofer wrote: > >> Hi Thomas, >> >> >> >> there is the tcp_option event, that might help you (see >> https://www.bro.org/sphinx/scripts/base/bif/plugins/Bro_TCP.events.bif.bro.html#id-tcp_option). >> If that does not fit for you, you might have a look into the TCPRS-plugin ( >> https://github.com/bro/bro-plugins/tree/master/tcprs/scripts/Bro/TCPRS). >> I have never used it but I think it also parses some TCP options and thus >> might be a good starting point. >> >> >> >> Best regards, >> >> Jan >> >> >> ------------------------------ >> *From:* bro-bounces at bro.org [bro-bounces at bro.org] on behalf of Thomas >> Tan [thomastan81 at gmail.com] >> *Sent:* Thursday, November 26, 2015 10:18 >> *To:* bro at bro.org >> *Subject:* [Bro] TCP options of a SYN packet >> >> Dear All, >> >> Just wondering if anyone knows a way (an event) to obtain TCP options of >> a SYN packet? >> >> Your help will be very much appreciated. >> >> Thank you. >> >> Best regards, >> >> Thomas >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151202/9a854ac0/attachment.html From bglaze at gmail.com Wed Dec 2 07:50:33 2015 From: bglaze at gmail.com (Brandon Glaze) Date: Wed, 2 Dec 2015 07:50:33 -0800 Subject: [Bro] surgical file extraction In-Reply-To: <4D2DAAA8-63D5-48C0-A9B5-80EFF91CE08B@icir.org> References: <51BCA40C-1E34-4FF0-876A-F8BBDFFB8374@gmail.com> <4D2DAAA8-63D5-48C0-A9B5-80EFF91CE08B@icir.org> Message-ID: Yeah, I think it was that missing closing bracket. I was stuck with using "vi" on one box, but once I opened it with "vim" it was clear. Taking the comma out in the table was just good practice, so thanks for that guys. I ran a check, and deployed it. Now I will have to monitor my test box to see what kind of performance hit this makes. Really appreciate the help. I am trying to show the benefits of using Bro in my work environment, and this is a huge win if I can get it working well. ===================== Brandon Glaze bglaze at gmail.com "Lead me, follow me, or get the hell out of my way." - General George Patton Jr On Wed, Dec 2, 2015 at 7:31 AM, Seth Hall wrote: > You are missing a right curly brace at the end. If you go through and > clean up your indentation it should quickly become clear what?s missing. :) > > .Seth > > > > On Dec 2, 2015, at 10:06 AM, Brandon Glaze wrote: > > > > Seth, > > Thanks, but I have pulled up quite a few other examples and dont see > where I am missing a semi-colon. I am wondering if its because I am running > 32+ worker nodes and the script isnt built correctly to utilize the > clustering (shot in the dark). Here is my script as I have it now. I was > thinking I may want to move it to use a URL instead of the IP, especially > if I start finding CnC comms to load balancer sites: > > > > global mime_to_ext: table[string] of string = { > > ["text/plain"] = "txt", > > ["text/html"] = "html", > > }; > > event file_sniff(f: fa_file, meta: fa_metadata) > > { > > for ( cid in f$conns ) > > { > > if ( f$conns[cid]$id$resp_h != 123.123.123.123 ) > > return; > > > > if ( f$source != "HTTP" ) > > return; > > > > if ( ! meta?$mime_type ) > > return; > > > > if ( meta$mime_type !in mime_to_ext ) > > return; > > > > local fname = fmt("%s-%s.%s", f$source, f$id, > mime_to_ext[meta$mime_type]); > > print fmt("Extracting file %s", fname); > > Files::add_analyzer(f, Files::ANALYZER_EXTRACT, > [$extract_filename=fname]); > > } > > > > > > ===================== > > Brandon Glaze > > bglaze at gmail.com > > > > "Lead me, follow me, or get the hell out of my way." > > - General George Patton Jr > > > > > > > > On Tue, Dec 1, 2015 at 9:28 PM, Seth Hall wrote: > > > > > On Nov 30, 2015, at 7:14 PM, Brandon Glaze wrote: > > > > > > I validated that if I comment out my @load line for the new file > extraction script this error goes away in a "broctl check" check. > > > > You may be missing a semi-colon somewhere near the end of your script. > Bro?s error reporting will accidentally report the problem on the first > line of the next script in this case on accident. > > > > .Seth > > > > -- > > Seth Hall > > International Computer Science Institute > > (Bro) because everyone has a network > > http://www.bro.org/ > > > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151202/8fbd6ea7/attachment-0001.html From seth at icir.org Wed Dec 2 08:34:54 2015 From: seth at icir.org (Seth Hall) Date: Wed, 2 Dec 2015 11:34:54 -0500 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: > On Dec 2, 2015, at 10:41 AM, Thomas Tan wrote: > > It cannot get TCP options and the order of the options down from a SYN packet. It sounds like you might want to write your own plugin but it might even be possible that that?s not enough and you?d have to add a feature to Bro?s core to generate an event only for SYN packets. (although you generally have to be very careful about even generating an event for a single packet). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From daniel.guerra69 at gmail.com Wed Dec 2 11:46:20 2015 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Wed, 2 Dec 2015 20:46:20 +0100 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: <049FFBD3-E8CD-41F3-96EE-A4C7EC274A4E@gmail.com> This probebly explains your problem in bro-plugins/tcprs/src/TCPRS.cc UsesTSOption = false; sack_in_use = false; AND in bro-plugins/tcprs/src/TCPRS_Endpoint.cc usesTimestamps = false; checkedForTSOptions = false; Regards, Daniel > On 02 Dec 2015, at 17:34, Seth Hall wrote: > > >> On Dec 2, 2015, at 10:41 AM, Thomas Tan wrote: >> >> It cannot get TCP options and the order of the options down from a SYN packet. > > It sounds like you might want to write your own plugin but it might even be possible that that?s not enough and you?d have to add a feature to Bro?s core to generate an event only for SYN packets. (although you generally have to be very careful about even generating an event for a single packet). > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From thomastan81 at gmail.com Wed Dec 2 12:39:41 2015 From: thomastan81 at gmail.com (Thomas Tan) Date: Wed, 2 Dec 2015 21:39:41 +0100 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: Dear Seth, Actually, I am writing a module using the outputs from Bro to detect Operating Systems running on remote host machines. I need to get the fingerprints of these OS for classification. I want to know if there is any means to obtain p0f-like OS fingerprints. Best regards, Thomas On 2 December 2015 at 17:34, Seth Hall wrote: > > > On Dec 2, 2015, at 10:41 AM, Thomas Tan wrote: > > > > It cannot get TCP options and the order of the options down from a SYN > packet. > > It sounds like you might want to write your own plugin but it might even > be possible that that?s not enough and you?d have to add a feature to Bro?s > core to generate an event only for SYN packets. (although you generally > have to be very careful about even generating an event for a single packet). > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151202/bb0f8f8d/attachment.html From thomastan81 at gmail.com Wed Dec 2 12:41:52 2015 From: thomastan81 at gmail.com (Thomas Tan) Date: Wed, 2 Dec 2015 21:41:52 +0100 Subject: [Bro] TCP options of a SYN packet In-Reply-To: <049FFBD3-E8CD-41F3-96EE-A4C7EC274A4E@gmail.com> References: <049FFBD3-E8CD-41F3-96EE-A4C7EC274A4E@gmail.com> Message-ID: Dear Daniel, Thanks for your reply. Could you give me an example or documentation detailing how I could achieve my goal? Best regards, Thomas On 2 December 2015 at 20:46, Daniel Guerra wrote: > This probebly explains your problem > > in bro-plugins/tcprs/src/TCPRS.cc > > UsesTSOption = false; > sack_in_use = false; > > AND > > in bro-plugins/tcprs/src/TCPRS_Endpoint.cc > > usesTimestamps = false; > checkedForTSOptions = false; > > Regards, > Daniel > > > On 02 Dec 2015, at 17:34, Seth Hall wrote: > > > > > >> On Dec 2, 2015, at 10:41 AM, Thomas Tan wrote: > >> > >> It cannot get TCP options and the order of the options down from a SYN > packet. > > > > It sounds like you might want to write your own plugin but it might even > be possible that that?s not enough and you?d have to add a feature to Bro?s > core to generate an event only for SYN packets. (although you generally > have to be very careful about even generating an event for a single packet). > > > > .Seth > > > > -- > > Seth Hall > > International Computer Science Institute > > (Bro) because everyone has a network > > http://www.bro.org/ > > > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151202/6317876b/attachment.html From vladg at illinois.edu Wed Dec 2 13:26:51 2015 From: vladg at illinois.edu (Vlad Grigorescu) Date: Wed, 02 Dec 2015 15:26:51 -0600 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: Thomas, Bro has p0f support built-in. See: https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html#id-OS_version_found That being said, the original p0f fingerprints are very out of date, and it's possible that Bro will stop supporting p0f in the future. I did some research on the fingerprints with the Windows XP end of life, and ended up leveraging some of Bro's other capabilities to write a much better detection: https://github.com/bro/bro/blob/master/scripts/policy/frameworks/software/windows-version-detection.bro Generally, I think the interest is in moving up the stack and performing this kind of fingerprinting at a higher, more reliable, layer. --Vlad Thomas Tan writes: > Dear Seth, > > Actually, I am writing a module using the outputs from Bro to detect > Operating Systems running on remote host machines. I need to get the > fingerprints of these OS for classification. I want to know if there is any > means to obtain p0f-like OS fingerprints. > > Best regards, > > Thomas > > On 2 December 2015 at 17:34, Seth Hall wrote: > >> >> > On Dec 2, 2015, at 10:41 AM, Thomas Tan wrote: >> > >> > It cannot get TCP options and the order of the options down from a SYN >> packet. >> >> It sounds like you might want to write your own plugin but it might even >> be possible that that?s not enough and you?d have to add a feature to Bro?s >> core to generate an event only for SYN packets. (although you generally >> have to be very careful about even generating an event for a single packet). >> >> .Seth >> >> -- >> Seth Hall >> International Computer Science Institute >> (Bro) because everyone has a network >> http://www.bro.org/ >> >> > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 800 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151202/e944453b/attachment.bin From thomastan81 at gmail.com Wed Dec 2 14:29:33 2015 From: thomastan81 at gmail.com (Thomas Tan) Date: Wed, 2 Dec 2015 23:29:33 +0100 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: Hi Vlad, Thanks for your reply. I am aware of the support of p0f in Bro. You are right. The original p0f v2 fingerprints are out-dated. In my work, I am not using the p0f v2 fingerprints but collecting OS fingerprints from network connections initiated by remote machines. A multi-class classifier will be applied to assign these OS fingerprints to their respective OS types. Best regards, Thomas On 2 December 2015 at 22:26, Vlad Grigorescu wrote: > Thomas, > > Bro has p0f support built-in. See: > > https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html#id-OS_version_found > > That being said, the original p0f fingerprints are very out of date, and > it's possible that Bro will stop supporting p0f in the future. I did > some research on the fingerprints with the Windows XP end of life, and > ended up leveraging some of Bro's other capabilities to write a much > better detection: > > https://github.com/bro/bro/blob/master/scripts/policy/frameworks/software/windows-version-detection.bro > > Generally, I think the interest is in moving up the stack and performing > this kind of fingerprinting at a higher, more reliable, layer. > > --Vlad > > Thomas Tan writes: > > > Dear Seth, > > > > Actually, I am writing a module using the outputs from Bro to detect > > Operating Systems running on remote host machines. I need to get the > > fingerprints of these OS for classification. I want to know if there is > any > > means to obtain p0f-like OS fingerprints. > > > > Best regards, > > > > Thomas > > > > On 2 December 2015 at 17:34, Seth Hall wrote: > > > >> > >> > On Dec 2, 2015, at 10:41 AM, Thomas Tan > wrote: > >> > > >> > It cannot get TCP options and the order of the options down from a > SYN > >> packet. > >> > >> It sounds like you might want to write your own plugin but it might even > >> be possible that that?s not enough and you?d have to add a feature to > Bro?s > >> core to generate an event only for SYN packets. (although you generally > >> have to be very careful about even generating an event for a single > packet). > >> > >> .Seth > >> > >> -- > >> Seth Hall > >> International Computer Science Institute > >> (Bro) because everyone has a network > >> http://www.bro.org/ > >> > >> > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151202/85caf8dd/attachment.html From michalpurzynski1 at gmail.com Wed Dec 2 15:44:21 2015 From: michalpurzynski1 at gmail.com (Michal Purzynski) Date: Thu, 3 Dec 2015 00:44:21 +0100 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: Indeed. Modern OS literally scream their versions over the network. Personally I find Bro's software detection capability to determine applications and libraries used on devices I cannot login to. Think IoT. Doing OS recognition per packet, based on IP options has always been a poor idea. Prone to false positives, difficult to update and does not scale. > On 02 Dec 2015, at 22:26, Vlad Grigorescu wrote: > > Thomas, > > Bro has p0f support built-in. See: > https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html#id-OS_version_found > > That being said, the original p0f fingerprints are very out of date, and > it's possible that Bro will stop supporting p0f in the future. I did > some research on the fingerprints with the Windows XP end of life, and > ended up leveraging some of Bro's other capabilities to write a much > better detection: > https://github.com/bro/bro/blob/master/scripts/policy/frameworks/software/windows-version-detection.bro > > Generally, I think the interest is in moving up the stack and performing > this kind of fingerprinting at a higher, more reliable, layer. > > --Vlad > > Thomas Tan writes: > >> Dear Seth, >> >> Actually, I am writing a module using the outputs from Bro to detect >> Operating Systems running on remote host machines. I need to get the >> fingerprints of these OS for classification. I want to know if there is any >> means to obtain p0f-like OS fingerprints. >> >> Best regards, >> >> Thomas >> >>> On 2 December 2015 at 17:34, Seth Hall wrote: >>> >>> >>>> On Dec 2, 2015, at 10:41 AM, Thomas Tan wrote: >>>> >>>> It cannot get TCP options and the order of the options down from a SYN >>> packet. >>> >>> It sounds like you might want to write your own plugin but it might even >>> be possible that that?s not enough and you?d have to add a feature to Bro?s >>> core to generate an event only for SYN packets. (although you generally >>> have to be very careful about even generating an event for a single packet). >>> >>> .Seth >>> >>> -- >>> Seth Hall >>> International Computer Science Institute >>> (Bro) because everyone has a network >>> http://www.bro.org/ >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From james.swaro at gmail.com Wed Dec 2 20:49:58 2015 From: james.swaro at gmail.com (James Swaro) Date: Wed, 2 Dec 2015 22:49:58 -0600 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: I'm sorry I didn't see this earlier. I'm curious. Why is the order of the options important? Are you searching for OS specific behavior? James Swaro On Wed, Dec 2, 2015 at 4:29 PM, Thomas Tan wrote: > Hi Vlad, > > Thanks for your reply. > > I am aware of the support of p0f in Bro. You are right. The original p0f > v2 fingerprints are out-dated. In my work, I am not using the p0f v2 > fingerprints but collecting OS fingerprints from network connections > initiated by remote machines. A multi-class classifier will be applied to > assign these OS fingerprints to their respective OS types. > > Best regards, > > Thomas > > On 2 December 2015 at 22:26, Vlad Grigorescu wrote: > >> Thomas, >> >> Bro has p0f support built-in. See: >> >> https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html#id-OS_version_found >> >> That being said, the original p0f fingerprints are very out of date, and >> it's possible that Bro will stop supporting p0f in the future. I did >> some research on the fingerprints with the Windows XP end of life, and >> ended up leveraging some of Bro's other capabilities to write a much >> better detection: >> >> https://github.com/bro/bro/blob/master/scripts/policy/frameworks/software/windows-version-detection.bro >> >> Generally, I think the interest is in moving up the stack and performing >> this kind of fingerprinting at a higher, more reliable, layer. >> >> --Vlad >> >> Thomas Tan writes: >> >> > Dear Seth, >> > >> > Actually, I am writing a module using the outputs from Bro to detect >> > Operating Systems running on remote host machines. I need to get the >> > fingerprints of these OS for classification. I want to know if there is >> any >> > means to obtain p0f-like OS fingerprints. >> > >> > Best regards, >> > >> > Thomas >> > >> > On 2 December 2015 at 17:34, Seth Hall wrote: >> > >> >> >> >> > On Dec 2, 2015, at 10:41 AM, Thomas Tan >> wrote: >> >> > >> >> > It cannot get TCP options and the order of the options down from a >> SYN >> >> packet. >> >> >> >> It sounds like you might want to write your own plugin but it might >> even >> >> be possible that that?s not enough and you?d have to add a feature to >> Bro?s >> >> core to generate an event only for SYN packets. (although you generally >> >> have to be very careful about even generating an event for a single >> packet). >> >> >> >> .Seth >> >> >> >> -- >> >> Seth Hall >> >> International Computer Science Institute >> >> (Bro) because everyone has a network >> >> http://www.bro.org/ >> >> >> >> >> > _______________________________________________ >> > Bro mailing list >> > bro at bro-ids.org >> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151202/c9f4db12/attachment.html From dnj0496 at gmail.com Wed Dec 2 21:16:21 2015 From: dnj0496 at gmail.com (Dnj) Date: Wed, 2 Dec 2015 21:16:21 -0800 Subject: [Bro] conditional loading In-Reply-To: <285BE09C-ED49-4D4D-9658-1FA9C26C6543@icir.org> References: <285BE09C-ED49-4D4D-9658-1FA9C26C6543@icir.org> Message-ID: Thanks Seth. > On Dec 1, 2015, at 9:27 PM, Seth Hall wrote: > > You already got the correct advice in this thread, but just to close it out I?ll go into a bit more detail on why it couldn?t work. > > Since ?@if? is a parser directive, that code is executed while the code is being parsed but the code within a ?when? block is executed asynchronously. There is higher-order problem that I?ll get to next, but conceptually that @if wouldn?t work anymore since it wouldn?t be executed until later when the when statement?s body executes. > > The high level problem is that it looks like you have a lot of code outside of event handlers. There isn?t much support for code outside of event handlers in Bro since that means the code would only be executed at parse time which is frequently not a useful time to accomplish things. If you want to do something at startup you would work in a bro_init event handler (which still wouldn?t work for what you want to do unfortunately). > > I would follow Anthony?s advice and do a condition in a script that just changes behavior based on the result of the command that gets executed. On the upside, this give you the flexibility to re-run the command later and have behavior change dynamically. > > .Seth > >> On Nov 30, 2015, at 9:43 PM, Dk Jack wrote: >> >> the directives don't seem to have file test operators. I tried to do the following... >> >> local c = "test -f filters.bro"; >> local cmd = Exec::Command($cmd=c); >> >> when (local res = Exec::run(cmd)) >> { >> @if (res$exit_code == 0) >> { >> @load filters.bro; >> } >> } >> >> >> However, I am getting parsing errors... >> >> --------------------------------------------------------------- >> referencing a local name in @if (res) >> invalid expression in @if (res$exit_code == 0) >> --------------------------------------------------------------- >> >> Dnj >> >> >> On Mon, Nov 30, 2015 at 4:41 PM, anthony kasza wrote: >> See here: >> https://www.bro.org/sphinx/script-reference/directives.html >> >> -AK >> >> On Nov 30, 2015 4:36 PM, "Dk Jack" wrote: >> Hi, >> Is it possible to perform conditional loading of bro script files ? >> In my local.bro file, I'd like to do something like this: >> >> if file 'filters.bro' exists >> { >> @load filters.bro >> } >> >> The file filters.bro may or may not exist because it's getting generated from an external program. >> Thanks for your help in advance. >> >> Dnj >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > From daniel.guerra69 at gmail.com Wed Dec 2 23:40:43 2015 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Thu, 3 Dec 2015 08:40:43 +0100 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: Hi Thomas, Tcp options and specific the option-timestamp is very important for device detection of linux-like kernels. P0f is really outdated and unuseable for mobile devices. We did some research on detecting devices behind a NAT router. Mac and windows can be followed by taking a look at source port behaviour (windows can be done with the ip-id field). But if you have a linux machine is behaves with random ports (see linux kernel source). The tcp-options ts uses the time since the device is up. So if you do a capture_ts - (tcp_ts*factor) you get a ?constant? number per device. The factor depends on the kernel source. I?ll have a look at the code, Thomas, I?ll get back later. Regards, Daniel > On 03 Dec 2015, at 05:49, James Swaro wrote: > > I'm sorry I didn't see this earlier. I'm curious. Why is the order of the options important? Are you searching for OS specific behavior? > > James Swaro > > > > On Wed, Dec 2, 2015 at 4:29 PM, Thomas Tan > wrote: > Hi Vlad, > > Thanks for your reply. > > I am aware of the support of p0f in Bro. You are right. The original p0f v2 fingerprints are out-dated. In my work, I am not using the p0f v2 fingerprints but collecting OS fingerprints from network connections initiated by remote machines. A multi-class classifier will be applied to assign these OS fingerprints to their respective OS types. > > Best regards, > > Thomas > > On 2 December 2015 at 22:26, Vlad Grigorescu > wrote: > Thomas, > > Bro has p0f support built-in. See: > https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html#id-OS_version_found > > That being said, the original p0f fingerprints are very out of date, and > it's possible that Bro will stop supporting p0f in the future. I did > some research on the fingerprints with the Windows XP end of life, and > ended up leveraging some of Bro's other capabilities to write a much > better detection: > https://github.com/bro/bro/blob/master/scripts/policy/frameworks/software/windows-version-detection.bro > > Generally, I think the interest is in moving up the stack and performing > this kind of fingerprinting at a higher, more reliable, layer. > > --Vlad > > Thomas Tan > writes: > > > Dear Seth, > > > > Actually, I am writing a module using the outputs from Bro to detect > > Operating Systems running on remote host machines. I need to get the > > fingerprints of these OS for classification. I want to know if there is any > > means to obtain p0f-like OS fingerprints. > > > > Best regards, > > > > Thomas > > > > On 2 December 2015 at 17:34, Seth Hall > wrote: > > > >> > >> > On Dec 2, 2015, at 10:41 AM, Thomas Tan > wrote: > >> > > >> > It cannot get TCP options and the order of the options down from a SYN > >> packet. > >> > >> It sounds like you might want to write your own plugin but it might even > >> be possible that that?s not enough and you?d have to add a feature to Bro?s > >> core to generate an event only for SYN packets. (although you generally > >> have to be very careful about even generating an event for a single packet). > >> > >> .Seth > >> > >> -- > >> Seth Hall > >> International Computer Science Institute > >> (Bro) because everyone has a network > >> http://www.bro.org/ > >> > >> > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151203/c142f204/attachment-0001.html From abdshah94 at gmail.com Thu Dec 3 05:44:16 2015 From: abdshah94 at gmail.com (Abdullah Shah) Date: Thu, 3 Dec 2015 18:44:16 +0500 Subject: [Bro] Not being able to install plugin. Message-ID: Trying to install a plugin into the running bro. when i initiate ./configure it says to specify --bro-dist directory which i am clueless about. Is there any one who can guide me to install a plugin the conventional way. Any help would be appreciated. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151203/967db9d0/attachment.html From David.deBruyn at jbssa.com Thu Dec 3 10:45:06 2015 From: David.deBruyn at jbssa.com (de Bruyn, David) Date: Thu, 3 Dec 2015 18:45:06 +0000 Subject: [Bro] SMTP File Hash not consistent Message-ID: <09eafd99a5e94f89bfecf7c08fdc4cab@USTXCR00EXC16I.global.corp.prod> Hi All! First off... I'm kinda new to Bro so please be gentle.... I've noticed some issues (strangeness?) with the file logging on Bro, in particular I would like for Bro to log an MD5 for all incomming files sent in through SMTP. At the moment it only seems to do it for some files and I can't seem to find a reason why some are getting hashed but others aren't... An extract from my files.log filtered by SMTP and pdf: 1449167625.904080 FS81ev 1449167623.516100 Fajnj71Xx6UprSmLef 198.22.115.26 208.33.144.195 C6pKQN2extOHQYZ4Fc SMTP 3 SHA1,MD5 application/pdf LoadTender3059527.pdf 0.015949 F T 57 - 1368 0 F - - - - - 1449167625.848077 FhU87R1PwGYciZcT2i 198.22.115.26 208.33.144.195 CkD4rQ1uG5VZhJL2v9 SMTP 1 SHA1,MD5 application/pdf 12.03.2015.pdf 0.016022 F T 456 - 1368 0 F - - - - -3MhA2vXGk5J8 198.22.115.26 208.33.145.195 CHB8Ew4kdUB3hDbkKl SMTP 3 SHA1,MD5 application/pdf Payment Advice Note from 12/03/2015.PDF 0.071983 F T 14535 - 0 0 F - ef853cc031d2abfbf6e0ec964163cd98 08eae5d275554f12d4783cb9c8be210d691f8db5 - - 1449167630.224049 FGUsvz3nDYqZlH56Y1 198.22.115.26 208.33.145.195 CK8Nwn4vGwpylAmpGj SMTP 3 SHA1,MD5 application/pdf PPC_LoadTender3057660.pdf 0.032006 F T 969 - 1544 0 F - - - - - 1449167631.024050 FiMmk5Zsczli9OGi7 198.22.115.26 208.33.144.195 CX4SUd3VDBBdYoXt0g SMTP 3 SHA1,MD5 application/pdf Payment Advice Note from 12/03/2015.PDF 0.011997 F T 171 - 1368 0 F - - - - - So basically about it won't create a file hash for a heap of file, then out of the blue it will create one, then no more for a while.... They all have the same mime type so I just can't seem to figure this out... any help or advice would be really appreciated... Cheers, David. The information in this email is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, re-transmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer. This email has been checked for viruses. However, JBS USA Holdings, Inc. and its constituent companies cannot accept responsibility for loss or damages arising from use of this email or attachments and we recommend that you subject these to your virus checking procedures prior to use. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151203/cdc446f3/attachment.html From james.swaro at gmail.com Thu Dec 3 11:14:14 2015 From: james.swaro at gmail.com (James Swaro) Date: Thu, 3 Dec 2015 13:14:14 -0600 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: As you noted already, TCPRS doesn't provide that type of behavior. The options of interest to TCPRS are limited and only reported via the TCPRS::conn_config event, which certainly does not fit your needs. That said, extending the TCPRS plugin, or the TCP analyzer within bro doesn't seem to be something that is out of the realm of possibility here. As Seth mentioned above, generating an event for every syn is potentially expensive. SYN retransmissions may generate duplicate records. You could create a new event in the source for syn options, but I'm not aware of any bro constructs that would (easily) allow for providing the options in the order they are observed in the header(Seth,Vlad,Daniel - please chime in here). It might be possible to send (ID, value) tuples to the script with a new event, given that the tuples are inserted into a vector in the order they are observed in the header. Once in the script, you could convert the ID (option identifier) into the string representation and create a single string that could easily be parsed. Example record: > 10.0.0.1 2000 10.0.0.2 2001 TCPOPT_MAXSEG=3,TCPOPT_WSCALE=1 Obviously, the output format is dictated by the bro script, so that isn't terribly important here since that would be up to your design. I'm sure you are already aware of issues with fingerprinting, but some values might differ within the same OS depending if the kernel has been modified or kernel config options have been modified from their default values. If this type of approach interests you, let me know. James Swaro On Thu, Dec 3, 2015 at 4:36 AM, Thomas Tan wrote: > Hi James, > > Thanks for you reply. You are right. I am looking into OS specific > behaviors. How can I use your plug-in to get TCP options from a SYN packet? > > Your help will be very much appreciated. > > Best regards, > > Thomas > > On 3 December 2015 at 05:49, James Swaro wrote: > >> I'm sorry I didn't see this earlier. I'm curious. Why is the order of the >> options important? Are you searching for OS specific behavior? >> >> James Swaro >> >> >> >> On Wed, Dec 2, 2015 at 4:29 PM, Thomas Tan wrote: >> >>> Hi Vlad, >>> >>> Thanks for your reply. >>> >>> I am aware of the support of p0f in Bro. You are right. The original p0f >>> v2 fingerprints are out-dated. In my work, I am not using the p0f v2 >>> fingerprints but collecting OS fingerprints from network connections >>> initiated by remote machines. A multi-class classifier will be applied to >>> assign these OS fingerprints to their respective OS types. >>> >>> Best regards, >>> >>> Thomas >>> >>> On 2 December 2015 at 22:26, Vlad Grigorescu wrote: >>> >>>> Thomas, >>>> >>>> Bro has p0f support built-in. See: >>>> >>>> https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html#id-OS_version_found >>>> >>>> That being said, the original p0f fingerprints are very out of date, and >>>> it's possible that Bro will stop supporting p0f in the future. I did >>>> some research on the fingerprints with the Windows XP end of life, and >>>> ended up leveraging some of Bro's other capabilities to write a much >>>> better detection: >>>> >>>> https://github.com/bro/bro/blob/master/scripts/policy/frameworks/software/windows-version-detection.bro >>>> >>>> Generally, I think the interest is in moving up the stack and performing >>>> this kind of fingerprinting at a higher, more reliable, layer. >>>> >>>> --Vlad >>>> >>>> Thomas Tan writes: >>>> >>>> > Dear Seth, >>>> > >>>> > Actually, I am writing a module using the outputs from Bro to detect >>>> > Operating Systems running on remote host machines. I need to get the >>>> > fingerprints of these OS for classification. I want to know if there >>>> is any >>>> > means to obtain p0f-like OS fingerprints. >>>> > >>>> > Best regards, >>>> > >>>> > Thomas >>>> > >>>> > On 2 December 2015 at 17:34, Seth Hall wrote: >>>> > >>>> >> >>>> >> > On Dec 2, 2015, at 10:41 AM, Thomas Tan >>>> wrote: >>>> >> > >>>> >> > It cannot get TCP options and the order of the options down from >>>> a SYN >>>> >> packet. >>>> >> >>>> >> It sounds like you might want to write your own plugin but it might >>>> even >>>> >> be possible that that?s not enough and you?d have to add a feature >>>> to Bro?s >>>> >> core to generate an event only for SYN packets. (although you >>>> generally >>>> >> have to be very careful about even generating an event for a single >>>> packet). >>>> >> >>>> >> .Seth >>>> >> >>>> >> -- >>>> >> Seth Hall >>>> >> International Computer Science Institute >>>> >> (Bro) because everyone has a network >>>> >> http://www.bro.org/ >>>> >> >>>> >> >>>> > _______________________________________________ >>>> > Bro mailing list >>>> > bro at bro-ids.org >>>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>> >>> >>> >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151203/d98e814a/attachment-0001.html From johanna at icir.org Thu Dec 3 12:01:28 2015 From: johanna at icir.org (Johanna Amann) Date: Thu, 3 Dec 2015 12:01:28 -0800 Subject: [Bro] OpenSSL security issue affecting Bro Message-ID: <20151203200124.GA36339@wifi74.sys.ICSI.Berkeley.EDU> Hello, The OpenSSL Project today published a security advisory, that affects users of Bro that are using the X.509 certificate validation functionality of Bro. Note that this functionality is not enabled by default - typically it is enabled by either loading the policy script protocols/ssl/validate-certs.bro or protocols/ssl/validate-ocsp.bro. The OpenSSL bug can cause a null-pointer exception when parsing certain malformed X.509 certificates and can potentially be used for DOS attacks. The issue affects OpenSSL 1.0.1 and 1.0.2 and was fixed in OpenSSL 1.0.1q and 1.0.2e respectively. If you use Bro and perform certificate validation, you should update as soon as possible. The original OpenSSL security advisory is available at https://www.openssl.org/news/secadv/20151203.txt. It also contains a few other issues that are not directly applicable to Bro. Johanna From sven at dreyer-net.de Thu Dec 3 12:41:10 2015 From: sven at dreyer-net.de (Sven Dreyer) Date: Thu, 3 Dec 2015 21:41:10 +0100 Subject: [Bro] Problem with connections in S1 and SF state In-Reply-To: References: <564B9060.1080304@dreyer-net.de> <5A214812-FF4A-4360-B643-EFCC1516F869@illinois.edu> <564DF725.6010604@dreyer-net.de> Message-ID: <5660A8E6.7070305@dreyer-net.de> Derek, thank you very much for your reply. I ran the command you mentioned for my pcap file and checked the conn.log written in my current directory. But this does not seem to change anything, source and destination for the connection I was watching at are still twisted. Best regards, Sven Am 25.11.2015 um 14:08 schrieb derek at criticalstack.com: > Sven, > > Try running the pcap through your local policy scripts and check the > output: > > bro -r file.pcap local > > I don't otherwise have a specific clue why this could happen, but it's > best to compare the same process. > > -Derek > >> From: Sven Dreyer >> Sent: Thursday, November 19, 10:34 >> Subject: Re: [Bro] Problem with connections in S1 and SF state >> To: bro at bro.org, Azoff, Justin S >> >> Justin, thanks for the hint. I should indeed have checked the history >> field. But even for connections that do not start with d or D in the >> history field, I see the same behaviour. Source and destination is >> still twisted: 1447675087.121817 CjRCD61gNErucciPb8 87.144.16.xxx >> 50993 192.168.100.yyy 26577 tcp ssl 83.596659 1432 2619 S1 F T 0 >> ShADad 18 2164 15 3231 (empty) Bro is configured to listen to a bridge >> interface (br0). But I also have running a dumpcap process writing all >> packets to pcap files. Interestingly, if I feed the corresponding pcap >> file to bro (bro -r file.pcap), I get source and destination in the >> right order: 1447675087.121817 C2AvJf3WgcdiBlYfS4 192.168.100.yyy >> 26577 87.144.16.xxx 50993 tcp ssl 83.596659 1432 2619 S1 - - 0 ShADad >> 18 2164 15 3231 (empty) Does anybody have an explanation for this? >> Thanks, Sven Am 17.11.2015 um 21:53 schrieb Azoff, Justin S: > You >> should really be looking at the history field: > > history: string >> &log &optional > Records the state history of connections as a string >> of letters. The meaning of those letters is: > > Letter Meaning > s a >> SYN w/o the ACK bit set > h a SYN+ACK (?handshake?) > a a pure ACK > d >> packet with payload (?data?) > f packet with FIN bit set > r packet >> with RST bit set > c packet with a bad checksum > i inconsistent >> packet (e.g. SYN+RST bits both set) > If the event comes from the >> originator, the letter is in upper-case; if it comes from the >> responder, it?s in lower-case. Multiple packets of the same type will >> only be noted once (e.g. we only record one ?d? in each direction, >> regardless of how many data packets were seen.) > > So any connection >> that starts with D or d means bro missed the initial syn handshake >> (Sh) > > _______________________________________________ Bro mailing >> list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> From daniel.guerra69 at gmail.com Thu Dec 3 15:16:14 2015 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Fri, 4 Dec 2015 00:16:14 +0100 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: I agree TCPRS is not the place. When i look at the analyzer TCP.cc , i miss some options available from /usr/include/netinet/tcp.h # define TCPOPT_EOL 0 # define TCPOPT_NOP 1 # define TCPOPT_MAXSEG 2 # define TCPOLEN_MAXSEG 4 # define TCPOPT_WINDOW 3 # define TCPOLEN_WINDOW 3 # define TCPOPT_SACK_PERMITTED 4 /* Experimental */ # define TCPOLEN_SACK_PERMITTED 2 # define TCPOPT_SACK 5 /* Experimental */ # define TCPOPT_TIMESTAMP 8 # define TCPOLEN_TIMESTAMP 10 # define TCPOLEN_TSTAMP_APPA (TCPOLEN_TIMESTAMP+2) /* appendix A */ It would be nice to have them available when needed. OS fingerprint can be done on how it was constructed, the order is free and each kernel treats the rules different. This can tell you what type of os you see, but some are universal. But if you want to detect two or more devices and want to relate the traffic to the device, you really need the timestamp. It could also provide a method to detect cooked packets or a virus that creates its own packets. And I think it is actually very cheap because you are sure you only get 1 syn (beside retrans) per connection and that is way less than HTTP for example. Regards, Daniel > On 03 Dec 2015, at 20:14, James Swaro wrote: > > As you noted already, TCPRS doesn't provide that type of behavior. The options of interest to TCPRS are limited and only reported via the TCPRS::conn_config event, which certainly does not fit your needs. That said, extending the TCPRS plugin, or the TCP analyzer within bro doesn't seem to be something that is out of the realm of possibility here. > > As Seth mentioned above, generating an event for every syn is potentially expensive. SYN retransmissions may generate duplicate records. > > You could create a new event in the source for syn options, but I'm not aware of any bro constructs that would (easily) allow for providing the options in the order they are observed in the header(Seth,Vlad,Daniel - please chime in here). It might be possible to send (ID, value) tuples to the script with a new event, given that the tuples are inserted into a vector in the order they are observed in the header. Once in the script, you could convert the ID (option identifier) into the string representation and create a single string that could easily be parsed. > > Example record: > > 10.0.0.1 2000 10.0.0.2 2001 TCPOPT_MAXSEG=3,TCPOPT_WSCALE=1 > > Obviously, the output format is dictated by the bro script, so that isn't terribly important here since that would be up to your design. > > I'm sure you are already aware of issues with fingerprinting, but some values might differ within the same OS depending if the kernel has been modified or kernel config options have been modified from their default values. > > If this type of approach interests you, let me know. > > > James Swaro > > > > On Thu, Dec 3, 2015 at 4:36 AM, Thomas Tan > wrote: > Hi James, > > Thanks for you reply. You are right. I am looking into OS specific behaviors. How can I use your plug-in to get TCP options from a SYN packet? > > Your help will be very much appreciated. > > Best regards, > > Thomas > > On 3 December 2015 at 05:49, James Swaro > wrote: > I'm sorry I didn't see this earlier. I'm curious. Why is the order of the options important? Are you searching for OS specific behavior? > > James Swaro > > > > On Wed, Dec 2, 2015 at 4:29 PM, Thomas Tan > wrote: > Hi Vlad, > > Thanks for your reply. > > I am aware of the support of p0f in Bro. You are right. The original p0f v2 fingerprints are out-dated. In my work, I am not using the p0f v2 fingerprints but collecting OS fingerprints from network connections initiated by remote machines. A multi-class classifier will be applied to assign these OS fingerprints to their respective OS types. > > Best regards, > > Thomas > > On 2 December 2015 at 22:26, Vlad Grigorescu > wrote: > Thomas, > > Bro has p0f support built-in. See: > https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html#id-OS_version_found > > That being said, the original p0f fingerprints are very out of date, and > it's possible that Bro will stop supporting p0f in the future. I did > some research on the fingerprints with the Windows XP end of life, and > ended up leveraging some of Bro's other capabilities to write a much > better detection: > https://github.com/bro/bro/blob/master/scripts/policy/frameworks/software/windows-version-detection.bro > > Generally, I think the interest is in moving up the stack and performing > this kind of fingerprinting at a higher, more reliable, layer. > > --Vlad > > Thomas Tan > writes: > > > Dear Seth, > > > > Actually, I am writing a module using the outputs from Bro to detect > > Operating Systems running on remote host machines. I need to get the > > fingerprints of these OS for classification. I want to know if there is any > > means to obtain p0f-like OS fingerprints. > > > > Best regards, > > > > Thomas > > > > On 2 December 2015 at 17:34, Seth Hall > wrote: > > > >> > >> > On Dec 2, 2015, at 10:41 AM, Thomas Tan > wrote: > >> > > >> > It cannot get TCP options and the order of the options down from a SYN > >> packet. > >> > >> It sounds like you might want to write your own plugin but it might even > >> be possible that that?s not enough and you?d have to add a feature to Bro?s > >> core to generate an event only for SYN packets. (although you generally > >> have to be very careful about even generating an event for a single packet). > >> > >> .Seth > >> > >> -- > >> Seth Hall > >> International Computer Science Institute > >> (Bro) because everyone has a network > >> http://www.bro.org/ > >> > >> > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151204/de46fd96/attachment-0001.html From james.swaro at gmail.com Thu Dec 3 16:07:32 2015 From: james.swaro at gmail.com (James Swaro) Date: Thu, 3 Dec 2015 18:07:32 -0600 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: > I agree TCPRS is not the place. Agreed. I would like TCPRS to be useful in this case, but the feature falls out of the scope of design of TCPRS. It might not be difficult to write a simple analyzer that looks only at syn packets and produces the events that Thomas is looking to generate. A simple analyzer may not need to retain packets like TCPRS does and would have a much faster execution and lower memory footprint because it doesn't need to do retrospective analysis on the connection. James Swaro On Thu, Dec 3, 2015 at 5:16 PM, Daniel Guerra wrote: > I agree TCPRS is not the place. > When i look at the analyzer TCP.cc , i miss > some options available from /usr/include/netinet/tcp.h > > # define TCPOPT_EOL 0 > # define TCPOPT_NOP 1 > # define TCPOPT_MAXSEG 2 > # define TCPOLEN_MAXSEG 4 > # define TCPOPT_WINDOW 3 > # define TCPOLEN_WINDOW 3 > # define TCPOPT_SACK_PERMITTED 4 /* Experimental */ > # define TCPOLEN_SACK_PERMITTED 2 > # define TCPOPT_SACK 5 /* Experimental */ > # define TCPOPT_TIMESTAMP 8 > # define TCPOLEN_TIMESTAMP 10 > # define TCPOLEN_TSTAMP_APPA (TCPOLEN_TIMESTAMP+2) /* appendix A */ > > It would be nice to have them available when needed. OS fingerprint can be > done > on how it was constructed, the order is free and each kernel treats the > rules different. > This can tell you what type of os you see, but some are universal. > But if you want to detect two or more devices and want to relate the > traffic to the device, > you really need the timestamp. > It could also provide a method to detect cooked packets or a virus that > creates its own packets. > And I think it is actually very cheap because you are sure you only get 1 > syn (beside retrans) > per connection and that is way less than HTTP for example. > > Regards, > > Daniel > > On 03 Dec 2015, at 20:14, James Swaro wrote: > > As you noted already, TCPRS doesn't provide that type of behavior. The > options of interest to TCPRS are limited and only reported via the > TCPRS::conn_config event, which certainly does not fit your needs. That > said, extending the TCPRS plugin, or the TCP analyzer within bro doesn't > seem to be something that is out of the realm of possibility here. > > As Seth mentioned above, generating an event for every syn is potentially > expensive. SYN retransmissions may generate duplicate records. > > You could create a new event in the source for syn options, but I'm not > aware of any bro constructs that would (easily) allow for providing the > options in the order they are observed in the header(Seth,Vlad,Daniel - > please chime in here). It might be possible to send (ID, value) tuples to > the script with a new event, given that the tuples are inserted into a > vector in the order they are observed in the header. Once in the script, > you could convert the ID (option identifier) into the string representation > and create a single string that could easily be parsed. > > Example record: > > 10.0.0.1 2000 10.0.0.2 2001 TCPOPT_MAXSEG=3,TCPOPT_WSCALE=1 > > Obviously, the output format is dictated by the bro script, so that isn't > terribly important here since that would be up to your design. > > I'm sure you are already aware of issues with fingerprinting, but some > values might differ within the same OS depending if the kernel has been > modified or kernel config options have been modified from their default > values. > > If this type of approach interests you, let me know. > > > James Swaro > > > > On Thu, Dec 3, 2015 at 4:36 AM, Thomas Tan wrote: > >> Hi James, >> >> Thanks for you reply. You are right. I am looking into OS specific >> behaviors. How can I use your plug-in to get TCP options from a SYN packet? >> >> Your help will be very much appreciated. >> >> Best regards, >> >> Thomas >> >> On 3 December 2015 at 05:49, James Swaro wrote: >> >>> I'm sorry I didn't see this earlier. I'm curious. Why is the order of >>> the options important? Are you searching for OS specific behavior? >>> >>> James Swaro >>> >>> >>> >>> On Wed, Dec 2, 2015 at 4:29 PM, Thomas Tan >>> wrote: >>> >>>> Hi Vlad, >>>> >>>> Thanks for your reply. >>>> >>>> I am aware of the support of p0f in Bro. You are right. The original >>>> p0f v2 fingerprints are out-dated. In my work, I am not using the p0f v2 >>>> fingerprints but collecting OS fingerprints from network connections >>>> initiated by remote machines. A multi-class classifier will be applied to >>>> assign these OS fingerprints to their respective OS types. >>>> >>>> Best regards, >>>> >>>> Thomas >>>> >>>> On 2 December 2015 at 22:26, Vlad Grigorescu >>>> wrote: >>>> >>>>> Thomas, >>>>> >>>>> Bro has p0f support built-in. See: >>>>> >>>>> https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html#id-OS_version_found >>>>> >>>>> That being said, the original p0f fingerprints are very out of date, >>>>> and >>>>> it's possible that Bro will stop supporting p0f in the future. I did >>>>> some research on the fingerprints with the Windows XP end of life, and >>>>> ended up leveraging some of Bro's other capabilities to write a much >>>>> better detection: >>>>> >>>>> https://github.com/bro/bro/blob/master/scripts/policy/frameworks/software/windows-version-detection.bro >>>>> >>>>> Generally, I think the interest is in moving up the stack and >>>>> performing >>>>> this kind of fingerprinting at a higher, more reliable, layer. >>>>> >>>>> --Vlad >>>>> >>>>> Thomas Tan writes: >>>>> >>>>> > Dear Seth, >>>>> > >>>>> > Actually, I am writing a module using the outputs from Bro to detect >>>>> > Operating Systems running on remote host machines. I need to get the >>>>> > fingerprints of these OS for classification. I want to know if there >>>>> is any >>>>> > means to obtain p0f-like OS fingerprints. >>>>> > >>>>> > Best regards, >>>>> > >>>>> > Thomas >>>>> > >>>>> > On 2 December 2015 at 17:34, Seth Hall wrote: >>>>> > >>>>> >> >>>>> >> > On Dec 2, 2015, at 10:41 AM, Thomas Tan >>>>> wrote: >>>>> >> > >>>>> >> > It cannot get TCP options and the order of the options down from >>>>> a SYN >>>>> >> packet. >>>>> >> >>>>> >> It sounds like you might want to write your own plugin but it might >>>>> even >>>>> >> be possible that that?s not enough and you?d have to add a feature >>>>> to Bro?s >>>>> >> core to generate an event only for SYN packets. (although you >>>>> generally >>>>> >> have to be very careful about even generating an event for a single >>>>> packet). >>>>> >> >>>>> >> .Seth >>>>> >> >>>>> >> -- >>>>> >> Seth Hall >>>>> >> International Computer Science Institute >>>>> >> (Bro) because everyone has a network >>>>> >> http://www.bro.org/ >>>>> >> >>>>> >> >>>>> > _______________________________________________ >>>>> > Bro mailing list >>>>> > bro at bro-ids.org >>>>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>> >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Bro mailing list >>>> bro at bro-ids.org >>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>> >>>> >>> >>> >> > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151203/789983c2/attachment.html From daniel.guerra69 at gmail.com Fri Dec 4 01:20:45 2015 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Fri, 4 Dec 2015 10:20:45 +0100 Subject: [Bro] OpenSSL security issue affecting Bro In-Reply-To: <20151203200124.GA36339@wifi74.sys.ICSI.Berkeley.EDU> References: <20151203200124.GA36339@wifi74.sys.ICSI.Berkeley.EDU> Message-ID: I think the main distro?s are not ready yet! Just got : jessie/main openssl amd64 1.0.1k-3+deb8u1 q is still in testing. > On 03 Dec 2015, at 21:01, Johanna Amann wrote: > > Hello, > > The OpenSSL Project today published a security advisory, that affects > users of Bro that are using the X.509 certificate validation functionality > of Bro. Note that this functionality is not enabled by default - typically > it is enabled by either loading the policy script > protocols/ssl/validate-certs.bro or protocols/ssl/validate-ocsp.bro. > > The OpenSSL bug can cause a null-pointer exception when parsing certain > malformed X.509 certificates and can potentially be used for DOS attacks. > > The issue affects OpenSSL 1.0.1 and 1.0.2 and was fixed in OpenSSL 1.0.1q > and 1.0.2e respectively. If you use Bro and perform certificate > validation, you should update as soon as possible. > > The original OpenSSL security advisory is available at > https://www.openssl.org/news/secadv/20151203.txt. It also contains a few > other issues that are not directly applicable to Bro. > > Johanna > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151204/fe5fe6e9/attachment-0001.html From struck at ICSI.Berkeley.EDU Fri Dec 4 07:30:26 2015 From: struck at ICSI.Berkeley.EDU (Christian Struck) Date: Fri, 4 Dec 2015 16:30:26 +0100 Subject: [Bro] OpenSSL security issue affecting Bro In-Reply-To: References: <20151203200124.GA36339@wifi74.sys.ICSI.Berkeley.EDU> Message-ID: <5661B192.7000104@icsi.berkeley.edu> Hi Daniel, On 04.12.2015 10:20, Daniel Guerra wrote: > I think the main distro?s are not ready yet! Just got : > jessie/main openssl amd64 1.0.1k-3+deb8u1 > q is still in testing. on debian stable they've ported the patch back into 1.0.1k-3+deb8u2 see DSA 3413-1 Best regards Christian From vladg at illinois.edu Thu Dec 3 10:45:06 2015 From: vladg at illinois.edu (Vlad Grigorescu) Date: Thu, 03 Dec 2015 18:45:06 +0000 Subject: [Bro] SMTP File Hash not consistent Message-ID: de Bruyn, David [ multipart/mixed ] [ multipart/alternative ] [ text/plain ] Hi All! First off... I'm kinda new to Bro so please be gentle.... I've noticed some issues (strangeness?) with the file logging on Bro, in particular I would like for Bro to log an MD5 for all incomming files sent in through SMTP. At the moment it only seems to do it for some files and I can't seem to find a reason why some are getting hashed but others aren't... An extract from my files.log filtered by SMTP and pdf: 1449167625.904080 FS81ev 1449167623.516100 Fajnj71Xx6UprSmLef 198.22.115.26 208.33.144.195 C6pKQN2extOHQYZ4Fc SMTP 3 SHA1,MD5 application/pdf LoadTender3059527.pdf 0.015949 F T 57 - 1368 0 F - - - - - 1449167625.848077 FhU87R1PwGYciZcT2i 198.22.115.26 208.33.144.195 CkD4rQ1uG5VZhJL2v9 SMTP 1 SHA1,MD5 application/pdf 12.03.2015.pdf 0.016022 F T 456 - 1368 0 F - - - - -3MhA2vXGk5J8 198.22.115.26 208.33.145.195 CHB8Ew4kdUB3hDbkKl SMTP 3 SHA1,MD5 application/pdf Payment Advice Note from 12/03/2015.PDF 0.071983 F T 14535 - 0 0 F - ef853cc031d2abfbf6e0ec964163cd98 08eae5d275554f12d4783cb9c8be210d691f8db5 - - 1449167630.224049 FGUsvz3nDYqZlH56Y1 198.22.115.26 208.33.145.195 CK8Nwn4vGwpylAmpGj SMTP 3 SHA1,MD5 application/pdf PPC_LoadTender3057660.pdf 0.032006 F T 969 - 1544 0 F - - - - - 1449167631.024050 FiMmk5Zsczli9OGi7 198.22.115.26 208.33.144.195 CX4SUd3VDBBdYoXt0g SMTP 3 SHA1,MD5 application/pdf Payment Advice Note from 12/03/2015.PDF 0.011997 F T 171 - 1368 0 F - - - - - So basically about it won't create a file hash for a heap of file, then out of the blue it will create one, then no more for a while.... They all have the same mime type so I just can't seem to figure this out... any help or advice would be really appreciated... Cheers, David. The information in this email is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, re-transmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer. This email has been checked for viruses. However, JBS USA Holdings, Inc. and its constituent companies cannot accept responsibility for loss or damages arising from use of this email or attachments and we recommend that you subject these to your virus checking procedures prior to use. [ text/html (hidden) ] [ text/plain ] [ 4-line signature. Click/Enter to show. ] _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro Vlad Grigorescu (8 mins. ago) (inbox signed) Subject: Re: [Bro] SMTP File Hash not consistent To: "de Bruyn, David" Date: Fri, 04 Dec 2015 10:42:58 -0600 [ multipart/signed ] [ Good signature by: Vlad Grigorescu ] [ text/plain ] Hi David, "de Bruyn, David" writes: > 1449167625.904080 FS81ev3MhA2vXGk5J8 198.22.115.26 208.33.145.195 CHB8Ew4kdUB3hDbkKl SMTP 3 SHA1,MD5 application/pdf Payment Advice Note from 12/03/2015.PDF 0.071983 F T 14535 - 0 0 F - ef853cc031d2abfbf6e0ec964163cd98 08eae5d275554f12d4783cb9c8be210d691f8db5 > - - [ 5 more citation lines. Click/Enter to show. ] > 1449167623.516100 Fajnj71Xx6UprSmLef 198.22.115.26 208.33.144.195 C6pKQN2extOHQYZ4Fc SMTP 3 SHA1,MD5 application/pdf LoadTender3059527.pdf 0.015949 F T 57 - 1368 0 F - - - - - > 1449167625.848077 FhU87R1PwGYciZcT2i 198.22.115.26 208.33.144.195 CkD4rQ1uG5VZhJL2v9 SMTP 1 SHA1,MD5 application/pdf 12.03.2015.pdf 0.016022 F T 456 - 1368 0 F - - - - - > 1449167630.224049 FGUsvz3nDYqZlH56Y1 198.22.115.26 208.33.145.195 CK8Nwn4vGwpylAmpGj SMTP 3 SHA1,MD5 application/pdf PPC_LoadTender3057660.pdf 0.032006 F T 969 - 1544 0 F - - - - - > 1449167631.024050 FiMmk5Zsczli9OGi7 198.22.115.26 208.33.144.195 CX4SUd3VDBBdYoXt0g SMTP 3 SHA1,MD5 application/pdf Payment Advice Note from 12/03/2015.PDF 0.011997 F T 171 - 1368 0 F - - - - - > > So basically about it won't create a file hash for a heap of file, then out of the blue it will create one, then no more for a while.... > > They all have the same mime type so I just can't seem to figure this out... any help or advice would be really appreciated... It's a bit hard to see, but the important field is missing_bytes: > fuid missing_bytes has_hash? > FS81ev3MhA2vXGk5J8 0 T > Fajnj71Xx6UprSmLef 1368 F > FhU87R1PwGYciZcT2i 1368 F > FGUsvz3nDYqZlH56Y1 1544 F > FiMmk5Zsczli9OGi7 1368 F If Bro has detected that it didn't see some bytes in the file, it won't generate a hash (why bother hashing an imcomplete file?). The interesting question becomes why is Bro not seeing those bytes? I have some slides about how to verify and troubleshoot your deployment available here: https://speakerdeck.com/vladg/bro-deployment-verification-and-troubleshooting Let me know what you find, and if you have any further questions. --Vlad [ signature.asc: application/pgp-signature ] From johanna at icir.org Fri Dec 4 15:51:03 2015 From: johanna at icir.org (Johanna Amann) Date: Fri, 4 Dec 2015 15:51:03 -0800 Subject: [Bro] Uodate: OpenSSL security issue affecting Bro In-Reply-To: <20151203200124.GA36339@wifi74.sys.ICSI.Berkeley.EDU> References: <20151203200124.GA36339@wifi74.sys.ICSI.Berkeley.EDU> Message-ID: <20151204235103.GA95375@wifi74.sys.ICSI.Berkeley.EDU> Hello, we just posted an updated blog post describing the problem to http://blog.bro.org/2015/12/openssl-security-issue-affecting-bro.html. Please note, that different from the original descriptions, default installations of Bro that use broctl are vulnerable; a quick fix is to not load protocols/ssl/validate-certs.bro in local.bro. The blog post also contains instructions on how to test if your local openssl installation is vulnerable. Johanna On Thu, Dec 03, 2015 at 12:01:28PM -0800, Johanna Amann wrote: > Hello, > > The OpenSSL Project today published a security advisory, that affects > users of Bro that are using the X.509 certificate validation functionality > of Bro. Note that this functionality is not enabled by default - typically > it is enabled by either loading the policy script > protocols/ssl/validate-certs.bro or protocols/ssl/validate-ocsp.bro. > > The OpenSSL bug can cause a null-pointer exception when parsing certain > malformed X.509 certificates and can potentially be used for DOS attacks. > > The issue affects OpenSSL 1.0.1 and 1.0.2 and was fixed in OpenSSL 1.0.1q > and 1.0.2e respectively. If you use Bro and perform certificate > validation, you should update as soon as possible. > > The original OpenSSL security advisory is available at > https://www.openssl.org/news/secadv/20151203.txt. It also contains a few > other issues that are not directly applicable to Bro. > > Johanna > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From robin at icir.org Fri Dec 4 17:51:00 2015 From: robin at icir.org (Robin Sommer) Date: Fri, 4 Dec 2015 17:51:00 -0800 Subject: [Bro] Plugin regex issue In-Reply-To: References: Message-ID: <20151205015100.GF15001@icir.org> On Tue, Dec 01, 2015 at 15:38 -0500, Josh Liburdi wrote: > In the record above, when used as a plugin, the magic_cookie is > skipped. When compiled, it works as expected. Turns out it's a problem with the order in which the BinPAC system is initialized. I just pushed a fix for Bro to git, that seems to solve it. Give it a try. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From daniel.guerra69 at gmail.com Sat Dec 5 06:32:00 2015 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Sat, 5 Dec 2015 15:32:00 +0100 Subject: [Bro] Uodate: OpenSSL security issue affecting Bro In-Reply-To: <20151204235103.GA95375@wifi74.sys.ICSI.Berkeley.EDU> References: <20151203200124.GA36339@wifi74.sys.ICSI.Berkeley.EDU> <20151204235103.GA95375@wifi74.sys.ICSI.Berkeley.EDU> Message-ID: <3249B5A3-5514-4D16-A074-BD18D07270D9@gmail.com> Hi Johanna, My latest docker project has been fixed for this. I tried your test before and after the update en can confirm it works on debian. Thanx > On 05 Dec 2015, at 00:51, Johanna Amann wrote: > > Hello, > > we just posted an updated blog post describing the problem to > http://blog.bro.org/2015/12/openssl-security-issue-affecting-bro.html. > > Please note, that different from the original descriptions, default > installations of Bro that use broctl are vulnerable; a quick fix is to not > load protocols/ssl/validate-certs.bro in local.bro. > > The blog post also contains instructions on how to test if your local > openssl installation is vulnerable. > > Johanna > > On Thu, Dec 03, 2015 at 12:01:28PM -0800, Johanna Amann wrote: >> Hello, >> >> The OpenSSL Project today published a security advisory, that affects >> users of Bro that are using the X.509 certificate validation functionality >> of Bro. Note that this functionality is not enabled by default - typically >> it is enabled by either loading the policy script >> protocols/ssl/validate-certs.bro or protocols/ssl/validate-ocsp.bro. >> >> The OpenSSL bug can cause a null-pointer exception when parsing certain >> malformed X.509 certificates and can potentially be used for DOS attacks. >> >> The issue affects OpenSSL 1.0.1 and 1.0.2 and was fixed in OpenSSL 1.0.1q >> and 1.0.2e respectively. If you use Bro and perform certificate >> validation, you should update as soon as possible. >> >> The original OpenSSL security advisory is available at >> https://www.openssl.org/news/secadv/20151203.txt. It also contains a few >> other issues that are not directly applicable to Bro. >> >> Johanna >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From liburdi.joshua at gmail.com Sat Dec 5 07:52:00 2015 From: liburdi.joshua at gmail.com (Josh Liburdi) Date: Sat, 5 Dec 2015 10:52:00 -0500 Subject: [Bro] Plugin regex issue In-Reply-To: <20151205015100.GF15001@icir.org> References: <20151205015100.GF15001@icir.org> Message-ID: <379F31EE-E721-401C-BCF3-802433B07350@gmail.com> Thanks Robin, it looks like it fixed the issue I was having with this analyzer. I have another analyzer that primarily uses regex for the protocol parsing, so I?ll try that later and verify it works as well. Will the fix you pushed to git be available in the Bro 2.5 release, or will it be packaged and available sooner than that? Thanks! Josh > On Dec 4, 2015, at 8:51 PM, Robin Sommer wrote: > > > > On Tue, Dec 01, 2015 at 15:38 -0500, Josh Liburdi wrote: > >> In the record above, when used as a plugin, the magic_cookie is >> skipped. When compiled, it works as expected. > > Turns out it's a problem with the order in which the BinPAC system is > initialized. I just pushed a fix for Bro to git, that seems to solve > it. Give it a try. > > Robin > > -- > Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From liburdi.joshua at gmail.com Sat Dec 5 07:59:42 2015 From: liburdi.joshua at gmail.com (Josh Liburdi) Date: Sat, 5 Dec 2015 10:59:42 -0500 Subject: [Bro] Plugin regex issue In-Reply-To: <379F31EE-E721-401C-BCF3-802433B07350@gmail.com> References: <20151205015100.GF15001@icir.org> <379F31EE-E721-401C-BCF3-802433B07350@gmail.com> Message-ID: <53D99B6A-1496-4FF6-8A3A-006E42E72131@gmail.com> Well, later turned out to come a lot sooner than I thought. I tested it with my second analyzer plugin and all is well. Thanks again! Josh > On Dec 5, 2015, at 10:52 AM, Josh Liburdi wrote: > > Thanks Robin, it looks like it fixed the issue I was having with this analyzer. I have another analyzer that primarily uses regex for the protocol parsing, so I?ll try that later and verify it works as well. > > Will the fix you pushed to git be available in the Bro 2.5 release, or will it be packaged and available sooner than that? > > Thanks! > Josh > >> On Dec 4, 2015, at 8:51 PM, Robin Sommer wrote: >> >> >> >> On Tue, Dec 01, 2015 at 15:38 -0500, Josh Liburdi wrote: >> >>> In the record above, when used as a plugin, the magic_cookie is >>> skipped. When compiled, it works as expected. >> >> Turns out it's a problem with the order in which the BinPAC system is >> initialized. I just pushed a fix for Bro to git, that seems to solve >> it. Give it a try. >> >> Robin >> >> -- >> Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin > From aniketpsavanand at gmail.com Sun Dec 6 00:07:57 2015 From: aniketpsavanand at gmail.com (Aniket Savanand) Date: Sun, 6 Dec 2015 00:07:57 -0800 Subject: [Bro] How BRO's in-built scripts are invoked in a flow one by one(one file after other file) Message-ID: Hello, I am a student new to BRO. I am learning it by solving any simple use case. I am exploring practically how BRO's internal function works . I could not find any links to internal working( practically, like where/when/which files are invoked one by one in general) Please redirect me to appropriate papers, links, documentation or examples Thanks Aniket Savanand San Jose State University (669-226-8162) -- *Regards, * *Aniket Savanand,* *MS Software Engineering 2016,* *San Jose State University, CA* *Email **Cellphone- +1-669-226-8162* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151206/96694650/attachment.html From pratikinamdar at gmail.com Sun Dec 6 00:12:31 2015 From: pratikinamdar at gmail.com (pratik inamdar) Date: Sun, 6 Dec 2015 00:12:31 -0800 Subject: [Bro] Exploring bro, need practical exposure Message-ID: Hello, I am a student at SJSU and working on a project related to network security. I am new to Bro IDS. I would like to know which algorithms does bro used for network detection? If not how can I debug and compile a bro source code? I would like to know how can I debug code in bro too. I have downloaded and installed successfully the BRO IDS -- Thanks & Regards. Pratik Inamdar -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151206/1c7684cb/attachment.html From anthony.kasza at gmail.com Sun Dec 6 10:28:40 2015 From: anthony.kasza at gmail.com (anthony kasza) Date: Sun, 6 Dec 2015 10:28:40 -0800 Subject: [Bro] How BRO's in-built scripts are invoked in a flow one by one(one file after other file) In-Reply-To: References: Message-ID: If you look at Bro's main.cc file, you can see the two .bro script files it uses to bootstrap the rest of the policy scripts < https://github.com/bro/bro/blob/master/src/main.cc>. -AK On Dec 6, 2015 12:16 AM, "Aniket Savanand" wrote: > Hello, > > I am a student new to BRO. I am learning it by solving any simple use case. > > I am exploring practically how BRO's internal function works . > > I could not find any links to internal working( practically, like > where/when/which files are invoked one by one in general) > > Please redirect me to appropriate papers, links, documentation or examples > > Thanks > Aniket Savanand > San Jose State University > (669-226-8162) > > > > -- > *Regards, * > *Aniket Savanand,* > *MS Software Engineering 2016,* > *San Jose State University, CA* > *Email **Cellphone- +1-669-226-8162 > <%2B1-669-226-8162>* > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151206/71118982/attachment.html From aniketpsavanand at gmail.com Sun Dec 6 17:07:03 2015 From: aniketpsavanand at gmail.com (Aniket Savanand) Date: Sun, 6 Dec 2015 17:07:03 -0800 Subject: [Bro] How BRO's in-built scripts are invoked in a flow one by one(one file after other file) In-Reply-To: References: Message-ID: Thanks for the reply. I got init-bare.bro and init-default.bro. Is there any way to see how these init-bare.bro and init-default.bro works stage by stage? I want to see how code gets run stepwise. Thanks Aniket Savanand San Jose State University (669-226-8162) On Sun, Dec 6, 2015 at 10:28 AM, anthony kasza wrote: > If you look at Bro's main.cc file, you can see the two .bro script files > it uses to bootstrap the rest of the policy scripts < > https://github.com/bro/bro/blob/master/src/main.cc>. > > -AK > On Dec 6, 2015 12:16 AM, "Aniket Savanand" > wrote: > >> Hello, >> >> I am a student new to BRO. I am learning it by solving any simple use >> case. >> >> I am exploring practically how BRO's internal function works . >> >> I could not find any links to internal working( practically, like >> where/when/which files are invoked one by one in general) >> >> Please redirect me to appropriate papers, links, documentation or >> examples >> >> Thanks >> Aniket Savanand >> San Jose State University >> (669-226-8162) >> >> >> >> -- >> *Regards, * >> *Aniket Savanand,* >> *MS Software Engineering 2016,* >> *San Jose State University, CA* >> *Email **Cellphone- +1-669-226-8162 >> <%2B1-669-226-8162>* >> >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > -- *Regards, * *Aniket Savanand,* *MS Software Engineering 2016,* *San Jose State University, CA* *Email **Cellphone- +1-669-226-8162* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151206/1945d113/attachment.html From anthony.kasza at gmail.com Sun Dec 6 17:52:54 2015 From: anthony.kasza at gmail.com (anthony kasza) Date: Sun, 6 Dec 2015 17:52:54 -0800 Subject: [Bro] How BRO's in-built scripts are invoked in a flow one by one(one file after other file) In-Reply-To: References: Message-ID: You could try removing @load statements from those files one by one and see what happens. -AK On Dec 6, 2015 5:07 PM, "Aniket Savanand" wrote: > Thanks for the reply. > > I got init-bare.bro and init-default.bro. > > Is there any way to see how these init-bare.bro and init-default.bro works > stage by stage? > > I want to see how code gets run stepwise. > > Thanks > Aniket Savanand > San Jose State University > (669-226-8162) > > > > On Sun, Dec 6, 2015 at 10:28 AM, anthony kasza > wrote: > >> If you look at Bro's main.cc file, you can see the two .bro script files >> it uses to bootstrap the rest of the policy scripts < >> https://github.com/bro/bro/blob/master/src/main.cc>. >> >> -AK >> On Dec 6, 2015 12:16 AM, "Aniket Savanand" >> wrote: >> >>> Hello, >>> >>> I am a student new to BRO. I am learning it by solving any simple use >>> case. >>> >>> I am exploring practically how BRO's internal function works . >>> >>> I could not find any links to internal working( practically, like >>> where/when/which files are invoked one by one in general) >>> >>> Please redirect me to appropriate papers, links, documentation or >>> examples >>> >>> Thanks >>> Aniket Savanand >>> San Jose State University >>> (669-226-8162) >>> >>> >>> >>> -- >>> *Regards, * >>> *Aniket Savanand,* >>> *MS Software Engineering 2016,* >>> *San Jose State University, CA* >>> *Email **Cellphone- +1-669-226-8162 >>> <%2B1-669-226-8162>* >>> >>> >>> >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> >> > > > -- > *Regards, * > *Aniket Savanand,* > *MS Software Engineering 2016,* > *San Jose State University, CA* > *Email **Cellphone- +1-669-226-8162 > <%2B1-669-226-8162>* > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151206/d9712c98/attachment.html From gc355804 at ohio.edu Sun Dec 6 20:51:14 2015 From: gc355804 at ohio.edu (Clark, Gilbert) Date: Mon, 7 Dec 2015 04:51:14 +0000 Subject: [Bro] How BRO's in-built scripts are invoked in a flow one by one(one file after other file) In-Reply-To: References: , Message-ID: In addition to what Anthony suggests: Bro has an option to trace execution and write the results to a file:?I think it's '-T' or something along those lines. ?The trace file generated by running bro with this option?can?show you which script functions were called and in which order they were called ... but this option generates a *lot* of output, and should therefore only be used offline and (probably) with a relatively small capture file. There's a benchmark script that ships with bro that also shows an example of incrementally running bro with 1 script loaded, 2 scripts loaded, etc to see how each script affects bro's runtime:?https://github.com/bro/bro-aux/blob/master/devel-tools/cpu-bench-with-trace Also, maybe try taking a look at try.bro.org: it's a pretty nice way to play with bro and become familiar with how things work. Cheers, Gilbert From aniketpsavanand at gmail.com Sun Dec 6 20:57:24 2015 From: aniketpsavanand at gmail.com (Aniket Savanand) Date: Sun, 6 Dec 2015 20:57:24 -0800 Subject: [Bro] How BRO's in-built scripts are invoked in a flow one by one(one file after other file) In-Reply-To: References: Message-ID: Thanks a lot. I will look into these files. Thanks Aniket Savanand On Sun, Dec 6, 2015 at 8:51 PM, Clark, Gilbert wrote: > In addition to what Anthony suggests: > > Bro has an option to trace execution and write the results to a file: I > think it's '-T' or something along those lines. The trace file generated > by running bro with this option can show you which script functions were > called and in which order they were called ... but this option generates a > *lot* of output, and should therefore only be used offline and (probably) > with a relatively small capture file. > > There's a benchmark script that ships with bro that also shows an example > of incrementally running bro with 1 script loaded, 2 scripts loaded, etc to > see how each script affects bro's runtime: > https://github.com/bro/bro-aux/blob/master/devel-tools/cpu-bench-with-trace > > Also, maybe try taking a look at try.bro.org: it's a pretty nice way to > play with bro and become familiar with how things work. > > Cheers, > Gilbert > -- *Regards, * *Aniket Savanand,* *MS Software Engineering 2016,* *San Jose State University, CA* *Email **Cellphone- +1-669-226-8162* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151206/1e43a87a/attachment-0001.html From sangeenjan at gmail.com Tue Dec 8 04:31:03 2015 From: sangeenjan at gmail.com (Sangeen Khan) Date: Tue, 8 Dec 2015 17:31:03 +0500 Subject: [Bro] Bro Digest, Vol 116, Issue 12 In-Reply-To: References: Message-ID: Dear Sir, I am trying to study and analysis the logs that generating against the attack. As there is a DoS attack hping3 i am trying to study logs that generating against this attack. The bro server is is deployed in the our Network and i am attacking from from then same network on the same network node . the particular kali linux command is following hping3 -S -a 172.20.16.105 --flood -p 80 172.20.16.74 Where i can find these logs that could generate against this attack??? On Sun, Dec 6, 2015 at 1:00 AM, wrote: > Send Bro mailing list submissions to > bro at bro.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > or, via email, send a message with subject or body 'help' to > bro-request at bro.org > > You can reach the person managing the list at > bro-owner at bro.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Bro digest..." > > > Today's Topics: > > 1. Uodate: OpenSSL security issue affecting Bro (Johanna Amann) > 2. Re: Plugin regex issue (Robin Sommer) > 3. Re: Uodate: OpenSSL security issue affecting Bro (Daniel Guerra) > 4. Re: Plugin regex issue (Josh Liburdi) > 5. Re: Plugin regex issue (Josh Liburdi) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 4 Dec 2015 15:51:03 -0800 > From: Johanna Amann > Subject: [Bro] Uodate: OpenSSL security issue affecting Bro > To: bro at bro.org > Message-ID: <20151204235103.GA95375 at wifi74.sys.ICSI.Berkeley.EDU> > Content-Type: text/plain; charset=us-ascii > > Hello, > > we just posted an updated blog post describing the problem to > http://blog.bro.org/2015/12/openssl-security-issue-affecting-bro.html. > > Please note, that different from the original descriptions, default > installations of Bro that use broctl are vulnerable; a quick fix is to not > load protocols/ssl/validate-certs.bro in local.bro. > > The blog post also contains instructions on how to test if your local > openssl installation is vulnerable. > > Johanna > > On Thu, Dec 03, 2015 at 12:01:28PM -0800, Johanna Amann wrote: > > Hello, > > > > The OpenSSL Project today published a security advisory, that affects > > users of Bro that are using the X.509 certificate validation > functionality > > of Bro. Note that this functionality is not enabled by default - > typically > > it is enabled by either loading the policy script > > protocols/ssl/validate-certs.bro or protocols/ssl/validate-ocsp.bro. > > > > The OpenSSL bug can cause a null-pointer exception when parsing certain > > malformed X.509 certificates and can potentially be used for DOS attacks. > > > > The issue affects OpenSSL 1.0.1 and 1.0.2 and was fixed in OpenSSL 1.0.1q > > and 1.0.2e respectively. If you use Bro and perform certificate > > validation, you should update as soon as possible. > > > > The original OpenSSL security advisory is available at > > https://www.openssl.org/news/secadv/20151203.txt. It also contains a few > > other issues that are not directly applicable to Bro. > > > > Johanna > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > > > ------------------------------ > > Message: 2 > Date: Fri, 4 Dec 2015 17:51:00 -0800 > From: Robin Sommer > Subject: Re: [Bro] Plugin regex issue > To: Josh Liburdi > Cc: bro at bro.org > Message-ID: <20151205015100.GF15001 at icir.org> > Content-Type: text/plain; charset=us-ascii > > > > On Tue, Dec 01, 2015 at 15:38 -0500, Josh Liburdi wrote: > > > In the record above, when used as a plugin, the magic_cookie is > > skipped. When compiled, it works as expected. > > Turns out it's a problem with the order in which the BinPAC system is > initialized. I just pushed a fix for Bro to git, that seems to solve > it. Give it a try. > > Robin > > -- > Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin > > > ------------------------------ > > Message: 3 > Date: Sat, 5 Dec 2015 15:32:00 +0100 > From: Daniel Guerra > Subject: Re: [Bro] Uodate: OpenSSL security issue affecting Bro > To: Johanna Amann > Cc: bro at bro.org > Message-ID: <3249B5A3-5514-4D16-A074-BD18D07270D9 at gmail.com> > Content-Type: text/plain; charset=us-ascii > > Hi Johanna, > > My latest docker project has been fixed for this. I tried your test before > and after the update en can confirm it works on debian. > > Thanx > > > On 05 Dec 2015, at 00:51, Johanna Amann wrote: > > > > Hello, > > > > we just posted an updated blog post describing the problem to > > http://blog.bro.org/2015/12/openssl-security-issue-affecting-bro.html. > > > > Please note, that different from the original descriptions, default > > installations of Bro that use broctl are vulnerable; a quick fix is to > not > > load protocols/ssl/validate-certs.bro in local.bro. > > > > The blog post also contains instructions on how to test if your local > > openssl installation is vulnerable. > > > > Johanna > > > > On Thu, Dec 03, 2015 at 12:01:28PM -0800, Johanna Amann wrote: > >> Hello, > >> > >> The OpenSSL Project today published a security advisory, that affects > >> users of Bro that are using the X.509 certificate validation > functionality > >> of Bro. Note that this functionality is not enabled by default - > typically > >> it is enabled by either loading the policy script > >> protocols/ssl/validate-certs.bro or protocols/ssl/validate-ocsp.bro. > >> > >> The OpenSSL bug can cause a null-pointer exception when parsing certain > >> malformed X.509 certificates and can potentially be used for DOS > attacks. > >> > >> The issue affects OpenSSL 1.0.1 and 1.0.2 and was fixed in OpenSSL > 1.0.1q > >> and 1.0.2e respectively. If you use Bro and perform certificate > >> validation, you should update as soon as possible. > >> > >> The original OpenSSL security advisory is available at > >> https://www.openssl.org/news/secadv/20151203.txt. It also contains a > few > >> other issues that are not directly applicable to Bro. > >> > >> Johanna > >> _______________________________________________ > >> Bro mailing list > >> bro at bro-ids.org > >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > >> > >> > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > ------------------------------ > > Message: 4 > Date: Sat, 5 Dec 2015 10:52:00 -0500 > From: Josh Liburdi > Subject: Re: [Bro] Plugin regex issue > To: Robin Sommer > Cc: bro at bro.org > Message-ID: <379F31EE-E721-401C-BCF3-802433B07350 at gmail.com> > Content-Type: text/plain; charset=utf-8 > > Thanks Robin, it looks like it fixed the issue I was having with this > analyzer. I have another analyzer that primarily uses regex for the > protocol parsing, so I?ll try that later and verify it works as well. > > Will the fix you pushed to git be available in the Bro 2.5 release, or > will it be packaged and available sooner than that? > > Thanks! > Josh > > > On Dec 4, 2015, at 8:51 PM, Robin Sommer wrote: > > > > > > > > On Tue, Dec 01, 2015 at 15:38 -0500, Josh Liburdi wrote: > > > >> In the record above, when used as a plugin, the magic_cookie is > >> skipped. When compiled, it works as expected. > > > > Turns out it's a problem with the order in which the BinPAC system is > > initialized. I just pushed a fix for Bro to git, that seems to solve > > it. Give it a try. > > > > Robin > > > > -- > > Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin > > > > > ------------------------------ > > Message: 5 > Date: Sat, 5 Dec 2015 10:59:42 -0500 > From: Josh Liburdi > Subject: Re: [Bro] Plugin regex issue > To: Robin Sommer > Cc: bro at bro.org > Message-ID: <53D99B6A-1496-4FF6-8A3A-006E42E72131 at gmail.com> > Content-Type: text/plain; charset=utf-8 > > Well, later turned out to come a lot sooner than I thought. I tested it > with my second analyzer plugin and all is well. Thanks again! > > Josh > > > On Dec 5, 2015, at 10:52 AM, Josh Liburdi > wrote: > > > > Thanks Robin, it looks like it fixed the issue I was having with this > analyzer. I have another analyzer that primarily uses regex for the > protocol parsing, so I?ll try that later and verify it works as well. > > > > Will the fix you pushed to git be available in the Bro 2.5 release, or > will it be packaged and available sooner than that? > > > > Thanks! > > Josh > > > >> On Dec 4, 2015, at 8:51 PM, Robin Sommer wrote: > >> > >> > >> > >> On Tue, Dec 01, 2015 at 15:38 -0500, Josh Liburdi wrote: > >> > >>> In the record above, when used as a plugin, the magic_cookie is > >>> skipped. When compiled, it works as expected. > >> > >> Turns out it's a problem with the order in which the BinPAC system is > >> initialized. I just pushed a fix for Bro to git, that seems to solve > >> it. Give it a try. > >> > >> Robin > >> > >> -- > >> Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin > > > > > > > ------------------------------ > > _______________________________________________ > Bro mailing list > Bro at bro.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > End of Bro Digest, Vol 116, Issue 12 > ************************************ > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151208/191fb7fd/attachment.html From jan.muthreich at consistec.de Tue Dec 8 07:35:24 2015 From: jan.muthreich at consistec.de (Jan Muthreich) Date: Tue, 8 Dec 2015 15:35:24 +0000 Subject: [Bro] Scheduling events are immediatly executed Message-ID: <853865C4C3106E4DA43737956F687BAA01126BB3@ex2010.ads.consistec.de> Hello, I'm working with BRO and have a problem. I want to use scheduling but it doesn't seem to work. Alle tests, that I wrote, are immidiatly ready. I have seen the ticket https://bro-tracker.atlassian.net/browse/BIT-747 and have tried to reschedule. But it doesn't work. The rescheduled event is either missing, when no files are read, or immidiatly ready when files are read. Have someone tipps for me or is there any documentation, how the scheduling and eventing are work in BRO? For example this is the test I use: #@TEST-EXEC: bro -b -C -r $TRACES/10000.pcapng %INPUT > output 2> output.err #@TEST-EXEC: test -f output #@TEST-EXEC: btest-diff output #@TEST-EXEC: test -f output.err #@TEST-EXEC: btest-diff output.err event e2() { print "e2"; } function scheduleEvent(){ print "f1"; schedule 100sec { e2() }; } event e1() { print "e1"; scheduleEvent(); } event bro_init() { schedule 100sec { e1() }; } The trace file can be any pcap file. consistec Engineering & Consulting GmbH Jan Muthreich - Software Engineer From robin at icir.org Tue Dec 8 08:23:03 2015 From: robin at icir.org (Robin Sommer) Date: Tue, 8 Dec 2015 08:23:03 -0800 Subject: [Bro] Scheduling events are immediatly executed In-Reply-To: <853865C4C3106E4DA43737956F687BAA01126BB3@ex2010.ads.consistec.de> References: <853865C4C3106E4DA43737956F687BAA01126BB3@ex2010.ads.consistec.de> Message-ID: <20151208162303.GK3718@icir.org> On Tue, Dec 08, 2015 at 15:35 +0000, Jan Muthreich wrote: > I'm working with BRO and have a problem. I want to use scheduling but > it doesn't seem to work. Alle tests, that I wrote, are immidiatly > ready. One thing to keep in mind for schedule is that it's relative to "network time", i.e., the packet timestamps in the trace. When you say 100s, it's not going to wait for 100s of wall clock time to pass, but will trigger the event once the packet timestmaps have covered 100s. When working offline from a trace, like in your case, that often feels like "immediately" if the input is short. Could that be it? Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From sangeenjan at gmail.com Tue Dec 8 12:20:29 2015 From: sangeenjan at gmail.com (Sangeen Khan) Date: Wed, 9 Dec 2015 01:20:29 +0500 Subject: [Bro] Bro Digest Message-ID: I am trying to study and analysis the logs that generating against the attack. As there is a DoS attack hping3 i am trying to study logs that generating against this attack. The bro server is is deployed in the our Network and i am attacking from the same network on the same network node . the particular kali linux command is following hping3 -S -a 172.X.X.X --flood -p 80 172.X.X.X Where i can find these logs that will generate against this attack??? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151209/f04a2c6f/attachment.html From jan.muthreich at consistec.de Tue Dec 8 23:51:42 2015 From: jan.muthreich at consistec.de (Jan Muthreich) Date: Wed, 9 Dec 2015 07:51:42 +0000 Subject: [Bro] Scheduling events are immediatly executed In-Reply-To: <20151208162303.GK3718@icir.org> References: <853865C4C3106E4DA43737956F687BAA01126BB3@ex2010.ads.consistec.de> <20151208162303.GK3718@icir.org> Message-ID: <853865C4C3106E4DA43737956F687BAA01126BE3@ex2010.ads.consistec.de> Thank you. I have an Input READER_ASCII in use, which need Input::force_update. It reads from a linux pipe. How can we schedule this operation if no network traffic is in the line? Mit freundlichen Gr??en Jan Muthreich -----Original Message----- From: Robin Sommer [mailto:robin at icir.org] Sent: Tuesday, December 8, 2015 5:23 PM To: Jan Muthreich Cc: bro at bro.org Subject: Re: [Bro] Scheduling events are immediatly executed On Tue, Dec 08, 2015 at 15:35 +0000, Jan Muthreich wrote: > I'm working with BRO and have a problem. I want to use scheduling but > it doesn't seem to work. Alle tests, that I wrote, are immidiatly > ready. One thing to keep in mind for schedule is that it's relative to "network time", i.e., the packet timestamps in the trace. When you say 100s, it's not going to wait for 100s of wall clock time to pass, but will trigger the event once the packet timestmaps have covered 100s. When working offline from a trace, like in your case, that often feels like "immediately" if the input is short. Could that be it? Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jeff61225 at gmail.com Wed Dec 9 20:44:08 2015 From: jeff61225 at gmail.com (Jeff H) Date: Wed, 9 Dec 2015 20:44:08 -0800 Subject: [Bro] Checking old Bro logs for new intel? Message-ID: Does Bro have a way to check old Bro logs for newly updated Intel? I read through the man page as well as the documentation for the Intel framework and didn't see anything like this. If this isn't supported, do anyone have any scripts they could share that do this? Jeff -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151209/14c5a8a0/attachment.html From hlin33 at illinois.edu Wed Dec 9 21:31:40 2015 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Wed, 9 Dec 2015 23:31:40 -0600 Subject: [Bro] udp event handlers not catching events Message-ID: Hi, I am analyzing a pcap which contains some UDP packets. I have redefined both "udp_content_deliver_all_orig" and "udp_content_deliver_all_resp" as true, but no events are caught by "udp_request", "upd_reply", and "udp_contents". However, I can use "packets_content" and "is_udp_port" to catch the udp communications. Can these udp event handlers still be used? Thanks and best, Hui Lin -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151209/f8d37d8d/attachment.html From thomastan81 at gmail.com Thu Dec 10 06:40:36 2015 From: thomastan81 at gmail.com (Thomas Tan) Date: Thu, 10 Dec 2015 15:40:36 +0100 Subject: [Bro] TCP options of a SYN packet In-Reply-To: References: Message-ID: Dear Daniel and Jame, Thank you so much for your comments and advice. I had been away for a quite few days and just got time to check the emails. I will try to write my simple analyzer and will come back to you later. Best regards, Thomas On 4 December 2015 at 01:07, James Swaro wrote: > > I agree TCPRS is not the place. > > Agreed. I would like TCPRS to be useful in this case, but the feature > falls out of the scope of design of TCPRS. > > It might not be difficult to write a simple analyzer that looks only at > syn packets and produces the events that Thomas is looking to generate. A > simple analyzer may not need to retain packets like TCPRS does and would > have a much faster execution and lower memory footprint because it doesn't > need to do retrospective analysis on the connection. > > James Swaro > > > > On Thu, Dec 3, 2015 at 5:16 PM, Daniel Guerra > wrote: > >> I agree TCPRS is not the place. >> When i look at the analyzer TCP.cc , i miss >> some options available from /usr/include/netinet/tcp.h >> >> # define TCPOPT_EOL 0 >> # define TCPOPT_NOP 1 >> # define TCPOPT_MAXSEG 2 >> # define TCPOLEN_MAXSEG 4 >> # define TCPOPT_WINDOW 3 >> # define TCPOLEN_WINDOW 3 >> # define TCPOPT_SACK_PERMITTED 4 /* Experimental */ >> # define TCPOLEN_SACK_PERMITTED 2 >> # define TCPOPT_SACK 5 /* Experimental */ >> # define TCPOPT_TIMESTAMP 8 >> # define TCPOLEN_TIMESTAMP 10 >> # define TCPOLEN_TSTAMP_APPA (TCPOLEN_TIMESTAMP+2) /* appendix A */ >> >> It would be nice to have them available when needed. OS fingerprint can >> be done >> on how it was constructed, the order is free and each kernel treats the >> rules different. >> This can tell you what type of os you see, but some are universal. >> But if you want to detect two or more devices and want to relate the >> traffic to the device, >> you really need the timestamp. >> It could also provide a method to detect cooked packets or a virus that >> creates its own packets. >> And I think it is actually very cheap because you are sure you only get 1 >> syn (beside retrans) >> per connection and that is way less than HTTP for example. >> >> Regards, >> >> Daniel >> >> On 03 Dec 2015, at 20:14, James Swaro wrote: >> >> As you noted already, TCPRS doesn't provide that type of behavior. The >> options of interest to TCPRS are limited and only reported via the >> TCPRS::conn_config event, which certainly does not fit your needs. That >> said, extending the TCPRS plugin, or the TCP analyzer within bro doesn't >> seem to be something that is out of the realm of possibility here. >> >> As Seth mentioned above, generating an event for every syn is potentially >> expensive. SYN retransmissions may generate duplicate records. >> >> You could create a new event in the source for syn options, but I'm not >> aware of any bro constructs that would (easily) allow for providing the >> options in the order they are observed in the header(Seth,Vlad,Daniel - >> please chime in here). It might be possible to send (ID, value) tuples to >> the script with a new event, given that the tuples are inserted into a >> vector in the order they are observed in the header. Once in the script, >> you could convert the ID (option identifier) into the string representation >> and create a single string that could easily be parsed. >> >> Example record: >> > 10.0.0.1 2000 10.0.0.2 2001 TCPOPT_MAXSEG=3,TCPOPT_WSCALE=1 >> >> Obviously, the output format is dictated by the bro script, so that isn't >> terribly important here since that would be up to your design. >> >> I'm sure you are already aware of issues with fingerprinting, but some >> values might differ within the same OS depending if the kernel has been >> modified or kernel config options have been modified from their default >> values. >> >> If this type of approach interests you, let me know. >> >> >> James Swaro >> >> >> >> On Thu, Dec 3, 2015 at 4:36 AM, Thomas Tan wrote: >> >>> Hi James, >>> >>> Thanks for you reply. You are right. I am looking into OS specific >>> behaviors. How can I use your plug-in to get TCP options from a SYN packet? >>> >>> Your help will be very much appreciated. >>> >>> Best regards, >>> >>> Thomas >>> >>> On 3 December 2015 at 05:49, James Swaro wrote: >>> >>>> I'm sorry I didn't see this earlier. I'm curious. Why is the order of >>>> the options important? Are you searching for OS specific behavior? >>>> >>>> James Swaro >>>> >>>> >>>> >>>> On Wed, Dec 2, 2015 at 4:29 PM, Thomas Tan >>>> wrote: >>>> >>>>> Hi Vlad, >>>>> >>>>> Thanks for your reply. >>>>> >>>>> I am aware of the support of p0f in Bro. You are right. The original >>>>> p0f v2 fingerprints are out-dated. In my work, I am not using the p0f v2 >>>>> fingerprints but collecting OS fingerprints from network connections >>>>> initiated by remote machines. A multi-class classifier will be applied to >>>>> assign these OS fingerprints to their respective OS types. >>>>> >>>>> Best regards, >>>>> >>>>> Thomas >>>>> >>>>> On 2 December 2015 at 22:26, Vlad Grigorescu >>>>> wrote: >>>>> >>>>>> Thomas, >>>>>> >>>>>> Bro has p0f support built-in. See: >>>>>> >>>>>> https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html#id-OS_version_found >>>>>> >>>>>> That being said, the original p0f fingerprints are very out of date, >>>>>> and >>>>>> it's possible that Bro will stop supporting p0f in the future. I did >>>>>> some research on the fingerprints with the Windows XP end of life, and >>>>>> ended up leveraging some of Bro's other capabilities to write a much >>>>>> better detection: >>>>>> >>>>>> https://github.com/bro/bro/blob/master/scripts/policy/frameworks/software/windows-version-detection.bro >>>>>> >>>>>> Generally, I think the interest is in moving up the stack and >>>>>> performing >>>>>> this kind of fingerprinting at a higher, more reliable, layer. >>>>>> >>>>>> --Vlad >>>>>> >>>>>> Thomas Tan writes: >>>>>> >>>>>> > Dear Seth, >>>>>> > >>>>>> > Actually, I am writing a module using the outputs from Bro to detect >>>>>> > Operating Systems running on remote host machines. I need to get the >>>>>> > fingerprints of these OS for classification. I want to know if >>>>>> there is any >>>>>> > means to obtain p0f-like OS fingerprints. >>>>>> > >>>>>> > Best regards, >>>>>> > >>>>>> > Thomas >>>>>> > >>>>>> > On 2 December 2015 at 17:34, Seth Hall wrote: >>>>>> > >>>>>> >> >>>>>> >> > On Dec 2, 2015, at 10:41 AM, Thomas Tan >>>>>> wrote: >>>>>> >> > >>>>>> >> > It cannot get TCP options and the order of the options down >>>>>> from a SYN >>>>>> >> packet. >>>>>> >> >>>>>> >> It sounds like you might want to write your own plugin but it >>>>>> might even >>>>>> >> be possible that that?s not enough and you?d have to add a feature >>>>>> to Bro?s >>>>>> >> core to generate an event only for SYN packets. (although you >>>>>> generally >>>>>> >> have to be very careful about even generating an event for a >>>>>> single packet). >>>>>> >> >>>>>> >> .Seth >>>>>> >> >>>>>> >> -- >>>>>> >> Seth Hall >>>>>> >> International Computer Science Institute >>>>>> >> (Bro) because everyone has a network >>>>>> >> http://www.bro.org/ >>>>>> >> >>>>>> >> >>>>>> > _______________________________________________ >>>>>> > Bro mailing list >>>>>> > bro at bro-ids.org >>>>>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>>> >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Bro mailing list >>>>> bro at bro-ids.org >>>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>>>> >>>>> >>>> >>>> >>> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151210/92e0aec3/attachment-0001.html From dirk.leinenbach at consistec.de Thu Dec 10 08:21:22 2015 From: dirk.leinenbach at consistec.de (Dirk Leinenbach) Date: Thu, 10 Dec 2015 17:21:22 +0100 Subject: [Bro] Scheduling events are immediatly executed In-Reply-To: <853865C4C3106E4DA43737956F687BAA01126BE3@ex2010.ads.consistec.de> References: <853865C4C3106E4DA43737956F687BAA01126BB3@ex2010.ads.consistec.de> <20151208162303.GK3718@icir.org> <853865C4C3106E4DA43737956F687BAA01126BE3@ex2010.ads.consistec.de> Message-ID: <5669A682.4040606@consistec.de> Hi all, is Bro's event schedule depending on "input" from the network / trace file? I.e., does it stop processing ASCII reader input if there's no more network activity? This might not be a big deal in production rollouts when bro is listing to real network interfaces, but in test scenarios (with btest) it looks to me as if bro stops processing other input, once the pcap files have been consumed completely. Is there any work around / best practice on how such situations can be handled in tests? Thanks for your help! Dirk On 09.12.2015 08:51, Jan Muthreich wrote: > Thank you. I have an Input READER_ASCII in use, which need Input::force_update. It reads from a linux pipe. How can we schedule this operation if no network traffic is in the line? > > Mit freundlichen Gr??en > Jan Muthreich > > -----Original Message----- > From: Robin Sommer [mailto:robin at icir.org] > Sent: Tuesday, December 8, 2015 5:23 PM > To: Jan Muthreich > Cc: bro at bro.org > Subject: Re: [Bro] Scheduling events are immediatly executed > > > > On Tue, Dec 08, 2015 at 15:35 +0000, Jan Muthreich wrote: > >> I'm working with BRO and have a problem. I want to use scheduling but >> it doesn't seem to work. Alle tests, that I wrote, are immidiatly >> ready. > One thing to keep in mind for schedule is that it's relative to "network time", i.e., the packet timestamps in the trace. When you say 100s, it's not going to wait for 100s of wall clock time to pass, but will trigger the event once the packet timestmaps have covered 100s. > When working offline from a trace, like in your case, that often feels like "immediately" if the input is short. Could that be it? > > Robin > > -- > Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Dr.-Ing. Dirk Leinenbach - Leitung Softwareentwicklung consistec Engineering & Consulting GmbH ------------------------------------------------------------------ Europaallee 5 Fon: +49 (0)681 / 959044-0 D-66113 Saarbr?cken Fax: +49 (0)681 / 959044-11 http://www.consistec.de e-mail: dirk.leinenbach at consistec.de Registergericht: Amtsgericht Saarbr?cken Registerblatt: HRB12003 Gesch?ftsf?hrer: Dr. Thomas Sinnwell, Volker Leiendecker, Stefan Sinnwell From jdopheid at illinois.edu Thu Dec 10 08:33:42 2015 From: jdopheid at illinois.edu (Dopheide, Jeannette M) Date: Thu, 10 Dec 2015 16:33:42 +0000 Subject: [Bro] Announcing new grant from Mozilla Message-ID: We have some exciting news! Bro has received a $200,000 grant from Mozilla to develop the Comprehensive Bro Archive Network (CBAN). To learn more about CBAN and the award, check out our blog post: http://blog.bro.org/2015/12/bro-receives-200k-grant-from-mozilla.html ------ Jeannette Dopheide Bro Outreach Coordinator National Center for Supercomputing Applications University of Illinois at Urbana-Champaign From robin at icir.org Thu Dec 10 12:33:30 2015 From: robin at icir.org (Robin Sommer) Date: Thu, 10 Dec 2015 12:33:30 -0800 Subject: [Bro] Scheduling events are immediatly executed In-Reply-To: <5669A682.4040606@consistec.de> References: <853865C4C3106E4DA43737956F687BAA01126BB3@ex2010.ads.consistec.de> <20151208162303.GK3718@icir.org> <853865C4C3106E4DA43737956F687BAA01126BE3@ex2010.ads.consistec.de> <5669A682.4040606@consistec.de> Message-ID: <20151210203330.GF69387@icir.org> On Thu, Dec 10, 2015 at 17:21 +0100, Dirk Leinenbach wrote: > Is there any work around / best practice on how such situations can be > handled in tests? Bro normally terminates once it has read the whole trace. You can prevent that by setting exit_only_after_terminate to true; then it will keep running (and proecessing other input) until you call the built-in function terminate(). You can see this in the input framework tests in testing/btest/scripts/base/frameworks/input. The other piece to this (and the original question) is that it's a fundamental challenge to process a trace while also doing things that happen in real-time (input framework, but also any communication with external processes). Bro can read a trace much more quickly than it would normally process the same traffic during live operation, meaning it will get out of sync with any other activity still happening "just" in real time. There's a work-around for testing purposes: Bro has a switch "--pseudo-realtime" that articifially delays processing a trace: after reading each packet, it inserts a delay corresponding to the timestamp gap to the next packet. In that way, it "simulates" real-time processing by not getting head of what a live Bro would do. For example, the test testing/btest//scripts/policy/protocols/ssl/validate-certs-cluster.bro uses this to set up a Bro cluster of multiple processes for X509 cert validation while reading the input from a trace. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From tlarson.hiscorp at gmail.com Fri Dec 11 10:52:58 2015 From: tlarson.hiscorp at gmail.com (Tim Larson) Date: Fri, 11 Dec 2015 12:52:58 -0600 Subject: [Bro] Transferring logs from bro Message-ID: Questions: What is the mechanism and commands within bro for scheduling the periodic transfer of conn.logs, protocol logs and notice.logs from each of a number of remotely distributed bro deployments to a central postgresql application running in a cloud service like AWS using a outbound port 443 connection? Can the scheduling of the bro log files be based on time and/or a specific log volume threshold being reached? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151211/9438a5e3/attachment.html From aniketpsavanand at gmail.com Sat Dec 12 19:42:39 2015 From: aniketpsavanand at gmail.com (Aniket Savanand) Date: Sat, 12 Dec 2015 19:42:39 -0800 Subject: [Bro] How BRO's in-built scripts are invoked in a flow one by one(one file after other file) In-Reply-To: References: Message-ID: Thanks Anthony, Clark for your replies. I got BRO installed as per https://www.digitalocean.com/community/tutorials/how-to-install-bro-ids-2-2-on-ubuntu-12-04 on my linux dual boot machine. Now, I am able to run the BRO using broctl and I can see log files generated. And I played with try.bro.org and saw how bro can capture traffic http, connection etc. I have succeeded doing above part only. Now at this stage, How do I proceed with suggestions you provided? I got many questions as ( 1 . So as Anthony suggested, to remove @load from these initial boot-strap files init-default.bro and init-bare.bro. But how do I that? I mean, where can I locate these files, and how do modify them to remove @load and make them run, but with my above installation. 2. As per Clark suggestion, I saw devel-tools list, but I could not figure out how do use : https://github.com/bro/bro-aux/blob/master/devel-tools/cpu-bench-with-trace in my current installation) Thanks Aniket Savanand San Jose State 669-226-8162 On Sun, Dec 6, 2015 at 8:57 PM, Aniket Savanand wrote: > Thanks a lot. > > I will look into these files. > > Thanks > Aniket Savanand > > On Sun, Dec 6, 2015 at 8:51 PM, Clark, Gilbert wrote: > >> In addition to what Anthony suggests: >> >> Bro has an option to trace execution and write the results to a file: I >> think it's '-T' or something along those lines. The trace file generated >> by running bro with this option can show you which script functions were >> called and in which order they were called ... but this option generates a >> *lot* of output, and should therefore only be used offline and (probably) >> with a relatively small capture file. >> >> There's a benchmark script that ships with bro that also shows an example >> of incrementally running bro with 1 script loaded, 2 scripts loaded, etc to >> see how each script affects bro's runtime: >> https://github.com/bro/bro-aux/blob/master/devel-tools/cpu-bench-with-trace >> >> Also, maybe try taking a look at try.bro.org: it's a pretty nice way to >> play with bro and become familiar with how things work. >> >> Cheers, >> Gilbert >> > > > > -- > *Regards, * > *Aniket Savanand,* > *MS Software Engineering 2016,* > *San Jose State University, CA* > *Email **Cellphone- +1-669-226-8162 > <%2B1-669-226-8162>* > > > -- *Regards, * *Aniket Savanand,* *MS Software Engineering 2016,* *San Jose State University, CA* *Email **Cellphone- +1-669-226-8162* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151212/dc8b33f4/attachment.html From sabiretude at gmail.com Sun Dec 13 12:00:35 2015 From: sabiretude at gmail.com (reda sabir) Date: Sun, 13 Dec 2015 21:00:35 +0100 Subject: [Bro] OSPF Dissector Message-ID: Hello everyone, I was wondering if is it possible to make an analyzer of OSPF with Binpac. The problem that I face is that OSPF is a layer 4 (there's no tcp or udp). Can anyone give me a solution of my problem? Thanks you, Reda Sabir -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151213/f53ef7f3/attachment.html From johanna at icir.org Mon Dec 14 13:34:49 2015 From: johanna at icir.org (Johanna Amann) Date: Mon, 14 Dec 2015 13:34:49 -0800 Subject: [Bro] Announcing the bro-announce email list Message-ID: <20151214213445.GA5773@wifi142.sys.ICSI.Berkeley.EDU> We just launched a new mailing list called bro-announce. This mailing list will be used to distribute information about new Bro versions, as well as information about security issues that affect Bro (like the recent OpenSSL Bug). As such, this is a very low volume list and we expect it to have less than one posting/month on average. All posts on bro-announce will be cross-posted to this mailing list (bro at bro.org); as long as you are subscribed to this mailing list, you will still get all new version information. You can subscribe to the list at http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-announce Johanna From johanna at icir.org Mon Dec 14 15:26:47 2015 From: johanna at icir.org (Johanna Amann) Date: Mon, 14 Dec 2015 15:26:47 -0800 Subject: [Bro] udp event handlers not catching events In-Reply-To: References: Message-ID: <20151214232647.GA11383@wifi142.sys.ICSI.Berkeley.EDU> > I am analyzing a pcap which contains some UDP packets. I have redefined > both "udp_content_deliver_all_orig" and "udp_content_deliver_all_resp" as > true, but no events are caught by "udp_request", "upd_reply", and > "udp_contents". However, I can use "packets_content" and "is_udp_port" to > catch the udp communications. Do you have a copy of the actual script that you are using? Trying the following on try.bro.org with exercise_traffic.pcap seems to work fine: ------ redef udp_content_deliver_all_orig=T; redef udp_content_deliver_all_resp=T; event udp_contents(u: connection, is_orig: bool, contents: string) { print contents; } ------ Johanna From cbarbaro at cert.unlp.edu.ar Tue Dec 15 14:10:22 2015 From: cbarbaro at cert.unlp.edu.ar (Cristian Barbaro) Date: Tue, 15 Dec 2015 19:10:22 -0300 Subject: [Bro] Scan ports doubt Message-ID: <56708FCE.5030207@cert.unlp.edu.ar> Hello, Community. I've a problem with scan ports: I'm working on a script to detect ports scan (horizontal and vertical) using scan.bro script and I send email when detected. I've a question with Notice::policy executions times. I do a scan to IP's ports (e.g. 10.10.10.10) from a specific IP (e.g. 10.10.1.2). If scan detected first time, send email, but if I do another scan to same IP (10.10.10.10) from 10.10.1.2, Notice::policy hook doesn't execute again. If I do nmap scan from another IP (e.g. 10.10.2.2) to 10.10.10.10, occurs same problem: It only detects first time executing Notice::policy. I would like to change this options but I can't find how to do it. Thank you and I'm sorry for my English. From jazoff at illinois.edu Tue Dec 15 14:27:50 2015 From: jazoff at illinois.edu (Azoff, Justin S) Date: Tue, 15 Dec 2015 22:27:50 +0000 Subject: [Bro] Scan ports doubt In-Reply-To: <56708FCE.5030207@cert.unlp.edu.ar> References: <56708FCE.5030207@cert.unlp.edu.ar> Message-ID: <4A1E921A-5692-4C86-A650-7389D27F72BC@illinois.edu> Duplicate notices are suppressed so that you don't get notified about the same event over and over again. Try something like this in your local.bro redef Notice::type_suppression_intervals += { [Scan::Port_Scan] = 60sec, [Scan::Address_Scan] = 60sec, }; -- - Justin Azoff > On Dec 15, 2015, at 5:10 PM, Cristian Barbaro wrote: > > Hello, Community. > > I've a problem with scan ports: > > I'm working on a script to detect ports scan (horizontal and > vertical) using scan.bro script and I send email when detected. I've a > question with Notice::policy executions times. > I do a scan to IP's ports (e.g. 10.10.10.10) from a specific IP (e.g. > 10.10.1.2). If scan detected first time, send email, but if I do another > scan to same IP (10.10.10.10) from 10.10.1.2, Notice::policy hook > doesn't execute again. > If I do nmap scan from another IP (e.g. 10.10.2.2) to 10.10.10.10, > occurs same problem: It only detects first time executing Notice::policy. > > I would like to change this options but I can't find how to do it. > > > Thank you and I'm sorry for my English. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From cbarbaro at cert.unlp.edu.ar Tue Dec 15 15:13:59 2015 From: cbarbaro at cert.unlp.edu.ar (Cristian Barbaro) Date: Tue, 15 Dec 2015 20:13:59 -0300 Subject: [Bro] Scan ports doubt In-Reply-To: <4A1E921A-5692-4C86-A650-7389D27F72BC@illinois.edu> References: <56708FCE.5030207@cert.unlp.edu.ar> <4A1E921A-5692-4C86-A650-7389D27F72BC@illinois.edu> Message-ID: <56709EB7.8090208@cert.unlp.edu.ar> Perfect. Works fine. Thank you. El 15/12/15 a las 19:27, Azoff, Justin S escribi?: > Duplicate notices are suppressed so that you don't get notified about the same event over and over again. > > Try something like this in your local.bro > > redef Notice::type_suppression_intervals += { > [Scan::Port_Scan] = 60sec, > [Scan::Address_Scan] = 60sec, > }; > From doris at bro.org Thu Dec 17 15:47:20 2015 From: doris at bro.org (Doris Schioberg) Date: Thu, 17 Dec 2015 15:47:20 -0800 Subject: [Bro] A new Bro Tutorial: Happy Holidays from the Bro Team Message-ID: <56734988.8030309@bro.org> We are happy to announce our special present to the Bro Community: Our new interactive Bro Tutorial: https://www.bro.org/documentation/tutorials/index.html Based on try.bro.org this tutorial leads you step by step through the Bro Script Language and allows you to interactively run and change all examples. The first lesson is complete, more lessons are in the works. Feedback and question are more than welcome on info at bro.org. We hope you enjoy our new little helper and wish you Happy Holidays. Your Bro Team -- Doris Schioberg Bro Outreach, Training, and Education Coordinator International Computer Science Institute (ICSI Berkeley) Phone: +1 (510) 289-8406 * doris at bro.org From robin.gruyters at gmail.com Fri Dec 18 08:51:30 2015 From: robin.gruyters at gmail.com (Robin Gruyters) Date: Fri, 18 Dec 2015 17:51:30 +0100 Subject: [Bro] Logging packet with mismatch content_size and data is being sent after reset Message-ID: Hi Bro'ers, I wonder if you could help me. I have created a policy that logs when a http stream has mismatch content-size versus body. This works fine but I need to add an extra check to see if data is being sent after a reset. I have uploaded my policy for you to see. https://rgruyters.stackstorage.com/index.php/s/JdNKlrxKWyzSMzB i know the weird.bro policy logs 'data_after_reset', but I don't know how to incorporate this in my policy. Could you please help me? Kind regards, Robin. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151218/1e7f6855/attachment.html From giedrius.ramas at gmail.com Sun Dec 20 23:27:54 2015 From: giedrius.ramas at gmail.com (Giedrius Ramas) Date: Mon, 21 Dec 2015 09:27:54 +0200 Subject: [Bro] issues with file extraction. Multiple files created. Message-ID: Hello, I have been experienced strange behavior of BRO file extraction . Here you can see what is extracted in bro extract directory. 0 Dec 20 07:42 HTTP-FZyPZr1vrwJ5czHazj.swf 0 Dec 20 07:42 HTTP-FZyPZr1vrwJ5czHazj.swf.1 0 Dec 20 07:42 HTTP-FZyPZr1vrwJ5czHazj.swf.1.1 0 Dec 13 06:42 HTTP-FZyPZr1vrwJ5czHazj.swf.1.1.1 0 Dec 6 06:57 HTTP-FZyPZr1vrwJ5czHazj.swf.1.2 0 Dec 20 07:42 HTTP-FZyPZr1vrwJ5czHazj.swf.2 0 Dec 13 06:42 HTTP-FZyPZr1vrwJ5czHazj.swf.2.1 359 Dec 1 09:36 HTTP-FZyPZr1vrwJ5czHazj.swf.3 Where should I check for troubleshoot ? I just expected to have one file extracted 359 Dec 1 09:36 HTTP-FZyPZr1vrwJ5czHazj.swf instead of those multiples with zero bytes. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151221/1356134d/attachment.html From hosom at battelle.org Mon Dec 21 05:06:28 2015 From: hosom at battelle.org (Hosom, Stephen M) Date: Mon, 21 Dec 2015 13:06:28 +0000 Subject: [Bro] issues with file extraction. Multiple files created. In-Reply-To: References: Message-ID: Do you have a packet capture that you can share that causes this? From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Giedrius Ramas Sent: Monday, December 21, 2015 2:28 AM To: bro at bro.org Subject: [Bro] issues with file extraction. Multiple files created. Hello, I have been experienced strange behavior of BRO file extraction . Here you can see what is extracted in bro extract directory. 0 Dec 20 07:42 HTTP-FZyPZr1vrwJ5czHazj.swf 0 Dec 20 07:42 HTTP-FZyPZr1vrwJ5czHazj.swf.1 0 Dec 20 07:42 HTTP-FZyPZr1vrwJ5czHazj.swf.1.1 0 Dec 13 06:42 HTTP-FZyPZr1vrwJ5czHazj.swf.1.1.1 0 Dec 6 06:57 HTTP-FZyPZr1vrwJ5czHazj.swf.1.2 0 Dec 20 07:42 HTTP-FZyPZr1vrwJ5czHazj.swf.2 0 Dec 13 06:42 HTTP-FZyPZr1vrwJ5czHazj.swf.2.1 359 Dec 1 09:36 HTTP-FZyPZr1vrwJ5czHazj.swf.3 Where should I check for troubleshoot ? I just expected to have one file extracted 359 Dec 1 09:36 HTTP-FZyPZr1vrwJ5czHazj.swf instead of those multiples with zero bytes. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151221/0a786f6e/attachment.html From michalpurzynski1 at gmail.com Tue Dec 22 07:05:50 2015 From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=) Date: Tue, 22 Dec 2015 16:05:50 +0100 Subject: [Bro] OSPF Dissector In-Reply-To: References: Message-ID: It sure is!! There's an excellent lesson how to write analyzers, and another routing protocol is dissected - RIP. https://www.youtube.com/watch?v=eZAgqSFd9-c That's BinPac, so something Bro uses now. On Sun, Dec 13, 2015 at 9:00 PM, reda sabir wrote: > Hello everyone, > > I was wondering if is it possible to make an analyzer of OSPF with Binpac. > The problem that I face is that OSPF is a layer 4 (there's no tcp or udp). > > Can anyone give me a solution of my problem? > > Thanks you, > > Reda Sabir > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151222/18c16c44/attachment.html From vitologrillo at gmail.com Wed Dec 23 13:15:31 2015 From: vitologrillo at gmail.com (Vito Logrillo) Date: Wed, 23 Dec 2015 22:15:31 +0100 Subject: [Bro] Bro dot problem Message-ID: Hi all, as you known, Elasticsearch is unable to menage fields with a dot separator. Until now I've used the Bro json output: the output logs were sent to Elastich through Logstash; from Elasticsearch 2.0 this is not possible. Is there a way to substitute a dot with another character? Thanks, Vito From tgdesrochers at gmail.com Wed Dec 23 13:36:17 2015 From: tgdesrochers at gmail.com (Tim Desrochers) Date: Wed, 23 Dec 2015 16:36:17 -0500 Subject: [Bro] Bro dot problem In-Reply-To: References: Message-ID: In logstash/elasticsearch there is a de_dot filter that works quite well. It has its bugs but it will get the work done. See link: https://www.elastic.co/guide/en/logstash/current/plugins-filters-de_dot.html On Dec 23, 2015 4:24 PM, "Vito Logrillo" wrote: > Hi all, > as you known, Elasticsearch is unable to menage fields with a dot > separator. > Until now I've used the Bro json output: the output logs were sent to > Elastich through Logstash; from Elasticsearch 2.0 this is not > possible. > Is there a way to substitute a dot with another character? > Thanks, > Vito > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151223/de03a9ce/attachment.html From aidaros.dev at gmail.com Wed Dec 23 18:55:08 2015 From: aidaros.dev at gmail.com (Hashem Alaidaros) Date: Thu, 24 Dec 2015 05:55:08 +0300 Subject: [Bro] Issue when Bro is reading a file which capturing live traffic Message-ID: Hi All, I run tcpdump live to capture the traffic into a file using "-w". Then I run bro to read that file offline using "-r". Both instances are running continuously. First it works fine but then bro stop generating results although it keep running, this means bro didn't continue reading from the file. Is it because bro -r is faster than the live capturing? How to let bro keep reading the file (this file is continuously increasing) My bro version: 2.3 running on ubuntu platform. Thanks -- A friend in need Is a friend indeed -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151224/43216ffa/attachment.html From aidaros.dev at gmail.com Thu Dec 24 06:26:19 2015 From: aidaros.dev at gmail.com (Hashem Alaidaros) Date: Thu, 24 Dec 2015 17:26:19 +0300 Subject: [Bro] How to let Packet filter reading from a file Message-ID: Dears I want to use packet filtering framework supported by Bro. It filters based on static IP address. But in my case, I want the filter to read IP address from from a dynamic file (a file that is updated from another bro instance). How to do that? My bro version: 2.3 running on ubuntu platform. Thanks in Advance -- A friend in need Is a friend indeed -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151224/c8c14ac8/attachment.html From daniel.guerra69 at gmail.com Mon Dec 28 04:05:17 2015 From: daniel.guerra69 at gmail.com (Daniel Guerra) Date: Mon, 28 Dec 2015 13:05:17 +0100 Subject: [Bro] Bro dot problem In-Reply-To: References: Message-ID: Check the patch in my repo https://github.com/danielguerra69/bro-debian-elasticsearch.git > On 23 Dec 2015, at 22:36, Tim Desrochers wrote: > > In logstash/elasticsearch there is a de_dot filter that works quite well. It has its bugs but it will get the work done. > > See link: > https://www.elastic.co/guide/en/logstash/current/plugins-filters-de_dot.html > On Dec 23, 2015 4:24 PM, "Vito Logrillo" > wrote: > Hi all, > as you known, Elasticsearch is unable to menage fields with a dot separator. > Until now I've used the Bro json output: the output logs were sent to > Elastich through Logstash; from Elasticsearch 2.0 this is not > possible. > Is there a way to substitute a dot with another character? > Thanks, > Vito > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151228/119e620b/attachment.html From franky.meier.1 at gmx.de Mon Dec 28 06:02:40 2015 From: franky.meier.1 at gmx.de (Frank Meier) Date: Mon, 28 Dec 2015 15:02:40 +0100 Subject: [Bro] Issue when Bro is reading a file which capturing live traffic In-Reply-To: References: Message-ID: <20151228150240.710c7e1e@NB181106> Hi Hashem, On Thu, 24 Dec 2015 05:55:08 +0300 Hashem Alaidaros wrote: > Hi All, > I run tcpdump live to capture the traffic into a file using "-w". > Then I run bro to read that file offline using "-r". > Both instances are running continuously. First it works fine but then > bro stop generating results although it keep running, this means bro > didn't continue reading from the file. Is it because bro -r is faster > than the live capturing? I guesst that is what's happening, but I did not test. Why don't you just let bro and tcpdump read from the network interface? Franky From aidaros.dev at gmail.com Mon Dec 28 19:35:29 2015 From: aidaros.dev at gmail.com (Hashem Alaidaros) Date: Tue, 29 Dec 2015 06:35:29 +0300 Subject: [Bro] Issue when Bro is reading a file which capturing live traffic In-Reply-To: <20151228150240.710c7e1e@NB181106> References: <20151228150240.710c7e1e@NB181106> Message-ID: Thanks Franky for your reply. in my research, I'm requested to run bro in offline mode for further analysis and forensics. Any idea how to resolve the issue. Thanks On Mon, Dec 28, 2015 at 5:02 PM, Frank Meier wrote: > > Hi Hashem, > > On Thu, 24 Dec 2015 05:55:08 +0300 > Hashem Alaidaros wrote: > > > Hi All, > > I run tcpdump live to capture the traffic into a file using "-w". > > Then I run bro to read that file offline using "-r". > > Both instances are running continuously. First it works fine but then > > bro stop generating results although it keep running, this means bro > > didn't continue reading from the file. Is it because bro -r is faster > > than the live capturing? > > I guesst that is what's happening, but I did not test. > Why don't you just let bro and tcpdump read from the network interface? > > Franky > -- A friend in need Is a friend indeed -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151229/98391bb6/attachment.html From ricardquentin at gmail.com Tue Dec 29 02:38:09 2015 From: ricardquentin at gmail.com (Quentin Ricard) Date: Tue, 29 Dec 2015 11:38:09 +0100 Subject: [Bro] [Bro advanced Architecture ] Message-ID: Dear broers, I'm currently working on a project involving Bro's core. The idea is to adapt Bro into another architecture and reusing the Protocol Analyser as well as the event engine (not only developing plugins). The problem is that I cannot find enough information on the GitHub/Bro.org web sites. Even In the Doxygen there is no information about the architecture in details (Classes involved etc). So I was wandering if some of you guys had those kind of documents ? If not I'll gladly edit them for you if someone helps me with the architecture details. Sincerely, Quentin Ricard. From ironholds at gmail.com Tue Dec 29 06:21:51 2015 From: ironholds at gmail.com (Oliver Keyes) Date: Tue, 29 Dec 2015 09:21:51 -0500 Subject: [Bro] Underscores in field names Message-ID: Heya, >From the guide to the various log files (https://www.bro.org/sphinx/script-reference/log-files.html) and some example files I've accumulated it looks like nested fields are represented in "flat" log files with period delimiters. So the orig_h field within the id field becomes id.orig_h. Is this correct? At the same time I'm seeing files with underscores instead of periods. >From what I can see on this mailing list and elsewhere, this is a logging setting - people can switch out periods for underscores to cover the situation where the software they read the logs /into/ does not like periods. My question: can I expect this to be consistent? In other words, for files to either use periods or underscores, but not both at once? From mirugy at gmail.com Tue Dec 29 06:34:21 2015 From: mirugy at gmail.com (=?UTF-8?B?R3nDtnJneSBNaXJ1?=) Date: Tue, 29 Dec 2015 15:34:21 +0100 Subject: [Bro] Fwd: log writer issue In-Reply-To: References: Message-ID: Hello List, I am developing a Bro analyzer plugin and I have the following issue. The analyzer logs events into three different logfiles, one of which keeps crashing with this error: Reporter::ERROR s7data/Log::WRITER_ASCII: terminating thread This happens before the first event is logged, however the headers are already written into the logfile. I am fairly new to the Bro development so it might be some obvious mistake I make, but I could not find any solution on the internet. I have attached the following files: init_part.bro: the relevant part of the script, used for the logging reporter.log: logfile that contains the error s7data.log: the logfile that causes the crash debug_s7data.log: relevant part of the debug.log file, when bro was run with -B threading switch strace_4938: relevant part of the strace -f output When creating the attached logs bro-2.4 was used, but I tested the analyzer with bro-2.4.1 as well and the problem persists. I hope someone can point out the origin of the error and help me resolve this issue. Thanks, Gyorgy Miru -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151229/67dc8ea0/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: debug_s7data.log Type: text/x-log Size: 327 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151229/67dc8ea0/attachment.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: init_part.bro Type: application/octet-stream Size: 2641 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151229/67dc8ea0/attachment.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: reporter.log Type: text/x-log Size: 288 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151229/67dc8ea0/attachment-0001.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: strace_4938 Type: application/octet-stream Size: 3236 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151229/67dc8ea0/attachment-0001.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: s7data.log Type: text/x-log Size: 327 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20151229/67dc8ea0/attachment-0002.bin From jazoff at illinois.edu Tue Dec 29 07:15:11 2015 From: jazoff at illinois.edu (Azoff, Justin S) Date: Tue, 29 Dec 2015 15:15:11 +0000 Subject: [Bro] log writer issue In-Reply-To: References: Message-ID: > On Dec 29, 2015, at 9:34 AM, Gy?rgy Miru wrote: > > This happens before the first event is logged, however the headers are already written into the logfile Was there a stderr.log ? Does it happen before the event would have been logged at all, or in the process of logging the event? If you add a print "This is siemenss7_write_data_unsigned"; #or siemenss7_read_data_unsigned print c$s7data; before the calls to Log::write(S7comm::LOG3, c$s7data); what gets output to stdout (or the stdout.log if you are using broctl)? I think this may be caused by one of the fields in one of your events being invalid somehow... > debug_s7data.log: relevant part of the debug.log file, when bro was run with -B threading switch You really want -B logging I have a feeling you'll see a "Field type doesn't match in WriterBackend::Write" message -- - Justin Azoff