From swells at cyberadapt.com Mon Mar 2 15:02:37 2015 From: swells at cyberadapt.com (Scott Wells) Date: Mon, 2 Mar 2015 23:02:37 +0000 Subject: [Bro] icmp events (custom and original) not firing? Message-ID: Hello. I am writing a C++ program to interface with Bro, using Broccoli. According to this [Bro] Mailing List message (http://mailman.icsi.berkeley.edu/pipermail/bro/2014-December/007844.html), I need to create my own events that only use certain fields from the conn_id bro record. I followed the example in the post, using the 'dns_message' event as a test. 1) In bro/share/bro/base/bif/plugins/Bro_DNS.events.bif.bro, I added this line after line 26: global dns_message_test: event(cid: conn_id, is_orig: bool , msg: dns_msg , len: count ); 2) In bro/share/bro/base/protocols/dns/main.bro, in the 'event dns_message' event (ln 286), I added this at the bottom of the event: event dns_message_test(c$id, is_orig, msg, len); Then, in my main function, I add it to the registry with something like: bro_event_registry_add_compact(f_broPtr, "dns_message_test", (BroCompactEventFunc) event_cb, NULL); And my 'event_cb' callback function is called and everything seems good. However, I then wanted to try with icmp_time_exceeded and icmp_packet_too_big (which is part of what I'm after): In bro/share/bro/base/bif/plugins/Bro_ICMP.events.bif.bro, I add a '_test' event to both: global icmp_packet_too_big_test: event(cid: conn_id , icmp: icmp_conn , code: count , context: icmp_context ); global icmp_time_exceeded_test: event(cid: conn_id , icmp: icmp_conn , code: count , context: icmp_context ); However: 1) There is only one 'event icmp*' function block, and that's in /bro/share/bro/policy/misc/detect-traceroute/main.bro for 'icmp_time_exceeded'. This has an associated 'event icmp_time_exceeded' (line 98). I add my 'icmp_time_exceeded_test(c$id, icmp, code, context)' line in that function and register it with another bro_event_registry_add_compact line in my C code, but the event (original and my _test version) never fires. I'm not sure I'm using the correct 'event icmp' since the detect-traceroute comments say it's for a Time Exceeded threshold, and I'm more interested in capturing ANY Time exceeded event. 2) Given 1), I cannot find a 'main.bro' file with 'event icmp_packet_too_big', and so that event (original and _test version) never fires, either. I am very new to DPI, so I may be missing something obvious. Any help greatly appreciated. Thanks! -Scott >>>CONFIDENTIALITY NOTICE>>> This electronic mail message, including any and/or all attachments, is for the sole use of the intended recipient(s), and may contain confidential and/or privileged information, pertaining to business conducted under the direction and supervision of the sending organization. All electronic mail messages, which may have been established as expressed views and/or opinions (stated either within the electronic mail message or any of its attachments), are left to the sole responsibility of that of the sender, and are not necessarily attributed to the sending organization. Unauthorized interception, review, use, disclosure or distribution of any such information contained within this electronic mail message and/or its attachment(s), is(are) strictly prohibited. If you are not the intended recipient, please contact the sender by replying to this electronic mail message, along with the destruction of all copies of the original electronic mail message (along with any attachments). -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150302/fb2f8fdd/attachment.html From swells at cyberadapt.com Mon Mar 2 15:16:26 2015 From: swells at cyberadapt.com (Scott Wells) Date: Mon, 2 Mar 2015 23:16:26 +0000 Subject: [Bro] icmp events (custom and original) not firing? In-Reply-To: References: Message-ID: I should also mention that we are firing pcaps that contain ICMP Oversize and Time Exceeded data, but it could be that they need to be configured in Bro. -Scott Ps: Pls excuse the Confidentiality Notice. It's auto-generated from my company. From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Scott Wells Sent: Monday, March 02, 2015 5:03 PM To: bro at bro.org Subject: [Bro] icmp events (custom and original) not firing? Hello. I am writing a C++ program to interface with Bro, using Broccoli. According to this [Bro] Mailing List message (http://mailman.icsi.berkeley.edu/pipermail/bro/2014-December/007844.html), I need to create my own events that only use certain fields from the conn_id bro record. I followed the example in the post, using the 'dns_message' event as a test. 1) In bro/share/bro/base/bif/plugins/Bro_DNS.events.bif.bro, I added this line after line 26: global dns_message_test: event(cid: conn_id, is_orig: bool , msg: dns_msg , len: count ); 2) In bro/share/bro/base/protocols/dns/main.bro, in the 'event dns_message' event (ln 286), I added this at the bottom of the event: event dns_message_test(c$id, is_orig, msg, len); Then, in my main function, I add it to the registry with something like: bro_event_registry_add_compact(f_broPtr, "dns_message_test", (BroCompactEventFunc) event_cb, NULL); And my 'event_cb' callback function is called and everything seems good. However, I then wanted to try with icmp_time_exceeded and icmp_packet_too_big (which is part of what I'm after): In bro/share/bro/base/bif/plugins/Bro_ICMP.events.bif.bro, I add a '_test' event to both: global icmp_packet_too_big_test: event(cid: conn_id , icmp: icmp_conn , code: count , context: icmp_context ); global icmp_time_exceeded_test: event(cid: conn_id , icmp: icmp_conn , code: count , context: icmp_context ); However: 1) There is only one 'event icmp*' function block, and that's in /bro/share/bro/policy/misc/detect-traceroute/main.bro for 'icmp_time_exceeded'. This has an associated 'event icmp_time_exceeded' (line 98). I add my 'icmp_time_exceeded_test(c$id, icmp, code, context)' line in that function and register it with another bro_event_registry_add_compact line in my C code, but the event (original and my _test version) never fires. I'm not sure I'm using the correct 'event icmp' since the detect-traceroute comments say it's for a Time Exceeded threshold, and I'm more interested in capturing ANY Time exceeded event. 2) Given 1), I cannot find a 'main.bro' file with 'event icmp_packet_too_big', and so that event (original and _test version) never fires, either. I am very new to DPI, so I may be missing something obvious. Any help greatly appreciated. Thanks! -Scott >>>CONFIDENTIALITY NOTICE>>> This electronic mail message, including any and/or all attachments, is for the sole use of the intended recipient(s), and may contain confidential and/or privileged information, pertaining to business conducted under the direction and supervision of the sending organization. All electronic mail messages, which may have been established as expressed views and/or opinions (stated either within the electronic mail message or any of its attachments), are left to the sole responsibility of that of the sender, and are not necessarily attributed to the sending organization. Unauthorized interception, review, use, disclosure or distribution of any such information contained within this electronic mail message and/or its attachment(s), is(are) strictly prohibited. If you are not the intended recipient, please contact the sender by replying to this electronic mail message, along with the destruction of all copies of the original electronic mail message (along with any attachments). >>>CONFIDENTIALITY NOTICE>>> This electronic mail message, including any and/or all attachments, is for the sole use of the intended recipient(s), and may contain confidential and/or privileged information, pertaining to business conducted under the direction and supervision of the sending organization. All electronic mail messages, which may have been established as expressed views and/or opinions (stated either within the electronic mail message or any of its attachments), are left to the sole responsibility of that of the sender, and are not necessarily attributed to the sending organization. Unauthorized interception, review, use, disclosure or distribution of any such information contained within this electronic mail message and/or its attachment(s), is(are) strictly prohibited. If you are not the intended recipient, please contact the sender by replying to this electronic mail message, along with the destruction of all copies of the original electronic mail message (along with any attachments). -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150302/42be9c90/attachment-0001.html From swells at cyberadapt.com Tue Mar 3 07:54:52 2015 From: swells at cyberadapt.com (Scott Wells) Date: Tue, 3 Mar 2015 15:54:52 +0000 Subject: [Bro] icmp events (custom and original) not firing? (now with line breaks!) Message-ID: It's been awhile since I've used a mailing list. Didn't add line breaks. Sorry. So let's try this again for better readability. -------- Hello. I am writing a C++ program to interface with Bro, using Broccoli. According to this [Bro] Mailing List message (http://mailman.icsi.berkeley.edu/pipermail/bro/2014-December/007844.html), I need to create my own events that only use certain fields from the conn_id bro record. I followed the example in the post, using the 'dns_message' event as a test. 1) In bro/share/bro/base/bif/plugins/Bro_DNS.events.bif.bro, I added this line after line 26: global dns_message_test: event(cid: conn_id, is_orig: bool , msg: dns_msg , len: count ); 2) In bro/share/bro/base/protocols/dns/main.bro, in the 'event dns_message' event (ln 286), I added this at the bottom of the event: event dns_message_test(c$id, is_orig, msg, len); 3) Then, in my main function, I add it to the registry with something like: bro_event_registry_add_compact(f_broPtr, "dns_message_test", (BroCompactEventFunc) event_cb, NULL); And my 'event_cb' callback function is called and everything seems good. However, I then wanted to try with icmp_time_exceeded and icmp_packet_too_big (which is part of what I'm after): In bro/share/bro/base/bif/plugins/Bro_ICMP.events.bif.bro, I add a '_test' event to both: global icmp_packet_too_big_test: event(cid: conn_id , icmp: icmp_conn , code: count , context: icmp_context ); global icmp_time_exceeded_test: event(cid: conn_id , icmp: icmp_conn , code: count , context: icmp_context ); However: 1) There is only one 'event icmp*' function block, and that's in /bro/share/bro/policy/misc/detect-traceroute/main.bro for 'icmp_time_exceeded'. This has an associated 'event icmp_time_exceeded' (line 98). I add my 'icmp_time_exceeded_test(c$id, icmp, code, context)' line in that function and register it with another bro_event_registry_add_compact line in my C code, but the event (original and my _test version) never fires. I'm not sure I'm using the correct 'event icmp' since the detect-traceroute comments say it's for a Time Exceeded threshold, and I'm more interested in capturing ANY Time exceeded event. 2) Given 1), I cannot find a 'main.bro' file with 'event icmp_packet_too_big', and so that event (original and _test version) never fires, either. I am very new to DPI, so I may be missing something obvious. Any help greatly appreciated. Thanks! -Scott >>>CONFIDENTIALITY NOTICE>>> This electronic mail message, including any and/or all attachments, is for the sole use of the intended recipient(s), and may contain confidential and/or privileged information, pertaining to business conducted under the direction and supervision of the sending organization. All electronic mail messages, which may have been established as expressed views and/or opinions (stated either within the electronic mail message or any of its attachments), are left to the sole responsibility of that of the sender, and are not necessarily attributed to the sending organization. Unauthorized interception, review, use, disclosure or distribution of any such information contained within this electronic mail message and/or its attachment(s), is(are) strictly prohibited. If you are not the intended recipient, please contact the sender by replying to this electronic mail message, along with the destruction of all copies of the original electronic mail message (along with any attachments). From jsiwek at illinois.edu Tue Mar 3 09:04:11 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Tue, 3 Mar 2015 17:04:11 +0000 Subject: [Bro] icmp events (custom and original) not firing? In-Reply-To: References: Message-ID: > On Mar 2, 2015, at 5:16 PM, Scott Wells wrote: > > However: > 1) There is only one ?event icmp*? function block, and that?s in /bro/share/bro/policy/misc/detect-traceroute/main.bro for ?icmp_time_exceeded?. This has an associated ?event icmp_time_exceeded? (line 98). I add my ?icmp_time_exceeded_test(c$id, icmp, code, context)? line in that function and register it with another bro_event_registry_add_compact line in my C code, but the event (original and my _test version) never fires. I?m not sure I?m using the correct ?event icmp? since the detect-traceroute comments say it?s for a Time Exceeded threshold, and I?m more interested in capturing ANY Time exceeded event. > 2) Given 1), I cannot find a ?main.bro? file with ?event icmp_packet_too_big?, and so that event (original and _test version) never fires, either. Are you sure that policy/misc/detect-traceroute/main.bro is loaded ? E.g. if you?re running from command line it won?t be loaded by default, you?d need something like: bro -r icmp.pcap misc/detect-traceroute/main.bro This is sort of a detour from the specific problem, but rather than modifying existing scripts, I?d suggest defining your own event handlers for the ones you need. I?d try to get something simple working from the command-line first. E.g. create a file called ?test.bro? containing: event icmp_time_exceeded(c: connection, icmp: icmp_conn, code: count, context: icmp_context) { print "icmp_time_exceeded", c$id, icmp, code, context; } Then run: bro -r icmp.pcap ./test.bro If you see output, then it works and you can modify the handler however you need. If not, then there?s something specific about the pcap you?re using that I?d probably need to see in order to tell what?s wrong, but sometimes weird.log might have hints as to what?s wrong. - Jon From jtaylor1024 at yahoo.com Tue Mar 3 11:12:44 2015 From: jtaylor1024 at yahoo.com (Jerome Taylor) Date: Tue, 3 Mar 2015 19:12:44 +0000 (UTC) Subject: [Bro] Typical Bro use case Message-ID: <673832571.3108370.1425409964750.JavaMail.yahoo@mail.yahoo.com> Hello all,?I am an applications engineer at a small start-up company located just North of Boston MA. Ihave been tasked to explore Bro and to write a follow on case study. I amsomewhat new to Bro. I have installed a small cluster and have been workingwith Bro for the past few months. I would like to find out how others in theBro community are using Bro. ?For instance:1.????Do most people use Bro stand-alone or are youusing it in conjunction with another IDP/IPS sensor such as Snort2.????What does a typical setup look like in terms ofequipmenta.????What does your engress network load look like(i.e. data rate, traffic mix, etc.)b.????How many cores are required to handle yourtraffic load/mix3.????How are you processing the log files4.????What is the ultimate problem that you are tryingto solve?I am more then happy to share my findings thus far with anyinterested party. Ultimately, I would like to turn this into a presentationthat I can share at the ?next BroCom.?If it makes more sense for me to take these types ofquestions off-line then I will gladly do so. Again, I am very interested infinding out how the rest of the community is using Bro so please feel free toreach out to me. Thanks in advance..?Regards,Jerome Taylor -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150303/e9b372c8/attachment.html From tonyceb at andrew.cmu.edu Wed Mar 4 13:44:53 2015 From: tonyceb at andrew.cmu.edu (Tony Cebzanov) Date: Wed, 04 Mar 2015 16:44:53 -0500 Subject: [Bro] multiple sqlite writers Message-ID: <54F77CD5.20705@andrew.cmu.edu> Is there any way to log multiple log streams to a single sqlite database as different tables, or does each logging stream / table have to be in a separate file? If the answer is the latter, how much work would be involved in adding support for multiple writers? From johanna at icir.org Wed Mar 4 14:03:25 2015 From: johanna at icir.org (Johanna Amann) Date: Wed, 4 Mar 2015 14:03:25 -0800 Subject: [Bro] multiple sqlite writers In-Reply-To: <54F77CD5.20705@andrew.cmu.edu> References: <54F77CD5.20705@andrew.cmu.edu> Message-ID: <20150304220325.GA67647@wifi86.sys.ICSI.Berkeley.EDU> Hello Tony, On Wed, Mar 04, 2015 at 04:44:53PM -0500, Tony Cebzanov wrote: > Is there any way to log multiple log streams to a single sqlite database > as different tables, or does each logging stream / table have to be in a > separate file? That should in principle work - I think you should be able to specify the same path for two logging filters while using different tablenames in the $config (see https://www.bro.org/sphinx-git/frameworks/logging-input-sqlite.html for an example). That being said, it might lead to a couple of problems. Each of the output streams will run in a separate thread and open the database individually. That means only one of the threads can write to the database simultaneously (see https://www.sqlite.org/faq.html#q5). This will probably have significant performance implications. I am also not quite sure how well the logging framework deals with the fact that a db is currently blocked and it is not able to write. I hope that helps, Johanna From johanna at icir.org Wed Mar 4 14:19:58 2015 From: johanna at icir.org (Johanna Amann) Date: Wed, 4 Mar 2015 14:19:58 -0800 Subject: [Bro] multiple sqlite writers In-Reply-To: <54F783E6.3010005@cert.org> References: <54F77CD5.20705@andrew.cmu.edu> <20150304220325.GA67647@wifi86.sys.ICSI.Berkeley.EDU> <54F783E6.3010005@cert.org> Message-ID: <20150304221958.GA68982@wifi86.sys.ICSI.Berkeley.EDU> On Wed, Mar 04, 2015 at 05:15:02PM -0500, Tony Cebzanov wrote: > I get the following error: > > 1425489173.970543 error: analysis/Log::WRITER_SQLITE: Error executing table > creation statement: database is locked > > This would appear to confirm your statement that the logging framework may > not deal well with locking. > > Should I file a bug? Please do -- we should probably at least try to acquire a lock for a bit before failing. Johanna From yardley at gmail.com Thu Mar 5 14:34:22 2015 From: yardley at gmail.com (Tim Yardley) Date: Thu, 5 Mar 2015 16:34:22 -0600 Subject: [Bro] Bro Network Appliance Options Message-ID: All, I'm gathering some requirements and vendor options associated with small scale Bro installations and looking for some input from the field. Namely, I have interest in systems that are known to work well with Bro, work reliably in the field, and that have the general community "stamp of approval". Features I'd like to have: - hardware bypass support (or ability to put in a card that does support it) - 4 interfaces - ability to scale to at least 100Mbit line rate, 1Gbit would be better but is not necessary Any thoughts on CPU and RAM are always appreciated as well. Recommended CPUs supporting any hardware options that are considered good to take advantage of, RAM sizing, etc. Thank you in advance for any input you can provide, as I am sure that it can provide value to others as well. Tim -- Tim Yardley Associate Director of Technology Information Trust Institute, University of Illinois yardley at illinois.edu From alajal at gmail.com Thu Mar 5 20:30:54 2015 From: alajal at gmail.com (Mustafa Qasim) Date: Fri, 6 Mar 2015 09:30:54 +0500 Subject: [Bro] Typical Bro use case In-Reply-To: <673832571.3108370.1425409964750.JavaMail.yahoo@mail.yahoo.com> References: <673832571.3108370.1425409964750.JavaMail.yahoo@mail.yahoo.com> Message-ID: Hi, I'm also exploring bro to be used as a core traffic intel framework. However, it's just a small single server setup. I've picked the Security Onion distro so, Snort is pre configured and running with that. I would also like to read complete case studies of other folks who've deployed it into production. ------ *Mustafa Qasim* GREM, GCFE On Wed, Mar 4, 2015 at 12:12 AM, Jerome Taylor wrote: > Hello all, > > I am an applications engineer at a small start-up company located just > North of Boston MA. I have been tasked to explore Bro and to write a follow > on case study. I am somewhat new to Bro. I have installed a small cluster > and have been working with Bro for the past few months. I would like to > find out how others in the Bro community are using Bro. > > For instance: > 1. Do most people use Bro stand-alone or are you using it in > conjunction with another IDP/IPS sensor such as Snort > 2. What does a typical setup look like in terms of equipment > a. What does your engress network load look like (i.e. data rate, > traffic mix, etc.) > b. How many cores are required to handle your traffic load/mix > 3. How are you processing the log files > 4. What is the ultimate problem that you are trying to solve > > I am more then happy to share my findings thus far with any interested > party. Ultimately, I would like to turn this into a presentation that I can > share at the next BroCom. > > If it makes more sense for me to take these types of questions off-line > then I will gladly do so. Again, I am very interested in finding out how > the rest of the community is using Bro so please feel free to reach out to > me. Thanks in advance.. > > Regards, > Jerome Taylor > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150306/eb38c999/attachment.html From seraphim6134 at hotmail.com Fri Mar 6 19:24:33 2015 From: seraphim6134 at hotmail.com (Dimitri H) Date: Fri, 6 Mar 2015 19:24:33 -0800 Subject: [Bro] Bro - posting authenticity Message-ID: Seraphim6134 at hotmail.com Sent from my iPhone From Emmanuel.TORQUATO at monext.net Mon Mar 9 06:09:47 2015 From: Emmanuel.TORQUATO at monext.net (Emmanuel TORQUATO) Date: Mon, 9 Mar 2015 14:09:47 +0100 Subject: [Bro] Error with a custom protocol decoder which returns an event with record type object Message-ID: Hello everyone, I am working on a banking protocol decoder and try to generate an event that return an object of type record. I assume the protocol decoding is ok because it works when I raise an event with a Val (count, or string) object. When I change the events.bif, the analyzer.pac for returning an event with a record object type, I have a core dump in Bro. The error is that CBCOM::Message type is unknown in file /opt/bro-2.3.2/share/bro/base/bif/plugins/Bro_CBCOM.events.bif.bro This type has been declared in the "events.bif" file I have followed the things done for radius protocol decoder under src/analyzer/protocol/radius in bro source files. ? Cbcom-protocol.pac: type CBCOM_PDU(is_orig: bool) = record { msg_length: uint32; pgi: uint8[1]; lgi: uint8; zp: CBCOM_TLV[] &length = lgi, &until($input.length() == 0); apdu: case apdudata of { 0 -> none: empty; default -> cb2a: CBCOM_APDU; }; } &let{ apdudata: int = msg_length - lgi - 2; } &byteorder=bigendian; type CBCOM_TLV = record { ti: uint8 &check(pi == 0x01 || pi == 0x05 || pi == 0x06 || pi == 0x07 || pi == 0x08 || pi == 0x0f || pi == 0x10 || pi == 0x11 || pi == 0x12 || pi == 0x19); li: uint8; vi: uint8[li]; } &byteorder=bigendian; ..... ? Cbcom-analyzer.pac generate event with result (record type): .... function proc_cbcom_message(msg: CBCOM_PDU): bool %{ RecordVal* result = new RecordVal(BifType::Record::CBCOM::Message); result->Assign(0, new Val(${msg.msg_length}, TYPE_COUNT)); uint length = ${msg.pgi}->size(); char buf[length]; for ( uint i = 0; i < length; ++i ) { buf[i] = ${msg.pgi[i]}; } result->Assign(1, string_to_val(get_hex_tostring(buf, length, true))); result->Assign(2, new Val(${msg.lgi}, TYPE_COUNT)); if ( ${msg.zp}->size() ) { TableVal* zp = new TableVal(BifType::Table::CBCOM::Zp); for ( uint i = 0; i < ${msg.zp}->size(); ++i ) { Val* index = new Val(${msg.zp[i].ti}, TYPE_COUNT); Val* current = zp->Lookup(index); uint length2 = ${msg.zp[i].vi}->size(); char char_vi[length2]; for ( uint j = 0; j < length2; ++j ) { char_vi[j] = ${msg.zp[i].vi[j]}; } Val* val = string_to_val(get_hex_tostring(char_vi, length2, true)); if ( current ) { VectorVal* vcurrent = current->AsVectorVal(); vcurrent->Assign(vcurrent->Size(), val); } else { VectorVal* parametres = new VectorVal(BifType::Vector::CBCOM::Parametres); parametres->Assign((unsigned int)0, val); zp->Assign(index, parametres); } Unref(index); } result->Assign(3, zp); } BifEvent::generate_cbcom_message(connection()->bro_analyzer(), connection()->bro_analyzer()->Conn(), result); return true; ..... ? event.bif with bro event with result as type record: event cbcom_message%(c: connection, result: CBCOM::Message%); event cbcom_apdu%(c: connection, itm: string%); event cbcom_pan%(c: connection, pan: string%); type CBCOM::Parametres: vector; type CBCOM::Zp: table; type CBCOM::Message: record; Anyone who has an idea or who has worked on the radius decoder could help me ? Thanks ! Cordialement, [cid:image001.jpg at 01D05A6F.BCA36940] T. +33 4 42 25 15 51 emmanuel.torquato at monext.net www.monext.net [cid:image002.jpg at 01D05A6F.BCA36940] [cid:image003.jpg at 01D05A6F.BCA36940] P Merci de penser ? l'environnement avant d'imprimer ce message. Please consider the environment before printing this email. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150309/21cc3dd3/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 4074 bytes Desc: image001.jpg Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150309/21cc3dd3/attachment-0003.jpg -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 950 bytes Desc: image002.jpg Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150309/21cc3dd3/attachment-0004.jpg -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.jpg Type: image/jpeg Size: 902 bytes Desc: image003.jpg Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150309/21cc3dd3/attachment-0005.jpg From vitologrillo at gmail.com Mon Mar 9 06:58:28 2015 From: vitologrillo at gmail.com (Vito Logrillo) Date: Mon, 9 Mar 2015 14:58:28 +0100 Subject: [Bro] SMB2 module Message-ID: Hi all, we are working on an application based on Bro: we need to implement some features based on SMB2. On Bitbucket, we have found this link: https://bitbucket.org/delbert77/bro/src/886ed4689c391129be3d45745a6c63e4033fd1c3/src/SMB2.cc?at=topic/seth/smb-smb2-work The link above seems connected to a previous Bro version: an updated version is present? If no, SMB2 will be implemented in next releases?When? Due to our requirements, we think to make that module from scratch if anyone is working on; otherwise, can we take part to your team for the development and testing of that module? Regards, Vito Logrillo From gregk at bivio.net Mon Mar 9 14:17:00 2015 From: gregk at bivio.net (Greg Kopchinski) Date: Mon, 9 Mar 2015 21:17:00 +0000 Subject: [Bro] Bro Network Appliance Options In-Reply-To: References: Message-ID: Hi Tim, Bivio provides a range of Linux platforms that are used extensively by federal government entities to host Bro as well as other cyber security centric applications. Our B7000 series platforms can support Bro running at line rates to multiple Gbps line rate, and include copper & fiber interfaces with hardware bypass support. It is easy to scale the platform by adding additional processor blades, so for your requirements as noted you could start with the base product (B7512/14), and then later add a processor blade to support full GbE line rates, or even scale to full 10 Gbps throughput. Our support and dev teams are very familiar with Bro as we are often called on by our customers to help optimize their systems for best performance, especially when running multiple co-tenant apps. The high level product info for the B7000 series is at http://www.bivio.net/products/b7000/, or I'd be happy to provide you with more information (PM me: gregk (at) bivio (dot) net). Note: I am employed by Bivio Networks. Regards, Greg Marketing Manager, Bivio Networks gregk at bivio.net -----Original Message----- From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Tim Yardley Sent: Thursday, March 05, 2015 2:34 PM To: bro at bro.org Subject: [Bro] Bro Network Appliance Options All, I'm gathering some requirements and vendor options associated with small scale Bro installations and looking for some input from the field. Namely, I have interest in systems that are known to work well with Bro, work reliably in the field, and that have the general community "stamp of approval". Features I'd like to have: - hardware bypass support (or ability to put in a card that does support it) - 4 interfaces - ability to scale to at least 100Mbit line rate, 1Gbit would be better but is not necessary Any thoughts on CPU and RAM are always appreciated as well. Recommended CPUs supporting any hardware options that are considered good to take advantage of, RAM sizing, etc. Thank you in advance for any input you can provide, as I am sure that it can provide value to others as well. Tim -- Tim Yardley Associate Director of Technology Information Trust Institute, University of Illinois yardley at illinois.edu _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From seth at icir.org Mon Mar 9 22:26:48 2015 From: seth at icir.org (Seth Hall) Date: Tue, 10 Mar 2015 01:26:48 -0400 Subject: [Bro] Error with a custom protocol decoder which returns an event with record type object In-Reply-To: References: Message-ID: > On Mar 9, 2015, at 9:09 AM, Emmanuel TORQUATO wrote: > > type CBCOM::Message: record; > > Anyone who has an idea or who has worked on the radius decoder could help me ? You probably just haven?t defined the structure of that record in a Bro script. Built in analyzers at the moment have their script land definitions provided in init-base.bro (for the most part), but if you are writing your analyzer as an external plugin (if you?re working with git master) you can provide the definition in a script shipped with the plugin. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From seth at icir.org Mon Mar 9 22:33:46 2015 From: seth at icir.org (Seth Hall) Date: Tue, 10 Mar 2015 01:33:46 -0400 Subject: [Bro] SMB2 module In-Reply-To: References: Message-ID: > On Mar 9, 2015, at 9:58 AM, Vito Logrillo wrote: > > The link above seems connected to a previous Bro version: an updated > version is present? If no, SMB2 will be implemented in next > releases?When? SMB will not be making it into the 2.4 release. It?s still too unstable. That branch you pointed to however is very old and no longer represents the current development state of the SMB analyzer. Probably the most up to date code today is in topic/vladg/smb but we know of a number of issues in that still. https://github.com/bro/bro/tree/topic/vladg/smb/src/analyzer/protocol/smb > Due to our requirements, we think to make that module from scratch if > anyone is working on; otherwise, can we take part to your team for the > development and testing of that module?  SMB might be a larger task than you wish it were. There are quite a number of dead ends and problems that you discover as you dig into the protocol more and more. If you have spare development cycles and qualified developers, we?re certainly willing to talk. :) Thanks, .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From Emmanuel.TORQUATO at monext.net Tue Mar 10 09:28:29 2015 From: Emmanuel.TORQUATO at monext.net (Emmanuel TORQUATO) Date: Tue, 10 Mar 2015 17:28:29 +0100 Subject: [Bro] Error with a custom protocol decoder which returns an event with record type object In-Reply-To: References: Message-ID: Thanks Seth, it's ok after adding my new type in my init-bare.bro file. It was the missing step ! Regards, T. +33 4 42 25 15 51 emmanuel.torquato at monext.net www.monext.net ? ? Merci de penser ? l'environnement avant d'imprimer ce message. Please consider the environment before printing this email. -----Message d'origine----- De?: Seth Hall [mailto:seth at icir.org] Envoy??: mardi 10 mars 2015 06:27 ??: Emmanuel TORQUATO Cc?: bro at bro.org Objet?: Re: [Bro] Error with a custom protocol decoder which returns an event with record type object > On Mar 9, 2015, at 9:09 AM, Emmanuel TORQUATO wrote: > > type CBCOM::Message: record; > > Anyone who has an idea or who has worked on the radius decoder could help me ? You probably just haven?t defined the structure of that record in a Bro script. Built in analyzers at the moment have their script land definitions provided in init-base.bro (for the most part), but if you are writing your analyzer as an external plugin (if you?re working with git master) you can provide the definition in a script shipped with the plugin. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From jdopheid at illinois.edu Tue Mar 10 10:05:21 2015 From: jdopheid at illinois.edu (Dopheide, Jeannette M) Date: Tue, 10 Mar 2015 17:05:21 +0000 Subject: [Bro] BroCon '15 Call for Presentations Message-ID: BroCon '15 is now accepting presentation proposals. This year is Bro's 20th Anniversary. To mark this special occasion we are looking for presentations that represent the diverse applications of Bro: * as a tool for solving problems; * interesting user stories, solutions, or research projects; * a postmortem analysis of a security incident, emphasizing Bro's contribution; * the value Bro brings to your professional work; * and, using Bro for more than intrusion detection. Send abstracts to info at bro.org with the subject "BroCon 2015 Call for Presentations" by Friday, May 29th. Plan on limiting your talk to 30-35 minutes with an additional 10 minutes for questions/comments. ------ Jeannette M. Dopheide Bro Outreach Coordinator National Center for Supercomputing Applications University of Illinois at Urbana-Champaign From bkellogg at dresser-rand.com Wed Mar 11 14:30:34 2015 From: bkellogg at dresser-rand.com (Kellogg, Brian D (OLN)) Date: Wed, 11 Mar 2015 21:30:34 +0000 Subject: [Bro] Bro detection scripts updates Message-ID: I'm going through and updating the NOTICEs for different detection scripts built into Bro. Trying to get the generated NOTICE logs set correctly for ELSA to parse. It is working but I'm not sure if I'm doing this the most Bro appropriate way. Couple questions: Is this the best way to accomplish this task? Secondly, if advisable, how do we get these script changes incorporated into Bro base? I'm not that experienced with git but willing to learn more if needed. These changes were made, again, to benefit ELSA searching/grouping and for the Bro correlation script recently released. Here's what I changed/add to some of the built-in detection scripts (Lines with "+" are what I changed/added): /opt/bro/share/bro/policy/protocols/ssh/detect-bruteforcing.bro NOTICE([$note=Password_Guessing, $msg=fmt("%s appears to be guessing SSH passwords (seen in %d connections).", key$host, r$num), $sub=sub_msg, + #$src=key$host, + $id=[$orig_h=key$host,$orig_p=0/tcp,$resp_h=0.0.0.0,$resp_p=0/tcp], $identifier=cat(key$host)]); }]); /opt/bro/share/bro/policy/protocols/ftp/detect-bruteforcing.bro NOTICE([$note=FTP::Bruteforcing, + #$src=key$host, + $id=[$orig_h=key$host,$orig_p=0/tcp,$resp_h=0.0.0.0,$resp_p=0/tcp], $msg=message, $identifier=cat(key$host)]); }]); /opt/bro/share/bro/policy/protocols/http/detect-sqli.bro NOTICE([$note=SQL_Injection_Attacker, $msg="An SQL injection attacker was discovered!", $email_body_sections=vector(format_sqli_samples(r$samples)), + #$src=key$host, + $id=[$orig_h=key$host,$orig_p=0/tcp,$resp_h=0.0.0.0,$resp_p=0/tcp], + $sub=cat(format_sqli_samples(r$samples)), $identifier=cat(key$host)]); }]); ? NOTICE([$note=SQL_Injection_Victim, $msg="An SQL injection victim was discovered!", $email_body_sections=vector(format_sqli_samples(r$samples)), + #$src=key$host, + $id=[$orig_h=0.0.0.0,$orig_p=0/tcp,$resp_h=key$host,$resp_p=0/tcp], + $sub=cat(format_sqli_samples(r$samples)), $identifier=cat(key$host)]); }]); /opt/bro/share/bro/policy/misc/scan.bro NOTICE([$note=Address_Scan, #$src=key$host, + $id=[$orig_h=key$host,$orig_p=0/tcp,$resp_h=0.0.0.0,$resp_p=key$str], + #$p=to_port(key$str), $sub=side, $msg=message, $identifier=cat(key$host)]); }]); ? NOTICE([$note=Port_Scan, #$src=key$host, + $id=[$orig_h=key$host,$orig_p=0/tcp,$resp_h=key$str,$resp_p=0/tcp], + #$dst=to_addr(key$str), $sub=side, $msg=message, $identifier=cat(key$host)]); }]); /opt/bro/share/bro/policy/misc/detect-traceroute/main.bro NOTICE([$note=Traceroute::Detected, $msg=fmt("%s seems to be running traceroute using %s", src, proto), + #$src=src, + $id=[$orig_h=src,$orig_p=0/icmp,$resp_h=dst,$resp_p=0/icmp], $identifier=cat(src,proto)]); }]); Thanks, -Brian From seth at icir.org Wed Mar 11 18:53:54 2015 From: seth at icir.org (Seth Hall) Date: Wed, 11 Mar 2015 21:53:54 -0400 Subject: [Bro] Bro detection scripts updates In-Reply-To: References: Message-ID: > On Mar 11, 2015, at 5:30 PM, Kellogg, Brian D (OLN) wrote: > > Is this the best way to accomplish this task? Secondly, if advisable, how do we get these script changes incorporated into Bro base? I'm not that experienced with git but willing to learn more if needed. These changes were made, again, to benefit ELSA searching/grouping and for the Bro correlation script recently released. You might be right in how you?ve done this. I never felt very comfortable with how this ended up getting implemented. I?ll file a ticket and see if I can address the ones you?ve pointed out and see if there are anymore. I?ll probably also mark the src and dst fields as deprecated and maybe do a reporter message if they?re ever used (to give people a bit of breathing room before we break any existing scripts they have). Here?s the ticket in case you or anyone else wants to comment on it: https://bro-tracker.atlassian.net/browse/BIT-1339  Great suggestion, thanks! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From troyj at maine.edu Fri Mar 13 04:39:47 2015 From: troyj at maine.edu (Troy Jordan) Date: Fri, 13 Mar 2015 07:39:47 -0400 Subject: [Bro] exercising binpac++/spicy parsers Message-ID: <5502CC83.9030701@maine.edu> Hello, There are some hilti-based parsers in the Bro docker image. When I run the pcaps for BACnet (/opt/hilti/bro/tests/Traces/bacnet/*.pcap) through Bro (eg bro -r NPDU.pcap) , no event logs are produced in /usr/local/bro/logs). How do I integrate these parsers into Bro? - Troy -- Troy Jordan t r o y j @ m a i n e . e d u GIAC GCIH,GCIA ------------------------------------------------------------ Network Systems Security Analyst Information Technology Security Office University of Maine System ------------------------------------------------------------ 233 Science Building | voice: 207.561.3590 Portland, ME 04103 | fax: 509.351.3650 "As you all know, Security Is Mortals chiefest Enemy" William Shakespeare, Macbeth From carlopmart at gmail.com Fri Mar 13 04:46:14 2015 From: carlopmart at gmail.com (C.L. Martinez) Date: Fri, 13 Mar 2015 11:46:14 +0000 Subject: [Bro] Disable some type of "alerts" for internal servers Message-ID: <5502CE06.3020801@gmail.com> Hi all, I am receiving a lot alerts like this: Bro SSL::Invalid_Server_Cert. 172.16.129.8 (Unknown):3040 -> 172.17.0.130 (Unknown):1610 which it is correct: we are using a lot of certs auto-signed in our infrastructure. Is it possible to disable this type of alert for an IP or a group of IP's? Thanks. From johanna at icir.org Fri Mar 13 07:39:19 2015 From: johanna at icir.org (Johanna Amann) Date: Fri, 13 Mar 2015 07:39:19 -0700 Subject: [Bro] exercising binpac++/spicy parsers In-Reply-To: <5502CC83.9030701@maine.edu> References: <5502CC83.9030701@maine.edu> Message-ID: <20150313143919.GB64511@Beezling.local> Hello Troy, > There are some hilti-based parsers in the Bro docker image. When I run > the pcaps for BACnet (/opt/hilti/bro/tests/Traces/bacnet/*.pcap) through > Bro (eg bro -r NPDU.pcap) , no event logs are produced in > /usr/local/bro/logs). You have to load the applicable scripts and pac files for Bro to be able to parse these protocols. Which files you have to load depends a bit on the protocol. The easiest way is to look at the tests that should be there for each of the protocols that is to look into the tests directory in hilti/bro/tests. In there, pac2/bacnet/npdu_nlmessages.bro shows that to parse the NPDUs, it loads bacnet.evt and bacnet.bro out of the hilti distribution and then defines a few custom events for output. Generally, none of the spicy parsers come with the Bro scripts to generate log output -- the parsers just create events. For some of the parsers replacing protocols (like DNS or HTTP), the events might be similar enough to the events emitted by the binpac parsers to already generate logs. For all new protocols implemented by spicy, you would first have to create such scripts. I hope that helps, Johanna From jsiwek at illinois.edu Fri Mar 13 07:40:50 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Fri, 13 Mar 2015 14:40:50 +0000 Subject: [Bro] Disable some type of "alerts" for internal servers In-Reply-To: <5502CE06.3020801@gmail.com> References: <5502CE06.3020801@gmail.com> Message-ID: <19661449-2183-40AF-B84C-0D7769E87A33@illinois.edu> > On Mar 13, 2015, at 6:46 AM, C.L. Martinez wrote: > > Hi all, > > I am receiving a lot alerts like this: > > Bro SSL::Invalid_Server_Cert. 172.16.129.8 (Unknown):3040 -> > 172.17.0.130 (Unknown):1610 > > which it is correct: we are using a lot of certs auto-signed in our > infrastructure. > > Is it possible to disable this type of alert for an IP or a group of IP's? A script like this may do what you want: const invalid_ssl_whitelist: set[addr] = { # Add IPs here } &redef; hook Notice::policy(n: Notice::Info) { if ( n$note == SSL::Invalid_Server_Cert && n$conn$id$resp_h in invalid_ssl_whitelist ) # Clear all actions for this notice. n$actions = Notice::ActionSet(); } You can probably also add logic to filter only if the reason it?s invalid is due to self-signing (e.g. as opposed to expired) by inspecting n$msg. Some related docs to reference: https://www.bro.org/sphinx/frameworks/notice.html - Jon From silusilusilu at gmail.com Fri Mar 13 10:16:30 2015 From: silusilusilu at gmail.com (fasf safas) Date: Fri, 13 Mar 2015 18:16:30 +0100 Subject: [Bro] What does PacketFilter::Dropped_Packets mean? Message-ID: Hi, i don't understand very well what PacketFilter::Dropped_Packets means. For example, i've found a notice like this: 2148880 packets dropped after filtering, 2233772 received, 2233380 on link Can you explain me better the meaning of these values? Thanks Fab -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150313/39e0f5dc/attachment.html From troyj at maine.edu Fri Mar 13 16:40:32 2015 From: troyj at maine.edu (Troy Jordan) Date: Fri, 13 Mar 2015 19:40:32 -0400 Subject: [Bro] exercising binpac++/spicy parsers In-Reply-To: <20150313143919.GB64511@Beezling.local> References: <5502CC83.9030701@maine.edu> <20150313143919.GB64511@Beezling.local> Message-ID: <55037570.4080607@maine.edu> Johanna, that does help, thank you. Was BACnet used simply as a test for spicy, or perhaps are there plans to develop it further? - Troy On 3/13/2015 10:39 AM, Johanna Amann wrote: > Hello Troy, > >> There are some hilti-based parsers in the Bro docker image. When I run >> the pcaps for BACnet (/opt/hilti/bro/tests/Traces/bacnet/*.pcap) through >> Bro (eg bro -r NPDU.pcap) , no event logs are produced in >> /usr/local/bro/logs). > > You have to load the applicable scripts and pac files for Bro to be able > to parse these protocols. Which files you have to load depends a bit on > the protocol. The easiest way is to look at the tests that should be there > for each of the protocols that is to look into the tests directory in > hilti/bro/tests. In there, pac2/bacnet/npdu_nlmessages.bro shows that to > parse the NPDUs, it loads bacnet.evt and bacnet.bro out of the hilti > distribution and then defines a few custom events for output. > > Generally, none of the spicy parsers come with the Bro scripts to generate > log output -- the parsers just create events. For some of the parsers > replacing protocols (like DNS or HTTP), the events might be similar enough > to the events emitted by the binpac parsers to already generate logs. For > all new protocols implemented by spicy, you would first have to create > such scripts. > > I hope that helps, > Johanna > -- Troy Jordan t r o y j @ m a i n e . e d u GIAC GCIH,GCIA ------------------------------------------------------------ Network Systems Security Analyst Information Technology Security Office University of Maine System ------------------------------------------------------------ 233 Science Building | voice: 207.561.3590 Portland, ME 04103 | fax: 509.351.3650 "As you all know, Security Is Mortals chiefest Enemy" William Shakespeare, Macbeth From carlopmart at gmail.com Sun Mar 15 10:06:07 2015 From: carlopmart at gmail.com (C.L. Martinez) Date: Sun, 15 Mar 2015 17:06:07 +0000 Subject: [Bro] Disable some type of "alerts" for internal servers In-Reply-To: <19661449-2183-40AF-B84C-0D7769E87A33@illinois.edu> References: <5502CE06.3020801@gmail.com> <19661449-2183-40AF-B84C-0D7769E87A33@illinois.edu> Message-ID: <5505BBFF.9000309@gmail.com> On 03/13/2015 02:40 PM, Siwek, Jon wrote: > >> On Mar 13, 2015, at 6:46 AM, C.L. Martinez wrote: >> >> Hi all, >> >> I am receiving a lot alerts like this: >> >> Bro SSL::Invalid_Server_Cert. 172.16.129.8 (Unknown):3040 -> >> 172.17.0.130 (Unknown):1610 >> >> which it is correct: we are using a lot of certs auto-signed in our >> infrastructure. >> >> Is it possible to disable this type of alert for an IP or a group of IP's? > > A script like this may do what you want: > > const invalid_ssl_whitelist: set[addr] = { > # Add IPs here > } &redef; > > hook Notice::policy(n: Notice::Info) > { > if ( n$note == SSL::Invalid_Server_Cert && > n$conn$id$resp_h in invalid_ssl_whitelist ) > # Clear all actions for this notice. > n$actions = Notice::ActionSet(); > } > > You can probably also add logic to filter only if the reason it?s invalid is due to self-signing (e.g. as opposed to expired) by inspecting n$msg. > > Some related docs to reference: > > https://www.bro.org/sphinx/frameworks/notice.html > > - Jon > Many thanks Jon, But I am doing something wrong. When I launch "bro check", I receive the following error: bro scripts failed. error in /data/config/etc/bro/policy/custom.bro, line 24: unknown identifier SSL::Invalid_Server_Cert, at or near "SSL::Invalid_Server_Cert" Actually: # Disable SSL::Invalid_Server_Cert alert for internal hosts const invalid_ssl_whitelist: set[addr] = { 10.19.0.12 } &redef; hook Notice::policy(n: Notice::Info) { if ( n$note == SSL::Invalid_Server_Cert && n$conn$id$resp_h in invalid_ssl_whitelist ) # Clear all actions for this notice. n$actions = Notice::ActionSet(); } # This script logs which scripts were loaded during each run. @load misc/loaded-scripts From carlopmart at gmail.com Sun Mar 15 10:15:46 2015 From: carlopmart at gmail.com (C.L. Martinez) Date: Sun, 15 Mar 2015 17:15:46 +0000 Subject: [Bro] Disable some type of "alerts" for internal servers In-Reply-To: <5505BBFF.9000309@gmail.com> References: <5502CE06.3020801@gmail.com> <19661449-2183-40AF-B84C-0D7769E87A33@illinois.edu> <5505BBFF.9000309@gmail.com> Message-ID: <5505BE42.1080006@gmail.com> On 03/15/2015 05:06 PM, C.L. Martinez wrote: > > > On 03/13/2015 02:40 PM, Siwek, Jon wrote: >> >>> On Mar 13, 2015, at 6:46 AM, C.L. Martinez wrote: >>> >>> Hi all, >>> >>> I am receiving a lot alerts like this: >>> >>> Bro SSL::Invalid_Server_Cert. 172.16.129.8 (Unknown):3040 -> >>> 172.17.0.130 (Unknown):1610 >>> >>> which it is correct: we are using a lot of certs auto-signed in our >>> infrastructure. >>> >>> Is it possible to disable this type of alert for an IP or a group >>> of IP's? >> >> A script like this may do what you want: >> >> const invalid_ssl_whitelist: set[addr] = { >> # Add IPs here >> } &redef; >> >> hook Notice::policy(n: Notice::Info) >> { >> if ( n$note == SSL::Invalid_Server_Cert && >> n$conn$id$resp_h in invalid_ssl_whitelist ) >> # Clear all actions for this notice. >> n$actions = Notice::ActionSet(); >> } >> >> You can probably also add logic to filter only if the reason it?s >> invalid is due to self-signing (e.g. as opposed to expired) by >> inspecting n$msg. >> >> Some related docs to reference: >> >> https://www.bro.org/sphinx/frameworks/notice.html >> >> - Jon >> > > Many thanks Jon, > > But I am doing something wrong. When I launch "bro check", I receive > the following error: > > bro scripts failed. > error in /data/config/etc/bro/policy/custom.bro, line 24: unknown > identifier SSL::Invalid_Server_Cert, at or near "SSL::Invalid_Server_Cert" > > Actually: > > # Disable SSL::Invalid_Server_Cert alert for internal hosts > const invalid_ssl_whitelist: set[addr] = { > 10.19.0.12 > } &redef; > > hook Notice::policy(n: Notice::Info) > { > if ( n$note == SSL::Invalid_Server_Cert && > n$conn$id$resp_h in invalid_ssl_whitelist ) > # Clear all actions for this notice. > n$actions = Notice::ActionSet(); > } > > # This script logs which scripts were loaded during each run. > @load misc/loaded-scripts Ok, problem solved. Forget it. Sorry for this last post. From jdopheid at illinois.edu Sun Mar 15 16:25:04 2015 From: jdopheid at illinois.edu (Dopheide, Jeannette M) Date: Sun, 15 Mar 2015 23:25:04 +0000 Subject: [Bro] forwarding Facebook inquiry Message-ID: <7EFD7D614A2BB84ABEA19B2CEDD2465801A4BA56@CITESMBX5.ad.uillinois.edu> Hello Bro Community, I'm forwarding along a Facebook post our page received: Can some one help me how to split connection from internal and external in a separate log even i dont want the traffic from out side to the inside? Find bellow the script im using: i have been looking for someone to help me for a month now its for educational purposes ----------------------------------------- global mime_to_ext: table[string] of string = { ["text/plain"] = "txt", ["text/html"] = "html", ["application/pdf"] = "pdf", ["application/x-pdf"] = "pdf", ["application/acrobat"] = "pdf", ["applications/vnd.pdf"] = "pdf", ["text/pdf"] = "pdf", ["text/x-pdf"] = "pdf", }; global sn_pro: table[string] of string = { ["SMTP"] = "SMTP", ["HTTP"] = "HTTP", }; event file_new(f: fa_file) { #if ( f$source !in ssl_ports ) if ( f$source !in sn_pro ) return; if ( ! f?$mime_type ) return; if ( f$mime_type !in mime_to_ext ) return; local fname = fmt("%s-%s.%s", f$source, f$id, mime_to_ext[f$mime_type]); print fmt("Extracting file %s", fname); Files::add_analyzer(f, Files::ANALYZER_EXTRACT, [$extract_filename=fname]); } -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150315/515b2504/attachment.html From liam.randall at gmail.com Sun Mar 15 21:36:36 2015 From: liam.randall at gmail.com (Liam Randall) Date: Mon, 16 Mar 2015 00:36:36 -0400 Subject: [Bro] forwarding Facebook inquiry In-Reply-To: <7EFD7D614A2BB84ABEA19B2CEDD2465801A4BA56@CITESMBX5.ad.uillinois.edu> References: <7EFD7D614A2BB84ABEA19B2CEDD2465801A4BA56@CITESMBX5.ad.uillinois.edu> Message-ID: Not 100% sure what the question is being asked, however I put together a simple script to log files based directionality: https://github.com/criticalstack/bro-scripts/blob/master/files-log-by-direction/files-log-by-direction.bro There are some subtleties here that may not be obvious: -- a file can have multiple transmitters or receivers; both tx_hosts and rx_hosts in the Files::Info record are a set [ADDR] -- files do not have to come from a connection In my example I just use the first TX and RX for making a decision. Thanks, Liam Randall On Sun, Mar 15, 2015 at 7:25 PM, Dopheide, Jeannette M < jdopheid at illinois.edu> wrote: > Hello Bro Community, > > I'm forwarding along a Facebook post our page received: > > Can some one help me how to split connection from internal and external > in a separate log even i dont want the traffic from out side to the inside? > > Find bellow the script im using: i have been looking for someone to help > me for a month now its for educational purposes > ----------------------------------------- > global mime_to_ext: table[string] of string = { > ["text/plain"] = "txt", > ["text/html"] = "html", > ["application/pdf"] = "pdf", > ["application/x-pdf"] = "pdf", > ["application/acrobat"] = "pdf", > ["applications/vnd.pdf"] = "pdf", > ["text/pdf"] = "pdf", > ["text/x-pdf"] = "pdf", > }; > global sn_pro: table[string] of string = { > ["SMTP"] = "SMTP", > ["HTTP"] = "HTTP", > }; > event file_new(f: fa_file) { > #if ( f$source !in ssl_ports ) > if ( f$source !in sn_pro ) return; > if ( ! f?$mime_type ) return; > if ( f$mime_type !in mime_to_ext ) return; > local fname = fmt("%s-%s.%s", f$source, f$id, mime_to_ext[f$mime_type]); > print fmt("Extracting file %s", fname); Files::add_analyzer(f, > Files::ANALYZER_EXTRACT, [$extract_filename=fname]); > } > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150316/1fe4a3ca/attachment.html From doris at bro.org Tue Mar 17 14:31:25 2015 From: doris at bro.org (Doris Schioberg) Date: Tue, 17 Mar 2015 14:31:25 -0700 Subject: [Bro] no newsletter this month Message-ID: <55089D2D.1070003@bro.org> For March, we decided to skip the newsletter. There is a lot going on at the moment, which is all still work in progress and we will have exciting news next month. Watch out for the April newsletter. -The Bro Team -- Doris Schioberg Bro Outreach, Training, and Education Coordinator International Computer Science Institute (ICSI Berkeley) Phone: +1 (510) 289-8406 * doris at bro.org -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 882 bytes Desc: OpenPGP digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150317/ad654bb2/attachment.bin From lubo.gusenica at zoznam.sk Tue Mar 17 17:14:26 2015 From: lubo.gusenica at zoznam.sk (Lubo Gusenica) Date: Wed, 18 Mar 2015 00:14:26 +0000 (UTC) Subject: [Bro] Problem with http request in Bro Message-ID: Hello, I'm trying to use this function in my script: function find_event(value: string) { local url = fmt("%s/%s/ip-src",prefix, value); local arg = fmt("--header \"Authorization: %s\"",auth); local req = ActiveHTTP::Request($url=url, $addl_curl_args=arg); when ( local resp = ActiveHTTP::request(req) ) { print resp; } } event Intel::match(s: Intel::Seen, items: set [Intel::Item]) { find_event(s$indicator); } Prefix and auth are global. Just sometimes it prints response. Most of times it prints rm: cannot remove `/tmp/bro-activehttp-_body': No such file or directory Can you tell me please what I'm doing wrong or why it is happenning ? Thank you From SHille at heartland.com Wed Mar 18 09:28:09 2015 From: SHille at heartland.com (Hille, Samson) Date: Wed, 18 Mar 2015 16:28:09 +0000 Subject: [Bro] Using Bro to detect DNS lookups in given timeframe Message-ID: <0B774F5E2B1584419DA3BCD9E8D1B4C477F1C691@HOMX03.hdcare.local> Does Bro have anything built-in for the following scenario: ? Detecting if a network device is looking up over 50 DNS entries in a 1 hour timeframe Samson Hille IT Security Analyst [http://heartland.com/sites/all/themes/heartland/images/external-files/HeartlandDentalLogo.png] ________________________________ Privacy Notice: This electronic mail message, and any attachments, are confidential and are intended for the exclusive use of the addressee(s) and may contain information that is proprietary and that may be Individually Identifiable or Protected Health Information under HIPAA. If you are not the intended recipient, please immediately contact the sender by telephone, or by email, and destroy all copies of this message. If you are a regular recipient of our electronic mail, please notify us promptly if you change your email address. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150318/a2324cd6/attachment.html From SHille at heartland.com Wed Mar 18 09:32:16 2015 From: SHille at heartland.com (Hille, Samson) Date: Wed, 18 Mar 2015 16:32:16 +0000 Subject: [Bro] Using Bro to detect DNS lookups in given timeframe Message-ID: <0B774F5E2B1584419DA3BCD9E8D1B4C477F1C6C0@HOMX03.hdcare.local> Does Bro have anything built-in for the following scenario: * Detecting if a network device is looking up over 50 DNS entries in a 1 hour timeframe Samson Hille IT Security Analyst [http://heartland.com/sites/all/themes/heartland/images/external-files/HeartlandDentalLogo.png] ________________________________ Privacy Notice: This electronic mail message, and any attachments, are confidential and are intended for the exclusive use of the addressee(s) and may contain information that is proprietary and that may be Individually Identifiable or Protected Health Information under HIPAA. If you are not the intended recipient, please immediately contact the sender by telephone, or by email, and destroy all copies of this message. If you are a regular recipient of our electronic mail, please notify us promptly if you change your email address. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150318/a7b9c6f3/attachment-0001.html From seth at icir.org Wed Mar 18 09:57:08 2015 From: seth at icir.org (Seth Hall) Date: Wed, 18 Mar 2015 12:57:08 -0400 Subject: [Bro] Using Bro to detect DNS lookups in given timeframe In-Reply-To: <0B774F5E2B1584419DA3BCD9E8D1B4C477F1C691@HOMX03.hdcare.local> References: <0B774F5E2B1584419DA3BCD9E8D1B4C477F1C691@HOMX03.hdcare.local> Message-ID: > On Mar 18, 2015, at 12:28 PM, Hille, Samson wrote: > > ? Detecting if a network device is looking up over 50 DNS entries in a 1 hour timeframe There is nothing built in right now, but it would be pretty easy to write a script to do it. Here?s a quick one... event bro_init() { local r1 = SumStats::Reducer($stream="too_much_dns.recursive_requests", $apply=set(SumStats::SUM)); SumStats::create([$name="too_much_dns", $epoch=1hr, $reducers=set(r1), $threshold_val(key: SumStats::Key, result: SumStats::Result) = { return result["too_much_dns.recursive_requests"]$sum; }, $threshold=5.0, $threshold_crossed(key: SumStats::Key, result: SumStats::Result) = { local r = result["too_much_dns.recursive_requests"]; local dur = duration_to_mins_secs(r$end-r$begin); local message = fmt("%s did at least %.0f recursive DNS requests in %s", key$host, r$sum, dur); print message; }]); } event dns_request(c: connection, msg: dns_msg, query: string, qtype: count, qclass: count) { if ( msg$RD ) SumStats::observe("too_much_dns.recursive_requests", [$host=c$id$orig_h], [$num=1]); } -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From jxbatchelor at gmail.com Wed Mar 18 10:59:20 2015 From: jxbatchelor at gmail.com (Jason Batchelor) Date: Wed, 18 Mar 2015 12:59:20 -0500 Subject: [Bro] ZWS File Magic Inclusion Message-ID: Hello all: I needed to extract from PCAP a malicious SWF that was compressed using LZMA, and thusly gave the SWF a 'ZWS' header instead of the normal 'CWS' you typically observe in a compressed SWF. While the general.sig file has signatures for CWS and FWS magic for SWF files, I did not see the presence of ZWF. I went ahead and created the following entry in libmagic.sig. signature file-magic-swf-zws { file-mime "application/x-shockwave-flash/lzma", 60 file-magic /(ZWS)/ } Then ran bro along side my extraction script on the PCAP and out the LZMA compressed SWF came. Just wanted to pass this along. It might be worth adding it to the sig files for a future release possibly? Thanks, Jason -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150318/d7702c3c/attachment.html From seth at icir.org Wed Mar 18 11:12:38 2015 From: seth at icir.org (Seth Hall) Date: Wed, 18 Mar 2015 14:12:38 -0400 Subject: [Bro] ZWS File Magic Inclusion In-Reply-To: References: Message-ID: <32943562-E9B0-45E2-AAD5-7075BFD41243@icir.org> > On Mar 18, 2015, at 1:59 PM, Jason Batchelor wrote: > > Then ran bro along side my extraction script on the PCAP and out the LZMA compressed SWF came. Just wanted to pass this along. It might be worth adding it to the sig files for a future release possibly? That was added a while ago and is queued up for inclusion into 2.4... signature file-swf { file-magic /^(F|C|Z)WS/ file-mime "application/x-shockwave-flash", 60 } 2.4 has a ton of updates to file type identification and it looks like there are going to be some more updates before the release too. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From dani.nicolo at gmail.com Thu Mar 19 08:48:12 2015 From: dani.nicolo at gmail.com (=?UTF-8?Q?Danilo_Nicol=C3=B2?=) Date: Thu, 19 Mar 2015 16:48:12 +0100 Subject: [Bro] SMB2 module In-Reply-To: References: Message-ID: Hi Seth, I'm a colleague of Vito and I'm trying to customize Bro with SMB2 protocol analyzer . I have got the latest version from GitHub and merged it with SMB2 version taken from Vladg topic; i've tried to run broctl after the merge, but later Bro crashes due to a SIGBUS event. I've substituted src/analyzer/protocol/smb, src/analyzer/protocol/netbios, init-bare.bro and init-default.bro from SMB2 version to master version. Below a snippet taken from "./broctl diag": Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/bro/bin/bro -i eth0 -U .status -p broctl -p broctl-live -p standalon'. Program terminated with signal SIGBUS, Bus error. #0 0x0000000000816193 in Serializer::Write (this=0x7fffc052fd00, v=35329, tag=0xb7a68f "stype") at /home/danko/bro/src/Serializer.h:57 57 DECLARE_IO(uint16) . . . Thread 1 (Thread 0x7f3337201780 (LWP 22674)): #0 0x0000000000816193 in Serializer::Write (this=0x7fffc052fd00, v=35329, tag=0xb7a68f "stype") at /home/danko/bro/src/Serializer.h:57 #1 0x0000000000815fdc in SerialObj::DoSerialize (this=0x2b2bf00, info=0x7fffc052fd60) at /home/danko/bro/src/SerialObj.cc:268 #2 0x00000000007df8f6 in BroObj::DoSerialize (this=0x2b2bf00, info=0x7fffc052fd60) at /home/danko/bro/src/Obj.cc:226 #3 0x0000000000843002 in BroType::DoSerialize (this=0x2b2bf00, info=0x7fffc052fd60) at /home/danko/bro/src/Type.cc:283 #4 0x000000000081585b in SerialObj::Serialize (this=0x2b2bf00, info=0x7fffc052fd60) at /home/danko/bro/src/SerialObj.cc:121 #5 0x0000000000842cce in BroType::Serialize (this=0x2b2bf00, info=0x7fffc052fd60) at /home/danko/bro/src/Type.cc:212 #6 0x00000000008438ec in TypeList::DoSerialize (this=0x2b402e0, info=0x7fffc052fd60) at /home/danko/bro/src/Type.cc:392 #7 0x000000000081585b in SerialObj::Serialize (this=0x2b402e0, info=0x7fffc052fd60) at /home/danko/bro/src/SerialObj.cc:121 . . . #81382 0x0000000000837f2a in ForStmt::DoExec (this=0x4c90610, f=0x6e5d9c0, v=0x740a610, flow=@0x7fffc0530080: FLOW_NEXT) at /home/danko/bro/src/Stmt.cc:1358 #81383 0x0000000000833db1 in ExprStmt::Exec (this=0x4c90610, f=0x6e5d9c0, flow=@0x7fffc0530080: FLOW_NEXT) at /home/danko/bro/src/Stmt.cc:373 #81384 0x0000000000839969 in StmtList::Exec (this=0x4c8f850, f=0x6e5d9c0, flow=@0x7fffc0530080: FLOW_NEXT) at /home/danko/bro/src/Stmt.cc:1764 #81385 0x0000000000839969 in StmtList::Exec (this=0x4c93a60, f=0x6e5d9c0, flow=@0x7fffc0530080: FLOW_NEXT) at /home/danko/bro/src/Stmt.cc:1764 #81386 0x00000000007a4828 in BroFunc::Call (this=0x4974a80, args=0x5acc3c0, parent=0x0) at /home/danko/bro/src/Func.cc:403 #81387 0x000000000077d5a4 in EventHandler::Call (this=0x49ae420, vl=0x5acc3c0, no_remote=false) at /home/danko/bro/src/EventHandler.cc:130 #81388 0x0000000000731ff1 in Event::Dispatch (this=0x70daec0, no_remote=false) at /home/danko/bro/src/Event.h:50 #81389 0x000000000077ccdd in EventMgr::Dispatch (this=0xf65e60 ) at /home/danko/bro/src/Event.cc:111 #81390 0x000000000077cde8 in EventMgr::Drain (this=0xf65e60 ) at /home/danko/bro/src/Event.cc:128 #81391 0x00000000007dbfa7 in net_run () at /home/danko/bro/src/Net.cc:374 #81392 0x000000000073105c in main (argc=19, argv=0x7fffc05309b8) at /home/danko/bro/src/main.cc:1212 ==== No reporter.log ==== stderr.log listening on eth0, capture length 8192 bytes send-mail: SENDMAIL-NOTFOUND not found /usr/local/bro/share/broctl/scripts/run-bro: line 100: 22674 Bus error (core dumped) nohup "$mybro" "$@" ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -i eth0 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto ==== .env_vars PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site CLUSTER_NODE= ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log I've pasted also the gdb log : [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/bro/bin/bro -i eth0 -U .status -p broctl -p broctl-live -p standalon'. Program terminated with signal SIGBUS, Bus error. #0 0x0000000000816193 in Serializer::Write (this=0x7fffc052fd00, v=35329, tag=0xb7a68f "stype") at /home/danko/bro/src/Serializer.h:57 57 DECLARE_IO(uint16) (gdb) p *this $1 = {_vptr.Serializer = 0xb83010 , static MAGIC = 1112691540, static DATA_FORMAT_VERSION = 25, io = 0x0, format = 0x73af900, current_cache = 0x0, error_descr = 0x0} (gdb) up #1 0x0000000000815fdc in SerialObj::DoSerialize (this=0x2b2bf00, info=0x7fffc052fd60) at /home/danko/bro/src/SerialObj.cc:268 268 bool ret = SERIALIZE(stype); (gdb) p *this $2 = {_vptr.SerialObj = 0xb82f70 , static NEVER = 0, static ALWAYS = 1, static factories = 0x2a8f1c0, static names = 0x2a8f200, static time_counter = 3480072, serial_type = 51713} (gdb) up #2 0x00000000007df8f6 in BroObj::DoSerialize (this=0x2b2bf00, info=0x7fffc052fd60) at /home/danko/bro/src/Obj.cc:226 226 DO_SERIALIZE(SER_BRO_OBJ, SerialObj); (gdb) Although Bro crashes, the module seems to work fine: in fact in a few minutes after I ran it, I can see the smb log files. Do you have any idea about this error? Kind regards, Danilo 2015-03-10 6:33 GMT+01:00 Seth Hall : > > > On Mar 9, 2015, at 9:58 AM, Vito Logrillo > wrote: > > > > The link above seems connected to a previous Bro version: an updated > > version is present? If no, SMB2 will be implemented in next > > releases?When? > > SMB will not be making it into the 2.4 release. It?s still too unstable. > That branch you pointed to however is very old and no longer represents the > current development state of the SMB analyzer. Probably the most up to > date code today is in topic/vladg/smb but we know of a number of issues in > that still. > > https://github.com/bro/bro/tree/topic/vladg/smb/src/analyzer/protocol/smb > > > Due to our requirements, we think to make that module from scratch if > > anyone is working on; otherwise, can we take part to your team for the > > development and testing of that module? > > SMB might be a larger task than you wish it were. There are quite a > number of dead ends and problems that you discover as you dig into the > protocol more and more. If you have spare development cycles and qualified > developers, we?re certainly willing to talk. :) > > Thanks, > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150319/93574aa1/attachment.html From gl89 at cornell.edu Fri Mar 20 08:18:29 2015 From: gl89 at cornell.edu (Glenn Forbes Fleming Larratt) Date: Fri, 20 Mar 2015 11:18:29 -0400 (EDT) Subject: [Bro] Trying to get Bro to share Myricom cards with tcpdump or Snort Message-ID: Folks, Can anyone point to a Bro+Snort HOWTO that would help me get Myricom cards to share? 1. Following the directions at https://www.myricom.com/software/sniffer10g/995-how-can-i-direct-sniffer10g-traffic-to-multiple-applications-using-snf-app-id.html doesn't really help, because my Bro deployment is a cluster, and the environmental variables don't propagate to my worker hosts - in fact, /proc/{bro_pid}/environ is 0-length on all the processes on the worker hosts. 2. I tried to reverse-engineer how Security Onion does it, but I didn't really glean anything that would help. Thanks for any info, -- Glenn Forbes Fleming Larratt Cornell University IT Security Office From latt0050 at umn.edu Fri Mar 20 08:24:44 2015 From: latt0050 at umn.edu (Brandon Lattin) Date: Fri, 20 Mar 2015 10:24:44 -0500 Subject: [Bro] Trying to get Bro to share Myricom cards with tcpdump or Snort In-Reply-To: References: Message-ID: Just to verify, you're using the Sniffer10G v3 driver, yes? Assuming you are, keep in mind that each interface is still limited to 32 ring buffers (this is what got me). So plan on running something like 16 for Bro and 16 for Snort/Suricata. On Fri, Mar 20, 2015 at 10:18 AM, Glenn Forbes Fleming Larratt < gl89 at cornell.edu> wrote: > Folks, > > Can anyone point to a Bro+Snort HOWTO that would help me get Myricom cards > to share? > > 1. Following the directions at > > > https://www.myricom.com/software/sniffer10g/995-how-can-i-direct-sniffer10g-traffic-to-multiple-applications-using-snf-app-id.html > > doesn't really help, because my Bro deployment is a cluster, and the > environmental variables don't propagate to my worker hosts - in fact, > /proc/{bro_pid}/environ is 0-length on all the processes on the worker > hosts. > > 2. I tried to reverse-engineer how Security Onion does it, but I didn't > really glean anything that would help. > > Thanks for any info, > -- > Glenn Forbes Fleming Larratt > Cornell University IT Security Office > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Brandon Lattin Security Analyst University of Minnesota - University Information Security Office: 612-626-6672 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150320/e37da3b3/attachment.html From gl89 at cornell.edu Fri Mar 20 08:31:46 2015 From: gl89 at cornell.edu (Glenn Forbes Fleming Larratt) Date: Fri, 20 Mar 2015 11:31:46 -0400 (EDT) Subject: [Bro] Trying to get Bro to share Myricom cards with tcpdump or Snort In-Reply-To: References: Message-ID: I am using the Sniffer10G driver, and in support of getting proof of concept, dropped the number of workers/host from 12 to 11 (we're actually RAM-limited because of the quantity of data we're trying to process). Abridged utput of myri_endpoint_info: The myri_snf driver is configured to support a maximum of: 160 endpoints per NIC, 32 NICs per host =================================================================== Endpoint PID Command Info none none 32 43305 bro rx handle (11 shared rings) 33 43304 bro rx handle (11 shared rings) 34 43300 bro rx handle (11 shared rings) 35 43302 bro rx handle (11 shared rings) 36 43307 bro rx handle (11 shared rings) 37 43303 bro rx handle (11 shared rings) 38 43301 bro rx handle (11 shared rings) 39 43306 bro rx handle (11 shared rings) 40 43308 bro rx handle (11 shared rings) 41 43310 bro rx handle (11 shared rings) 42 43309 bro rx handle (11 shared rings) 64 43306 bro rx ring 0 65 43305 bro rx ring 1 66 43307 bro rx ring 2 67 43303 bro rx ring 3 68 43302 bro rx ring 4 69 43308 bro rx ring 5 70 43309 bro rx ring 6 71 43301 bro rx ring 7 72 43300 bro rx ring 8 73 43310 bro rx ring 9 74 43304 bro rx ring 10 There are currently 22 regular endpoints open -- Glenn Forbes Fleming Larratt Cornell University IT Security Office On Fri, 20 Mar 2015, Brandon Lattin wrote: > Just to verify, you're using the Sniffer10G v3 driver, yes? > Assuming you are, keep in mind that each interface is still limited to 32 ring buffers (this is what got me). So > plan on running something like 16 for Bro and 16 for Snort/Suricata.? > > On Fri, Mar 20, 2015 at 10:18 AM, Glenn Forbes Fleming Larratt wrote: > Folks, > > Can anyone point to a Bro+Snort HOWTO that would help me get Myricom cards > to share? > > 1. Following the directions at > > ??https://www.myricom.com/software/sniffer10g/995-how-can-i-direct-sniffer10g-traffic-to-multiple-applications-us > ing-snf-app-id.html > > doesn't really help, because my Bro deployment is a cluster, and the > environmental variables don't propagate to my worker hosts - in fact, > /proc/{bro_pid}/environ is 0-length on all the processes on the worker > hosts. > > 2. I tried to reverse-engineer how Security Onion does it, but I didn't > really glean anything that would help. > > Thanks for any info, > -- > Glenn Forbes Fleming Larratt > Cornell University IT Security Office > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > -- > Brandon LattinSecurity Analyst > University of Minnesota - University Information Security > Office: 612-626-6672 > > From asharma at lbl.gov Fri Mar 20 08:34:57 2015 From: asharma at lbl.gov (Aashish Sharma) Date: Fri, 20 Mar 2015 08:34:57 -0700 Subject: [Bro] Trying to get Bro to share Myricom cards with tcpdump or Snort In-Reply-To: References: Message-ID: <20150320153455.GD3785@yaksha.lbl.gov> Hello Glenn: (You'd need myricom sniffer V3 drivers inorder to run multiple applications. V2 only allows one application to listen. ) for Bro workers: Try setting your worker nodes like the following: [worker-1] type=worker host=bro-worker.site.edu interface=myri0 lb_method=myricom lb_procs=10 pin_cpus=3,5,7,9,11,13,15,17,19,21 env_vars="LD_LIBRARY_PATH=/usr/local/opt/snf/lib:$PATH SNF_DATARING_SIZE=0x20000000 SNF_NUM_RINGS=10 SNF_FLAGS=0x1" - Also, I have put the following in broctl.cfg : env_vars="LD_LIBRARY_PATH=/usr/local/opt/snf/lib" - And have LD_LIBRARY_PATH=/usr/local/opt/snf/lib in my .bash_profile of bro user. (depending on what shell you are using) This config works quite alright. Others can chimein if they have a more optimal config. Oh btw, CPU numbering is different on FreeBSD vs linux so depending on your OS, make sure you are running a worker on each core, instead of 2 workers on core+hyperthread leaving buch of other cores free. Above pin_cpu scheme is for FreeBSD. I believe linux is 1,2,3,4,5,6,7...... (not sure). Hope this helps. Thanks, Aashish On Fri, Mar 20, 2015 at 11:18:29AM -0400, Glenn Forbes Fleming Larratt wrote: > Folks, > > Can anyone point to a Bro+Snort HOWTO that would help me get Myricom cards > to share? > > 1. Following the directions at > > https://www.myricom.com/software/sniffer10g/995-how-can-i-direct-sniffer10g-traffic-to-multiple-applications-using-snf-app-id.html > > doesn't really help, because my Bro deployment is a cluster, and the > environmental variables don't propagate to my worker hosts - in fact, > /proc/{bro_pid}/environ is 0-length on all the processes on the worker > hosts. > > 2. I tried to reverse-engineer how Security Onion does it, but I didn't > really glean anything that would help. > > Thanks for any info, > -- > Glenn Forbes Fleming Larratt > Cornell University IT Security Office > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Lawrence Berkeley National Laboratory http://go.lbl.gov/pgp-aashish Office: (510)-495-2680 Cell: (510)-612-7971 From michalpurzynski1 at gmail.com Fri Mar 20 08:38:53 2015 From: michalpurzynski1 at gmail.com (=?UTF-8?B?TWljaGHFgiBQdXJ6ecWEc2tp?=) Date: Fri, 20 Mar 2015 16:38:53 +0100 Subject: [Bro] Trying to get Bro to share Myricom cards with tcpdump or Snort In-Reply-To: References: Message-ID: >From my Bro's node.cfg - look at the "env_vars". I don't use the multi-application mode but that is how you pass the variables to Bro workers. [nsm7-eth4] type=worker host=a.b.c.d interface=eth4 lb_method=myricom lb_procs=12 pin_cpus=1,2,3,4,5,6,7,8,9,10,11,12 env_vars=SNF_DEBUG_MASK=0x3,SNF_DESCRING_SIZE=4294967296,SNF_DATARING_SIZE=17179869184 For other applications you will most likely have the modify the SO startup/stop scripts, putting variables just before the application is started. On Fri, Mar 20, 2015 at 4:31 PM, Glenn Forbes Fleming Larratt wrote: > I am using the Sniffer10G driver, and in support of getting proof of > concept, dropped the number of workers/host from 12 to 11 (we're actually > RAM-limited because of the quantity of data we're trying to process). > > Abridged utput of myri_endpoint_info: > The myri_snf driver is configured to support a maximum of: > 160 endpoints per NIC, 32 NICs per host > =================================================================== > Endpoint PID Command Info > none none > 32 43305 bro rx handle (11 shared rings) > 33 43304 bro rx handle (11 shared rings) > 34 43300 bro rx handle (11 shared rings) > 35 43302 bro rx handle (11 shared rings) > 36 43307 bro rx handle (11 shared rings) > 37 43303 bro rx handle (11 shared rings) > 38 43301 bro rx handle (11 shared rings) > 39 43306 bro rx handle (11 shared rings) > 40 43308 bro rx handle (11 shared rings) > 41 43310 bro rx handle (11 shared rings) > 42 43309 bro rx handle (11 shared rings) > 64 43306 bro rx ring 0 > 65 43305 bro rx ring 1 > 66 43307 bro rx ring 2 > 67 43303 bro rx ring 3 > 68 43302 bro rx ring 4 > 69 43308 bro rx ring 5 > 70 43309 bro rx ring 6 > 71 43301 bro rx ring 7 > 72 43300 bro rx ring 8 > 73 43310 bro rx ring 9 > 74 43304 bro rx ring 10 > There are currently 22 regular endpoints open > > > -- > Glenn Forbes Fleming Larratt > Cornell University IT Security Office > > On Fri, 20 Mar 2015, Brandon Lattin wrote: > >> Just to verify, you're using the Sniffer10G v3 driver, yes? >> Assuming you are, keep in mind that each interface is still limited to 32 >> ring buffers (this is what got me). So >> plan on running something like 16 for Bro and 16 for Snort/Suricata. >> >> On Fri, Mar 20, 2015 at 10:18 AM, Glenn Forbes Fleming Larratt >> wrote: >> Folks, >> >> Can anyone point to a Bro+Snort HOWTO that would help me get Myricom >> cards >> to share? >> >> 1. Following the directions at >> >> >> https://www.myricom.com/software/sniffer10g/995-how-can-i-direct-sniffer10g-traffic-to-multiple-applications-us >> ing-snf-app-id.html >> >> doesn't really help, because my Bro deployment is a cluster, and the >> environmental variables don't propagate to my worker hosts - in >> fact, >> /proc/{bro_pid}/environ is 0-length on all the processes on the >> worker >> hosts. >> >> 2. I tried to reverse-engineer how Security Onion does it, but I >> didn't >> really glean anything that would help. >> >> Thanks for any info, >> -- >> Glenn Forbes Fleming Larratt >> Cornell University IT Security Office >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> >> >> >> -- >> Brandon LattinSecurity Analyst >> University of Minnesota - University Information Security >> Office: 612-626-6672 >> > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From vlad at grigorescu.org Fri Mar 20 10:46:20 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Fri, 20 Mar 2015 12:46:20 -0500 Subject: [Bro] SMB2 module In-Reply-To: References: Message-ID: Hi Danilo, One of the bottlenecks of SMB development has been a lack of real-world testing, so I'd definitely appreciate any bugs or feedback you run into. On Thu, Mar 19, 2015 at 10:48 AM, Danilo Nicol? wrote: I've substituted src/analyzer/protocol/smb, src/analyzer/protocol/netbios, > init-bare.bro and init-default.bro from SMB2 version to master version. > I don't quite understand this - can you elaborate on what specifically you did here? If you did the git merge topic/vladg/smb, that should replace everything for you. Were you seeing merge conflicts? I can get those cleaned up, if so. > Thread 1 (Thread 0x7f3337201780 (LWP 22674)): > #0 0x0000000000816193 in Serializer::Write (this=0x7fffc052fd00, v=35329, > tag=0xb7a68f "stype") at /home/danko/bro/src/Serializer.h:57 > #1 0x0000000000815fdc in SerialObj::DoSerialize (this=0x2b2bf00, > info=0x7fffc052fd60) at /home/danko/bro/src/SerialObj.cc:268 > #2 0x00000000007df8f6 in BroObj::DoSerialize (this=0x2b2bf00, > info=0x7fffc052fd60) at /home/danko/bro/src/Obj.cc:226 > #3 0x0000000000843002 in BroType::DoSerialize (this=0x2b2bf00, > info=0x7fffc052fd60) at /home/danko/bro/src/Type.cc:283 > #4 0x000000000081585b in SerialObj::Serialize (this=0x2b2bf00, > info=0x7fffc052fd60) at /home/danko/bro/src/SerialObj.cc:121 > #5 0x0000000000842cce in BroType::Serialize (this=0x2b2bf00, > info=0x7fffc052fd60) at /home/danko/bro/src/Type.cc:212 > #6 0x00000000008438ec in TypeList::DoSerialize (this=0x2b402e0, > info=0x7fffc052fd60) at /home/danko/bro/src/Type.cc:392 > #7 0x000000000081585b in SerialObj::Serialize (this=0x2b402e0, > info=0x7fffc052fd60) at /home/danko/bro/src/SerialObj.cc:121 > ... > #81382 0x0000000000837f2a in ForStmt::DoExec (this=0x4c90610, f=0x6e5d9c0, > v=0x740a610, flow=@0x7fffc0530080: FLOW_NEXT) at > /home/danko/bro/src/Stmt.cc:1358 > #81383 0x0000000000833db1 in ExprStmt::Exec (this=0x4c90610, f=0x6e5d9c0, > flow=@0x7fffc0530080: FLOW_NEXT) at /home/danko/bro/src/Stmt.cc:373 > #81384 0x0000000000839969 in StmtList::Exec (this=0x4c8f850, f=0x6e5d9c0, > flow=@0x7fffc0530080: FLOW_NEXT) at /home/danko/bro/src/Stmt.cc:1764 > #81385 0x0000000000839969 in StmtList::Exec (this=0x4c93a60, f=0x6e5d9c0, > flow=@0x7fffc0530080: FLOW_NEXT) at /home/danko/bro/src/Stmt.cc:1764 > #81386 0x00000000007a4828 in BroFunc::Call (this=0x4974a80, > args=0x5acc3c0, parent=0x0) at /home/danko/bro/src/Func.cc:403 > #81387 0x000000000077d5a4 in EventHandler::Call (this=0x49ae420, > vl=0x5acc3c0, no_remote=false) at /home/danko/bro/src/EventHandler.cc:130 > #81388 0x0000000000731ff1 in Event::Dispatch (this=0x70daec0, > no_remote=false) at /home/danko/bro/src/Event.h:50 > #81389 0x000000000077ccdd in EventMgr::Dispatch (this=0xf65e60 ) at > /home/danko/bro/src/Event.cc:111 > #81390 0x000000000077cde8 in EventMgr::Drain (this=0xf65e60 ) at > /home/danko/bro/src/Event.cc:128 > #81391 0x00000000007dbfa7 in net_run () at /home/danko/bro/src/Net.cc:374 > #81392 0x000000000073105c in main (argc=19, argv=0x7fffc05309b8) at > /home/danko/bro/src/main.cc:1212 > I've seen errors similar to this before, but I'm not sure it's related to SMB. Usually the cause of this is that Bro can't do DNS queries (there are a few scripts that do reverse lookups). Do you see the same behavior if you run git/master on this system (with no SMB changes)? Thanks, --Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150320/7f8a6f9b/attachment.html From gl89 at cornell.edu Fri Mar 20 12:00:26 2015 From: gl89 at cornell.edu (Glenn Forbes Fleming Larratt) Date: Fri, 20 Mar 2015 15:00:26 -0400 (EDT) Subject: [Bro] Trying to get Bro to share Myricom cards with tcpdump or Snort In-Reply-To: References: Message-ID: Folks, What a resource this list is! Thanks to Brandon, Aashish, and Micha?, I have the answer I need - configuring "env_vars=" in Bro's node.cfg file did the trick. Many thanks! -g -- Glenn Forbes Fleming Larratt Cornell University IT Security Office On Fri, 20 Mar 2015, Glenn Forbes Fleming Larratt wrote: > Folks, > > Can anyone point to a Bro+Snort HOWTO that would help me get Myricom cards > to share? > > 1. Following the directions at > > https://www.myricom.com/software/sniffer10g/995-how-can-i-direct-sniffer10g-traffic-to-multiple-applications-using-snf-app-id.html > > doesn't really help, because my Bro deployment is a cluster, and the > environmental variables don't propagate to my worker hosts - in fact, > /proc/{bro_pid}/environ is 0-length on all the processes on the worker > hosts. > > 2. I tried to reverse-engineer how Security Onion does it, but I didn't > really glean anything that would help. > > Thanks for any info, > From vlad at grigorescu.org Fri Mar 20 12:08:38 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Fri, 20 Mar 2015 14:08:38 -0500 Subject: [Bro] SMB2 module In-Reply-To: References: Message-ID: Please try to keep the Bro list CC-ed on this, as it might be useful to others. That error location makes sense - it's where I would expect to see problems if there's an issue with DNS. What I'm confused about is that SMB and SSH should be completely unrelated. How exactly are you disabling the SMB plugin when you don't see any errors? You might just want to comment out the following lines in your local.bro: @load protocols/ssh/interesting-hostnames @load frameworks/files/detect-MHR Of course, the "better" solution would be to fix the system so that it can do reverse DNS lookups (and TXT queries for detect-MHR) :-) --Vlad On Fri, Mar 20, 2015 at 1:55 PM, Danilo Nicol? wrote: > Hi Vlad, > > Thanks for your reply. > Yes, I did the git merge and I didn't have any conflicts. > > About the stacktrace, that error is raised up when Bro logs SSH packets > with SMB plugin active. In particular, the SIGBUS error is catched when the > script > /usr/local/bro/share/bro/policy/protocols/ssh/interesting-hostnames.bro is > launched. > I've debugged the stacktrace and I found the row when the SIGBUS is raised > up on the script: > > 'when ( local hostname = lookup_addr(host) )' > > This is the gdb analysis: > > Reading symbols from ./bro...done. > (gdb) r > Starting program: /usr/local/bro/bin/bro -i eth0 -U .status -p broctl -p > broctl-live -p standalon > /usr/local/bro/share/bro/policy/protocols/ssh/interesting-hostnames.bro > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > listening on eth0, capture length 8192 bytes > > [New Thread 0x7ffff588e700 (LWP 5824)] > [New Thread 0x7ffff508d700 (LWP 5825)] > tcmalloc: large alloc 1562501120 bytes == 0x42420000 @ 0x7ffff70f1d9c > 0x7ffff7111845 0x5e5846 0x5e5a5b 0x600f0c 0x603958 0x5e620d 0x60352c > 0x5e620d 0x6038c2 0x603a25 0x5e620d 0x60317e 0x603209 0x5e620d 0x547d31 > 0x5e620d 0x6038a4 0x603a25 0x5e620d 0x547d31 0x5e620d 0x6038a4 0x603a25 > 0x5e620d 0x547d31 0x5e620d 0x6038a4 0x603a25 0x5e620d 0x547d31 > > Program received signal SIGBUS, Bus error. > 0x00000000005e58c7 in BinarySerializationFormat::Write (this=0x427a090, > v=, tag=) at > /home/danko/bro_stable/bro/src/SerializationFormat.cc:311 > 311 return WriteData(&v, sizeof(v)); > (gdb) > > > If I try to disable smb plugin I don't receive any errors. > > > Best regards, > > Danilo > > > > 2015-03-20 18:46 GMT+01:00 Vlad Grigorescu : > >> Hi Danilo, >> >> One of the bottlenecks of SMB development has been a lack of real-world >> testing, so I'd definitely appreciate any bugs or feedback you run into. >> >> On Thu, Mar 19, 2015 at 10:48 AM, Danilo Nicol? >> wrote: >> >> I've substituted src/analyzer/protocol/smb, >>> src/analyzer/protocol/netbios, init-bare.bro and init-default.bro from SMB2 >>> version to master version. >>> >> >> I don't quite understand this - can you elaborate on what specifically >> you did here? If you did the git merge topic/vladg/smb, that should replace >> everything for you. Were you seeing merge conflicts? I can get those >> cleaned up, if so. >> >> >>> Thread 1 (Thread 0x7f3337201780 (LWP 22674)): >>> #0 0x0000000000816193 in Serializer::Write (this=0x7fffc052fd00, >>> v=35329, tag=0xb7a68f "stype") at /home/danko/bro/src/Serializer.h:57 >>> #1 0x0000000000815fdc in SerialObj::DoSerialize (this=0x2b2bf00, >>> info=0x7fffc052fd60) at /home/danko/bro/src/SerialObj.cc:268 >>> #2 0x00000000007df8f6 in BroObj::DoSerialize (this=0x2b2bf00, >>> info=0x7fffc052fd60) at /home/danko/bro/src/Obj.cc:226 >>> #3 0x0000000000843002 in BroType::DoSerialize (this=0x2b2bf00, >>> info=0x7fffc052fd60) at /home/danko/bro/src/Type.cc:283 >>> #4 0x000000000081585b in SerialObj::Serialize (this=0x2b2bf00, >>> info=0x7fffc052fd60) at /home/danko/bro/src/SerialObj.cc:121 >>> #5 0x0000000000842cce in BroType::Serialize (this=0x2b2bf00, >>> info=0x7fffc052fd60) at /home/danko/bro/src/Type.cc:212 >>> #6 0x00000000008438ec in TypeList::DoSerialize (this=0x2b402e0, >>> info=0x7fffc052fd60) at /home/danko/bro/src/Type.cc:392 >>> #7 0x000000000081585b in SerialObj::Serialize (this=0x2b402e0, >>> info=0x7fffc052fd60) at /home/danko/bro/src/SerialObj.cc:121 >>> ... >>> #81382 0x0000000000837f2a in ForStmt::DoExec (this=0x4c90610, >>> f=0x6e5d9c0, v=0x740a610, flow=@0x7fffc0530080: FLOW_NEXT) at >>> /home/danko/bro/src/Stmt.cc:1358 >>> #81383 0x0000000000833db1 in ExprStmt::Exec (this=0x4c90610, >>> f=0x6e5d9c0, flow=@0x7fffc0530080: FLOW_NEXT) at >>> /home/danko/bro/src/Stmt.cc:373 >>> #81384 0x0000000000839969 in StmtList::Exec (this=0x4c8f850, >>> f=0x6e5d9c0, flow=@0x7fffc0530080: FLOW_NEXT) at >>> /home/danko/bro/src/Stmt.cc:1764 >>> #81385 0x0000000000839969 in StmtList::Exec (this=0x4c93a60, >>> f=0x6e5d9c0, flow=@0x7fffc0530080: FLOW_NEXT) at >>> /home/danko/bro/src/Stmt.cc:1764 >>> #81386 0x00000000007a4828 in BroFunc::Call (this=0x4974a80, >>> args=0x5acc3c0, parent=0x0) at /home/danko/bro/src/Func.cc:403 >>> #81387 0x000000000077d5a4 in EventHandler::Call (this=0x49ae420, >>> vl=0x5acc3c0, no_remote=false) at /home/danko/bro/src/EventHandler.cc:130 >>> #81388 0x0000000000731ff1 in Event::Dispatch (this=0x70daec0, >>> no_remote=false) at /home/danko/bro/src/Event.h:50 >>> #81389 0x000000000077ccdd in EventMgr::Dispatch (this=0xf65e60 ) at >>> /home/danko/bro/src/Event.cc:111 >>> #81390 0x000000000077cde8 in EventMgr::Drain (this=0xf65e60 ) at >>> /home/danko/bro/src/Event.cc:128 >>> #81391 0x00000000007dbfa7 in net_run () at /home/danko/bro/src/Net.cc:374 >>> #81392 0x000000000073105c in main (argc=19, argv=0x7fffc05309b8) at >>> /home/danko/bro/src/main.cc:1212 >>> >> >> I've seen errors similar to this before, but I'm not sure it's related >> to SMB. Usually the cause of this is that Bro can't do DNS queries (there >> are a few scripts that do reverse lookups). Do you see the same behavior if >> you run git/master on this system (with no SMB changes)? >> >> Thanks, >> >> --Vlad >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150320/640cb9a3/attachment.html From seth at icir.org Fri Mar 20 12:27:17 2015 From: seth at icir.org (Seth Hall) Date: Fri, 20 Mar 2015 15:27:17 -0400 Subject: [Bro] SMB2 module In-Reply-To: References: Message-ID: > On Mar 20, 2015, at 3:08 PM, Vlad Grigorescu wrote: > > Of course, the "better" solution would be to fix the system so that it can do reverse DNS lookups (and TXT queries for detect-MHR) :-) Another option here is to force Bro into a mode where it fakes DNS responses internally. Unfortunately there isn?t a switch to enable this in scripts, but you can change the behavior with an environment variable: BRO_DNS_FAKE=1 bro -r somepackets.pcap .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From dani.nicolo at gmail.com Fri Mar 20 16:35:41 2015 From: dani.nicolo at gmail.com (=?UTF-8?Q?Danilo_Nicol=C3=B2?=) Date: Sat, 21 Mar 2015 00:35:41 +0100 Subject: [Bro] SMB2 module In-Reply-To: References: Message-ID: Hi, 2015-03-20 20:27 GMT+01:00 Seth Hall : > > > On Mar 20, 2015, at 3:08 PM, Vlad Grigorescu > wrote: > > > > Of course, the "better" solution would be to fix the system so that it > can do reverse DNS lookups (and TXT queries for detect-MHR) :-) > At the line 35 of /usr/local/bro/share/bro/policy/protocols/ssh/interesting-hostnames.bro script there's the function lookup_addr(host) that invoke DNS lookup, so I think definitely that the problem is in this function. > Another option here is to force Bro into a mode where it fakes DNS > responses internally. Unfortunately there isn?t a switch to enable this in > scripts, but you can change the behavior with an environment variable: > > BRO_DNS_FAKE=1 bro -r somepackets.pcap > I've tried to run bro with BRO_DNS_FAKE=1 env but unfortunately it didn't work. I've received the SIGSEV signal, below you can see the gdb log Program received signal SIGSEGV, Segmentation fault. 0x000000000060a5d9 in SerializationFormat::WriteData (this=0x7ffff001b780, b=b at entry=0x7fffff7ff03c, count=count at entry=2) at /home/danko/bro_smb/bro/src/SerializationFormat.cc:87 87 memcpy(output + output_pos, b, count); (gdb) p output $1 = 0x7fff51d14010 "\001" As Vlad as suggested to me, I'm going to disable these scripts and I'll let you know asap. Thank you so much. Regards, Danilo -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150321/0de6842f/attachment-0001.html From dani.nicolo at gmail.com Mon Mar 23 03:07:15 2015 From: dani.nicolo at gmail.com (=?UTF-8?Q?Danilo_Nicol=C3=B2?=) Date: Mon, 23 Mar 2015 11:07:15 +0100 Subject: [Bro] SMB2 module In-Reply-To: References: Message-ID: Hi Seth, Sorry, I've to rectify my latest reply: inserting the environment variable BRO_DNS_FAKE Bro seems to work now. I've tried also to disable the affected scripts and Bro works too. I'm little confused about the different behavior: if I set BRO_DNS_FAKE=1, Will dns logs be altered significantly? Thank you so much. Best regards, Danilo 2015-03-21 0:35 GMT+01:00 Danilo Nicol? : > Hi, > > > 2015-03-20 20:27 GMT+01:00 Seth Hall : > >> >> > On Mar 20, 2015, at 3:08 PM, Vlad Grigorescu >> wrote: >> > >> > Of course, the "better" solution would be to fix the system so that it >> can do reverse DNS lookups (and TXT queries for detect-MHR) :-) >> > > At the line 35 of > /usr/local/bro/share/bro/policy/protocols/ssh/interesting-hostnames.bro > script there's the function > > lookup_addr(host) > > that invoke DNS lookup, so I think definitely that the problem is in this > function. > > >> Another option here is to force Bro into a mode where it fakes DNS >> responses internally. Unfortunately there isn?t a switch to enable this in >> scripts, but you can change the behavior with an environment variable: >> >> BRO_DNS_FAKE=1 bro -r somepackets.pcap >> > > I've tried to run bro with BRO_DNS_FAKE=1 env but unfortunately it didn't > work. > I've received the SIGSEV signal, below you can see the gdb log > > Program received signal SIGSEGV, Segmentation fault. > 0x000000000060a5d9 in SerializationFormat::WriteData (this=0x7ffff001b780, > b=b at entry=0x7fffff7ff03c, count=count at entry=2) > at /home/danko/bro_smb/bro/src/SerializationFormat.cc:87 > 87 memcpy(output + output_pos, b, count); > (gdb) p output > $1 = 0x7fff51d14010 "\001" > > As Vlad as suggested to me, I'm going to disable these scripts and I'll > let you know asap. > > Thank you so much. > > Regards, > Danilo > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150323/3316280f/attachment.html From seth at icir.org Mon Mar 23 06:21:50 2015 From: seth at icir.org (Seth Hall) Date: Mon, 23 Mar 2015 09:21:50 -0400 Subject: [Bro] SMB2 module In-Reply-To: References: Message-ID: <44908704-0F9E-4F47-8E85-08DEBDB76604@icir.org> > On Mar 23, 2015, at 6:07 AM, Danilo Nicol? wrote: > > I'm little confused about the different behavior: if I set BRO_DNS_FAKE=1, Will dns logs be altered significantly? No, BRO_DNS_FAKE only changes active DNS lookups. When Bro itself goes out to lookup a name in DNS it will return junk information. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From carlopmart at gmail.com Tue Mar 24 03:57:05 2015 From: carlopmart at gmail.com (C. L. Martinez) Date: Tue, 24 Mar 2015 10:57:05 +0000 Subject: [Bro] Convert utf8 in smtp.log Message-ID: Hi all, Is it possible to convert utf8 characters in subject emails logged by bro to a "readable" format? Thanks. From seth at icir.org Tue Mar 24 08:32:10 2015 From: seth at icir.org (Seth Hall) Date: Tue, 24 Mar 2015 11:32:10 -0400 Subject: [Bro] Convert utf8 in smtp.log In-Reply-To: References: Message-ID: <74FDF2CA-4E22-4539-B257-46ACFB909DE4@icir.org> > On Mar 24, 2015, at 6:57 AM, C. L. Martinez wrote: > > Is it possible to convert utf8 characters in subject emails logged by > bro to a "readable" format? There is a script! (i?ve heard that it has caused problems for some people though) :) It adds a new fields named ?decoded_subject? to your smtp log: https://github.com/sethhall/bro-junk-drawer/blob/master/smtp-decode-encoded-word-subjects.bro .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From provnov at solidex.by Wed Mar 25 05:11:04 2015 From: provnov at solidex.by (Rovnov Pavel) Date: Wed, 25 Mar 2015 15:11:04 +0300 Subject: [Bro] One-way TCP session to handle HTTP requests only Message-ID: <2D80CFEBA87EC04487AA762153342454010757B7@mercury.solidex.minsk.by> Hello! I'm looking for a monitoring solution that will give me an instrument to log all HTTP requests (including HTTPS). I see that Bro does this really well by default. But as soon as I will have huge amount of web traffic (like 10Gb/s+) I would like to process HTTP requests only by mirroring only one-way of TCP sessions. That will save a lot of processing power since HTTP request << HTTP response. I found only one reference to my idea that say that handling one-way TCP at best will slow down Bro (http://mailman.icsi.berkeley.edu/pipermail/bro/2006-October/001853.html ). So the questions are: 1) Can anyone confirm that using Bro to handle one-way TCP session is a bad idea? 2) Does anyone have any experience of tuning Bro to handle one-way TCP sessions? We might turn off unnecessary processing (e. g. policies that need 2-way session) to solve the task... Thanks! Pavel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150325/69a6b5df/attachment.html From doug.burks at gmail.com Wed Mar 25 05:52:39 2015 From: doug.burks at gmail.com (Doug Burks) Date: Wed, 25 Mar 2015 08:52:39 -0400 Subject: [Bro] Bro --> Google Safe Browsing API? Message-ID: Hello all, Has anybody developed a script to have Bro query the Google Safe Browsing API? http://googleonlinesecurity.blogspot.com/2015/03/even-more-unwanted-software-protection.html https://developers.google.com/safe-browsing/ Thanks! -- Doug Burks Need Security Onion Training or Commercial Support? http://securityonionsolutions.com From hosom at battelle.org Wed Mar 25 06:13:30 2015 From: hosom at battelle.org (Hosom, Stephen M) Date: Wed, 25 Mar 2015 13:13:30 +0000 Subject: [Bro] Bro --> Google Safe Browsing API? In-Reply-To: References: Message-ID: I was actually looking at this yesterday, however, because of the way that Google implements the API, this is non-trivial and would not really be something that I would feel comfortable using the current active http function(s) for. Basically, the API has requirements that you implement their rate limiting at the client level... so under certain conditions, Google could tell you 'Do not query again for another hour' and you're supposed to play along with their request. Rumor has it that someone is working on the active http module, so, I haven't looked into doing any of that myself. I'd love to take on Safe Browsing integration though. Maybe I'll just look into making Safe Browsing its own full blown plugin? Querying safe-browsing for at least the links that I parse from emails would be extremely desirable from my perspective. If you want to talk about it, feel free to ping me on IRC, since I'm always logged in during the day anyway, or, we can just keep the discussion on the mailing list so everyone can feel free to chime in. -----Original Message----- From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Doug Burks Sent: Wednesday, March 25, 2015 8:53 AM To: Subject: [Bro] Bro --> Google Safe Browsing API? Hello all, Has anybody developed a script to have Bro query the Google Safe Browsing API? http://googleonlinesecurity.blogspot.com/2015/03/even-more-unwanted-software-protection.html https://developers.google.com/safe-browsing/ Thanks! -- Doug Burks Need Security Onion Training or Commercial Support? http://securityonionsolutions.com _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From doug.burks at gmail.com Wed Mar 25 06:22:56 2015 From: doug.burks at gmail.com (Doug Burks) Date: Wed, 25 Mar 2015 09:22:56 -0400 Subject: [Bro] Bro --> Google Safe Browsing API? In-Reply-To: References: Message-ID: Hi Stephen, Does the rate limiting apply to the new "API v3"? "The Safe Browsing API is an experimental API that enables applications to download an encrypted table for local, client-side lookups of URLs that you would like to check. In 2014, we published a new version (v3) of the Safe Browsing API, which adds features and efficiency improvements to the previous v2. The Safe Browsing API is used by several browsers, including Google Chrome and Mozilla Firefox. You can start using the Safe Browsing API v3 now." https://developers.google.com/safe-browsing/ On Wed, Mar 25, 2015 at 9:13 AM, Hosom, Stephen M wrote: > I was actually looking at this yesterday, however, because of the way that Google implements the API, this is non-trivial and would not really be something that I would feel comfortable using the current active http function(s) for. Basically, the API has requirements that you implement their rate limiting at the client level... so under certain conditions, Google could tell you 'Do not query again for another hour' and you're supposed to play along with their request. > > Rumor has it that someone is working on the active http module, so, I haven't looked into doing any of that myself. I'd love to take on Safe Browsing integration though. Maybe I'll just look into making Safe Browsing its own full blown plugin? Querying safe-browsing for at least the links that I parse from emails would be extremely desirable from my perspective. > > If you want to talk about it, feel free to ping me on IRC, since I'm always logged in during the day anyway, or, we can just keep the discussion on the mailing list so everyone can feel free to chime in. > > -----Original Message----- > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Doug Burks > Sent: Wednesday, March 25, 2015 8:53 AM > To: > Subject: [Bro] Bro --> Google Safe Browsing API? > > Hello all, > > Has anybody developed a script to have Bro query the Google Safe Browsing API? > > http://googleonlinesecurity.blogspot.com/2015/03/even-more-unwanted-software-protection.html > > https://developers.google.com/safe-browsing/ > > Thanks! > > -- > Doug Burks > Need Security Onion Training or Commercial Support? > http://securityonionsolutions.com > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Doug Burks Need Security Onion Training or Commercial Support? http://securityonionsolutions.com From davidvasil at gmail.com Wed Mar 25 06:42:38 2015 From: davidvasil at gmail.com (David Vasil) Date: Wed, 25 Mar 2015 08:42:38 -0500 Subject: [Bro] Bro --> Google Safe Browsing API? In-Reply-To: References: Message-ID: I believe it still does: https://developers.google.com/safe-browsing/developers_guide_v3#RequestFrequency On Wed, Mar 25, 2015 at 8:22 AM, Doug Burks wrote: > Hi Stephen, > > Does the rate limiting apply to the new "API v3"? > > "The Safe Browsing API is an experimental API that enables > applications to download an encrypted table for local, client-side > lookups of URLs that you would like to check. In 2014, we published a > new version (v3) of the Safe Browsing API, which adds features and > efficiency improvements to the previous v2. The Safe Browsing API is > used by several browsers, including Google Chrome and Mozilla Firefox. > You can start using the Safe Browsing API v3 now." > > https://developers.google.com/safe-browsing/ > > On Wed, Mar 25, 2015 at 9:13 AM, Hosom, Stephen M > wrote: > > I was actually looking at this yesterday, however, because of the way > that Google implements the API, this is non-trivial and would not really be > something that I would feel comfortable using the current active http > function(s) for. Basically, the API has requirements that you implement > their rate limiting at the client level... so under certain conditions, > Google could tell you 'Do not query again for another hour' and you're > supposed to play along with their request. > > > > Rumor has it that someone is working on the active http module, so, I > haven't looked into doing any of that myself. I'd love to take on Safe > Browsing integration though. Maybe I'll just look into making Safe Browsing > its own full blown plugin? Querying safe-browsing for at least the links > that I parse from emails would be extremely desirable from my perspective. > > > > If you want to talk about it, feel free to ping me on IRC, since I'm > always logged in during the day anyway, or, we can just keep the discussion > on the mailing list so everyone can feel free to chime in. > > > > -----Original Message----- > > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of > Doug Burks > > Sent: Wednesday, March 25, 2015 8:53 AM > > To: > > Subject: [Bro] Bro --> Google Safe Browsing API? > > > > Hello all, > > > > Has anybody developed a script to have Bro query the Google Safe > Browsing API? > > > > > http://googleonlinesecurity.blogspot.com/2015/03/even-more-unwanted-software-protection.html > > > > https://developers.google.com/safe-browsing/ > > > > Thanks! > > > > -- > > Doug Burks > > Need Security Onion Training or Commercial Support? > > http://securityonionsolutions.com > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > -- > Doug Burks > Need Security Onion Training or Commercial Support? > http://securityonionsolutions.com > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150325/0639df22/attachment-0001.html From john at giggled.org Wed Mar 25 06:44:27 2015 From: john at giggled.org (John Green) Date: Wed, 25 Mar 2015 13:44:27 +0000 Subject: [Bro] Bro --> Google Safe Browsing API? In-Reply-To: References: Message-ID: On 25 March 2015 at 12:52, Doug Burks wrote: > Hello all, > > Has anybody developed a script to have Bro query the Google Safe Browsing > API? > For bulk lookups you need to maintain a local copy of the chunks which are basically black/white lists of hash prefixes of the canonicalised URL (Bloom filter). This is the same data Chrome/Firefox use for safe browsing. There is a reference implementation available which maintains a local copy. Then your script just needs to hash the URL (or one of a number of different permutations) and check the prefix if it is present in both lists. If it is present in the blacklist then followup with a query to Google for the full hash and compare. I wrote some shoddy code a while ago against v2 of this spec to maintain a local copy of the partial hashes within postgres. https://github.com/j-o-h-n-g/Mortimer/blob/master/importgoogle.py John -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150325/d90ee03c/attachment.html From vlad at grigorescu.org Wed Mar 25 06:57:19 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Wed, 25 Mar 2015 08:57:19 -0500 Subject: [Bro] Bro --> Google Safe Browsing API? In-Reply-To: References: Message-ID: On Wed, Mar 25, 2015 at 8:22 AM, Doug Burks wrote: > Hi Stephen, > > Does the rate limiting apply to the new "API v3"? > > "The Safe Browsing API is an experimental API that enables > applications to download an encrypted table for local, client-side > lookups of URLs that you would like to check. In 2014, we published a > new version (v3) of the Safe Browsing API, which adds features and > efficiency improvements to the previous v2. The Safe Browsing API is > used by several browsers, including Google Chrome and Mozilla Firefox. > You can start using the Safe Browsing API v3 now." > > https://developers.google.com/safe-browsing/ Yes, see: https://developers.google.com/safe-browsing/developers_guide_v3#RequestFrequency Essentially the issue is that there needs to be a piece between Bro and the API, which is handling downloading/updating the prefixsets, and ensuring that the request frequency is observed. It'd be interesting, but Bro integration with v3 is a difficult task. --Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150325/c08c0e85/attachment.html From dani.nicolo at gmail.com Wed Mar 25 10:08:48 2015 From: dani.nicolo at gmail.com (=?UTF-8?Q?Danilo_Nicol=C3=B2?=) Date: Wed, 25 Mar 2015 18:08:48 +0100 Subject: [Bro] SMB2 module In-Reply-To: <44908704-0F9E-4F47-8E85-08DEBDB76604@icir.org> References: <44908704-0F9E-4F47-8E85-08DEBDB76604@icir.org> Message-ID: Hi Seth, I've tested much Bro and in many cases it crashes with SIGSEV or SIGBUS (with BRO_DNS_FAKE=1). The problem is always in ssh/interesting-hostnames.bro in the when condition.. so I modified this script: before trying to remove the when condition and Bro didn't crash, later trying to remove lookup_addr function keeping the when condition and Bro crashed. Below you can see a snippet of the modified script with when condition local hostname : string; when ( hostname == "10.1.2.3" ) { if ( interesting_hostnames in hostname ) { NOTICE([$note=Interesting_Hostname_Login, $msg=fmt("Possible SSH login involving a %s %s with an interesting hostname.", Site::is_local_addr(host) ? "local" : "remote", host == c$id$orig_h ? "client" : "server"), $sub=hostname, $conn=c]); } } hostname = "10.1.2.3"; The gdb log is : Starting program: /usr/local/bro/bin/bro -i eth0 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro /usr/local/bro/share/bro/policy/protocols/ssh/interesting-hostnames.bro [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". listening on eth0, capture length 8192 bytes [New Thread 0x7ffff5a9a700 (LWP 6916)] WARNING: No Site::local_nets have been defined. It's usually a good idea to define your local networks. [New Thread 0x7ffff5299700 (LWP 6917)] [New Thread 0x7ffff4a98700 (LWP 6918)] [New Thread 0x7ffff4297700 (LWP 6919)] [New Thread 0x7ffff3a96700 (LWP 6920)] [New Thread 0x7ffff3295700 (LWP 6921)] 1427301195.578365 error in /usr/local/bro/share/bro/policy/protocols/ssh/geo-data.bro, line 30: Bro was not configured for GeoIP support (lookup_location(SSH::lookup_ip)) Program received signal SIGSEGV, Segmentation fault. 0x00000000007423e2 in Serializer::Write (this=0x7fffffffd860, v=true, tag=0xb7a563 "full") at /home/danko/bro/src/Serializer.h:62 62 DECLARE_IO(bool) (gdb) up 81400 #81400 0x0000000000851a58 in MutableVal::DoSerialize (this=0x44a79e0, info=0x7fffffffd8c0) at /home/danko/bro/src/Val.cc:656 656 DO_SERIALIZE(SER_MUTABLE_VAL, Val); (gdb) up #81401 0x000000000085a732 in RecordVal::DoSerialize (this=0x44a79e0, info=0x7fffffffd8c0) at /home/danko/bro/src/Val.cc:2813 2813 DO_SERIALIZE(SER_RECORD_VAL, MutableVal); (gdb) up #81402 0x000000000081587b in SerialObj::Serialize (this=0x44a79e0, info=0x7fffffffd8c0) at /home/danko/bro/src/SerialObj.cc:121 121 bool ret = DoSerialize(info); (gdb) up #81403 0x000000000084fdcb in Val::Serialize (this=0x44a79e0, info=0x7fffffffd8c0) at /home/danko/bro/src/Val.cc:100 100 return SerialObj::Serialize(info); (gdb) up #81404 0x000000000084fc7d in Val::Clone (this=0x44a79e0) at /home/danko/bro/src/Val.cc:83 83 if ( ! this->Serialize(&sinfo) ) (gdb) up #81405 0x00000000007a2fb3 in Frame::Clone (this=0x4482bd0) at /home/danko/bro/src/Frame.cc:78 78 f->frame[i] = frame[i] ? frame[i]->Clone() : 0; (gdb) up #81406 0x000000000083ee4c in Trigger::Trigger (this=0x18031e0, arg_cond=0x35ab080, arg_body=0x35ab020, arg_timeout_stmts=0x0, arg_timeout=0x0, arg_frame=0x4482bd0, arg_is_return=false, arg_location=0x35b94c0) at /home/danko/bro/src/Trigger.cc:108 108 frame = arg_frame->Clone(); (gdb) up #81407 0x000000000083b302 in WhenStmt::Exec (this=0x35b18e0, f=0x4482bd0, flow=@0x7fffffffdbe0: FLOW_NEXT) at /home/danko/bro/src/Stmt.cc:2166 2166 new Trigger(cond, s1, s2, timeout, f, is_return, location); (gdb) p *this.location $1 = { = {_vptr.SerialObj = 0xb71e30 , static NEVER = 0, static ALWAYS = 1, static factories = 0x1786000, static names = 0x1786060, static time_counter = 19515, serial_type = 0}, filename = 0x3586500 "/usr/local/bro/share/bro/policy/protocols/ssh/interesting-hostnames.bro", first_line = 36, last_line = 46, first_column = 0, last_column = 0, delete_data = false, timestamp = 0, text = 0x0, static register_type = {}, tid = {id = 417376, static counter = 455184}} (gdb) I've tried to search anything problem about the when condition like this but I haven't found nothing similar issues. Have you ever had this kind of problem? I hope I was helpful. Best regards, Danilo 2015-03-23 14:21 GMT+01:00 Seth Hall : > > > On Mar 23, 2015, at 6:07 AM, Danilo Nicol? > wrote: > > > > I'm little confused about the different behavior: if I set > BRO_DNS_FAKE=1, Will dns logs be altered significantly? > > No, BRO_DNS_FAKE only changes active DNS lookups. When Bro itself goes > out to lookup a name in DNS it will return junk information. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150325/b1601b34/attachment.html From provnov at solidex.by Wed Mar 25 10:11:34 2015 From: provnov at solidex.by (Rovnov Pavel) Date: Wed, 25 Mar 2015 20:11:34 +0300 Subject: [Bro] One-way TCP session to handle HTTP requests only Message-ID: <2D80CFEBA87EC04487AA762153342454010757C8@mercury.solidex.minsk.by> Hello again! I'm trying to run installation with client-to-server only traffic visible to Bro. This seems not to break Bro however the following messages fill weird.log: 1427302895.156616 C50xd821xHdTYgVRWj 172.x.x.x 33468 87.252.227.138 41223 data_before_established - F bro 1427302895.228297 CqeQYQ1Q4MgbwupuR8 172.x.x.x 45107 62.84.63.46 13871 possible_split_routing - F bro 1427302895.228985 CqeQYQ1Q4MgbwupuR8 172.x.x.x 45107 62.84.63.46 13871 data_before_established - F bro 1427302895.782191 CiSuNR2tWAfGBpuSxe 172.x.x.x 55007 80.249.82.211 11898 possible_split_routing - F bro 1427302895.783376 CiSuNR2tWAfGBpuSxe 172.x.x.x 55007 80.249.82.211 11898 data_before_established Does anyone know how to switch Bro into asymmetric mode? At least can I disable notices that need 2-way session? Thanks! Pavel From: Rovnov Pavel Sent: Wednesday, March 25, 2015 3:11 PM To: 'bro at bro.org' Subject: One-way TCP session to handle HTTP requests only Hello! I'm looking for a monitoring solution that will give me an instrument to log all HTTP requests (including HTTPS). I see that Bro does this really well by default. But as soon as I will have huge amount of web traffic (like 10Gb/s+) I would like to process HTTP requests only by mirroring only one-way of TCP sessions. That will save a lot of processing power since HTTP request << HTTP response. I found only one reference to my idea that say that handling one-way TCP at best will slow down Bro (http://mailman.icsi.berkeley.edu/pipermail/bro/2006-October/001853.html ). So the questions are: 1) Can anyone confirm that using Bro to handle one-way TCP session is a bad idea? 2) Does anyone have any experience of tuning Bro to handle one-way TCP sessions? We might turn off unnecessary processing (e. g. policies that need 2-way session) to solve the task... Thanks! Pavel -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150325/fce0c40e/attachment-0001.html From seth at icir.org Wed Mar 25 10:42:21 2015 From: seth at icir.org (Seth Hall) Date: Wed, 25 Mar 2015 13:42:21 -0400 Subject: [Bro] Bro --> Google Safe Browsing API? In-Reply-To: References: Message-ID: > On Mar 25, 2015, at 9:57 AM, Vlad Grigorescu wrote: > > Essentially the issue is that there needs to be a piece between Bro and the API, which is handling downloading/updating the prefixsets, and ensuring that the request frequency is observed. It'd be interesting, but Bro integration with v3 is a difficult task. Yeah, I agree. Google has been moving the service toward needing more frequent touches with them to get an accurate picture of matches against their list. This works perfectly fine for desktops that might see a maximum of 1000 URLs being requested per hour or something, but on a Bro cluster, there could be thousands per second. I had an implementation of the v1 of that API running with Bro years ago, but even that didn?t work well enough that I could ever distribute it. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From seth at icir.org Wed Mar 25 11:27:13 2015 From: seth at icir.org (Seth Hall) Date: Wed, 25 Mar 2015 14:27:13 -0400 Subject: [Bro] One-way TCP session to handle HTTP requests only In-Reply-To: <2D80CFEBA87EC04487AA762153342454010757C8@mercury.solidex.minsk.by> References: <2D80CFEBA87EC04487AA762153342454010757C8@mercury.solidex.minsk.by> Message-ID: <616FA81D-6C7D-49AE-BE18-E76928A6BC2D@icir.org> > On Mar 25, 2015, at 1:11 PM, Rovnov Pavel wrote: > > Does anyone know how to switch Bro into asymmetric mode? At least can I disable notices that need 2-way session? Unfortunately at this time, we don?t put much attention to asymmetric traffic analysis. This is something I?ve been wanting to do for a long time, but it hasn?t bubbled up high enough on the priority list yet. Any results you get from asymmetric traffic processing are coincidental, we don?t have any tests or anything that validate that Bro works in any particular scenario with asymmetric traffic. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From provnov at solidex.by Wed Mar 25 23:58:32 2015 From: provnov at solidex.by (Rovnov Pavel) Date: Thu, 26 Mar 2015 09:58:32 +0300 Subject: [Bro] One-way TCP session to handle HTTP requests only In-Reply-To: <616FA81D-6C7D-49AE-BE18-E76928A6BC2D@icir.org> References: <2D80CFEBA87EC04487AA762153342454010757C8@mercury.solidex.minsk.by> <616FA81D-6C7D-49AE-BE18-E76928A6BC2D@icir.org> Message-ID: <2D80CFEBA87EC04487AA762153342454010757C9@mercury.solidex.minsk.by> Hello Seth, To bubble up asymmetric traffic analysis higher in the list let me describe our scenario. We would like to analyze ~55Gb/s+ (5Gb/s upstream, 50Gb/s downstream) of web traffic (both HTTP and HTTPS). At layer 7 we need to know hostnames and perhaps URLs visited. In case we analyze upstream only we can reduce hardware requirements greatly. What causes Bro to be asymmetric intolerant: rule, BinPac,...? What is we disable all rules and leave only rules that solve the task? Will the result be still coincidental? Thanks for answers! Pavel -----Original Message----- From: Seth Hall [mailto:seth at icir.org] Sent: Wednesday, March 25, 2015 9:27 PM To: Rovnov Pavel Cc: bro at bro.org Subject: Re: [Bro] One-way TCP session to handle HTTP requests only > On Mar 25, 2015, at 1:11 PM, Rovnov Pavel wrote: > > Does anyone know how to switch Bro into asymmetric mode? At least can I disable notices that need 2-way session? Unfortunately at this time, we don?t put much attention to asymmetric traffic analysis. This is something I?ve been wanting to do for a long time, but it hasn?t bubbled up high enough on the priority list yet. Any results you get from asymmetric traffic processing are coincidental, we don?t have any tests or anything that validate that Bro works in any particular scenario with asymmetric traffic. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From carlopmart at gmail.com Thu Mar 26 03:01:37 2015 From: carlopmart at gmail.com (C. L. Martinez) Date: Thu, 26 Mar 2015 10:01:37 +0000 Subject: [Bro] Question about packet-bricks Message-ID: Hi all, I am trying to use packet-bricks in a FreeBSD 10.1 host with netmap enabled. Following README, I have executed the following commands: [ pmain(): line 463] Executing (null) [print_version(): line 345] BRICKS Version 0.1-alpha bricks> BRICKS.help() BRICKS Commands: help() print_status() show_stats() shutdown() Available subsystems within BRICKS have their own help() methods: PktEngine bricks> utilObj:enable_nmpipes() [ report(): line 222] stdin:1: attempt to index global 'utilObj' (a nil value) stack traceback: stdin:1: in main chunk [C]: ?bricks> pe = PktEngine.new("e0") bricks> print_status() [ report(): line 222] stdin:1: attempt to call global 'print_status' (a nil value) stack traceback: stdin:1: in main chunk [C]: ?bricks> pe = PktEngine.new("e0", 1024, 1) [pktengine_new(): line 178] Engine with name: e0 already exists bricks> lb = Brick.new("LoadBalancer", 2) bricks> lb:connect_input("vtnet2") bricks> lb:connect_output("vtnet2{0", "vtnet2{1", "em0") bricks> pe:link(lb) 546.982488 nm_open [444] overriding ARG3 0 546.982547 nm_open [457] overriding ifname vtnet2 ringid 0x0 flags 0x1 [netmap_link_iface(): line 175] Wait for 2 secs for phy reset [brick_link(): line 101] Linking e0 with link vtnet2 with batch size: 512 and qid: -1 549.055808 nm_open [444] overriding ARG3 0 [netmap_create_channel(): line 761] zerocopy for vtnet2 --> vtnet2{0 (index: 0) enabled [netmap_create_channel(): line 766] Created netmap:vtnet2{0 interface 549.101244 nm_open [444] overriding ARG3 0 [netmap_create_channel(): line 761] zerocopy for vtnet2 --> vtnet2{1 (index: 1) enabled [netmap_create_channel(): line 766] Created netmap:vtnet2{1 interface 549.142483 nm_open [444] overriding ARG3 0 [netmap_create_channel(): line 761] zerocopy for vtnet2 --> em0 (index: 2) enabled [netmap_create_channel(): line 766] Created netmap:em0 interface bricks> pe:start() bricks> [initiate_backend(): line 346] Engine e0 is listening on port 1239 pe.show_stats() [ report(): line 222] stdin:1: bad argument #1 to 'show_stats' (userdata expected, got no value) stack traceback: [C]: in function 'show_stats' stdin:1: in main chunk [C]: ?bricks> show.stats() [ report(): line 222] stdin:1: attempt to index global 'show' (a nil value) stack traceback: stdin:1: in main chunk [C]: ?bricks> dmesg shows me the following: 544.880658 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 544.946020 [ 676] netmap_update_config configuration changed (but fine) 544.981566 [1219] netmap_mem_global_config reconfiguring 545.014116 [ 818] netmap_config_obj_allocator XXX aligning object by 1 bytes 546.288980 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 546.353092 [ 79] vtnet_netmap_free_bufs freed 256 mbufs, 0 netmap bufs on 1 queues 546.982572 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 547.011973 [ 407] vtnet_netmap_config vtnet config txq=1, txd=512 rxq=1, rxd=512 549.055837 [ 325] netmap_pipe_krings_create 0xfffff80110f9ac00: case 1, create everything 549.101272 [ 325] netmap_pipe_krings_create 0xfffff801105e0800: case 1, create everything em0: link state changed to DOWN em0: link state changed to UP As you can see, stats failed. When I try to use tcpdump: root at nsm02:/dev # tcpdump -i 'netmap:vtnet2{0' tcpdump: netmap:vtnet2{0: No such device exists (BIOCSETIF failed: Device not configured) Please, any help?? From ajamshed at ICSI.Berkeley.EDU Thu Mar 26 03:24:25 2015 From: ajamshed at ICSI.Berkeley.EDU (Asim Jamshed) Date: Thu, 26 Mar 2015 19:24:25 +0900 Subject: [Bro] Question about packet-bricks In-Reply-To: References: Message-ID: <5513DE59.4000708@icsi.berkeley.edu> Hi, The utilObj functions are actually referenced in scripts/startup-one-thread.lua startup script file. So you should start your packet-bricks using the following command: # bin/bricks -f scripts/startup-one-thread.lua I suggest that you go through the scripts/startup-one-thread.lua and scripts/utils.lua files (after reading the README file) before using packet-bricks. Thanks, --Asim On 03/26/2015 07:01 PM, C. L. Martinez wrote: > Hi all, > > I am trying to use packet-bricks in a FreeBSD 10.1 host with netmap > enabled. Following README, I have executed the following commands: > > [ pmain(): line 463] Executing (null) > [print_version(): line 345] BRICKS Version 0.1-alpha > bricks> BRICKS.help() > BRICKS Commands: > help() > print_status() > show_stats() > shutdown() > Available subsystems within BRICKS have their own help() methods: > PktEngine > bricks> utilObj:enable_nmpipes() > [ report(): line 222] stdin:1: attempt to index global 'utilObj' > (a nil value) > stack traceback: > stdin:1: in main chunk > [C]: ?bricks> pe = PktEngine.new("e0") > bricks> print_status() > [ report(): line 222] stdin:1: attempt to call global > 'print_status' (a nil value) > stack traceback: > stdin:1: in main chunk > [C]: ?bricks> pe = PktEngine.new("e0", 1024, 1) > [pktengine_new(): line 178] Engine with name: e0 already exists > bricks> lb = Brick.new("LoadBalancer", 2) > bricks> lb:connect_input("vtnet2") > bricks> lb:connect_output("vtnet2{0", "vtnet2{1", "em0") > bricks> pe:link(lb) > 546.982488 nm_open [444] overriding ARG3 0 > 546.982547 nm_open [457] overriding ifname vtnet2 ringid 0x0 flags 0x1 > [netmap_link_iface(): line 175] Wait for 2 secs for phy reset > [brick_link(): line 101] Linking e0 with link vtnet2 with batch size: > 512 and qid: -1 > 549.055808 nm_open [444] overriding ARG3 0 > [netmap_create_channel(): line 761] zerocopy for vtnet2 --> vtnet2{0 > (index: 0) enabled > [netmap_create_channel(): line 766] Created netmap:vtnet2{0 interface > 549.101244 nm_open [444] overriding ARG3 0 > [netmap_create_channel(): line 761] zerocopy for vtnet2 --> vtnet2{1 > (index: 1) enabled > [netmap_create_channel(): line 766] Created netmap:vtnet2{1 interface > 549.142483 nm_open [444] overriding ARG3 0 > [netmap_create_channel(): line 761] zerocopy for vtnet2 --> em0 > (index: 2) enabled > [netmap_create_channel(): line 766] Created netmap:em0 interface > bricks> pe:start() > bricks> [initiate_backend(): line 346] Engine e0 is listening on port 1239 > pe.show_stats() > [ report(): line 222] stdin:1: bad argument #1 to 'show_stats' > (userdata expected, got no value) > stack traceback: > [C]: in function 'show_stats' > stdin:1: in main chunk > [C]: ?bricks> show.stats() > [ report(): line 222] stdin:1: attempt to index global 'show' (a nil value) > stack traceback: > stdin:1: in main chunk > [C]: ?bricks> > > dmesg shows me the following: > > 544.880658 [ 407] vtnet_netmap_config vtnet config txq=1, > txd=512 rxq=1, rxd=512 > 544.946020 [ 676] netmap_update_config configuration changed (but fine) > 544.981566 [1219] netmap_mem_global_config reconfiguring > 545.014116 [ 818] netmap_config_obj_allocator XXX aligning object by 1 bytes > 546.288980 [ 407] vtnet_netmap_config vtnet config txq=1, > txd=512 rxq=1, rxd=512 > 546.353092 [ 79] vtnet_netmap_free_bufs freed 256 mbufs, 0 netmap > bufs on 1 queues > 546.982572 [ 407] vtnet_netmap_config vtnet config txq=1, > txd=512 rxq=1, rxd=512 > 547.011973 [ 407] vtnet_netmap_config vtnet config txq=1, > txd=512 rxq=1, rxd=512 > 549.055837 [ 325] netmap_pipe_krings_create 0xfffff80110f9ac00: case > 1, create everything > 549.101272 [ 325] netmap_pipe_krings_create 0xfffff801105e0800: case > 1, create everything > em0: link state changed to DOWN > em0: link state changed to UP > > As you can see, stats failed. When I try to use tcpdump: > > root at nsm02:/dev # tcpdump -i 'netmap:vtnet2{0' > tcpdump: netmap:vtnet2{0: No such device exists > (BIOCSETIF failed: Device not configured) > > Please, any help?? > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From carlopmart at gmail.com Thu Mar 26 03:52:40 2015 From: carlopmart at gmail.com (C. L. Martinez) Date: Thu, 26 Mar 2015 10:52:40 +0000 Subject: [Bro] Question about packet-bricks In-Reply-To: <5513DE59.4000708@icsi.berkeley.edu> References: <5513DE59.4000708@icsi.berkeley.edu> Message-ID: On Thu, Mar 26, 2015 at 10:24 AM, Asim Jamshed wrote: > Hi, > > The utilObj functions are actually referenced in > scripts/startup-one-thread.lua > startup script file. So you should start your packet-bricks using the > following > command: > > # bin/bricks -f scripts/startup-one-thread.lua > > I suggest that you go through the scripts/startup-one-thread.lua and > scripts/utils.lua files (after reading the README file) before using > packet-bricks. > > Thanks, > --Asim > Nothing ... root at nsm02:/data/config/etc/packet-bricks # /opt/packet-bricks/bin/bricks -f startup-one-thread.lua [ pmain(): line 463] Executing startup-one-thread.lua [print_version(): line 345] BRICKS Version 0.1-alpha bricks> pe:start() [ report(): line 222] stdin:1: attempt to index global 'pe' (a nil value) stack traceback: stdin:1: in main chunk [C]: ?bricks> bricks> show_stats() [ report(): line 222] stdin:2: ambiguous syntax (function call x new statement) near '('bricks> bricks> I have attached my actual lua scripts -------------- next part -------------- A non-text attachment was scrubbed... Name: setup.lua Type: text/x-lua Size: 218 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150326/26021c0e/attachment.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: startup-one-thread.lua Type: text/x-lua Size: 2068 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150326/26021c0e/attachment-0001.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: utils.lua Type: text/x-lua Size: 2108 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150326/26021c0e/attachment-0002.bin From ajamshed at ICSI.Berkeley.EDU Thu Mar 26 05:03:15 2015 From: ajamshed at ICSI.Berkeley.EDU (Asim Jamshed) Date: Thu, 26 Mar 2015 21:03:15 +0900 Subject: [Bro] Question about packet-bricks In-Reply-To: References: <5513DE59.4000708@icsi.berkeley.edu> Message-ID: <5513F583.1010800@icsi.berkeley.edu> pe is not a global variable. You should type the following commands: bricks> init() . . . bricks> start() . . . -- to stop packet-bricks bricks> stop() --Asim On 03/26/2015 07:52 PM, C. L. Martinez wrote: > On Thu, Mar 26, 2015 at 10:24 AM, Asim Jamshed > wrote: >> Hi, >> >> The utilObj functions are actually referenced in >> scripts/startup-one-thread.lua >> startup script file. So you should start your packet-bricks using the >> following >> command: >> >> # bin/bricks -f scripts/startup-one-thread.lua >> >> I suggest that you go through the scripts/startup-one-thread.lua and >> scripts/utils.lua files (after reading the README file) before using >> packet-bricks. >> >> Thanks, >> --Asim >> > Nothing ... > > root at nsm02:/data/config/etc/packet-bricks # > /opt/packet-bricks/bin/bricks -f startup-one-thread.lua > [ pmain(): line 463] Executing startup-one-thread.lua > [print_version(): line 345] BRICKS Version 0.1-alpha > bricks> pe:start() > [ report(): line 222] stdin:1: attempt to index global 'pe' (a nil value) > stack traceback: > stdin:1: in main chunk > [C]: ?bricks> > bricks> show_stats() > [ report(): line 222] stdin:2: ambiguous syntax (function call x > new statement) near '('bricks> > bricks> > > I have attached my actual lua scripts > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From franky.meier.1 at gmx.de Fri Mar 27 06:35:29 2015 From: franky.meier.1 at gmx.de (Frank Meier) Date: Fri, 27 Mar 2015 14:35:29 +0100 Subject: [Bro] http incomplete file extraction (Files::ANALYZER_EXTRACT) Message-ID: Hi! I am relatively new to bro so please excuse me, if I missed the obvious solution. I want to extract files downloaded via http from a pcap-file, but the files I download are never extracted completely. They seem to be truncated at ~1 mb. My bro-script is quite simple: event file_new(f: fa_file) { Files::add_analyzer(f, Files::ANALYZER_EXTRACT); } Are there any other events I have to catch to get the complete file? When I download a test file from [1] with size 3521964 bytes, only 960204 bytes are extracted. I checked with wireshark and tcpflow, that the download was completely captured in the pcap, I tested with Bro 2.3.2 and the current dev version from git. have a nice weekend! Franky [1] http://ipv4.download.thinkbroadband.com/5MB.zip From dani.nicolo at gmail.com Fri Mar 27 10:22:50 2015 From: dani.nicolo at gmail.com (=?UTF-8?Q?Danilo_Nicol=C3=B2?=) Date: Fri, 27 Mar 2015 18:22:50 +0100 Subject: [Bro] Error in Netbios analyzer Message-ID: Hi all, I'm working on an application based on Bro and we need to use SMB and Netbios analyzers at the moment. If I try to use the NetBIOS SSN analyzer found in the master git, I receive an error when I try to enable the analyzer. Analyzer::enable_analyzer(Analyzer::ANALYZER_NETBIOSSSN); Analyzer::register_for_ports(Analyzer::ANALYZER_NETBIOSSSN, ports); This is the received error: Internal error: unknown analyzer name NETBIOS; mismatch with tag analyzer::Component? So I have seen the NetbiosSSN.cc file and I have found the line that should be fixed. At line 458 you can see that NetbiosSSN_Analyzer::NetbiosSSN_Analyzer(Connection* conn) :tcp::TCP_ApplicationAnalyzer("NETBIOS", conn) I've changed "NETBIOS" string to "NETBIOSSN", then re-built it and Bro worked. Can you let me know that this fix is correct? Thank you. Best regards, Danilo -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150327/7d166c39/attachment.html From seth at icir.org Fri Mar 27 11:41:47 2015 From: seth at icir.org (Seth Hall) Date: Fri, 27 Mar 2015 14:41:47 -0400 Subject: [Bro] http incomplete file extraction (Files::ANALYZER_EXTRACT) In-Reply-To: References: Message-ID: <69C06BAB-6847-4284-88A4-4201BDC703E4@icir.org> > On Mar 27, 2015, at 9:35 AM, Frank Meier wrote: > > event file_new(f: fa_file) > { > Files::add_analyzer(f, Files::ANALYZER_EXTRACT); > } Nope, that should work. > Are there any other events I have to catch to get the complete file? > > When I download a test file from [1] with size 3521964 bytes, only 960204 bytes are extracted. I checked with > wireshark and tcpflow, that the download was completely captured in the pcap, Could you show me the files.log entry and the associated conn.log entry? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From seth at icir.org Fri Mar 27 11:43:15 2015 From: seth at icir.org (Seth Hall) Date: Fri, 27 Mar 2015 14:43:15 -0400 Subject: [Bro] Error in Netbios analyzer In-Reply-To: References: Message-ID: > On Mar 27, 2015, at 1:22 PM, Danilo Nicol? wrote: > > So I have seen the NetbiosSSN.cc file and I have found the line that should be fixed. > > At line 458 you can see that > > NetbiosSSN_Analyzer::NetbiosSSN_Analyzer(Connection* conn) :tcp::TCP_ApplicationAnalyzer("NETBIOS", conn) > > I've changed "NETBIOS" string to "NETBIOSSN", then re-built it and Bro worked. If you?re working with master, that code is no longer current. You also don?t want to rely on Bro for SMB analysis right now either as the analyzer is incomplete and broken. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From Ted.Llewellyn at ftr.com Sun Mar 29 08:12:07 2015 From: Ted.Llewellyn at ftr.com (Llewellyn, Ted) Date: Sun, 29 Mar 2015 15:12:07 +0000 Subject: [Bro] New installation crashes appear to be ssh-related Message-ID: We have a new Bro installation, built from source on Debian wheezy, that keeps core dumping. It looks like it's choking on some code related to ssh. Here is the diag for the latest crash. It is identical to the other one I have: [BroControl] > diag [bro] Bro 2.3-633 Linux 3.2.0-4-686-pae No gdb installed. ==== No reporter.log ==== stderr.log listening on eth1, capture length 8192 bytes bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion `t_dataptr_after_cookie <= t_end_of_data' failed. /usr/local/bro/share/broctl/scripts/run-bro: line 100: 10307 Aborted (core dumped) nohup "$mybro" "$@" ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto ==== .env_vars PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local/bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site CLUSTER_NODE= ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log [BroControl] > This is just running the default setup, with the local subnets configured, as we are just starting with Bro. This is a really low end server, but the capture interface is only running at 100 meg so there are really no resource issues. (Yes, this is a 32-bit box. It's pretty old. That's why I built from source.) The first crash occurred after a few minutes. Then it ran for nearly 24 hours before this crash. Is there something I can tweak to prevent this? Thanks, Ted Llewellyn -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150329/c133c6eb/attachment.html From vlad at grigorescu.org Sun Mar 29 18:30:54 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Sun, 29 Mar 2015 20:30:54 -0500 Subject: [Bro] New installation crashes appear to be ssh-related In-Reply-To: References: Message-ID: Hi Ted, Thanks for reporting this. I'll look into it. --Vlad On Sun, Mar 29, 2015 at 10:12 AM, Llewellyn, Ted wrote: > We have a new Bro installation, built from source on Debian wheezy, > that keeps core dumping. It looks like it?s choking on some code related to > ssh. Here is the diag for the latest crash. It is identical to the other > one I have: > > > > [BroControl] > diag > > [bro] > > > > Bro 2.3-633 > > Linux 3.2.0-4-686-pae > > > > No gdb installed. > > > > ==== No reporter.log > > > > ==== stderr.log > > listening on eth1, capture length 8192 bytes > > > > bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int > binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, > binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion > `t_dataptr_after_cookie <= t_end_of_data' failed. > > /usr/local/bro/share/broctl/scripts/run-bro: line 100: 10307 > Aborted (core dumped) nohup "$mybro" "$@" > > > > ==== stdout.log > > max memory size (kbytes, -m) unlimited > > data seg size (kbytes, -d) unlimited > > virtual memory (kbytes, -v) unlimited > > core file size (blocks, -c) unlimited > > > > ==== .cmdline > > -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro > local.bro broctl broctl/standalone broctl/auto > > > > ==== .env_vars > > > PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local/bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > > > BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site > > CLUSTER_NODE= > > > > ==== .status > > RUNNING [net_run] > > > > ==== No prof.log > > > > ==== No packet_filter.log > > > > ==== No loaded_scripts.log > > [BroControl] > > > > > This is just running the default setup, with the local subnets configured, > as we are just starting with Bro. This is a really low end server, but the > capture interface is only running at 100 meg so there are really no > resource issues. (Yes, this is a 32-bit box. It?s pretty old. That?s why I > built from source.) > > The first crash occurred after a few minutes. Then it ran for nearly 24 > hours before this crash. Is there something I can tweak to prevent this? > > > > Thanks, > > Ted Llewellyn > > > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150329/a31afc4e/attachment.html From richardqa at gmail.com Sun Mar 29 20:55:44 2015 From: richardqa at gmail.com (Javier Richard Quinto Ancieta) Date: Mon, 30 Mar 2015 00:55:44 -0300 Subject: [Bro] Field value missing Message-ID: Greetings all, I am new to Bro, and I hope you can help me. I read the following documentation: https://www.bro.org/sphinx-git/frameworks/notice.html Exactly, this part of the code: ... hook Notice::policy(n: Notice::Info) { if ( n$note == SSH::Password_Guessing && n$id$resp_h == 10.0.0.1 ) add n$actions[Notice::ACTION_EMAIL]; } ... And write it in the file ../local.bro But, when I generate an attack to IP (10.0.0.1), and I got an error: "*field value missing [n$id]*" . I use *bro -i eth0 local *to debug logs in live I did many changes, also I use "$id?$resp_h" to check errors, and i got the same error. I am sorry but I am new with Bro and I would like to know How can I fix that?. Thank you Javier -- Saludos Cordiales Javier -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150330/5df6bd42/attachment.html From franky.meier.1 at gmx.de Sun Mar 29 23:31:19 2015 From: franky.meier.1 at gmx.de (Frank Meier) Date: Mon, 30 Mar 2015 08:31:19 +0200 Subject: [Bro] http incomplete file extraction (Files::ANALYZER_EXTRACT) In-Reply-To: <69C06BAB-6847-4284-88A4-4201BDC703E4@icir.org> References: , <69C06BAB-6847-4284-88A4-4201BDC703E4@icir.org> Message-ID: An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150330/4b820a7f/attachment.html From dani.nicolo at gmail.com Mon Mar 30 02:06:25 2015 From: dani.nicolo at gmail.com (=?UTF-8?Q?Danilo_Nicol=C3=B2?=) Date: Mon, 30 Mar 2015 11:06:25 +0200 Subject: [Bro] Error in Netbios analyzer In-Reply-To: References: Message-ID: Hi Seth, Sorry, but I don't understand.I'm only using your NetBIOS analyzer in the master branch (without SMB) and I've found that code in the tree master at https://github.com/bro/bro/blob/master/src/analyzer/protocol/netbios/NetbiosSSN.cc line 459. Could you explain me where I'm making mistakes? Thank you very much. Regards, Danilo 2015-03-27 19:43 GMT+01:00 Seth Hall : > > > On Mar 27, 2015, at 1:22 PM, Danilo Nicol? > wrote: > > > > So I have seen the NetbiosSSN.cc file and I have found the line that > should be fixed. > > > > At line 458 you can see that > > > > NetbiosSSN_Analyzer::NetbiosSSN_Analyzer(Connection* conn) > :tcp::TCP_ApplicationAnalyzer("NETBIOS", conn) > > > > I've changed "NETBIOS" string to "NETBIOSSN", then re-built it and Bro > worked. > > If you?re working with master, that code is no longer current. You also > don?t want to rely on Bro for SMB analysis right now either as the analyzer > is incomplete and broken. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150330/fb35ff3c/attachment-0001.html From franky.meier.1 at gmx.de Mon Mar 30 02:24:18 2015 From: franky.meier.1 at gmx.de (Frank Meier) Date: Mon, 30 Mar 2015 11:24:18 +0200 Subject: [Bro] http incomplete file extraction (Files::ANALYZER_EXTRACT) In-Reply-To: References: , Message-ID: An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150330/b32a5611/attachment.html From Ted.Llewellyn at ftr.com Mon Mar 30 05:23:16 2015 From: Ted.Llewellyn at ftr.com (Llewellyn, Ted) Date: Mon, 30 Mar 2015 12:23:16 +0000 Subject: [Bro] New installation crashes appear to be ssh-related In-Reply-To: References: Message-ID: Vlad, I happened to run across HILTI while I was looking at something not related to Bro. My output seems to come from binpac, and according to the HILTI folks they develop on 64-bit platforms and promise nothing if it?s run on 32-bit hardware. I thought binpac was just supposed to be a plugin, which says to me it can be turned off or I could rebuild without it, if I could find out how. ?./configure ?help? wasn?t very helpful about this. Does this sound plausible? Thanks, Ted From: grigorescu at gmail.com [mailto:grigorescu at gmail.com] On Behalf Of Vlad Grigorescu Sent: Sunday, March 29, 2015 9:31 PM To: Llewellyn, Ted Cc: bro at bro.org Subject: Re: [Bro] New installation crashes appear to be ssh-related Hi Ted, Thanks for reporting this. I'll look into it. --Vlad On Sun, Mar 29, 2015 at 10:12 AM, Llewellyn, Ted > wrote: We have a new Bro installation, built from source on Debian wheezy, that keeps core dumping. It looks like it?s choking on some code related to ssh. Here is the diag for the latest crash. It is identical to the other one I have: [BroControl] > diag [bro] Bro 2.3-633 Linux 3.2.0-4-686-pae No gdb installed. ==== No reporter.log ==== stderr.log listening on eth1, capture length 8192 bytes bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion `t_dataptr_after_cookie <= t_end_of_data' failed. /usr/local/bro/share/broctl/scripts/run-bro: line 100: 10307 Aborted (core dumped) nohup "$mybro" "$@" ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto ==== .env_vars PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local/bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site CLUSTER_NODE= ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log [BroControl] > This is just running the default setup, with the local subnets configured, as we are just starting with Bro. This is a really low end server, but the capture interface is only running at 100 meg so there are really no resource issues. (Yes, this is a 32-bit box. It?s pretty old. That?s why I built from source.) The first crash occurred after a few minutes. Then it ran for nearly 24 hours before this crash. Is there something I can tweak to prevent this? Thanks, Ted Llewellyn _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150330/d649d641/attachment-0001.html From robin at icir.org Mon Mar 30 07:39:20 2015 From: robin at icir.org (Robin Sommer) Date: Mon, 30 Mar 2015 07:39:20 -0700 Subject: [Bro] New installation crashes appear to be ssh-related In-Reply-To: References: Message-ID: <20150330143920.GD22580@icir.org> Ted, mind filing a ticket so that we track this one? Robin On Sun, Mar 29, 2015 at 15:12 +0000, you wrote: > We have a new Bro installation, built from source on Debian wheezy, that keeps core dumping. It looks like it's choking on some code related to ssh. Here is the diag for the latest crash. It is identical to the other one I have: > > [BroControl] > diag > [bro] > > Bro 2.3-633 > Linux 3.2.0-4-686-pae > > No gdb installed. > > ==== No reporter.log > > ==== stderr.log > listening on eth1, capture length 8192 bytes > > bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion `t_dataptr_after_cookie <= t_end_of_data' failed. > /usr/local/bro/share/broctl/scripts/run-bro: line 100: 10307 Aborted (core dumped) nohup "$mybro" "$@" > > ==== stdout.log > max memory size (kbytes, -m) unlimited > data seg size (kbytes, -d) unlimited > virtual memory (kbytes, -v) unlimited > core file size (blocks, -c) unlimited > > ==== .cmdline > -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto > > ==== .env_vars > PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local/bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site > CLUSTER_NODE= > > ==== .status > RUNNING [net_run] > > ==== No prof.log > > ==== No packet_filter.log > > ==== No loaded_scripts.log > [BroControl] > > > This is just running the default setup, with the local subnets configured, as we are just starting with Bro. This is a really low end server, but the capture interface is only running at 100 meg so there are really no resource issues. (Yes, this is a 32-bit box. It's pretty old. That's why I built from source.) > The first crash occurred after a few minutes. Then it ran for nearly 24 hours before this crash. Is there something I can tweak to prevent this? > > Thanks, > Ted Llewellyn > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From vlad at grigorescu.org Mon Mar 30 08:05:54 2015 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Mon, 30 Mar 2015 10:05:54 -0500 Subject: [Bro] New installation crashes appear to be ssh-related In-Reply-To: <20150330143920.GD22580@icir.org> References: <20150330143920.GD22580@icir.org> Message-ID: Also, do you happen to have a core dump of this? It would help with debugging. To answer your question about BinPAC - BinPAC is a Binary Protocol Analyzer Compiler. Some analyzers in Bro are written in a language that BinPAC will compile to C++. When you compile Bro, this compilation happens, and then that C++ code gets compiled with the rest of Bro. So, it's not really a plugin - you could technically build Bro without BinPAC, but in practice, you wouldn't want to do that. Hope that makes sense, --Vlad On Mon, Mar 30, 2015 at 9:39 AM, Robin Sommer wrote: > Ted, mind filing a ticket so that we track this one? > > Robin > > On Sun, Mar 29, 2015 at 15:12 +0000, you wrote: > > > We have a new Bro installation, built from source on Debian wheezy, > that keeps core dumping. It looks like it's choking on some code related to > ssh. Here is the diag for the latest crash. It is identical to the other > one I have: > > > > [BroControl] > diag > > [bro] > > > > Bro 2.3-633 > > Linux 3.2.0-4-686-pae > > > > No gdb installed. > > > > ==== No reporter.log > > > > ==== stderr.log > > listening on eth1, capture length 8192 bytes > > > > bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int > binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, > binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion > `t_dataptr_after_cookie <= t_end_of_data' failed. > > /usr/local/bro/share/broctl/scripts/run-bro: line 100: 10307 Aborted > (core dumped) nohup "$mybro" "$@" > > > > ==== stdout.log > > max memory size (kbytes, -m) unlimited > > data seg size (kbytes, -d) unlimited > > virtual memory (kbytes, -v) unlimited > > core file size (blocks, -c) unlimited > > > > ==== .cmdline > > -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p > bro local.bro broctl broctl/standalone broctl/auto > > > > ==== .env_vars > > > PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local/bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > > > BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site > > CLUSTER_NODE= > > > > ==== .status > > RUNNING [net_run] > > > > ==== No prof.log > > > > ==== No packet_filter.log > > > > ==== No loaded_scripts.log > > [BroControl] > > > > > This is just running the default setup, with the local subnets > configured, as we are just starting with Bro. This is a really low end > server, but the capture interface is only running at 100 meg so there are > really no resource issues. (Yes, this is a 32-bit box. It's pretty old. > That's why I built from source.) > > The first crash occurred after a few minutes. Then it ran for nearly 24 > hours before this crash. Is there something I can tweak to prevent this? > > > > Thanks, > > Ted Llewellyn > > > > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > -- > Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150330/ba999835/attachment.html From jsiwek at illinois.edu Mon Mar 30 08:19:45 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Mon, 30 Mar 2015 15:19:45 +0000 Subject: [Bro] Field value missing In-Reply-To: References: Message-ID: > On Mar 29, 2015, at 10:55 PM, Javier Richard Quinto Ancieta wrote: > > ... > hook Notice::policy(n: Notice::Info) > { > if > ( n$note == SSH::Password_Guessing && n$id$resp_h == 10.0.0.1 > ) > add n$actions[Notice::ACTION_EMAIL]; > } > ... > > And write it in the file ../local.bro > > But, when I generate an attack to IP (10.0.0.1), and I got an error: "field value missing [n$id]" . Sorry, the documentation here is out of date. The SSH::Password_Guessing notice now seems able to track a guesser across multiple servers, so in order to check if they attempted a certain server, you?d have to inspect the value of n$sub to see if it?s mentioned there. E.g.: hook Notice::policy(n: Notice::Info) { if ( n$note == SSH::Password_Guessing && /10\.0\.0\.1/ in n$sub ) add n$actions[Notice::ACTION_EMAIL]; } - Jon From dopheide at gmail.com Mon Mar 30 08:49:52 2015 From: dopheide at gmail.com (Mike Dopheide) Date: Mon, 30 Mar 2015 10:49:52 -0500 Subject: [Bro] Field value missing In-Reply-To: References: Message-ID: Javier, To add to what Jon said... In this case you're hitting a situation where not all Notices are created equal. I believe, for SSH::Password_Guessing, the connection 'id' itself isn't populated, so the n$id isn't there to reference n$id$resp_h from. It will have an n$src if you wanted the originator, but for recipient you need to look at the notice subject (see Jon's message). The recipients listed there are a sampled set. -Dop On Sun, Mar 29, 2015 at 10:55 PM, Javier Richard Quinto Ancieta < richardqa at gmail.com> wrote: > Greetings all, > > I am new to Bro, and I hope you can help me. > > I read the following documentation: > https://www.bro.org/sphinx-git/frameworks/notice.html > > Exactly, this part of the code: > > ... > hook Notice::policy(n: Notice::Info) > { > if > ( n$note == SSH::Password_Guessing && n$id$resp_h == 10.0.0.1 > ) > add n$actions[Notice::ACTION_EMAIL]; > } > ... > > And write it in the file ../local.bro > > But, when I generate an attack to IP (10.0.0.1), and I got an error: "*field > value missing [n$id]*" . > > I use *bro -i eth0 local *to debug logs in live > > I did many changes, also I use "$id?$resp_h" to check errors, and i got > the same error. I am sorry but I am new with Bro and I would like to know > How can I fix that?. > > Thank you > Javier > > -- > Saludos Cordiales > Javier > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150330/fd526440/attachment.html From Ted.Llewellyn at ftr.com Mon Mar 30 08:59:10 2015 From: Ted.Llewellyn at ftr.com (Llewellyn, Ted) Date: Mon, 30 Mar 2015 15:59:10 +0000 Subject: [Bro] New installation crashes appear to be ssh-related In-Reply-To: References: <20150330143920.GD22580@icir.org> Message-ID: Yes, I have core dumps. Is there an upload site? I?m concerned about information leakage, also. This is a lab environment, but I still don?t want too much information about it being distributed in public forums. Ted Llewellyn Sr. Network Planning Engineer VoIP Engineering Frontier Communications 120 Plymouth Ave. N. Rochester, NY 14608 585-413-9743 From: grigorescu at gmail.com [mailto:grigorescu at gmail.com] On Behalf Of Vlad Grigorescu Sent: Monday, March 30, 2015 11:06 AM To: Llewellyn, Ted Cc: bro at bro.org Subject: Re: [Bro] New installation crashes appear to be ssh-related Also, do you happen to have a core dump of this? It would help with debugging. To answer your question about BinPAC - BinPAC is a Binary Protocol Analyzer Compiler. Some analyzers in Bro are written in a language that BinPAC will compile to C++. When you compile Bro, this compilation happens, and then that C++ code gets compiled with the rest of Bro. So, it's not really a plugin - you could technically build Bro without BinPAC, but in practice, you wouldn't want to do that. Hope that makes sense, --Vlad On Mon, Mar 30, 2015 at 9:39 AM, Robin Sommer > wrote: Ted, mind filing a ticket so that we track this one? Robin On Sun, Mar 29, 2015 at 15:12 +0000, you wrote: > We have a new Bro installation, built from source on Debian wheezy, that keeps core dumping. It looks like it's choking on some code related to ssh. Here is the diag for the latest crash. It is identical to the other one I have: > > [BroControl] > diag > [bro] > > Bro 2.3-633 > Linux 3.2.0-4-686-pae > > No gdb installed. > > ==== No reporter.log > > ==== stderr.log > listening on eth1, capture length 8192 bytes > > bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion `t_dataptr_after_cookie <= t_end_of_data' failed. > /usr/local/bro/share/broctl/scripts/run-bro: line 100: 10307 Aborted (core dumped) nohup "$mybro" "$@" > > ==== stdout.log > max memory size (kbytes, -m) unlimited > data seg size (kbytes, -d) unlimited > virtual memory (kbytes, -v) unlimited > core file size (blocks, -c) unlimited > > ==== .cmdline > -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto > > ==== .env_vars > PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local/bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site > CLUSTER_NODE= > > ==== .status > RUNNING [net_run] > > ==== No prof.log > > ==== No packet_filter.log > > ==== No loaded_scripts.log > [BroControl] > > > This is just running the default setup, with the local subnets configured, as we are just starting with Bro. This is a really low end server, but the capture interface is only running at 100 meg so there are really no resource issues. (Yes, this is a 32-bit box. It's pretty old. That's why I built from source.) > The first crash occurred after a few minutes. Then it ran for nearly 24 hours before this crash. Is there something I can tweak to prevent this? > > Thanks, > Ted Llewellyn > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150330/73e17245/attachment-0001.html From jsiwek at illinois.edu Mon Mar 30 09:54:38 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Mon, 30 Mar 2015 16:54:38 +0000 Subject: [Bro] http incomplete file extraction (Files::ANALYZER_EXTRACT) In-Reply-To: References: <,> <69C06BAB-6847-4284-88A4-4201BDC703E4@icir.org> Message-ID: <7A038474-58CC-43D7-B337-7B7C4024C248@illinois.edu> > On Mar 30, 2015, at 1:31 AM, Frank Meier wrote: > > I was also wondering, why the correct size is in the logs. If data was missing I would > at least have exspected a warning or some missing_bytes. In files.log, the value of total_bytes is just taken from the HTTP Content-Length header. Since the value of seen_bytes is less than total_bytes, you can suspect Bro didn?t see the full file for some reason. Do you have a weird.log containing any obvious clues? Else, I may need the original pcap to understand what went wrong. - Jon From richardqa at gmail.com Mon Mar 30 10:10:24 2015 From: richardqa at gmail.com (Javier Richard Quinto Ancieta) Date: Mon, 30 Mar 2015 14:10:24 -0300 Subject: [Bro] Field value missing In-Reply-To: References: Message-ID: Hi, Thank you for your answers, clarify part of my doubt. I got successful results using the two methods: e.g., if (n$note == SSH::Password_Guessing && \11\.1\.1\.7/ in n$sub) print fmt ("testing1"); or if (n$note == SSH::Password_Guessing && n$src = ) print fmt ("testing2"); I also saw logs notice.log and I understand because I get these values (sub, src). But, I'd like to understand because the notice.log dont populate fields "id.orig_h, id.resp_h". You told me that "the connection 'id' itself isn't populated, so the n$id isn't there to reference n$id$resp_h from" but I wonder if there is some way to populate these fields (id.orig_h, id.resp_h, ...) ? for this type of event (SSH::Password_Guessing). Thank you, Javier 2015-03-30 12:49 GMT-03:00 Mike Dopheide : > Javier, > > To add to what Jon said... > > In this case you're hitting a situation where not all Notices are created > equal. > > I believe, for SSH::Password_Guessing, the connection 'id' itself isn't > populated, so the n$id isn't there to reference n$id$resp_h from. It will > have an n$src if you wanted the originator, but for recipient you need to > look at the notice subject (see Jon's message). The recipients listed > there are a sampled set. > > -Dop > > > > > On Sun, Mar 29, 2015 at 10:55 PM, Javier Richard Quinto Ancieta < > richardqa at gmail.com> wrote: > >> Greetings all, >> >> I am new to Bro, and I hope you can help me. >> >> I read the following documentation: >> https://www.bro.org/sphinx-git/frameworks/notice.html >> >> Exactly, this part of the code: >> >> ... >> hook Notice::policy(n: Notice::Info) >> { >> if >> ( n$note == SSH::Password_Guessing && n$id$resp_h == 10.0.0.1 >> ) >> add n$actions[Notice::ACTION_EMAIL]; >> } >> ... >> >> And write it in the file ../local.bro >> >> But, when I generate an attack to IP (10.0.0.1), and I got an error: "*field >> value missing [n$id]*" . >> >> I use *bro -i eth0 local *to debug logs in live >> >> I did many changes, also I use "$id?$resp_h" to check errors, and i got >> the same error. I am sorry but I am new with Bro and I would like to know >> How can I fix that?. >> >> Thank you >> Javier >> >> -- >> Saludos Cordiales >> Javier >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > > -- Saludos Cordiales Javier Richard Quinto Ancieta Est. maestr?a en Ing. de Computaci?n-UNICAMP Br http://www.linkedin.com/in/richardqa CELL: +51 972205099 (Lima), +55 19 99033699 (Campinas-SP) Fingerprint: 52C8 9361 B7B1 0CDE A7FF 0AAF 6911 459E F588 ACFD -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150330/2c7e0dc0/attachment.html From dopheide at gmail.com Mon Mar 30 12:06:25 2015 From: dopheide at gmail.com (Mike Dopheide) Date: Mon, 30 Mar 2015 14:06:25 -0500 Subject: [Bro] Field value missing In-Reply-To: References: Message-ID: There's a couple things going on that make this tricky. 1) First, the SSH::Password_Guessing notice is built up using SumStats in: share/bro/policy/protocols/ssh/detect-bruteforcing.bro It's only keeping track of 5 sample targets and the default limit to cause a notice is 30 attempts (which you can change). 2) The other problem is that a notice line with a connection identifier (c$id) can only have one connection id that populates orig_h, resp_h, etc. For the code that creates SSH::Password_Guessing, you're dealing with lots of different connections. ... However, with Bro, most things are typically possible, we just need to think about it in a roundabout sort of way. My two immediate thoughts are you could write your own policies that keeps all of the target information (more memory intensive) and write out the c$id info for each target, or you can get more aggressive with your guess limit and just set the limit at 5. Then the notice subject sample size will (usually?) be all of your targets. That'd be something like this in your local.bro: redef SSH::password_guesses_limit = 5; -Dop On Mon, Mar 30, 2015 at 12:10 PM, Javier Richard Quinto Ancieta < richardqa at gmail.com> wrote: > Hi, > > Thank you for your answers, clarify part of my doubt. I got successful > results using the two methods: > > e.g., > > if (n$note == SSH::Password_Guessing && \11\.1\.1\.7/ in n$sub) > print fmt ("testing1"); > > or > > if (n$note == SSH::Password_Guessing && n$src = ) > print fmt ("testing2"); > > I also saw logs notice.log and I understand because I get these values > (sub, src). > > But, I'd like to understand because the notice.log dont populate fields > "id.orig_h, id.resp_h". > > You told me that "the connection 'id' itself isn't populated, so the n$id > isn't there to reference n$id$resp_h from" > but I wonder if there is some way to populate these fields (id.orig_h, > id.resp_h, ...) ? for this type of event (SSH::Password_Guessing). > > Thank you, > Javier > > 2015-03-30 12:49 GMT-03:00 Mike Dopheide : > > Javier, >> >> To add to what Jon said... >> >> In this case you're hitting a situation where not all Notices are created >> equal. >> >> I believe, for SSH::Password_Guessing, the connection 'id' itself isn't >> populated, so the n$id isn't there to reference n$id$resp_h from. It will >> have an n$src if you wanted the originator, but for recipient you need to >> look at the notice subject (see Jon's message). The recipients listed >> there are a sampled set. >> >> -Dop >> >> >> >> >> On Sun, Mar 29, 2015 at 10:55 PM, Javier Richard Quinto Ancieta < >> richardqa at gmail.com> wrote: >> >>> Greetings all, >>> >>> I am new to Bro, and I hope you can help me. >>> >>> I read the following documentation: >>> https://www.bro.org/sphinx-git/frameworks/notice.html >>> >>> Exactly, this part of the code: >>> >>> ... >>> hook Notice::policy(n: Notice::Info) >>> { >>> if >>> ( n$note == SSH::Password_Guessing && n$id$resp_h == 10.0.0.1 >>> ) >>> add n$actions[Notice::ACTION_EMAIL]; >>> } >>> ... >>> >>> And write it in the file ../local.bro >>> >>> But, when I generate an attack to IP (10.0.0.1), and I got an error: "*field >>> value missing [n$id]*" . >>> >>> I use *bro -i eth0 local *to debug logs in live >>> >>> I did many changes, also I use "$id?$resp_h" to check errors, and i got >>> the same error. I am sorry but I am new with Bro and I would like to know >>> How can I fix that?. >>> >>> Thank you >>> Javier >>> >>> -- >>> Saludos Cordiales >>> Javier >>> >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >>> >> >> > > > -- > Saludos Cordiales > Javier Richard Quinto Ancieta > Est. maestr?a en Ing. de Computaci?n-UNICAMP Br > http://www.linkedin.com/in/richardqa > CELL: +51 972205099 (Lima), +55 19 99033699 (Campinas-SP) > Fingerprint: 52C8 9361 B7B1 0CDE A7FF 0AAF 6911 459E F588 ACFD > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150330/092ec702/attachment.html From Ted.Llewellyn at ftr.com Mon Mar 30 16:54:17 2015 From: Ted.Llewellyn at ftr.com (Llewellyn, Ted) Date: Mon, 30 Mar 2015 23:54:17 +0000 Subject: [Bro] New installation crashes appear to be ssh-related In-Reply-To: <20150330143920.GD22580@icir.org> References: <20150330143920.GD22580@icir.org> Message-ID: Robin, I submitted a ticket, 1361. It won't let me attach the core dump as it's too big. How do I upload that? Thanks, Ted -----Original Message----- From: Robin Sommer [mailto:robin at icir.org] Sent: Monday, March 30, 2015 10:39 AM To: Llewellyn, Ted Cc: bro at bro.org Subject: Re: [Bro] New installation crashes appear to be ssh-related Ted, mind filing a ticket so that we track this one? Robin On Sun, Mar 29, 2015 at 15:12 +0000, you wrote: > We have a new Bro installation, built from source on Debian wheezy, that keeps core dumping. It looks like it's choking on some code related to ssh. Here is the diag for the latest crash. It is identical to the other one I have: > > [BroControl] > diag > [bro] > > Bro 2.3-633 > Linux 3.2.0-4-686-pae > > No gdb installed. > > ==== No reporter.log > > ==== stderr.log > listening on eth1, capture length 8192 bytes > > bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion `t_dataptr_after_cookie <= t_end_of_data' failed. > /usr/local/bro/share/broctl/scripts/run-bro: line 100: 10307 Aborted (core dumped) nohup "$mybro" "$@" > > ==== stdout.log > max memory size (kbytes, -m) unlimited > data seg size (kbytes, -d) unlimited > virtual memory (kbytes, -v) unlimited > core file size (blocks, -c) unlimited > > ==== .cmdline > -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p > bro local.bro broctl broctl/standalone broctl/auto > > ==== .env_vars > PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local > /bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr > /local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/sh > are/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site > CLUSTER_NODE= > > ==== .status > RUNNING [net_run] > > ==== No prof.log > > ==== No packet_filter.log > > ==== No loaded_scripts.log > [BroControl] > > > This is just running the default setup, with the local subnets > configured, as we are just starting with Bro. This is a really low end server, but the capture interface is only running at 100 meg so there are really no resource issues. (Yes, this is a 32-bit box. It's pretty old. That's why I built from source.) The first crash occurred after a few minutes. Then it ran for nearly 24 hours before this crash. Is there something I can tweak to prevent this? > > Thanks, > Ted Llewellyn > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From Ted.Llewellyn at ftr.com Tue Mar 31 05:57:02 2015 From: Ted.Llewellyn at ftr.com (Llewellyn, Ted) Date: Tue, 31 Mar 2015 12:57:02 +0000 Subject: [Bro] New installation crashes appear to be ssh-related In-Reply-To: References: <20150330143920.GD22580@icir.org> Message-ID: Vlad, It crashed again this morning. The crash on 3/29 was at 6:29 local time, and the crash this morning was at 6:27 local time. I?m not aware of anything that happens here around that time on a regular basis. The diag looks pretty much the same: [BroControl] > diag [bro] Bro 2.3-633 Linux 3.2.0-4-686-pae No gdb installed. ==== No reporter.log ==== stderr.log listening on eth1, capture length 8192 bytes bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion `t_dataptr_after_cookie <= t_end_of_data' failed. /usr/local/bro/share/broctl/scripts/run-bro: line 100: 24675 Aborted (core dumped) nohup "$mybro" "$@" ==== stdout.log max memory size (kbytes, -m) unlimited data seg size (kbytes, -d) unlimited virtual memory (kbytes, -v) unlimited core file size (blocks, -c) unlimited ==== .cmdline -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto ==== .env_vars PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local/bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site CLUSTER_NODE= ==== .status RUNNING [net_run] ==== No prof.log ==== No packet_filter.log ==== No loaded_scripts.log [BroControl] > Here is the ssh.log, with the local addresses obfuscated: #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path ssh #open 2015-03-31-06-12-54 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version auth_success direction client server cipher_alg mac_alg compression_alg kex_alg host_key_alg host_key remote_location.country_code remote_location.region remote_location.city remote_location.latitude remote_location.longitude #types time string addr port addr port count bool enum string string string string string string string string string string string double double 1427796767.723015 CekWob4QEqOlP0oqp8 115.239.230.133 57922 10.10.20.217 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss 92:fe:da:65:a4:2e:ae:30:a4:26:a9:62:56:35:30:37 - - - - - 1427796768.761095 Ctm96W1UH7UUMJkEhk 115.239.230.133 42380 10.10.24.233 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss 92:fe:da:65:a4:2e:ae:30:a4:26:a9:62:56:35:30:37 - - - - - 1427796773.022767 CBJCTy0vfPn8efye4 115.239.230.133 45326 10.10.20.194 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss 92:fe:da:65:a4:2e:ae:30:a4:26:a9:62:56:35:30:37 - - - - - 1427796998.016420 CPC3hO10j08ML06CRj 115.231.218.130 56223 10.10.20.217 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss - - - - - - 1427796998.641613 CMUo9V3XqIY3J45Arl 115.231.218.130 51297 10.10.20.194 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss 92:fe:da:65:a4:2e:ae:30:a4:26:a9:62:56:35:30:37 - - - - - 1427797000.236567 C4F5Ca2TZOVL55re0i 115.231.218.130 60792 10.10.24.233 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss 92:fe:da:65:a4:2e:ae:30:a4:26:a9:62:56:35:30:37 - - - - - 1427797056.937244 CeElA5RdppTwHbR6b 183.136.216.4 34758 10.10.24.233 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss 92:fe:da:65:a4:2e:ae:30:a4:26:a9:62:56:35:30:37 - - - - - 1427797056.134247 CDKUcz2vwqwCQ6FMP 183.136.216.4 57005 10.10.20.217 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss 92:fe:da:65:a4:2e:ae:30:a4:26:a9:62:56:35:30:37 - - - - - 1427797314.991566 CPkA7E3jOaA4O3n6Zj 115.239.248.238 46652 10.10.20.217 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss 92:fe:da:65:a4:2e:ae:30:a4:26:a9:62:56:35:30:37 - - - - - 1427797315.312565 CF4kqy4fSKVNiRwHKa 115.239.248.238 34778 10.10.20.194 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss 92:fe:da:65:a4:2e:ae:30:a4:26:a9:62:56:35:30:37 - - - - - 1427797316.044014 CfKqmt3d5HTfWS7xyc 115.239.248.238 50058 10.10.24.233 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss 92:fe:da:65:a4:2e:ae:30:a4:26:a9:62:56:35:30:37 - - - - - 1427797665.315966 CUdfQY3IPL1xx4UtY7 115.231.218.131 57464 10.10.20.194 22 2 - - SSH-2.0-PUTTY SSH-2.0-2.0.12 3des-cbc hmac-sha1 none diffie-hellman-group1-sha1 ssh-dss 92:fe:da:65:a4:2e:ae:30:a4:26:a9:62:56:35:30:37 - - - - - I can only get the core files down to about 15 meg, so they won?t attach to the ticket. Should I try sending it directly to your gmail account, so the whole list doesn?t get it? Ted From: grigorescu at gmail.com [mailto:grigorescu at gmail.com] On Behalf Of Vlad Grigorescu Sent: Monday, March 30, 2015 11:06 AM To: Llewellyn, Ted Cc: bro at bro.org Subject: Re: [Bro] New installation crashes appear to be ssh-related Also, do you happen to have a core dump of this? It would help with debugging. To answer your question about BinPAC - BinPAC is a Binary Protocol Analyzer Compiler. Some analyzers in Bro are written in a language that BinPAC will compile to C++. When you compile Bro, this compilation happens, and then that C++ code gets compiled with the rest of Bro. So, it's not really a plugin - you could technically build Bro without BinPAC, but in practice, you wouldn't want to do that. Hope that makes sense, --Vlad On Mon, Mar 30, 2015 at 9:39 AM, Robin Sommer > wrote: Ted, mind filing a ticket so that we track this one? Robin On Sun, Mar 29, 2015 at 15:12 +0000, you wrote: > We have a new Bro installation, built from source on Debian wheezy, that keeps core dumping. It looks like it's choking on some code related to ssh. Here is the diag for the latest crash. It is identical to the other one I have: > > [BroControl] > diag > [bro] > > Bro 2.3-633 > Linux 3.2.0-4-686-pae > > No gdb installed. > > ==== No reporter.log > > ==== stderr.log > listening on eth1, capture length 8192 bytes > > bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion `t_dataptr_after_cookie <= t_end_of_data' failed. > /usr/local/bro/share/broctl/scripts/run-bro: line 100: 10307 Aborted (core dumped) nohup "$mybro" "$@" > > ==== stdout.log > max memory size (kbytes, -m) unlimited > data seg size (kbytes, -d) unlimited > virtual memory (kbytes, -v) unlimited > core file size (blocks, -c) unlimited > > ==== .cmdline > -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto > > ==== .env_vars > PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local/bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr/local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/share/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site > CLUSTER_NODE= > > ==== .status > RUNNING [net_run] > > ==== No prof.log > > ==== No packet_filter.log > > ==== No loaded_scripts.log > [BroControl] > > > This is just running the default setup, with the local subnets configured, as we are just starting with Bro. This is a really low end server, but the capture interface is only running at 100 meg so there are really no resource issues. (Yes, this is a 32-bit box. It's pretty old. That's why I built from source.) > The first crash occurred after a few minutes. Then it ran for nearly 24 hours before this crash. Is there something I can tweak to prevent this? > > Thanks, > Ted Llewellyn > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20150331/69898ebe/attachment-0001.html From robin at icir.org Tue Mar 31 08:42:24 2015 From: robin at icir.org (Robin Sommer) Date: Tue, 31 Mar 2015 08:42:24 -0700 Subject: [Bro] New installation crashes appear to be ssh-related In-Reply-To: References: <20150330143920.GD22580@icir.org> Message-ID: <20150331154224.GM31994@icir.org> Thanks for filing the ticket. For the core, actually what would be most helpful right now I believe is a stack backtrace. Your crash report didn't have that, it looks like there's no gdb installed. Can you install gdb and then run "gdb bro core" + "bt" as described here: https://www.bro.org/support/reporting-problems.html#getting-more-information-after-acrash For the core itself, I think the best thing might be to hold on to it for now, just the core won't be useful for others much anyways, as one also needs to the binary and potentially a similar system to use it. So if you could keep binary and core somewhere until this is resolved, that would be best for now. Robin On Mon, Mar 30, 2015 at 23:54 +0000, you wrote: > Robin, > > I submitted a ticket, 1361. It won't let me attach the core dump as it's too big. How do I upload that? > > Thanks, > Ted > > > -----Original Message----- > From: Robin Sommer [mailto:robin at icir.org] > Sent: Monday, March 30, 2015 10:39 AM > To: Llewellyn, Ted > Cc: bro at bro.org > Subject: Re: [Bro] New installation crashes appear to be ssh-related > > Ted, mind filing a ticket so that we track this one? > > Robin > > On Sun, Mar 29, 2015 at 15:12 +0000, you wrote: > > > We have a new Bro installation, built from source on Debian wheezy, that keeps core dumping. It looks like it's choking on some code related to ssh. Here is the diag for the latest crash. It is identical to the other one I have: > > > > [BroControl] > diag > > [bro] > > > > Bro 2.3-633 > > Linux 3.2.0-4-686-pae > > > > No gdb installed. > > > > ==== No reporter.log > > > > ==== stderr.log > > listening on eth1, capture length 8192 bytes > > > > bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion `t_dataptr_after_cookie <= t_end_of_data' failed. > > /usr/local/bro/share/broctl/scripts/run-bro: line 100: 10307 Aborted (core dumped) nohup "$mybro" "$@" > > > > ==== stdout.log > > max memory size (kbytes, -m) unlimited > > data seg size (kbytes, -d) unlimited > > virtual memory (kbytes, -v) unlimited > > core file size (blocks, -c) unlimited > > > > ==== .cmdline > > -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p > > bro local.bro broctl broctl/standalone broctl/auto > > > > ==== .env_vars > > PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/local > > /bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > > BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/usr > > /local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/sh > > are/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/site > > CLUSTER_NODE= > > > > ==== .status > > RUNNING [net_run] > > > > ==== No prof.log > > > > ==== No packet_filter.log > > > > ==== No loaded_scripts.log > > [BroControl] > > > > > This is just running the default setup, with the local subnets > > configured, as we are just starting with Bro. This is a really low end server, but the capture interface is only running at 100 meg so there are really no resource issues. (Yes, this is a 32-bit box. It's pretty old. That's why I built from source.) The first crash occurred after a few minutes. Then it ran for nearly 24 hours before this crash. Is there something I can tweak to prevent this? > > > > Thanks, > > Ted Llewellyn > > > > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > -- > Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin > -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From Ted.Llewellyn at ftr.com Tue Mar 31 09:36:58 2015 From: Ted.Llewellyn at ftr.com (Llewellyn, Ted) Date: Tue, 31 Mar 2015 16:36:58 +0000 Subject: [Bro] New installation crashes appear to be ssh-related In-Reply-To: <20150331154224.GM31994@icir.org> References: <20150330143920.GD22580@icir.org> <20150331154224.GM31994@icir.org> Message-ID: Robin, I have attached the backtrace to the ticket, but here it is also: (gdb) bt #0 0xb76e6424 in __kernel_vsyscall () #1 0xb71b4661 in raise () from /lib/i386-linux-gnu/i686/cmov/libc.so.6 #2 0xb71b7a92 in abort () from /lib/i386-linux-gnu/i686/cmov/libc.so.6 #3 0xb71ad878 in __assert_fail () from /lib/i386-linux-gnu/i686/cmov/libc.so.6 #4 0x083eaabe in binpac::SSH::SSH2_KEXINIT::Parse (this=0xac7ac978, t_begin_of_data=t_begin_of_data at entry=0xac533ff6 "", t_end_of_data=t_end_of_data at entry=0xac534008 "\210>%\255\035", t_context=t_context at entry=0xad9419e8, t_byteorder=t_byteorder at entry=0) at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382 #5 0x083eac60 in binpac::SSH::SSH2_Message::Parse (this=0xad22d938, t_begin_of_data=t_begin_of_data at entry=0xac533ff6 "", t_end_of_data=t_end_of_data at entry=0xac534008 "\210>%\255\035", t_context=t_context at entry=0xad9419e8, t_byteorder=t_byteorder at entry=0) at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1216 #6 0x083eb160 in binpac::SSH::SSH2_Key_Exchange::ParseBuffer ( this=0xab743610, t_flow_buffer=0xafd04dc0, t_context=0xad9419e8, t_byteorder=0) at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1090 #7 0x083eb4d6 in binpac::SSH::SSH_Key_Exchange::ParseBuffer (this=0xaeb2e878, t_flow_buffer=0xafd04dc0, t_context=0xad9419e8, t_byteorder=0) at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:520 #8 0x083eb6ff in binpac::SSH::SSH_PDU::ParseBuffer (this=0xaeb323f8, t_flow_buffer=0xafd04dc0, t_context=0xad9419e8) at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:360 ---Type to continue, or q to quit--- #9 0x083eb982 in binpac::SSH::SSH_Flow::NewData (this=0xafd635b8, t_begin_of_data=0xac533ff0 "", t_end_of_data=0xac534008 "\210>%\255\035") at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:2913 #10 0x083e2855 in analyzer::SSH::SSH_Analyzer::DeliverStream (this=0xadc3e1f0, len=24, data=0xac533ff0 "", orig=true) at /root/bro/src/analyzer/protocol/ssh/SSH.cc:71 #11 0x08479f34 in analyzer::Analyzer::NextStream (this=0xadc3e1f0, len=24, data=0xac533ff0 "", is_orig=true) at /root/bro/src/analyzer/Analyzer.cc:245 #12 0x0847a72c in analyzer::Analyzer::ForwardStream (this=0xae014040, len=24, data=0xac533ff0 "", is_orig=true) at /root/bro/src/analyzer/Analyzer.cc:331 #13 0x0840ddec in analyzer::tcp::TCP_Reassembler::DeliverBlock ( this=this at entry=0xadbc1cb0, seq=16, len=len at entry=24, data=0xac533ff0 "") at /root/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:647 #14 0x0840e2cc in BlockInserted (start_block=, this=) at /root/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:393 #15 analyzer::tcp::TCP_Reassembler::BlockInserted (this=0xadbc1cb0, start_block=0xac648218) at /root/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:368 #16 0x0840db2e in analyzer::tcp::TCP_Reassembler::DataSent (this=0xadbc1cb0, t=1427797676.2736609, seq=16, len=, data=0xa8d1a4a "", replaying=true) at /root/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:492 ---Type to continue, or q to quit--- #17 0x0840beeb in analyzer::tcp::TCP_Endpoint::DataSent (this=0xadc74340, t=1427797676.2736609, seq=16, len=24, caplen=24, data=0xa8d1a4a "", ip=0xbfbeacac, tp=0xa8d1a2a) at /root/bro/src/analyzer/protocol/tcp/TCP_Endpoint.cc:205 #18 0x08408c76 in DeliverData (flags=..., is_orig=, rel_data_seq=16, endpoint=0xadc74340, tp=0xa8d1a2a, ip=0xbfbeacac, caplen=, len=, data=, t=, this=0xae014040) at /root/bro/src/analyzer/protocol/tcp/TCP.cc:947 #19 analyzer::tcp::TCP_Analyzer::DeliverPacket (this=0xae014040, len=24, data=0xa8d1a4a "", is_orig=true, seq=18446744073709551615, ip=0xbfbeacac, caplen=24) at /root/bro/src/analyzer/protocol/tcp/TCP.cc:1347 #20 0x0847a118 in analyzer::Analyzer::NextPacket (this=0xae014040, len=56, data=0xa8d1a2a "\230", , is_orig=true, seq=18446744073709551615, ip=0xbfbeacac, caplen=56) at /root/bro/src/analyzer/Analyzer.cc:222 #21 0x081951c4 in Connection::NextPacket (this=0xafd52858, t=1427797676.2736609, is_orig=1, ip=0xbfbeacac, len=56, caplen=56, data=@0xbfbeaa68: 0xa8d1a2a "\230", , record_packet=@0xbfbeaa70: 1, record_content=@0xbfbeaa74: 1, hdr=0xa097074, pkt=0xa8d1a08 "", hdr_size=14) at /root/bro/src/Conn.cc:260 #22 0x08238ca0 in NetSessions::DoNextPacket (this=this at entry=0xa8d3a10, t=1427797676.2736609, ---Type to continue, or q to quit--- t at entry=, hdr=hdr at entry=0xa097074, ip_hdr=ip_hdr at entry=0xbfbeacac, pkt=pkt at entry=0xa8d1a08 "", hdr_size=hdr_size at entry=14, encapsulation=encapsulation at entry=0x0) at /root/bro/src/Sessions.cc:760 #23 0x0823a3bc in NetSessions::NextPacket (this=0xa8d3a10, t=1427797676.2736609, hdr=0xa097074, pkt=0xa8d1a08 "", hdr_size=14) at /root/bro/src/Sessions.cc:231 #24 0x08205de6 in net_packet_dispatch (t=1427797676.2736609, hdr=0xa097074, pkt=0xa8d1a08 "", hdr_size=14, src_ps=0xa096f88) at /root/bro/src/Net.cc:281 #25 0x0844d5ce in iosource::PktSrc::Process (this=0xa096f88) at /root/bro/src/iosource/PktSrc.cc:411 #26 0x0820631a in net_run () at /root/bro/src/Net.cc:329 #27 0x0815e588 in main (argc=19, argv=0xbfbeb214) at /root/bro/src/main.cc:1212 (gdb) Ted -----Original Message----- From: Robin Sommer [mailto:robin at icir.org] Sent: Tuesday, March 31, 2015 11:42 AM To: Llewellyn, Ted Cc: bro at bro.org Subject: Re: [Bro] New installation crashes appear to be ssh-related Thanks for filing the ticket. For the core, actually what would be most helpful right now I believe is a stack backtrace. Your crash report didn't have that, it looks like there's no gdb installed. Can you install gdb and then run "gdb bro core" + "bt" as described here: https://www.bro.org/support/reporting-problems.html#getting-more-information-after-acrash For the core itself, I think the best thing might be to hold on to it for now, just the core won't be useful for others much anyways, as one also needs to the binary and potentially a similar system to use it. So if you could keep binary and core somewhere until this is resolved, that would be best for now. Robin On Mon, Mar 30, 2015 at 23:54 +0000, you wrote: > Robin, > > I submitted a ticket, 1361. It won't let me attach the core dump as it's too big. How do I upload that? > > Thanks, > Ted > > > -----Original Message----- > From: Robin Sommer [mailto:robin at icir.org] > Sent: Monday, March 30, 2015 10:39 AM > To: Llewellyn, Ted > Cc: bro at bro.org > Subject: Re: [Bro] New installation crashes appear to be ssh-related > > Ted, mind filing a ticket so that we track this one? > > Robin > > On Sun, Mar 29, 2015 at 15:12 +0000, you wrote: > > > We have a new Bro installation, built from source on Debian wheezy, that keeps core dumping. It looks like it's choking on some code related to ssh. Here is the diag for the latest crash. It is identical to the other one I have: > > > > [BroControl] > diag > > [bro] > > > > Bro 2.3-633 > > Linux 3.2.0-4-686-pae > > > > No gdb installed. > > > > ==== No reporter.log > > > > ==== stderr.log > > listening on eth1, capture length 8192 bytes > > > > bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion `t_dataptr_after_cookie <= t_end_of_data' failed. > > /usr/local/bro/share/broctl/scripts/run-bro: line 100: 10307 Aborted (core dumped) nohup "$mybro" "$@" > > > > ==== stdout.log > > max memory size (kbytes, -m) unlimited > > data seg size (kbytes, -d) unlimited > > virtual memory (kbytes, -v) unlimited > > core file size (blocks, -c) unlimited > > > > ==== .cmdline > > -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local > > -p bro local.bro broctl broctl/standalone broctl/auto > > > > ==== .env_vars > > PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/loc > > al > > /bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bi > > n > > BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/u > > sr > > /local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/ > > sh > > are/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/sit > > e > > CLUSTER_NODE= > > > > ==== .status > > RUNNING [net_run] > > > > ==== No prof.log > > > > ==== No packet_filter.log > > > > ==== No loaded_scripts.log > > [BroControl] > > > > > This is just running the default setup, with the local subnets > > configured, as we are just starting with Bro. This is a really low end server, but the capture interface is only running at 100 meg so there are really no resource issues. (Yes, this is a 32-bit box. It's pretty old. That's why I built from source.) The first crash occurred after a few minutes. Then it ran for nearly 24 hours before this crash. Is there something I can tweak to prevent this? > > > > Thanks, > > Ted Llewellyn > > > > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > -- > Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin > -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin