From jbabio at po-box.esu.edu Sun Sep 1 16:48:59 2013 From: jbabio at po-box.esu.edu (John Babio) Date: Sun, 1 Sep 2013 23:48:59 +0000 Subject: [Bro] bro notices to snorby Message-ID: Is anyone working on something like this? From rickv9 at live.com Tue Sep 3 11:26:13 2013 From: rickv9 at live.com (Rick Vnine) Date: Tue, 3 Sep 2013 14:26:13 -0400 Subject: [Bro] To bro, or not to Bro? Message-ID: Hey all, I'm trying to follow and correlate web sessions that go through a proxy server pool and then through a NATing firewall - and I'm curious if Bro would be the tool to use? My immediate need is to troubleshoot a nagging performance issue with our web browsing. I'd like to be able to analyze all our web sessions during a period and discover those that show slow responses and trace back to where the delay is located.What makes this tricky is that the sample would be 1000's of clients, 100's of GB per hour in traffic, a pool of proxy servers, and a FW that is doing NATing/PATing and protocol inspection (it doesn't blindly forward every packet, it is in the middle of the TCP sessions randomizing SEQ numbers and such).Specifically, we have a few thousand web clients that are transparently proxied (WCCP) for HTTP/S. After their traffic leaves the proxy, it goes through a FW and out to the Internet.I've analyzed specific flows, manually tracing through the hops by inserting unique headers, but that is fairly laborious and I don't know for sure if I'm truly capturing what the clients are seeing.I can capture traffic at each stage simultaneously, so I should be able to get a good view.I believe I would need to:-Create a list of all client->proxy TCP sessions--Match them to any DNS queries--understand what a complete page/request is (i.e. a complete yahoo.com request goes to dozens of sites to build the page)-Create a list of proxy->FW sessions--Correlate to client->proxy session list via destination, client headers, time, etc.-Create a list of ASA->Internet sessions via destination, client headers, time, etc.--Correlate to client->proxy->ASA listOnce I have a complete flow from client to Internet, then I need to analyze and look for patterns of delays. Are delays caused by one or more proxy servers all the time, periodically, based on certain destinations, etc? I noticed the FW normally forwards packets one for one unless a packet is dropped from the Internet - then it buffers the packets until it can deliver a complete response back to the proxy. I assume the FW wants a complete response so that it can do its protocol inspection.(After I typed this out, I'm thinking it'll probably be quicker to use PhantomJS/other and drive some test workstations, but I am curious how/if Bro can do this because analysis of actual traffic would be most accurate imho). Thanks for any input -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130903/86ca0c3d/attachment.html From nuno at ipb.pt Wed Sep 4 10:14:07 2013 From: nuno at ipb.pt (Nuno Rodrigues) Date: Wed, 04 Sep 2013 18:14:07 +0100 Subject: [Bro] Specification of log fields Message-ID: <52276A5F.8030103@ipb.pt> Dears Sirs, I'm a recent user of Bro, so excuse me if this is a basic question... I need to find the exact meaning of each field of log files. For example, "duration is in seconds, miliseconds...?", "difference between orig_bytes and orig_ip_bytes" and so on. Still exist this information published in any place? Thanks in advance! Best regards, Nuno. -- ................................................................. Nuno Rodrigues : e-mail e msn: nuno at ipb.pt : http://www.ipb.pt/~nuno Eq. Assistente 2o Tri?nio : Dep. Inform?tica e Comunica??es : ESTiG/IPB Coordenador do Centro de Comunica??es do IPB ................................................................. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3257 bytes Desc: S/MIME Cryptographic Signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130904/2bc63569/attachment.bin From jsiwek at illinois.edu Wed Sep 4 11:00:35 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 4 Sep 2013 18:00:35 +0000 Subject: [Bro] Specification of log fields In-Reply-To: <52276A5F.8030103@ipb.pt> References: <52276A5F.8030103@ipb.pt> Message-ID: > I need to find the exact meaning of each field of log files. For example, "duration is in seconds, miliseconds...?", "difference between orig_bytes and orig_ip_bytes" and so on. You can usually find descriptions if you browse the script reference documentation like at [1]. Generally, for fields of type "interval" (like "duration"), the unit is seconds. - Jon [1] http://bro.org/sphinx/scripts/base/protocols/conn/main.html#type-Conn::Info From jpl at didconcept.com Wed Sep 4 17:22:09 2013 From: jpl at didconcept.com (Jean-Philippe Luiggi) Date: Wed, 04 Sep 2013 20:22:09 -0400 Subject: [Bro] Redux - source build on OpenBSD (5.3) and BIND libs In-Reply-To: References: <521B70E9.1040206@louruppert.com> Message-ID: <5227CEB1.6000202@didconcept.com> Hi everybody, As Lou said, getting libbind6 is the solution. Here is the process I followed today : 1) Packages I had to install some of them (maybe others are needed but as my VPS was up and running...) # pkg_add flex bison libmagic swig ruby python gmake Note : ruby is ruby 1.9 and python is 2.7 2) Libbind 6.0 - Get libbind 6 from : http://www.isc.org/downloads/ - Extract it somewhere then compile/install it. Note that the directory I used (/opt/share/libbind-6.0) could be another one. # tar zxvf libbind-6.0.tar.gz # cd libbind-6.0 # ./configure --prefix=/opt/share/libbind-6.0; make; make install 3) Bro 2.1 - Extract bro-2.1 then compile it # tar zxvf bro-2.1.tar.gz # cd bro-2.1 # CFLAGS="-I/usr/include/ -I/usr/local/include/ -I/usr/local/include/ruby-1.9.1/ruby/backward/" ./configure --prefix=/opt/share/bro-2.1 --with-bind=/opt/share/libbind-6.0/ # gmake; gmake install Please note I use "/opt/share/bro-2.1" as install directory and I use "gmake". The whole process should work but if not, just send a mail, I will be happy helping about it :) Cheers. Jean-Philippe. PS: I will try to provide a complete document in the next weeks Le 27/08/2013 02:00, Darren Spruell a ?crit : > Indeed, appears that the libbind port disappeared for 5.2 and 5.3 but > has since been reinstated. > > http://www.openbsd.org/cgi-bin/cvsweb/ports/net/libbind/ > http://marc.info/?l=openbsd-ports&m=136465613319561&w=2 > > I'll give it a go with the source package, thanks for the help. > -- Ce message a ?t? v?rifi? par MailScanner pour des virus ou des polluriels et rien de suspect n'a ?t? trouv?. For all your IT requirements visit: http://www.transtec.co.uk From phatbuckett at gmail.com Wed Sep 4 23:22:42 2013 From: phatbuckett at gmail.com (Darren Spruell) Date: Wed, 4 Sep 2013 23:22:42 -0700 Subject: [Bro] Redux - source build on OpenBSD (5.3) and BIND libs In-Reply-To: <5227CEB1.6000202@didconcept.com> References: <521B70E9.1040206@louruppert.com> <5227CEB1.6000202@didconcept.com> Message-ID: Very nice. I completed install in very similar manner, although things appeared to work without requiring CFLAGS to be set for the configure script; my built completed with: ./configure --with-bind=/opt/libbind gmake; gmake install That said, I do notice that when invoking some bro commands I get the following output about symbol size mismatches: $ bro --help bro:/usr/lib/libc.so.66.2: bro : WARNING: symbol(__p_class_syms) size mismatch, relink your program bro:/usr/lib/libc.so.66.2: bro : WARNING: symbol(_res) size mismatch, relink your program bro:/usr/lib/libc.so.66.2: bro : WARNING: symbol(__p_type_syms) size mismatch, relink your program bro version 2.1 [...] I can see this has been encountered several times over the years: (2003) http://comments.gmane.org/gmane.comp.security.detection.bro/222 (2006) http://comments.gmane.org/gmane.comp.security.detection.bro/1382 (2007) http://comments.gmane.org/gmane.comp.security.detection.bro/1890 (2008) http://comments.gmane.org/gmane.os.openbsd.ports/18520 (2012) http://comments.gmane.org/gmane.comp.security.detection.bro/4254 Someone suggested it was related to issue between libbind and libc resolver routines. Anyone know more? On Wed, Sep 4, 2013 at 5:22 PM, Jean-Philippe Luiggi wrote: > Hi everybody, > > As Lou said, getting libbind6 is the solution. > > Here is the process I followed today : > > 1) Packages > > I had to install some of them (maybe others are needed but as my VPS was up > and running...) > > # pkg_add flex bison libmagic swig ruby python gmake > > Note : ruby is ruby 1.9 and python is 2.7 > > > 2) Libbind 6.0 > > - Get libbind 6 from : http://www.isc.org/downloads/ > - Extract it somewhere then compile/install it. Note that the directory I > used (/opt/share/libbind-6.0) could be another one. > > # tar zxvf libbind-6.0.tar.gz > # cd libbind-6.0 > # ./configure --prefix=/opt/share/libbind-6.0; make; make install > > 3) Bro 2.1 > > - Extract bro-2.1 then compile it > > # tar zxvf bro-2.1.tar.gz > # cd bro-2.1 > # CFLAGS="-I/usr/include/ -I/usr/local/include/ > -I/usr/local/include/ruby-1.9.1/ruby/backward/" ./configure > --prefix=/opt/share/bro-2.1 --with-bind=/opt/share/libbind-6.0/ > # gmake; gmake install > > Please note I use "/opt/share/bro-2.1" as install directory and I use > "gmake". > > The whole process should work but if not, just send a mail, I will be happy > helping about it :) > > Cheers. > > Jean-Philippe. > > PS: I will try to provide a complete document in the next weeks > > > Le 27/08/2013 02:00, Darren Spruell a ?crit : > >> Indeed, appears that the libbind port disappeared for 5.2 and 5.3 but >> has since been reinstated. >> >> http://www.openbsd.org/cgi-bin/cvsweb/ports/net/libbind/ >> http://marc.info/?l=openbsd-ports&m=136465613319561&w=2 >> >> I'll give it a go with the source package, thanks for the help. >> > > > -- > Ce message a ?t? v?rifi? par MailScanner > pour des virus ou des polluriels et rien de > suspect n'a ?t? trouv?. > For all your IT requirements visit: http://www.transtec.co.uk > -- Darren Spruell phatbuckett at gmail.com From jpl at didconcept.com Thu Sep 5 06:14:12 2013 From: jpl at didconcept.com (Jean-Philippe Luiggi) Date: Thu, 05 Sep 2013 09:14:12 -0400 Subject: [Bro] Redux - source build on OpenBSD (5.3) and BIND libs In-Reply-To: References: <521B70E9.1040206@louruppert.com> <5227CEB1.6000202@didconcept.com> Message-ID: <522883A4.9070002@didconcept.com> Hi, I encountered the same messages but didn't have the time to find the reason and explain it. I put this problem on hold and try to get a response as soon as possible. Speaking of "CFLAGS", I had to use it because of the Ruby binding. I think Broccoli wants to use "rubyio.h" which is in "/usr/local/include/ruby-1.9.1/ruby/backward/". This is the only reason I need to use this specific flag. Cheers. Jean-Philippe. Le 2013-09-05 02:22, Darren Spruell a ?crit : > Very nice. > > I completed install in very similar manner, although things appeared > to work without requiring CFLAGS to be set for the configure script; > my built completed with: > > ./configure --with-bind=/opt/libbind > gmake; gmake install > > That said, I do notice that when invoking some bro commands I get the > following output about symbol size mismatches: > > $ bro --help > bro:/usr/lib/libc.so.66.2: bro : WARNING: symbol(__p_class_syms) size > mismatch, relink your program > bro:/usr/lib/libc.so.66.2: bro : WARNING: symbol(_res) size mismatch, > relink your program > bro:/usr/lib/libc.so.66.2: bro : WARNING: symbol(__p_type_syms) size > mismatch, relink your program > bro version 2.1 > [...] > > I can see this has been encountered several times over the years: > > (2003) http://comments.gmane.org/gmane.comp.security.detection.bro/222 > (2006) http://comments.gmane.org/gmane.comp.security.detection.bro/1382 > (2007) http://comments.gmane.org/gmane.comp.security.detection.bro/1890 > (2008) http://comments.gmane.org/gmane.os.openbsd.ports/18520 > (2012) http://comments.gmane.org/gmane.comp.security.detection.bro/4254 > > Someone suggested it was related to issue between libbind and libc > resolver routines. Anyone know more? > > > On Wed, Sep 4, 2013 at 5:22 PM, Jean-Philippe Luiggi wrote: >> Hi everybody, >> >> As Lou said, getting libbind6 is the solution. >> >> Here is the process I followed today : >> >> 1) Packages >> >> I had to install some of them (maybe others are needed but as my VPS was up >> and running...) >> >> # pkg_add flex bison libmagic swig ruby python gmake >> >> Note : ruby is ruby 1.9 and python is 2.7 >> >> >> 2) Libbind 6.0 >> >> - Get libbind 6 from : http://www.isc.org/downloads/ >> - Extract it somewhere then compile/install it. Note that the directory I >> used (/opt/share/libbind-6.0) could be another one. >> >> # tar zxvf libbind-6.0.tar.gz >> # cd libbind-6.0 >> # ./configure --prefix=/opt/share/libbind-6.0; make; make install >> >> 3) Bro 2.1 >> >> - Extract bro-2.1 then compile it >> >> # tar zxvf bro-2.1.tar.gz >> # cd bro-2.1 >> # CFLAGS="-I/usr/include/ -I/usr/local/include/ >> -I/usr/local/include/ruby-1.9.1/ruby/backward/" ./configure >> --prefix=/opt/share/bro-2.1 --with-bind=/opt/share/libbind-6.0/ >> # gmake; gmake install >> >> Please note I use "/opt/share/bro-2.1" as install directory and I use >> "gmake". >> >> The whole process should work but if not, just send a mail, I will be happy >> helping about it :) >> >> Cheers. >> >> Jean-Philippe. >> >> PS: I will try to provide a complete document in the next weeks >> >> >> Le 27/08/2013 02:00, Darren Spruell a ?crit : >> >>> Indeed, appears that the libbind port disappeared for 5.2 and 5.3 but >>> has since been reinstated. >>> >>> http://www.openbsd.org/cgi-bin/cvsweb/ports/net/libbind/ >>> http://marc.info/?l=openbsd-ports&m=136465613319561&w=2 >>> >>> I'll give it a go with the source package, thanks for the help. >>> >> >> -- >> Ce message a ?t? v?rifi? par MailScanner >> pour des virus ou des polluriels et rien de >> suspect n'a ?t? trouv?. >> For all your IT requirements visit: http://www.transtec.co.uk >> > > -- Ce message a ?t? v?rifi? par MailScanner pour des virus ou des polluriels et rien de suspect n'a ?t? trouv?. For all your IT requirements visit: http://www.transtec.co.uk From seth at icir.org Thu Sep 5 08:06:43 2013 From: seth at icir.org (Seth Hall) Date: Thu, 5 Sep 2013 11:06:43 -0400 Subject: [Bro] Redux - source build on OpenBSD (5.3) and BIND libs In-Reply-To: <522883A4.9070002@didconcept.com> References: <521B70E9.1040206@louruppert.com> <5227CEB1.6000202@didconcept.com> <522883A4.9070002@didconcept.com> Message-ID: <0C5A93C3-CC44-40B0-B1B3-BAA0F71693E3@icir.org> On Sep 5, 2013, at 9:14 AM, Jean-Philippe Luiggi wrote: > Speaking of "CFLAGS", I had to use it because of the Ruby binding. > I think Broccoli wants to use "rubyio.h" which is in > "/usr/local/include/ruby-1.9.1/ruby/backward/". > This is the only reason I need to use this specific flag. Yeah, that's probably because the ruby bindings are still stuck at 1.8.x support only. I've never had the motivation to update them to a newer version of Ruby. On the other hand, the ruby bindings aren't necessary for anything unless you happen to be using them in scripts you've written locally. BroControl only uses the python bindings. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130905/415465f6/attachment.bin From jeppojeps at gmail.com Thu Sep 5 08:38:26 2013 From: jeppojeps at gmail.com (Antonio Nappa) Date: Thu, 5 Sep 2013 17:38:26 +0200 Subject: [Bro] File carving Message-ID: Hello, I'm quite new to bro, so please be kind :) I'm using this script: #######################################! module HTTP; export { # NOTICE Type redef enum Notice::Type += { Exe_File_Capture, }; # File Magic Bytes to look for const file_magic_bytes = /^\x50\x4B\x03\x04\x14\x00\x08\x00\x08\x00/ &redef; # MIME types to look for const extract_mime_types = /application\/java-archive/ &redef; redef HTTP::extract_mime_types += /application\/x-java-archive/; # redef HTTP::extract_mime_types += /application\/x-msdownload/; # redef HTTP::extract_mime_types += /application\/exe/; # redef HTTP::extract_mime_types += /application\/x-exe/; # redef HTTP::extract_mime_types += /application\/dos-exe/; # redef HTTP::extract_mime_types += /application\/x-winexe/; # redef HTTP::extract_mime_types += /application\/msdos-windows/; # redef HTTP::extract_mime_types += /application\/x-msdos-program/; # File extensions to look for const extract_file_extensions = /\.[jJ][aA][rR]$/ &redef; redef HTTP::extract_file_extensions += /\.[sS][cC][rR]$/; # Size constraints of file to extract (TODO) const minimum_size = 10240 &redef; # 10K const maximum_size = 8388608 &redef; # 8MB const ufile = open("malware_urls.txt"); # URL patterns to whitelist const whitelist_url_patterns = /^http:\/\/[^\/]*\.windowsupdate\.com\// &redef; redef HTTP::whitelist_url_patterns += /^http:\/\/[^\/]*\.microsoft\.com\//; redef HTTP::whitelist_url_patterns += /^http:\/\/[^\/]*\.google\.com\//; const hd: table[count] of mime_header_rec ={}; const chk = F; #type mime_header_list: table[count] of mime_header_rec; # Information to track throughout session redef record Info += { extraction_prefix: string &optional; extraction_file: file &log &optional; extract_file: bool &default=F; extracted_size: count &default=0; }; } event http_all_headers (c: connection, is_orig: bool, hlist: mime_header_list) &priority=-1 { hd = hlist; } event http_entity_data(c: connection, is_orig: bool, length: count, data: string) &priority=-5 { # Ignore client communication if ( is_orig ) return; # If in first chunk of data if ( c$http$first_chunk ) { # Get the URL for whitelisting and extension matching local url = build_url_http(c$http); # Check for file magic byte matches if ( HTTP::file_magic_bytes in data ) { c$http$extraction_prefix = "magic-match"; c$http$extract_file = T; } #Check for MIME type matches else if ( HTTP::extract_mime_types in c$http$mime_type ) { c$http$extraction_prefix = "mime-match"; c$http$extract_file = T; } # Check for file extension matches else if ( HTTP::extract_file_extensions in url ) { c$http$extraction_prefix = "extension-match"; c$http$extract_file = T; } # Content Disposition HTTP Header String - TODO? # If a magic byte, MIME, or Ext match... if ( c$http$extract_file ) { # Check against whitelist if ( HTTP::whitelist_url_patterns in url ) { c$http$extract_file = F; } else { # Open file to capture data local suffix = fmt("%s_%d.jar", is_orig ? "orig" : "resp", c$http_state$current_response); local fname = generate_extraction_filename(c$http$extraction_prefix, c, suffix); c$http$extraction_file = open(fname); enable_raw_output(c$http$extraction_file); c$http$extracted_size = 0; local output=fmt("%s,%s", c$id$resp_h,c$http$uri); #local reshdr= fmt("%s %s %s", c$http$status_code, c$http$status_msg, c$http$info_msg); print ufile,"########################"; print ufile, output; print ufile,c$http$status_code,c$http$status_msg; for (j in hd) { print ufile, hd[j]; } print ufile,"########################"; local message = fmt("Storing %s to %s", url, c$http$extraction_file); local method = "UNKNOWN"; if ( c$http?$method ) method = c$http$method; NOTICE([$note=Exe_File_Capture, $msg=message, $conn=c, $method=method, $URL=url]); } } } # End first chunk if # Do the capture when a capture file is open if ( c$http?$extraction_file ) { print c$http$extraction_file, data; c$http$extracted_size += length; } } # End HTTP entity data event http_end_entity(c: connection, is_orig: bool) { if ( c$http?$extraction_file ) { close(c$http$extraction_file); } } ############################################################################## to carve jar files from pcaps, the problem is that if I manually carve a file from wireshark the size and the hash of this file are different from the ones that I get if I use bro. What is funny is that in the http.log I get the right md5 of the jar file, but then if I use the md5sum utility on the file extracted with bro they don't match, and actually the content-lengh does not match with size of the extracted file. I've already checked if there are multiple jar archives in the pcap that I'm using as example but there's only one. So I wondering what can be wrong. Thank you for you help. Cheers Antonio -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130905/aa2fc093/attachment.html From seth at icir.org Thu Sep 5 09:14:57 2013 From: seth at icir.org (Seth Hall) Date: Thu, 5 Sep 2013 12:14:57 -0400 Subject: [Bro] File carving In-Reply-To: References: Message-ID: <9611F285-35E3-4DF9-87D3-B1282C42766C@icir.org> On Sep 5, 2013, at 11:38 AM, Antonio Nappa wrote: > to carve jar files from pcaps, the problem is that if I manually carve a file from wireshark the size and the hash of this file are different from the ones that I get if I use bro. What is funny is that in the http.log I get the right md5 of the jar file, but then if I use the md5sum utility on the file extracted with bro they don't match There is some problem with doing file extraction in 2.1 that pops up from time to time. I don't think there is anyone that is totally clear what the problem is and we've completely revamped file handling for 2.2. The video where I discuss file handling in the upcoming 2.2 release was just released today and there are some exercises available too. Video: http://security.ncsa.illinois.edu/BroExchange2013/Hall-File_Analysis-NCSA%20DSS%20H264%201.25Mbps.mp4 Exercises: http://bro.org/bro-exchange-2013/exercises/faf.html .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130905/6762e333/attachment.bin From christianbuia at gmail.com Tue Sep 10 16:04:57 2013 From: christianbuia at gmail.com (Christian Buia) Date: Tue, 10 Sep 2013 19:04:57 -0400 Subject: [Bro] Log an Arbitrarily Long Collection Message-ID: Greetings Brofolk, I have become increasingly interested in Bro lately, and I am starting to explore how my organization can use it as a general network processor to generate some verbose logs that we can then export for indexing and analytics. The first use case I would like to explore involves generating verbose HTTP logs so that we can identify suspicious characteristics such as direct-to-IP host headers, missing headers, unexpected ordering of headers, RFC compliance issues, etc. I spent some time auditing the main.bro script for the HTTP protocol, and then proceeded to make some edits to add additional record fields. Specifically, I created a new record: type HeaderValue: record { h: string &optional; v: string &optional; }; and within the Info type, I added the following members: client_headers: vector of HeaderValue &log &optional; client_headers_count: count &default=0; server_headers: vector of HeaderValue &log &optional; server_headers_count: count &default=0; I decided to use a vector because I want to keep track of header order. Further down on the http_header event handler, I add each header to the appropriate vector, indexed by the count field which increments. When I fire up bro, I get the following error message: error in /usr/local/bro/share/bro/base/protocols/http/./main.bro, line 83: &log applied to a type that cannot be logged (&log) So presumably Bro doesn't like the idea of generating a log entry that includes a vector type (no less consisting of record members, I'm not even sure what that would have looked like but was hoping to find out). The next best thing I can think of doing is recording this information with some custom delimeter in a single string field, such as: Accept:*/*|||Accept-Language:en-US|||User-Agent:Mozilla 4.0|||Host: somehost.com||||Connection:Keep-Alive Further downstream I plan to convert the tab-delimited content to JSON anyways prior to indexing. Is this a good solution for including an arbitrarily long collection field in my HTTP logs? Is there a better way to accomplish this? Also, I have a feeling that directly editing http/main.bro is a bad practice. Should I instead be adding this script to the policy branch, redefining the HTTP Info object and handling the http header event in there? Thanks for your attention! Christian -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130910/03e9a5d9/attachment.html From jp.bourget at gmail.com Wed Sep 11 07:34:40 2013 From: jp.bourget at gmail.com (JP Bourget) Date: Wed, 11 Sep 2013 15:34:40 +0100 Subject: [Bro] updating dev from git repo Message-ID: Hello, I'm trying to update to the latest version. (from 2.1.1052) Few questions: Can I just do a git pull origin master git pull origin master - it seems to grab all the submodules do I need to: git submodule foreach git pull origin master? If I just do the first one, and run ./configure I get the following to files missing: missing - sqllite3.c and CMake Warning at aux/broctl/aux/trace-summary/CMakeLists.txt:30 (message): Could not find dependency: 'ipsumdump', trace-summary will not be able to read pcap traces I see that at least the sqllite3.c is available from the repo - how can I fix this so that it's fixed properly - should I just copy the files from the repo? Or should I actually run a different git command? Thanks in advance! -- JP -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130911/924a6df7/attachment.html From seth at icir.org Wed Sep 11 07:48:37 2013 From: seth at icir.org (Seth Hall) Date: Wed, 11 Sep 2013 10:48:37 -0400 Subject: [Bro] updating dev from git repo In-Reply-To: References: Message-ID: <3C690D02-71FE-44C6-90E4-5595FFD309A2@icir.org> On Sep 11, 2013, at 10:34 AM, JP Bourget wrote: > Few questions: > Can I just do a git pull origin master > > git pull origin master - it seems to grab all the submodules When you update, sometimes (if we update submodules) you will also need to run: git submodule update --init --recursive .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130911/b18e5463/attachment.bin From jsiwek at illinois.edu Wed Sep 11 07:54:56 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 11 Sep 2013 14:54:56 +0000 Subject: [Bro] Log an Arbitrarily Long Collection In-Reply-To: References: Message-ID: > server_headers: vector of HeaderValue &log &optional; > server_headers_count: count &default=0; > > I decided to use a vector because I want to keep track of header order. Further down on the http_header event handler, I add each header to the appropriate vector, indexed by the count field which increments. A minor point: you don't really have to store the count since it's implicit in the vector if you use `|?|` to get the size of the container. E.g. |c$http$server_headers| in this case would be equivalent to: c$http$server_headers_count > error in /usr/local/bro/share/bro/base/protocols/http/./main.bro, line 83: &log applied to a type that cannot be logged (&log) > > So presumably Bro doesn't like the idea of generating a log entry that includes a vector type (no less consisting of record members, I'm not even sure what that would have looked like but was hoping to find out). A vector of an atomic type, say `vector of string`, should work. So an option would be to store the header names and values in two different vectors. > The next best thing I can think of doing is recording this information with some custom delimeter in a single string field That could work also and might be better if you're more concerned w/ human readability. > Also, I have a feeling that directly editing http/main.bro is a bad practice. Should I instead be adding this script to the policy branch, redefining the HTTP Info object and handling the http header event in there? Take a look at "policy/protocols/http/header-names.bro". Copy that to your own script (possibly storing it in $prefix/share/bro/site) and modify it to do what you want (log header name+value in whatever format). Then have bro load that script: if you're already loading local.bro, adding an @load in there for the new script is one way. - Jon From seth at icir.org Wed Sep 11 08:01:07 2013 From: seth at icir.org (Seth Hall) Date: Wed, 11 Sep 2013 11:01:07 -0400 Subject: [Bro] Log an Arbitrarily Long Collection In-Reply-To: References: Message-ID: <81D100AC-5779-466C-BAF8-8C01A26FDC75@icir.org> On Sep 10, 2013, at 7:04 PM, Christian Buia wrote: > So presumably Bro doesn't like the idea of generating a log entry that includes a vector type (no less consisting of record members, I'm not even sure what that would have looked like but was hoping to find out). We don't allow logging collections of complex types. You can log vectors, but only vectors of simple atomic types (addr, string, subnet, count, double, etc) which doesn't include records. You can log records too but they can't be part of a collection type (set, table, vector). If we had allowed the logging framework to log those types we would have been introducing a lot of headaches for ourselves down the road. > Accept:*/*|||Accept-Language:en-US|||User-Agent:Mozilla 4.0|||Host:somehost.com||||Connection:Keep-Alive That would work but it's just as nasty as I'm guessing you feel like it is. :) > Is this a good solution for including an arbitrarily long collection field in my HTTP logs? Is there a better way to accomplish this?  There are a number of ways of doing, each with upsides and downsides. They way I have done it (I have a script floating around somewhere?) is to add fields to HTTP::Info.. client_header_names: vector of string &log &optional; client_header_values: vector of string &log &optional; server_header_names: vector of string &log &optional; server_header_values: vector of string &log &optional; > Also, I have a feeling that directly editing http/main.bro is a bad practice. Yes, it's bad practice. > Should I instead be adding this script to the policy branch, redefining the HTTP Info object and handling the http header event in there? Yep. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130911/69b0cb0d/attachment.bin From jsiwek at illinois.edu Wed Sep 11 08:06:04 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 11 Sep 2013 15:06:04 +0000 Subject: [Bro] updating dev from git repo In-Reply-To: References: Message-ID: > I'm trying to update to the latest version. (from 2.1.1052) > > Few questions: > Can I just do a git pull origin master > > git pull origin master - it seems to grab all the submodules > > do I need to: git submodule foreach git pull origin master? Every time you pull from bro's master branch, you should make sure the submodules are updated. So the typical way to use Bro git repos is to get it for the first time: git clone --recursive git://git.bro.org/bro Then sometime later update the master branch git pull # Assuming master branch is already checked out locally. git submodule update --recursive --init # Update submodules. > and CMake Warning at aux/broctl/aux/trace-summary/CMakeLists.txt:30 (message): > > Could not find dependency: 'ipsumdump', trace-summary will not be able to read pcap traces This is unrelated to how you update from git, it's just pointing out an optional dependency, ipsumdump, wasn't found. - Jon From jp.bourget at gmail.com Wed Sep 11 08:28:27 2013 From: jp.bourget at gmail.com (JP Gmail) Date: Wed, 11 Sep 2013 16:28:27 +0100 Subject: [Bro] updating dev from git repo In-Reply-To: References: Message-ID: <5E8C821C-C666-4013-A9A5-4DD39281E17C@gmail.com> Thanks all, I understand! Appreciate the quick response! JP On Sep 11, 2013, at 4:06 PM, "Siwek, Jonathan Luke" wrote: > >> I'm trying to update to the latest version. (from 2.1.1052) >> >> Few questions: >> Can I just do a git pull origin master >> >> git pull origin master - it seems to grab all the submodules >> >> do I need to: git submodule foreach git pull origin master? > > Every time you pull from bro's master branch, you should make sure the submodules are updated. So the typical way to use Bro git repos is to get it for the first time: > > git clone --recursive git://git.bro.org/bro > > Then sometime later update the master branch > > git pull # Assuming master branch is already checked out locally. > git submodule update --recursive --init # Update submodules. > >> and CMake Warning at aux/broctl/aux/trace-summary/CMakeLists.txt:30 (message): >> >> Could not find dependency: 'ipsumdump', trace-summary will not be able to read pcap traces > > This is unrelated to how you update from git, it's just pointing out an optional dependency, ipsumdump, wasn't found. > > - Jon From jthoel at gmail.com Wed Sep 11 15:53:25 2013 From: jthoel at gmail.com (Jeremy Hoel) Date: Wed, 11 Sep 2013 22:53:25 +0000 Subject: [Bro] DNS query logging Message-ID: So I'm testing out bro for a limited use on recording dns queries and responses. I have the logs coming in and that's great, but I don't think I'm not seeing all the dns traffic. Example: via tcpdump with a BPF for just a client I get: 22:44:26.342201 IP 10.10.189.40.36221 > 10.10.189.225.53: 58059+ A? nike.com. (26) 22:44:26.412863 IP 10.10.189.225.53 > 10.10.189.40.36221: 58059 1/0/0 A 66.54.56.30 (42) That makes sense.. request, and reply. Yet in the dns.log I see 1378939466.342353 10.10.189.225 64592 10.85.30.71 53 udp 11033 nike.com 0 NOERROR F T T 66.54.56.30 1378939466.342201 10.10.189.40 36221 10.10.189.225 53 udp 58059 nike.com 0 NOERROR F T T 66.54.56.30 which shows the dns server talking to it's upstream server (expected) and then issues the answer to the client, but the original request isn't in the dns log. So assuming you get a response back from an upstream server, you can infer that the original requester was the second entry, but I was expecting to see an entry for the actual request to the 189.225 server. Or am I not understanding something right? I could probably look at the conn.log, but I am trying to just log the dns request, so I have conn.log turned off. From anthony.kasza at gmail.com Wed Sep 11 18:27:29 2013 From: anthony.kasza at gmail.com (anthony kasza) Date: Wed, 11 Sep 2013 18:27:29 -0700 Subject: [Bro] DNS query logging In-Reply-To: References: Message-ID: Are you using broctl or the bro binary? What scripts do you have loaded that affect DNS and DNS logging? Are you running against live traffic or a trace file? If on live traffic, I assume you're running Bro on the DNS server at 10.85.30.71, are inbound client queries+responses and outbound upstream queries+responses happening on the same interface? -AK On Wed, Sep 11, 2013 at 3:53 PM, Jeremy Hoel wrote: > So I'm testing out bro for a limited use on recording dns queries and > responses. I have the logs coming in and that's great, but I don't > think I'm not seeing all the dns traffic. > > Example: > > via tcpdump with a BPF for just a client I get: > > 22:44:26.342201 IP 10.10.189.40.36221 > 10.10.189.225.53: 58059+ A? > nike.com. (26) > 22:44:26.412863 IP 10.10.189.225.53 > 10.10.189.40.36221: 58059 1/0/0 > A 66.54.56.30 (42) > > That makes sense.. request, and reply. > > Yet in the dns.log I see > > 1378939466.342353 10.10.189.225 64592 10.85.30.71 53 udp 11033 > nike.com 0 NOERROR F T T 66.54.56.30 > 1378939466.342201 10.10.189.40 36221 10.10.189.225 53 udp 58059 > nike.com 0 NOERROR F T T 66.54.56.30 > > which shows the dns server talking to it's upstream server (expected) > and then issues the answer to the client, but the original request > isn't in the dns log. > > So assuming you get a response back from an upstream server, you can > infer that the original requester was the second entry, but I was > expecting to see an entry for the actual request to the 189.225 > server. > > Or am I not understanding something right? I could probably look at > the conn.log, but I am trying to just log the dns request, so I have > conn.log turned off. > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130911/8d0b04c7/attachment.html From anthony.kasza at gmail.com Wed Sep 11 18:28:20 2013 From: anthony.kasza at gmail.com (anthony kasza) Date: Wed, 11 Sep 2013 18:28:20 -0700 Subject: [Bro] DNS query logging In-Reply-To: References: Message-ID: *on the DNS server at 10.10.189.225 On Wed, Sep 11, 2013 at 6:27 PM, anthony kasza wrote: > Are you using broctl or the bro binary? > What scripts do you have loaded that affect DNS and DNS logging? > Are you running against live traffic or a trace file? If on live traffic, > I assume you're running Bro on the DNS server at 10.85.30.71, are inbound > client queries+responses and outbound upstream queries+responses happening > on the same interface? > > -AK > > > On Wed, Sep 11, 2013 at 3:53 PM, Jeremy Hoel wrote: > >> So I'm testing out bro for a limited use on recording dns queries and >> responses. I have the logs coming in and that's great, but I don't >> think I'm not seeing all the dns traffic. >> >> Example: >> >> via tcpdump with a BPF for just a client I get: >> >> 22:44:26.342201 IP 10.10.189.40.36221 > 10.10.189.225.53: 58059+ A? >> nike.com. (26) >> 22:44:26.412863 IP 10.10.189.225.53 > 10.10.189.40.36221: 58059 1/0/0 >> A 66.54.56.30 (42) >> >> That makes sense.. request, and reply. >> >> Yet in the dns.log I see >> >> 1378939466.342353 10.10.189.225 64592 10.85.30.71 53 udp 11033 >> nike.com 0 NOERROR F T T 66.54.56.30 >> 1378939466.342201 10.10.189.40 36221 10.10.189.225 53 udp 58059 >> nike.com 0 NOERROR F T T 66.54.56.30 >> >> which shows the dns server talking to it's upstream server (expected) >> and then issues the answer to the client, but the original request >> isn't in the dns log. >> >> So assuming you get a response back from an upstream server, you can >> infer that the original requester was the second entry, but I was >> expecting to see an entry for the actual request to the 189.225 >> server. >> >> Or am I not understanding something right? I could probably look at >> the conn.log, but I am trying to just log the dns request, so I have >> conn.log turned off. >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130911/853ec793/attachment.html From christianbuia at gmail.com Wed Sep 11 18:34:34 2013 From: christianbuia at gmail.com (Christian Buia) Date: Wed, 11 Sep 2013 21:34:34 -0400 Subject: [Bro] Log an Arbitrarily Long Collection In-Reply-To: References: Message-ID: Jon, Seth, Thank you both for the responses. I have a much better sense now for how to organize my scripts within the folder structure, and have found the relevant documentation on this subject. Jon, I took your advice and used header-names.bro as a template. However, it seems to be that header-names.bro in the policy folder has a couple of key logic flaws. The const type boolean types allow both client and server headers to be enumerated. However, if you look at the "http_header" handler, you can see that this function will return immediately for any event that is not "is_orig", in other words it will return for server responses with no work done. Furthermore, if you set the const boolean value for server responses to True (T), the logic in the event handler is such that you will just get the client header names populated in the server header names vector. I have attached a (lightly tested) modified and expanded version of this script called header-names-and-vals.bro. Happy to hear if I am mistaken in my understanding of the bro code, as this is the first time I write my own. Also, happy to hear feedback/best practices on how to escape commas in a vector listing (I just gsubbed them with c). Thanks for your attention. Regards, Christian On Wed, Sep 11, 2013 at 10:54 AM, Siwek, Jonathan Luke wrote: > > server_headers: vector of HeaderValue &log &optional; > > server_headers_count: count &default=0; > > > > I decided to use a vector because I want to keep track of header order. > Further down on the http_header event handler, I add each header to the > appropriate vector, indexed by the count field which increments. > > A minor point: you don't really have to store the count since it's > implicit in the vector if you use `|?|` to get the size of the container. > E.g. > > |c$http$server_headers| > > in this case would be equivalent to: > > c$http$server_headers_count > > > error in /usr/local/bro/share/bro/base/protocols/http/./main.bro, line > 83: &log applied to a type that cannot be logged (&log) > > > > So presumably Bro doesn't like the idea of generating a log entry that > includes a vector type (no less consisting of record members, I'm not even > sure what that would have looked like but was hoping to find out). > > A vector of an atomic type, say `vector of string`, should work. So an > option would be to store the header names and values in two different > vectors. > > > The next best thing I can think of doing is recording this information > with some custom delimeter in a single string field > > That could work also and might be better if you're more concerned w/ human > readability. > > > Also, I have a feeling that directly editing http/main.bro is a bad > practice. Should I instead be adding this script to the policy branch, > redefining the HTTP Info object and handling the http header event in there? > > Take a look at "policy/protocols/http/header-names.bro". Copy that to > your own script (possibly storing it in $prefix/share/bro/site) and modify > it to do what you want (log header name+value in whatever format). Then > have bro load that script: if you're already loading local.bro, adding an > @load in there for the new script is one way. > > - Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130911/5c932134/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: header-names-and-vals.bro Type: application/octet-stream Size: 1720 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130911/5c932134/attachment.obj From jthoel at gmail.com Thu Sep 12 08:07:07 2013 From: jthoel at gmail.com (Jeremy Hoel) Date: Thu, 12 Sep 2013 15:07:07 +0000 Subject: [Bro] DNS query logging In-Reply-To: References: Message-ID: I had only sent this to Liam.. not the list; and he responded, so that helped. So this is just an FYI. Also, I just noticed the timestamps in the entries.. >From the example, the client request (1378939466.342201) happens before the forwarding lookup (1378939466.342353) but gets written after (since it's waiting for the results. So, now that makes sense why it logs the way it does. Thanks! On Thu, Sep 12, 2013 at 2:28 AM, Jeremy Hoel wrote: > Liam - It's 2.1 from the stable tgz. I can test the git also. Is > there a rough ETA for 2.2? > > > Anthony - Running bro via broctl on the 189.225 server. This is off > live traffic, sniffing a span port that is from the 189.225 server. > This is a basic fresh install with nothing special except for turning > off the logs I don't want; so no extra scripts. The 10.85.30.71 is a > server that requests are forwarded to from the 189.225 server (a > windows dc) and yes, all traffic in and out is on one same interface. > > And I guess the other odd part is that in the log, the source of the > second log is the original requesting client. I guess I expected to > see this more like how tcpdump would show the output vs how the logs > are recording it. maybe the git will make more sense. I'll test it > out tomorrow. > > Thanks for the responses! > > On Wed, Sep 11, 2013 at 7:52 PM, Liam Randall wrote: >> Hey Jeremy, >> >> Are you on Bro 2.1 or SecurityOnion? >> >> Upgrade to Git Master; there was a bug fixed already: >> >> https://bro-tracker.atlassian.net/browse/BIT-1064 >> >> Thanks, >> >> Liam Randall >> >> >> On Wed, Sep 11, 2013 at 6:53 PM, Jeremy Hoel wrote: >>> >>> So I'm testing out bro for a limited use on recording dns queries and >>> responses. I have the logs coming in and that's great, but I don't >>> think I'm not seeing all the dns traffic. >>> >>> Example: >>> >>> via tcpdump with a BPF for just a client I get: >>> >>> 22:44:26.342201 IP 10.10.189.40.36221 > 10.10.189.225.53: 58059+ A? >>> nike.com. (26) >>> 22:44:26.412863 IP 10.10.189.225.53 > 10.10.189.40.36221: 58059 1/0/0 >>> A 66.54.56.30 (42) >>> >>> That makes sense.. request, and reply. >>> >>> Yet in the dns.log I see >>> >>> 1378939466.342353 10.10.189.225 64592 10.85.30.71 53 udp 11033 >>> nike.com 0 NOERROR F T T 66.54.56.30 >>> 1378939466.342201 10.10.189.40 36221 10.10.189.225 53 udp 58059 >>> nike.com 0 NOERROR F T T 66.54.56.30 >>> >>> which shows the dns server talking to it's upstream server (expected) >>> and then issues the answer to the client, but the original request >>> isn't in the dns log. >>> >>> So assuming you get a response back from an upstream server, you can >>> infer that the original requester was the second entry, but I was >>> expecting to see an entry for the actual request to the 189.225 >>> server. >>> >>> Or am I not understanding something right? I could probably look at >>> the conn.log, but I am trying to just log the dns request, so I have >>> conn.log turned off. >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> From jsiwek at illinois.edu Thu Sep 12 08:13:17 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Thu, 12 Sep 2013 15:13:17 +0000 Subject: [Bro] Log an Arbitrarily Long Collection In-Reply-To: References: Message-ID: > Jon, I took your advice and used header-names.bro as a template. However, it seems to be that header-names.bro in the policy folder has a couple of key logic flaws. The const type boolean types allow both client and server headers to be enumerated. However, if you look at the "http_header" handler, you can see that this function will return immediately for any event that is not "is_orig", in other words it will return for server responses with no work done. Furthermore, if you set the const boolean value for server responses to True (T), the logic in the event handler is such that you will just get the client header names populated in the server header names vector. I think your read is correct. The tracking of server headers was probably something that got added in as an after-thought and never tested since that script isn't loaded by default anywhere. Thanks for pointing that out. > I have attached a (lightly tested) modified and expanded version of this script called header-names-and-vals.bro. Your version looks good, though the initial check for c?$http being set is still a nice thing to leave in. > Also, happy to hear feedback/best practices on how to escape commas in a vector listing (I just gsubbed them with c). I think they should automatically get escaped when appearing within a container value and you shouldn't have to worry about it. Did you see differently? - Jon From christianbuia at gmail.com Thu Sep 12 17:36:05 2013 From: christianbuia at gmail.com (Christian Buia) Date: Thu, 12 Sep 2013 20:36:05 -0400 Subject: [Bro] Log an Arbitrarily Long Collection In-Reply-To: References: Message-ID: Thanks for your response Jon. You're right, commas are already escaped as "\x2c", so gsub is not needed. Thank you for the heads up! Regards, Christian On Thu, Sep 12, 2013 at 11:13 AM, Siwek, Jonathan Luke wrote: > > > Jon, I took your advice and used header-names.bro as a template. > However, it seems to be that header-names.bro in the policy folder has a > couple of key logic flaws. The const type boolean types allow both client > and server headers to be enumerated. However, if you look at the > "http_header" handler, you can see that this function will return > immediately for any event that is not "is_orig", in other words it will > return for server responses with no work done. Furthermore, if you set the > const boolean value for server responses to True (T), the logic in the > event handler is such that you will just get the client header names > populated in the server header names vector. > > I think your read is correct. The tracking of server headers was probably > something that got added in as an after-thought and never tested since that > script isn't loaded by default anywhere. Thanks for pointing that out. > > > I have attached a (lightly tested) modified and expanded version of this > script called header-names-and-vals.bro. > > Your version looks good, though the initial check for c?$http being set is > still a nice thing to leave in. > > > Also, happy to hear feedback/best practices on how to escape commas in a > vector listing (I just gsubbed them with c). > > I think they should automatically get escaped when appearing within a > container value and you shouldn't have to worry about it. Did you see > differently? > > - Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130912/5f77d5fa/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: header-names-and-vals.bro Type: application/octet-stream Size: 1618 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130912/5f77d5fa/attachment.obj From sconzo at visiblerisk.com Fri Sep 13 06:58:08 2013 From: sconzo at visiblerisk.com (Mike Sconzo) Date: Fri, 13 Sep 2013 08:58:08 -0500 Subject: [Bro] rescan_state()? Message-ID: I've been trying to play with persistent variables with the following: module PTEST; global a = 4 &persistent; event bro_init() { if ( rescan_state() ) print "Success!"; print a; } event bro_done() { if ( checkpoint_state() ) print "Success!"; } event connection_state_remove(c: connection) { a = 5; } but the "print a" always prints "4" (regardless of how many times I run it), what am I missing? Any thoughts? Bro is being run with bro -C -r test.pcap persistent.bro (from the same directory each time, and I can see the .state/state.bst file being created. Thanks. -- cat ~/.bash_history > documentation.txt -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130913/b9f5c123/attachment.html From benson.mathews at gmail.com Fri Sep 13 08:28:53 2013 From: benson.mathews at gmail.com (Benson Mathews) Date: Fri, 13 Sep 2013 11:28:53 -0400 Subject: [Bro] SSH heuristic Message-ID: I had a question about the SSH analyzer and how it determines a successful connection. I have a bro notification for a successful SSH login to a system on my network for a connection originating from outside US. Log entry: conn.log 2013-09-01 10:04:31 KIDipWlFWSi y.y.y.y 40014 x.x.x.x 22 tcp ssh 147.140508 1160 2377 S3 F 0 ShAadDf 21 2664 48 8221 ssh.log 2013-09-01 10:04:33 KIDipWlFWSi y.y.y.y 40014 x.x.x.x 22 success INBOUND SSH-2.0-libssh-0.2 SSH-2.0-OpenSSH_5.9 5725 CN - - - - The log entry is dated on the 1st and my client side logs have rolled over, so I can't valid this with syslog on the client. I wanted to check if there's a chance this could be a false positive. Also, the switch providing the feed itself is dropping packets and I see alerts for CaptureLoss::Too_Much_Loss and PacketFilter::Dropped_Packets. Would this loss cause Bro to misinterpret a brute forcing/scan attempt to a successful login? I'm running Bro 2.0. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130913/c295a460/attachment.html From soehlert at illinois.edu Fri Sep 13 09:42:46 2013 From: soehlert at illinois.edu (Oehlert, Samuel) Date: Fri, 13 Sep 2013 11:42:46 -0500 Subject: [Bro] SSH heuristic In-Reply-To: References: Message-ID: <52334086.2000705@illinois.edu> For the first part, that "5725" is the number of response bytes. The threshold for what is considered successful is: const authentication_data_size, which can be found in: /bro/share/bro/base/protocols/ssh and I believe it's default value was either 5000 or 5500 on 2.0. Though if you want to redef this you'll want to do it elsewhere such as your local.bro. Seth spent lots of time figuring out the best threshold, but it is still only a guess based on bytes, so there will be false positives. We find that many of them just around 5000 bytes were false positives. I don't know about your second part off the top of my head. -Sam On 9/13/13 10:28 AM, Benson Mathews wrote: > I had a question about the SSH analyzer and how it determines a > successful connection. I have a bro notification for a successful SSH > login to a system on my network for a connection originating from > outside US. Log entry: > conn.log 2013-09-01 10:04:31 KIDipWlFWSi y.y.y.y 40014 > x.x.x.x 22 tcp ssh 147.140508 1160 2377 > S3 F 0 ShAadDf 21 2664 48 8221 > ssh.log 2013-09-01 10:04:33 KIDipWlFWSi y.y.y.y 40014 > x.x.x.x 22 success INBOUND SSH-2.0-libssh-0.2 > SSH-2.0-OpenSSH_5.9 5725 CN - - - - > > The log entry is dated on the 1st and my client side logs have rolled > over, so I can't valid this with syslog on the client. > > I wanted to check if there's a chance this could be a false positive. > Also, the switch providing the feed itself is dropping packets and I > see alerts for CaptureLoss::Too_Much_Loss and > PacketFilter::Dropped_Packets. > > Would this loss cause Bro to misinterpret a brute forcing/scan attempt > to a successful login? > > I'm running Bro 2.0. > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Sam Oehlert Security Engineer NCSA soehlert at illinois.edu (217)300-1076 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130913/1e93db0f/attachment.html From alexwis at gmail.com Fri Sep 13 10:47:10 2013 From: alexwis at gmail.com (Alex Waher) Date: Fri, 13 Sep 2013 10:47:10 -0700 Subject: [Bro] SSH heuristic In-Reply-To: <52334086.2000705@illinois.edu> References: <52334086.2000705@illinois.edu> Message-ID: Right, if the ssh connection's response bytes are above this threshold it'll log the session as having a successful login. If you have a large pre-auth login banner (usually for compliance reasons) it will very likely report as a false positive. On Fri, Sep 13, 2013 at 9:42 AM, Oehlert, Samuel wrote: > For the first part, that "5725" is the number of response bytes. The > threshold for what is considered successful is: const > authentication_data_size, which can be found in: > /bro/share/bro/base/protocols/ssh and I believe it's default value was > either 5000 or 5500 on 2.0. Though if you want to redef this you'll want to > do it elsewhere such as your local.bro. Seth spent lots of time figuring > out the best threshold, but it is still only a guess based on bytes, so > there will be false positives. We find that many of them just around 5000 > bytes were false positives. > > I don't know about your second part off the top of my head. > > -Sam > > > > > On 9/13/13 10:28 AM, Benson Mathews wrote: > > I had a question about the SSH analyzer and how it determines a successful > connection. I have a bro notification for a successful SSH login to a > system on my network for a connection originating from outside US. Log > entry: > conn.log 2013-09-01 10:04:31 KIDipWlFWSi y.y.y.y 40014 > x.x.x.x 22 tcp ssh 147.140508 1160 2377 S3 > F 0 ShAadDf 21 2664 48 8221 > ssh.log 2013-09-01 10:04:33 KIDipWlFWSi y.y.y.y 40014 > x.x.x.x 22 success INBOUND SSH-2.0-libssh-0.2 > SSH-2.0-OpenSSH_5.9 5725 CN - - - - > > The log entry is dated on the 1st and my client side logs have rolled > over, so I can't valid this with syslog on the client. > > I wanted to check if there's a chance this could be a false positive. > Also, the switch providing the feed itself is dropping packets and I see > alerts for CaptureLoss::Too_Much_Loss and PacketFilter::Dropped_Packets. > > Would this loss cause Bro to misinterpret a brute forcing/scan attempt to > a successful login? > > I'm running Bro 2.0. > > > _______________________________________________ > Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > -- > Sam Oehlert > Security Engineer > NCSAsoehlert at illinois.edu(217)300-1076 > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130913/0f9773b3/attachment.html From jsiwek at illinois.edu Fri Sep 13 10:48:30 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Fri, 13 Sep 2013 17:48:30 +0000 Subject: [Bro] rescan_state()? In-Reply-To: References: Message-ID: > but the "print a" always prints "4" (regardless of how many times I run it), what am I missing? Any thoughts? Not entirely sure, but an observation is that it does seem to work when handling "new_connection" and changing the value there instead. So maybe "connection_state_remove" is being raised as a result of reaching the end of the pcap and persistent state tracking/changes doesn't work as well when Bro is in the process of shutting down. - Jon From jlay at slave-tothe-box.net Fri Sep 13 11:12:37 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 13 Sep 2013 12:12:37 -0600 Subject: [Bro] HTTP not being seen In-Reply-To: <52334086.2000705@illinois.edu> References: <52334086.2000705@illinois.edu> Message-ID: Hey all, Topic says it...it's very strange..new install on a different box..I see the below: [12:08:56 gateway:~/current$] ls -l total 64 -rw-r--r-- 1 root root 3914 Sep 13 12:08 communication.log -rw-r--r-- 1 root root 4082 Sep 13 12:08 conn.log -rw-r--r-- 1 root root 12521 Sep 13 12:08 dns.log -rw-r--r-- 1 root root 396 Sep 13 12:08 dpd.log -rw-r--r-- 1 root root 8691 Sep 13 12:05 loaded_scripts.log -rw-r--r-- 1 root root 1101 Sep 13 12:05 notice_policy.log -rw-r--r-- 1 root root 224 Sep 13 12:05 packet_filter.log -rw-r--r-- 1 root root 699 Sep 13 12:08 ssl.log -rw-r--r-- 1 root root 46 Sep 13 12:05 stderr.log -rw-r--r-- 1 root root 30 Sep 13 12:05 stdout.log -rw-r--r-- 1 root root 717 Sep 13 12:07 weird.log I even see: 2013-09-13T12:05:46-0600 8GxbB0zXe0g x.x.x.x 53547 74.125.129.99 80 tcp - - - - OTH F 0 C 0 0 0 0 (empty) 2013-09-13T12:05:46-0600 HQym3XmcURj x.x.x.x 36086 205.171.2.25 53 udp 59556 www.google.com 1 C_INTERNET 1 A 0 NOERROR F F T T 0 74.125.129.99,74.125.129.104,74.125.129.105,74.125.129.147,74.125.129.103,74.125.129.106 297.000000,297.000000,297.000000,297.000000,297.000000,297.000000 loaded_scripts.log shows: [12:10:26 gateway:~/current$] grep http loaded_scripts.log /usr/local/bro/share/bro/base/protocols/http/__load__.bro /usr/local/bro/share/bro/base/protocols/http/./main.bro /usr/local/bro/share/bro/base/protocols/http/./utils.bro /usr/local/bro/share/bro/base/protocols/http/./file-ident.bro /usr/local/bro/share/bro/base/protocols/http/./file-hash.bro /usr/local/bro/share/bro/base/protocols/http/./file-extract.bro /usr/local/bro/share/bro/policy/protocols/http/software.bro /usr/local/bro/share/bro/policy/protocols/http/detect-MHR.bro But http.log is still not created. Anything I'm missing here or something I can do to troubleshoot on this end? This is running on ppp0. Thank you. James From jlay at slave-tothe-box.net Fri Sep 13 11:33:50 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 13 Sep 2013 12:33:50 -0600 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> Message-ID: <394689255bdaceba8e2deae69a1fb942@localhost> On 2013-09-13 12:12, James Lay wrote: > Hey all, > > Topic says it...it's very strange..new install on a different box..I > see the below: > > [12:08:56 gateway:~/current$] ls -l > total 64 > -rw-r--r-- 1 root root 3914 Sep 13 12:08 communication.log > -rw-r--r-- 1 root root 4082 Sep 13 12:08 conn.log > -rw-r--r-- 1 root root 12521 Sep 13 12:08 dns.log > -rw-r--r-- 1 root root 396 Sep 13 12:08 dpd.log > -rw-r--r-- 1 root root 8691 Sep 13 12:05 loaded_scripts.log > -rw-r--r-- 1 root root 1101 Sep 13 12:05 notice_policy.log > -rw-r--r-- 1 root root 224 Sep 13 12:05 packet_filter.log > -rw-r--r-- 1 root root 699 Sep 13 12:08 ssl.log > -rw-r--r-- 1 root root 46 Sep 13 12:05 stderr.log > -rw-r--r-- 1 root root 30 Sep 13 12:05 stdout.log > -rw-r--r-- 1 root root 717 Sep 13 12:07 weird.log > > I even see: > > 2013-09-13T12:05:46-0600 8GxbB0zXe0g x.x.x.x 53547 > 74.125.129.99 80 tcp - - - - OTH > F 0 C 0 0 0 0 (empty) > > 2013-09-13T12:05:46-0600 HQym3XmcURj x.x.x.x 36086 > 205.171.2.25 53 udp 59556 www.google.com 1 > C_INTERNET 1 A 0 NOERROR F F T > T 0 > > 74.125.129.99,74.125.129.104,74.125.129.105,74.125.129.147,74.125.129.103,74.125.129.106 > > 297.000000,297.000000,297.000000,297.000000,297.000000,297.000000 > > loaded_scripts.log shows: > > [12:10:26 gateway:~/current$] grep http loaded_scripts.log > /usr/local/bro/share/bro/base/protocols/http/__load__.bro > /usr/local/bro/share/bro/base/protocols/http/./main.bro > /usr/local/bro/share/bro/base/protocols/http/./utils.bro > /usr/local/bro/share/bro/base/protocols/http/./file-ident.bro > /usr/local/bro/share/bro/base/protocols/http/./file-hash.bro > /usr/local/bro/share/bro/base/protocols/http/./file-extract.bro > /usr/local/bro/share/bro/policy/protocols/http/software.bro > /usr/local/bro/share/bro/policy/protocols/http/detect-MHR.bro > > But http.log is still not created. Anything I'm missing here or > something I can do to troubleshoot on this end? This is running on > ppp0. Thank you. Well...it IS being created...however, certain things don't seem to be logged. Example: [12:27:34 gateway:~$] wget www.amazon.com --2013-09-13 12:30:59-- http://www.amazon.com/ Resolving www.amazon.com (www.amazon.com)... 72.21.194.212 Connecting to www.amazon.com (www.amazon.com)|72.21.194.212|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: `index.html.1' [ <=> ] 233,456 271K/s in 0.8s 2013-09-13 12:31:01 (271 KB/s) - `index.html' saved [233456] And bro shows the below: 2013-09-13T12:30:59-0600 51V4hVkz1le x.x.x.x 36471 72.21.194.212 80 tcp - - - - OTH F 0 C 0 0 0 0 (empty) 2013-09-13T12:30:59-0600 gSRjffzGzGj x.x.x.x 52571 205.171.2.25 53 udp 311 www.amazon.com 1 C_INTERNET 1 A 0 NOERROR F F T T 0 72.21.194.212 34.000000 2013-09-13T12:31:00-0600 51V4hVkz1le x.x.x.x 36471 72.21.194.212 80 active_connection_reuse - F bro Not sure why it's telling me it's an active connection reuse instead of logging the http traffic. James From doug.burks at gmail.com Fri Sep 13 11:45:47 2013 From: doug.burks at gmail.com (Doug Burks) Date: Fri, 13 Sep 2013 14:45:47 -0400 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> Message-ID: Hi James, Is it possible you're seeing the effects of NIC offloading features? http://securityonion.blogspot.com/2011/10/when-is-full-packet-capture-not-full.html On Fri, Sep 13, 2013 at 2:12 PM, James Lay wrote: > Hey all, > > Topic says it...it's very strange..new install on a different box..I > see the below: > > [12:08:56 gateway:~/current$] ls -l > total 64 > -rw-r--r-- 1 root root 3914 Sep 13 12:08 communication.log > -rw-r--r-- 1 root root 4082 Sep 13 12:08 conn.log > -rw-r--r-- 1 root root 12521 Sep 13 12:08 dns.log > -rw-r--r-- 1 root root 396 Sep 13 12:08 dpd.log > -rw-r--r-- 1 root root 8691 Sep 13 12:05 loaded_scripts.log > -rw-r--r-- 1 root root 1101 Sep 13 12:05 notice_policy.log > -rw-r--r-- 1 root root 224 Sep 13 12:05 packet_filter.log > -rw-r--r-- 1 root root 699 Sep 13 12:08 ssl.log > -rw-r--r-- 1 root root 46 Sep 13 12:05 stderr.log > -rw-r--r-- 1 root root 30 Sep 13 12:05 stdout.log > -rw-r--r-- 1 root root 717 Sep 13 12:07 weird.log > > I even see: > > 2013-09-13T12:05:46-0600 8GxbB0zXe0g x.x.x.x 53547 > 74.125.129.99 80 tcp - - - - OTH > F 0 C 0 0 0 0 (empty) > > 2013-09-13T12:05:46-0600 HQym3XmcURj x.x.x.x 36086 > 205.171.2.25 53 udp 59556 www.google.com 1 > C_INTERNET 1 A 0 NOERROR F F T > T 0 > 74.125.129.99,74.125.129.104,74.125.129.105,74.125.129.147,74.125.129.103,74.125.129.106 > 297.000000,297.000000,297.000000,297.000000,297.000000,297.000000 > > loaded_scripts.log shows: > > [12:10:26 gateway:~/current$] grep http loaded_scripts.log > /usr/local/bro/share/bro/base/protocols/http/__load__.bro > /usr/local/bro/share/bro/base/protocols/http/./main.bro > /usr/local/bro/share/bro/base/protocols/http/./utils.bro > /usr/local/bro/share/bro/base/protocols/http/./file-ident.bro > /usr/local/bro/share/bro/base/protocols/http/./file-hash.bro > /usr/local/bro/share/bro/base/protocols/http/./file-extract.bro > /usr/local/bro/share/bro/policy/protocols/http/software.bro > /usr/local/bro/share/bro/policy/protocols/http/detect-MHR.bro > > But http.log is still not created. Anything I'm missing here or > something I can do to troubleshoot on this end? This is running on > ppp0. Thank you. > > James > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Doug Burks http://securityonion.blogspot.com From liam at broala.com Fri Sep 13 12:31:18 2013 From: liam at broala.com (Liam Randall) Date: Fri, 13 Sep 2013 15:31:18 -0400 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> Message-ID: Let's enable your capture loss and see what happens: Add the following to your local.bro; on security onion it will be located at /opt/bro/share/bro/site/ # count the ACKs, tell me the # and % I am missing @load misc/capture-loss.bro # By default capture-loss reports every 15 minutes, let's turn it up redef CaptureLoss::watch_interval = 1 min; Give it a couple of minutes and see what the log says under: /nsm/bro/logs/capture_loss.log You will see per worker statistics written every minute. Let us know. Thanks, Liam Randall On Fri, Sep 13, 2013 at 2:45 PM, Doug Burks wrote: > Hi James, > > Is it possible you're seeing the effects of NIC offloading features? > > http://securityonion.blogspot.com/2011/10/when-is-full-packet-capture-not-full.html > > On Fri, Sep 13, 2013 at 2:12 PM, James Lay > wrote: > > Hey all, > > > > Topic says it...it's very strange..new install on a different box..I > > see the below: > > > > [12:08:56 gateway:~/current$] ls -l > > total 64 > > -rw-r--r-- 1 root root 3914 Sep 13 12:08 communication.log > > -rw-r--r-- 1 root root 4082 Sep 13 12:08 conn.log > > -rw-r--r-- 1 root root 12521 Sep 13 12:08 dns.log > > -rw-r--r-- 1 root root 396 Sep 13 12:08 dpd.log > > -rw-r--r-- 1 root root 8691 Sep 13 12:05 loaded_scripts.log > > -rw-r--r-- 1 root root 1101 Sep 13 12:05 notice_policy.log > > -rw-r--r-- 1 root root 224 Sep 13 12:05 packet_filter.log > > -rw-r--r-- 1 root root 699 Sep 13 12:08 ssl.log > > -rw-r--r-- 1 root root 46 Sep 13 12:05 stderr.log > > -rw-r--r-- 1 root root 30 Sep 13 12:05 stdout.log > > -rw-r--r-- 1 root root 717 Sep 13 12:07 weird.log > > > > I even see: > > > > 2013-09-13T12:05:46-0600 8GxbB0zXe0g x.x.x.x 53547 > > 74.125.129.99 80 tcp - - - - OTH > > F 0 C 0 0 0 0 (empty) > > > > 2013-09-13T12:05:46-0600 HQym3XmcURj x.x.x.x 36086 > > 205.171.2.25 53 udp 59556 www.google.com 1 > > C_INTERNET 1 A 0 NOERROR F F T > > T 0 > > > 74.125.129.99,74.125.129.104,74.125.129.105,74.125.129.147,74.125.129.103,74.125.129.106 > > 297.000000,297.000000,297.000000,297.000000,297.000000,297.000000 > > > > loaded_scripts.log shows: > > > > [12:10:26 gateway:~/current$] grep http loaded_scripts.log > > /usr/local/bro/share/bro/base/protocols/http/__load__.bro > > /usr/local/bro/share/bro/base/protocols/http/./main.bro > > /usr/local/bro/share/bro/base/protocols/http/./utils.bro > > /usr/local/bro/share/bro/base/protocols/http/./file-ident.bro > > /usr/local/bro/share/bro/base/protocols/http/./file-hash.bro > > /usr/local/bro/share/bro/base/protocols/http/./file-extract.bro > > /usr/local/bro/share/bro/policy/protocols/http/software.bro > > /usr/local/bro/share/bro/policy/protocols/http/detect-MHR.bro > > > > But http.log is still not created. Anything I'm missing here or > > something I can do to troubleshoot on this end? This is running on > > ppp0. Thank you. > > > > James > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > -- > Doug Burks > http://securityonion.blogspot.com > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Liam Randall Managing Partner 510-281-0760 www.Broala.com >From the creators of Bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130913/55d828ca/attachment.html From jlay at slave-tothe-box.net Fri Sep 13 12:50:12 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 13 Sep 2013 13:50:12 -0600 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> Message-ID: On 2013-09-13 13:31, Liam Randall wrote: > Lets enable your capture loss and see what happens: > > Add the following to your local.bro; on security onion it will be > located at /opt/bro/share/bro/site/ > > # count the ACKs, tell me the # and % I am missing > @load misc/capture-loss.bro > > # By default capture-loss reports every 15 minutes, lets turn it up > redef CaptureLoss::watch_interval = 1 min; > > Give it a couple of minutes and see what the log says under: > > /nsm/bro/logs/capture_loss.log > > You will see per worker statistics written every minute. > > Let us know. > > Thanks, > > Liam Randall Thanks Liam...here's what I got: #fields ts ts_delta peer gaps acks percent_lost #types time interval string count count string 2013-09-13T13:34:59-0600 60.000084 bro 0 16 0.000% 2013-09-13T13:35:59-0600 60.000044 bro 0 0 0.000% 2013-09-13T13:36:59-0600 60.000048 bro 0 14 0.000% 2013-09-13T13:37:59-0600 60.000048 bro 1 1 100.000% 2013-09-13T13:38:59-0600 60.000038 bro 0 0 0.000% 2013-09-13T13:39:59-0600 60.000050 bro 0 0 0.000% 2013-09-13T13:40:59-0600 60.000093 bro 0 0 0.000% 2013-09-13T13:41:59-0600 60.000023 bro 0 0 0.000% 2013-09-13T13:42:59-0600 60.000022 bro 0 0 0.000% 2013-09-13T13:43:59-0600 60.000023 bro 0 0 0.000% 2013-09-13T13:44:59-0600 60.000089 bro 0 0 0.000% 2013-09-13T13:45:59-0600 60.000073 bro 0 0 0.000% 2013-09-13T13:46:59-0600 60.000011 bro 0 0 0.000% Bro entries: 2013-09-13T13:45:33-0600 dPCMEyJBiU7 x.x.x.x 47285 50.18.192.250 80 tcp - - - - OTH - 0 C 0 0 0 0 (empty) 2013-09-13T13:45:33-0600 PFUbImVSSZ2 x.x.x.x 35306 205.171.2.25 53 udp 63725 www.duckduckgo.com 1 C_INTERNET 1 A 0 NOERROR F F T T 0 duckduckgo.com,50.18.192.250,50.18.192.251 900.000000,25.000000,25.000000 2013-09-13T13:45:34-0600 8ZduhgTSjm6 x.x.x.x 37025 205.171.2.25 53 udp 35309 duckduckgo.com 1 C_INTERNET 1 A 0 NOERROR F F T T 0 50.18.192.251,50.18.192.250 24.000000,24.000000 2013-09-13T13:45:34-0600 dPCMEyJBiU7 x.x.x.x 47285 50.18.192.250 80 active_connection_reuse - F bro Wget info: [13:45:20 gateway:~$] wget www.duckduckgo.com --2013-09-13 13:45:33-- http://www.duckduckgo.com/ Resolving www.duckduckgo.com (www.duckduckgo.com)... 50.18.192.250, 50.18.192.251 Connecting to www.duckduckgo.com (www.duckduckgo.com)|50.18.192.250|:80... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://duckduckgo.com/ [following] --2013-09-13 13:45:34-- https://duckduckgo.com/ Resolving duckduckgo.com (duckduckgo.com)... 50.18.192.251, 50.18.192.250 Connecting to duckduckgo.com (duckduckgo.com)|50.18.192.251|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 8646 (8.4K) [text/html] Saving to: `index.html' 100%[=========================>] 8,646 --.-K/s in 0s 2013-09-13 13:45:34 (86.8 MB/s) - `index.html' saved [8646/8646] Tshark info: 2013-09-13 13:45:33.991079 x.x.x.x -> 50.18.192.250 TCP 76 47285 > 80 [SYN] Seq=0 Win=14520 Len=0 MSS=1452 SACK_PERM=1 TSval=147157135 TSecr=0 WS=16 2013-09-13 13:45:34.035256 50.18.192.250 -> x.x.x.x TCP 76 80 > 47285 [SYN, ACK] Seq=0 Ack=1 Win=7240 Len=0 MSS=1460 SACK_PERM=1 TSval=23107600 TSecr=147157135 WS=1 2013-09-13 13:45:34.035375 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 [ACK] Seq=1 Ack=1 Win=14528 Len=0 TSval=147157146 TSecr=23107600 2013-09-13 13:45:34.035595 x.x.x.x -> 50.18.192.250 HTTP 186 GET / HTTP/1.1 2013-09-13 13:45:34.082121 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 [ACK] Seq=1 Ack=119 Win=7122 Len=0 TSval=23107612 TSecr=147157146 2013-09-13 13:45:34.082132 50.18.192.250 -> x.x.x.x HTTP 503 HTTP/1.1 301 Moved Permanently (text/html) 2013-09-13 13:45:34.082241 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 [ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157158 TSecr=23107612 2013-09-13 13:45:34.446981 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 [FIN, ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157249 TSecr=23107612 2013-09-13 13:45:34.492112 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 [FIN, ACK] Seq=436 Ack=120 Win=7121 Len=0 TSval=23107714 TSecr=147157249 2013-09-13 13:45:34.492164 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 [ACK] Seq=120 Ack=437 Win=15600 Len=0 TSval=147157260 TSecr=23107714 Not sure what to think...it's very strange. Thanks again. James From sconzo at visiblerisk.com Fri Sep 13 13:12:23 2013 From: sconzo at visiblerisk.com (Mike Sconzo) Date: Fri, 13 Sep 2013 15:12:23 -0500 Subject: [Bro] rescan_state()? In-Reply-To: References: Message-ID: Thanks for the feedback! I found this: "Bro generates this event reliably once for every connection when it is about to delete the internal state. As such, the event is well-suited for script-level cleanup that needs to be performed for every connection." The oddities seem to exist around UDP, I split it up into connection_finished and udp_session_done. The TCP stuff seems to always work/get persisted and the UDP stuff never does. The expanded script (for anybody interested): module DF; # IP -> DHH -> proto -> [0] = number of connections [1] = bytes transferred global ip_to_connection_info: table[addr] of table[int] of table[string] of vector of count &persistent &synchronized; event bro_init() { if ( rescan_state() ) print "Success!"; print ip_to_connection_info; print "------"; } event bro_done() { if ( checkpoint_state() ) print "Success!"; print ip_to_connection_info; } #event connection_state_remove(c: connection) event connection_finished(c: connection) { local net_time = network_time(); local date = to_int(strftime("%w%H", net_time)); #local proto = fmt("%s", c$conn$proto); local proto = "tcp"; local ip = c$id$orig_h; if ( ip !in ip_to_connection_info ) { ip_to_connection_info[ip] = table(); } if ( ip in ip_to_connection_info && date !in ip_to_connection_info[ip] ) { ip_to_connection_info[ip][date] = table(); } if ( ip in ip_to_connection_info && date in ip_to_connection_info[ip] && proto !in ip_to_connection_info[ip][date] ) { ip_to_connection_info[ip][date][proto] = vector(); ip_to_connection_info[ip][date][proto][0] = 1; ip_to_connection_info[ip][date][proto][1] = c$orig$num_bytes_ip; } else { ip_to_connection_info[ip][date][proto][0] = ip_to_connection_info[ip][date][proto][0] + 1; ip_to_connection_info[ip][date][proto][1] = ip_to_connection_info[ip][date][proto][1] + c$orig$num_bytes_ip; } print fmt("ips[%s][%d][%s][connections] = %d", ip, date, proto, ip_to_connection_info[ip][date][proto][0]); print fmt("ips[%s][%d][%s][total_bytes] = %d", ip, date, proto, ip_to_connection_info[ip][date][proto][1]); print "-------"; } event udp_session_done(u: connection) { local net_time = network_time(); local date = to_int(strftime("%w%H", net_time)); local proto = "udp"; local ip = u$id$orig_h; if ( ip !in ip_to_connection_info ) { ip_to_connection_info[ip] = table(); } if ( ip in ip_to_connection_info && date !in ip_to_connection_info[ip] ) { ip_to_connection_info[ip][date] = table(); } if ( ip in ip_to_connection_info && date in ip_to_connection_info[ip] && proto !in ip_to_connection_info[ip][date] ) { ip_to_connection_info[ip][date][proto] = vector(); ip_to_connection_info[ip][date][proto][0] = 1; ip_to_connection_info[ip][date][proto][1] = u$orig$num_bytes_ip; } else { ip_to_connection_info[ip][date][proto][0] = ip_to_connection_info[ip][date][proto][0] + 1; ip_to_connection_info[ip][date][proto][1] = ip_to_connection_info[ip][date][proto][1] + u$orig$num_bytes_ip; } print fmt("ips[%s][%d][%s][connections] = %d", ip, date, proto, ip_to_connection_info[ip][date][proto][0]); print fmt("ips[%s][%d][%s][total_bytes] = %d", ip, date, proto, ip_to_connection_info[ip][date][proto][1]); print "-------"; } On Fri, Sep 13, 2013 at 12:48 PM, Siwek, Jonathan Luke wrote: >> but the "print a" always prints "4" (regardless of how many times I run it), what am I missing? Any thoughts? > > Not entirely sure, but an observation is that it does seem to work when handling "new_connection" and changing the value there instead. So maybe "connection_state_remove" is being raised as a result of reaching the end of the pcap and persistent state tracking/changes doesn't work as well when Bro is in the process of shutting down. > > - Jon -- cat ~/.bash_history > documentation.txt From liam at broala.com Fri Sep 13 14:23:32 2013 From: liam at broala.com (Liam Randall) Date: Fri, 13 Sep 2013 17:23:32 -0400 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> Message-ID: How is the data getting to bro? Span port? Tap? On Fri, Sep 13, 2013 at 3:50 PM, James Lay wrote: > On 2013-09-13 13:31, Liam Randall wrote: > >> Lets enable your capture loss and see what happens: >> >> >> Add the following to your local.bro; on security onion it will be >> located at /opt/bro/share/bro/site/ >> >> # count the ACKs, tell me the # and % I am missing >> @load misc/capture-loss.bro >> >> # By default capture-loss reports every 15 minutes, lets turn it up >> >> redef CaptureLoss::watch_interval = 1 min; >> >> Give it a couple of minutes and see what the log says under: >> >> /nsm/bro/logs/capture_loss.log >> >> You will see per worker statistics written every minute. >> >> Let us know. >> >> Thanks, >> >> Liam Randall >> > > > Thanks Liam...here's what I got: > #fields ts ts_delta peer gaps acks percent_lost > #types time interval string count count string > 2013-09-13T13:34:59-0600 60.000084 bro 0 16 > 0.000% > 2013-09-13T13:35:59-0600 60.000044 bro 0 0 > 0.000% > 2013-09-13T13:36:59-0600 60.000048 bro 0 14 > 0.000% > 2013-09-13T13:37:59-0600 60.000048 bro 1 1 > 100.000% > 2013-09-13T13:38:59-0600 60.000038 bro 0 0 > 0.000% > 2013-09-13T13:39:59-0600 60.000050 bro 0 0 > 0.000% > 2013-09-13T13:40:59-0600 60.000093 bro 0 0 > 0.000% > 2013-09-13T13:41:59-0600 60.000023 bro 0 0 > 0.000% > 2013-09-13T13:42:59-0600 60.000022 bro 0 0 > 0.000% > 2013-09-13T13:43:59-0600 60.000023 bro 0 0 > 0.000% > 2013-09-13T13:44:59-0600 60.000089 bro 0 0 > 0.000% > 2013-09-13T13:45:59-0600 60.000073 bro 0 0 > 0.000% > 2013-09-13T13:46:59-0600 60.000011 bro 0 0 > 0.000% > > Bro entries: > 2013-09-13T13:45:33-0600 dPCMEyJBiU7 x.x.x.x 47285 > 50.18.192.250 80 tcp - - - - OTH - > 0 C 0 0 0 0 (empty) > 2013-09-13T13:45:33-0600 PFUbImVSSZ2 x.x.x.x 35306 > 205.171.2.25 53 udp 63725 www.duckduckgo.com 1 > C_INTERNET 1 A 0 NOERROR F F T T > 0 duckduckgo.com,50.18.192.250,50.18.192.251 > 900.000000,25.000000,25.000000 > 2013-09-13T13:45:34-0600 8ZduhgTSjm6 x.x.x.x 37025 > 205.171.2.25 53 udp 35309 duckduckgo.com 1 > C_INTERNET 1 A 0 NOERROR F F T T > 0 50.18.192.251,50.18.192.250 24.000000,24.000000 > 2013-09-13T13:45:34-0600 dPCMEyJBiU7 x.x.x.x 47285 > 50.18.192.250 80 active_connection_reuse - F bro > > > Wget info: > [13:45:20 gateway:~$] wget www.duckduckgo.com > --2013-09-13 13:45:33-- http://www.duckduckgo.com/ > Resolving www.duckduckgo.com (www.duckduckgo.com)... 50.18.192.250, > 50.18.192.251 > Connecting to www.duckduckgo.com (www.duckduckgo.com)|50.18.192.250|:80... > connected. > HTTP request sent, awaiting response... 301 Moved Permanently > Location: https://duckduckgo.com/ [following] > --2013-09-13 13:45:34-- https://duckduckgo.com/ > Resolving duckduckgo.com (duckduckgo.com)... 50.18.192.251, 50.18.192.250 > Connecting to duckduckgo.com (duckduckgo.com)|50.18.192.251|:443... > connected. > > HTTP request sent, awaiting response... 200 OK > Length: 8646 (8.4K) [text/html] > Saving to: `index.html' > > 100%[=========================**>] 8,646 --.-K/s in 0s > > 2013-09-13 13:45:34 (86.8 MB/s) - `index.html' saved [8646/8646] > > > Tshark info: > 2013-09-13 13:45:33.991079 x.x.x.x -> 50.18.192.250 TCP 76 47285 > 80 > [SYN] Seq=0 Win=14520 Len=0 MSS=1452 SACK_PERM=1 TSval=147157135 TSecr=0 > WS=16 > 2013-09-13 13:45:34.035256 50.18.192.250 -> x.x.x.x TCP 76 80 > 47285 > [SYN, ACK] Seq=0 Ack=1 Win=7240 Len=0 MSS=1460 SACK_PERM=1 TSval=23107600 > TSecr=147157135 WS=1 > 2013-09-13 13:45:34.035375 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 > [ACK] Seq=1 Ack=1 Win=14528 Len=0 TSval=147157146 TSecr=23107600 > 2013-09-13 13:45:34.035595 x.x.x.x -> 50.18.192.250 HTTP 186 GET / > HTTP/1.1 > 2013-09-13 13:45:34.082121 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 > [ACK] Seq=1 Ack=119 Win=7122 Len=0 TSval=23107612 TSecr=147157146 > 2013-09-13 13:45:34.082132 50.18.192.250 -> x.x.x.x HTTP 503 HTTP/1.1 301 > Moved Permanently (text/html) > 2013-09-13 13:45:34.082241 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 > [ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157158 TSecr=23107612 > 2013-09-13 13:45:34.446981 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 > [FIN, ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157249 TSecr=23107612 > 2013-09-13 13:45:34.492112 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 > [FIN, ACK] Seq=436 Ack=120 Win=7121 Len=0 TSval=23107714 TSecr=147157249 > 2013-09-13 13:45:34.492164 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 > [ACK] Seq=120 Ack=437 Win=15600 Len=0 TSval=147157260 TSecr=23107714 > > Not sure what to think...it's very strange. Thanks again. > > James > > -- Liam Randall Managing Partner 510-281-0760 www.Broala.com >From the creators of Bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130913/1e8d4198/attachment.html From kebutler at gmail.com Fri Sep 13 14:27:29 2013 From: kebutler at gmail.com (Keith Butler) Date: Fri, 13 Sep 2013 17:27:29 -0400 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> Message-ID: <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> In line with Doug's suggestion, can you try a known good source of traffic? For example a packet trace from Bro Workshop: The first exercise on the following page has an http.pcap file: http://www.bro.org/bro-workshop-2011/exercises/logs/index.html Here is the direct link: http://www.bro.org/bro-workshop-2011/exercises/logs/http.pcap Try running: $ /path/to/your/bro -r http.pcap and see what happens? -kb On Sep 13, 2013, at 3:50 PM, James Lay wrote: > On 2013-09-13 13:31, Liam Randall wrote: >> Lets enable your capture loss and see what happens: >> >> Add the following to your local.bro; on security onion it will be >> located at /opt/bro/share/bro/site/ >> >> # count the ACKs, tell me the # and % I am missing >> @load misc/capture-loss.bro >> >> # By default capture-loss reports every 15 minutes, lets turn it up >> redef CaptureLoss::watch_interval = 1 min; >> >> Give it a couple of minutes and see what the log says under: >> >> /nsm/bro/logs/capture_loss.log >> >> You will see per worker statistics written every minute. >> >> Let us know. >> >> Thanks, >> >> Liam Randall > > > Thanks Liam...here's what I got: > #fields ts ts_delta peer gaps acks percent_lost > #types time interval string count count string > 2013-09-13T13:34:59-0600 60.000084 bro 0 16 > 0.000% > 2013-09-13T13:35:59-0600 60.000044 bro 0 0 > 0.000% > 2013-09-13T13:36:59-0600 60.000048 bro 0 14 > 0.000% > 2013-09-13T13:37:59-0600 60.000048 bro 1 1 > 100.000% > 2013-09-13T13:38:59-0600 60.000038 bro 0 0 > 0.000% > 2013-09-13T13:39:59-0600 60.000050 bro 0 0 > 0.000% > 2013-09-13T13:40:59-0600 60.000093 bro 0 0 > 0.000% > 2013-09-13T13:41:59-0600 60.000023 bro 0 0 > 0.000% > 2013-09-13T13:42:59-0600 60.000022 bro 0 0 > 0.000% > 2013-09-13T13:43:59-0600 60.000023 bro 0 0 > 0.000% > 2013-09-13T13:44:59-0600 60.000089 bro 0 0 > 0.000% > 2013-09-13T13:45:59-0600 60.000073 bro 0 0 > 0.000% > 2013-09-13T13:46:59-0600 60.000011 bro 0 0 > 0.000% > > Bro entries: > 2013-09-13T13:45:33-0600 dPCMEyJBiU7 x.x.x.x 47285 > 50.18.192.250 80 tcp - - - - OTH > - 0 C 0 0 0 0 (empty) > 2013-09-13T13:45:33-0600 PFUbImVSSZ2 x.x.x.x 35306 > 205.171.2.25 53 udp 63725 www.duckduckgo.com 1 > C_INTERNET 1 A 0 NOERROR F F T > T 0 duckduckgo.com,50.18.192.250,50.18.192.251 > 900.000000,25.000000,25.000000 > 2013-09-13T13:45:34-0600 8ZduhgTSjm6 x.x.x.x 37025 > 205.171.2.25 53 udp 35309 duckduckgo.com 1 > C_INTERNET 1 A 0 NOERROR F F T > T 0 50.18.192.251,50.18.192.250 24.000000,24.000000 > 2013-09-13T13:45:34-0600 dPCMEyJBiU7 x.x.x.x 47285 > 50.18.192.250 80 active_connection_reuse - F bro > > > Wget info: > [13:45:20 gateway:~$] wget www.duckduckgo.com > --2013-09-13 13:45:33-- http://www.duckduckgo.com/ > Resolving www.duckduckgo.com (www.duckduckgo.com)... 50.18.192.250, > 50.18.192.251 > Connecting to www.duckduckgo.com > (www.duckduckgo.com)|50.18.192.250|:80... connected. > HTTP request sent, awaiting response... 301 Moved Permanently > Location: https://duckduckgo.com/ [following] > --2013-09-13 13:45:34-- https://duckduckgo.com/ > Resolving duckduckgo.com (duckduckgo.com)... 50.18.192.251, > 50.18.192.250 > Connecting to duckduckgo.com (duckduckgo.com)|50.18.192.251|:443... > connected. > HTTP request sent, awaiting response... 200 OK > Length: 8646 (8.4K) [text/html] > Saving to: `index.html' > > 100%[=========================>] 8,646 --.-K/s in 0s > > 2013-09-13 13:45:34 (86.8 MB/s) - `index.html' saved [8646/8646] > > > Tshark info: > 2013-09-13 13:45:33.991079 x.x.x.x -> 50.18.192.250 TCP 76 47285 > 80 > [SYN] Seq=0 Win=14520 Len=0 MSS=1452 SACK_PERM=1 TSval=147157135 TSecr=0 > WS=16 > 2013-09-13 13:45:34.035256 50.18.192.250 -> x.x.x.x TCP 76 80 > 47285 > [SYN, ACK] Seq=0 Ack=1 Win=7240 Len=0 MSS=1460 SACK_PERM=1 > TSval=23107600 TSecr=147157135 WS=1 > 2013-09-13 13:45:34.035375 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 > [ACK] Seq=1 Ack=1 Win=14528 Len=0 TSval=147157146 TSecr=23107600 > 2013-09-13 13:45:34.035595 x.x.x.x -> 50.18.192.250 HTTP 186 GET / > HTTP/1.1 > 2013-09-13 13:45:34.082121 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 > [ACK] Seq=1 Ack=119 Win=7122 Len=0 TSval=23107612 TSecr=147157146 > 2013-09-13 13:45:34.082132 50.18.192.250 -> x.x.x.x HTTP 503 HTTP/1.1 > 301 Moved Permanently (text/html) > 2013-09-13 13:45:34.082241 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 > [ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157158 TSecr=23107612 > 2013-09-13 13:45:34.446981 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 > [FIN, ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157249 > TSecr=23107612 > 2013-09-13 13:45:34.492112 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 > [FIN, ACK] Seq=436 Ack=120 Win=7121 Len=0 TSval=23107714 TSecr=147157249 > 2013-09-13 13:45:34.492164 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 > [ACK] Seq=120 Ack=437 Win=15600 Len=0 TSval=147157260 TSecr=23107714 > > Not sure what to think...it's very strange. Thanks again. > > James > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From jlay at slave-tothe-box.net Fri Sep 13 14:31:57 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 13 Sep 2013 15:31:57 -0600 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> Message-ID: <8B70F282-21F1-4739-8E33-86B9CE5D7BF4@slave-tothe-box.net> Liam, The I have a DSL modem in dummy mode connecting direct to the machine running bro. That wget is seen running on that machine. Thank you. James On Sep 13, 2013, at 15:23, Liam Randall wrote: > How is the data getting to bro? Span port? Tap? > > > On Fri, Sep 13, 2013 at 3:50 PM, James Lay wrote: > On 2013-09-13 13:31, Liam Randall wrote: > Lets enable your capture loss and see what happens: > > > Add the following to your local.bro; on security onion it will be > located at /opt/bro/share/bro/site/ > > # count the ACKs, tell me the # and % I am missing > @load misc/capture-loss.bro > > # By default capture-loss reports every 15 minutes, lets turn it up > > redef CaptureLoss::watch_interval = 1 min; > > Give it a couple of minutes and see what the log says under: > > /nsm/bro/logs/capture_loss.log > > You will see per worker statistics written every minute. > > Let us know. > > Thanks, > > Liam Randall > > > Thanks Liam...here's what I got: > #fields ts ts_delta peer gaps acks percent_lost > #types time interval string count count string > 2013-09-13T13:34:59-0600 60.000084 bro 0 16 0.000% > 2013-09-13T13:35:59-0600 60.000044 bro 0 0 0.000% > 2013-09-13T13:36:59-0600 60.000048 bro 0 14 0.000% > 2013-09-13T13:37:59-0600 60.000048 bro 1 1 100.000% > 2013-09-13T13:38:59-0600 60.000038 bro 0 0 0.000% > 2013-09-13T13:39:59-0600 60.000050 bro 0 0 0.000% > 2013-09-13T13:40:59-0600 60.000093 bro 0 0 0.000% > 2013-09-13T13:41:59-0600 60.000023 bro 0 0 0.000% > 2013-09-13T13:42:59-0600 60.000022 bro 0 0 0.000% > 2013-09-13T13:43:59-0600 60.000023 bro 0 0 0.000% > 2013-09-13T13:44:59-0600 60.000089 bro 0 0 0.000% > 2013-09-13T13:45:59-0600 60.000073 bro 0 0 0.000% > 2013-09-13T13:46:59-0600 60.000011 bro 0 0 0.000% > > Bro entries: > 2013-09-13T13:45:33-0600 dPCMEyJBiU7 x.x.x.x 47285 50.18.192.250 80 tcp - - - - OTH - 0 C 0 0 0 0 (empty) > 2013-09-13T13:45:33-0600 PFUbImVSSZ2 x.x.x.x 35306 205.171.2.25 53 udp 63725 www.duckduckgo.com 1 C_INTERNET 1 A 0 NOERROR F F T T 0 duckduckgo.com,50.18.192.250,50.18.192.251 900.000000,25.000000,25.000000 > 2013-09-13T13:45:34-0600 8ZduhgTSjm6 x.x.x.x 37025 205.171.2.25 53 udp 35309 duckduckgo.com 1 C_INTERNET 1 A 0 NOERROR F F T T 0 50.18.192.251,50.18.192.250 24.000000,24.000000 > 2013-09-13T13:45:34-0600 dPCMEyJBiU7 x.x.x.x 47285 50.18.192.250 80 active_connection_reuse - F bro > > > Wget info: > [13:45:20 gateway:~$] wget www.duckduckgo.com > --2013-09-13 13:45:33-- http://www.duckduckgo.com/ > Resolving www.duckduckgo.com (www.duckduckgo.com)... 50.18.192.250, 50.18.192.251 > Connecting to www.duckduckgo.com (www.duckduckgo.com)|50.18.192.250|:80... connected. > HTTP request sent, awaiting response... 301 Moved Permanently > Location: https://duckduckgo.com/ [following] > --2013-09-13 13:45:34-- https://duckduckgo.com/ > Resolving duckduckgo.com (duckduckgo.com)... 50.18.192.251, 50.18.192.250 > Connecting to duckduckgo.com (duckduckgo.com)|50.18.192.251|:443... connected. > > HTTP request sent, awaiting response... 200 OK > Length: 8646 (8.4K) [text/html] > Saving to: `index.html' > > 100%[=========================>] 8,646 --.-K/s in 0s > > 2013-09-13 13:45:34 (86.8 MB/s) - `index.html' saved [8646/8646] > > > Tshark info: > 2013-09-13 13:45:33.991079 x.x.x.x -> 50.18.192.250 TCP 76 47285 > 80 [SYN] Seq=0 Win=14520 Len=0 MSS=1452 SACK_PERM=1 TSval=147157135 TSecr=0 WS=16 > 2013-09-13 13:45:34.035256 50.18.192.250 -> x.x.x.x TCP 76 80 > 47285 [SYN, ACK] Seq=0 Ack=1 Win=7240 Len=0 MSS=1460 SACK_PERM=1 TSval=23107600 TSecr=147157135 WS=1 > 2013-09-13 13:45:34.035375 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 [ACK] Seq=1 Ack=1 Win=14528 Len=0 TSval=147157146 TSecr=23107600 > 2013-09-13 13:45:34.035595 x.x.x.x -> 50.18.192.250 HTTP 186 GET / HTTP/1.1 > 2013-09-13 13:45:34.082121 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 [ACK] Seq=1 Ack=119 Win=7122 Len=0 TSval=23107612 TSecr=147157146 > 2013-09-13 13:45:34.082132 50.18.192.250 -> x.x.x.x HTTP 503 HTTP/1.1 301 Moved Permanently (text/html) > 2013-09-13 13:45:34.082241 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 [ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157158 TSecr=23107612 > 2013-09-13 13:45:34.446981 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 [FIN, ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157249 TSecr=23107612 > 2013-09-13 13:45:34.492112 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 [FIN, ACK] Seq=436 Ack=120 Win=7121 Len=0 TSval=23107714 TSecr=147157249 > 2013-09-13 13:45:34.492164 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 [ACK] Seq=120 Ack=437 Win=15600 Len=0 TSval=147157260 TSecr=23107714 > > Not sure what to think...it's very strange. Thanks again. > > James > > > > > -- > Liam Randall > Managing Partner > 510-281-0760 > www.Broala.com > From the creators of Bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130913/c5a5af0d/attachment.html From jlay at slave-tothe-box.net Fri Sep 13 14:36:30 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 13 Sep 2013 15:36:30 -0600 Subject: [Bro] HTTP not being seen In-Reply-To: <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> References: <52334086.2000705@illinois.edu> <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> Message-ID: Hi Keith, Just ran that http.pcap?looked great in my http.log: #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1320279566.452687 JIDwiHjbv85 192.168.2.76 52026 132.235.215.119 80 1 GET www.reddit.com / - Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 0 109978 200 OK - - - (empty) - - - text/html - - 1320279566.831619 VKdyryoPlil 192.168.2.76 52030 72.21.211.173 80 1 GET e.thumbs.redditmedia.com /E-pbDbmiBclPkDaX.jpg http://www.reddit.com/ Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 0 2300 200 OK - - - (empty) - - - image/jpeg - - Thank you. James On Sep 13, 2013, at 15:27, Keith Butler wrote: > In line with Doug's suggestion, can you try a known good source of traffic? For example a packet trace from Bro Workshop: > The first exercise on the following page has an http.pcap file: > http://www.bro.org/bro-workshop-2011/exercises/logs/index.html > > Here is the direct link: > http://www.bro.org/bro-workshop-2011/exercises/logs/http.pcap > > Try running: > $ /path/to/your/bro -r http.pcap > > and see what happens? > > -kb > > > On Sep 13, 2013, at 3:50 PM, James Lay wrote: > >> On 2013-09-13 13:31, Liam Randall wrote: >>> Lets enable your capture loss and see what happens: >>> >>> Add the following to your local.bro; on security onion it will be >>> located at /opt/bro/share/bro/site/ >>> >>> # count the ACKs, tell me the # and % I am missing >>> @load misc/capture-loss.bro >>> >>> # By default capture-loss reports every 15 minutes, lets turn it up >>> redef CaptureLoss::watch_interval = 1 min; >>> >>> Give it a couple of minutes and see what the log says under: >>> >>> /nsm/bro/logs/capture_loss.log >>> >>> You will see per worker statistics written every minute. >>> >>> Let us know. >>> >>> Thanks, >>> >>> Liam Randall >> >> >> Thanks Liam...here's what I got: >> #fields ts ts_delta peer gaps acks percent_lost >> #types time interval string count count string >> 2013-09-13T13:34:59-0600 60.000084 bro 0 16 >> 0.000% >> 2013-09-13T13:35:59-0600 60.000044 bro 0 0 >> 0.000% >> 2013-09-13T13:36:59-0600 60.000048 bro 0 14 >> 0.000% >> 2013-09-13T13:37:59-0600 60.000048 bro 1 1 >> 100.000% >> 2013-09-13T13:38:59-0600 60.000038 bro 0 0 >> 0.000% >> 2013-09-13T13:39:59-0600 60.000050 bro 0 0 >> 0.000% >> 2013-09-13T13:40:59-0600 60.000093 bro 0 0 >> 0.000% >> 2013-09-13T13:41:59-0600 60.000023 bro 0 0 >> 0.000% >> 2013-09-13T13:42:59-0600 60.000022 bro 0 0 >> 0.000% >> 2013-09-13T13:43:59-0600 60.000023 bro 0 0 >> 0.000% >> 2013-09-13T13:44:59-0600 60.000089 bro 0 0 >> 0.000% >> 2013-09-13T13:45:59-0600 60.000073 bro 0 0 >> 0.000% >> 2013-09-13T13:46:59-0600 60.000011 bro 0 0 >> 0.000% >> >> Bro entries: >> 2013-09-13T13:45:33-0600 dPCMEyJBiU7 x.x.x.x 47285 >> 50.18.192.250 80 tcp - - - - OTH >> - 0 C 0 0 0 0 (empty) >> 2013-09-13T13:45:33-0600 PFUbImVSSZ2 x.x.x.x 35306 >> 205.171.2.25 53 udp 63725 www.duckduckgo.com 1 >> C_INTERNET 1 A 0 NOERROR F F T >> T 0 duckduckgo.com,50.18.192.250,50.18.192.251 >> 900.000000,25.000000,25.000000 >> 2013-09-13T13:45:34-0600 8ZduhgTSjm6 x.x.x.x 37025 >> 205.171.2.25 53 udp 35309 duckduckgo.com 1 >> C_INTERNET 1 A 0 NOERROR F F T >> T 0 50.18.192.251,50.18.192.250 24.000000,24.000000 >> 2013-09-13T13:45:34-0600 dPCMEyJBiU7 x.x.x.x 47285 >> 50.18.192.250 80 active_connection_reuse - F bro >> >> >> Wget info: >> [13:45:20 gateway:~$] wget www.duckduckgo.com >> --2013-09-13 13:45:33-- http://www.duckduckgo.com/ >> Resolving www.duckduckgo.com (www.duckduckgo.com)... 50.18.192.250, >> 50.18.192.251 >> Connecting to www.duckduckgo.com >> (www.duckduckgo.com)|50.18.192.250|:80... connected. >> HTTP request sent, awaiting response... 301 Moved Permanently >> Location: https://duckduckgo.com/ [following] >> --2013-09-13 13:45:34-- https://duckduckgo.com/ >> Resolving duckduckgo.com (duckduckgo.com)... 50.18.192.251, >> 50.18.192.250 >> Connecting to duckduckgo.com (duckduckgo.com)|50.18.192.251|:443... >> connected. >> HTTP request sent, awaiting response... 200 OK >> Length: 8646 (8.4K) [text/html] >> Saving to: `index.html' >> >> 100%[=========================>] 8,646 --.-K/s in 0s >> >> 2013-09-13 13:45:34 (86.8 MB/s) - `index.html' saved [8646/8646] >> >> >> Tshark info: >> 2013-09-13 13:45:33.991079 x.x.x.x -> 50.18.192.250 TCP 76 47285 > 80 >> [SYN] Seq=0 Win=14520 Len=0 MSS=1452 SACK_PERM=1 TSval=147157135 TSecr=0 >> WS=16 >> 2013-09-13 13:45:34.035256 50.18.192.250 -> x.x.x.x TCP 76 80 > 47285 >> [SYN, ACK] Seq=0 Ack=1 Win=7240 Len=0 MSS=1460 SACK_PERM=1 >> TSval=23107600 TSecr=147157135 WS=1 >> 2013-09-13 13:45:34.035375 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >> [ACK] Seq=1 Ack=1 Win=14528 Len=0 TSval=147157146 TSecr=23107600 >> 2013-09-13 13:45:34.035595 x.x.x.x -> 50.18.192.250 HTTP 186 GET / >> HTTP/1.1 >> 2013-09-13 13:45:34.082121 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 >> [ACK] Seq=1 Ack=119 Win=7122 Len=0 TSval=23107612 TSecr=147157146 >> 2013-09-13 13:45:34.082132 50.18.192.250 -> x.x.x.x HTTP 503 HTTP/1.1 >> 301 Moved Permanently (text/html) >> 2013-09-13 13:45:34.082241 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >> [ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157158 TSecr=23107612 >> 2013-09-13 13:45:34.446981 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >> [FIN, ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157249 >> TSecr=23107612 >> 2013-09-13 13:45:34.492112 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 >> [FIN, ACK] Seq=436 Ack=120 Win=7121 Len=0 TSval=23107714 TSecr=147157249 >> 2013-09-13 13:45:34.492164 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >> [ACK] Seq=120 Ack=437 Win=15600 Len=0 TSval=147157260 TSecr=23107714 >> >> Not sure what to think...it's very strange. Thanks again. >> >> James >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From Keith_Schoenefeld at baylor.edu Fri Sep 13 14:41:57 2013 From: Keith_Schoenefeld at baylor.edu (Schoenefeld, Keith P.) Date: Fri, 13 Sep 2013 21:41:57 +0000 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> Message-ID: I'd point out that the conn.log file lines included in his email indicate that the received and sent bytes are 0, and the connection state is borked -- I believe this would prevent it from being detected as http traffic, and therefore not create an http.log file. -- KS -----Original Message----- From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of James Lay Sent: Friday, September 13, 2013 4:37 PM To: Keith Butler Cc: Subject: Re: [Bro] HTTP not being seen Hi Keith, Just ran that http.pcap...looked great in my http.log: #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1320279566.452687 JIDwiHjbv85 192.168.2.76 52026 132.235.215.119 80 1 GET www.reddit.com / - Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 0 109978 200 OK - - - (empty) - - - text/html - - 1320279566.831619 VKdyryoPlil 192.168.2.76 52030 72.21.211.173 80 1 GET e.thumbs.redditmedia.com /E-pbDbmiBclPkDaX.jpg http://www.reddit.com/ Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 0 2300 200 OK - - - (empty) - - - image/jpeg - - Thank you. James On Sep 13, 2013, at 15:27, Keith Butler wrote: > In line with Doug's suggestion, can you try a known good source of traffic? For example a packet trace from Bro Workshop: > The first exercise on the following page has an http.pcap file: > http://www.bro.org/bro-workshop-2011/exercises/logs/index.html > > Here is the direct link: > http://www.bro.org/bro-workshop-2011/exercises/logs/http.pcap > > Try running: > $ /path/to/your/bro -r http.pcap > > and see what happens? > > -kb > > > On Sep 13, 2013, at 3:50 PM, James Lay wrote: > >> On 2013-09-13 13:31, Liam Randall wrote: >>> Lets enable your capture loss and see what happens: >>> >>> Add the following to your local.bro; on security onion it will be >>> located at /opt/bro/share/bro/site/ >>> >>> # count the ACKs, tell me the # and % I am missing @load >>> misc/capture-loss.bro >>> >>> # By default capture-loss reports every 15 minutes, lets turn it up >>> redef CaptureLoss::watch_interval = 1 min; >>> >>> Give it a couple of minutes and see what the log says under: >>> >>> /nsm/bro/logs/capture_loss.log >>> >>> You will see per worker statistics written every minute. >>> >>> Let us know. >>> >>> Thanks, >>> >>> Liam Randall >> >> >> Thanks Liam...here's what I got: >> #fields ts ts_delta peer gaps acks percent_lost >> #types time interval string count count string >> 2013-09-13T13:34:59-0600 60.000084 bro 0 16 >> 0.000% >> 2013-09-13T13:35:59-0600 60.000044 bro 0 0 >> 0.000% >> 2013-09-13T13:36:59-0600 60.000048 bro 0 14 >> 0.000% >> 2013-09-13T13:37:59-0600 60.000048 bro 1 1 >> 100.000% >> 2013-09-13T13:38:59-0600 60.000038 bro 0 0 >> 0.000% >> 2013-09-13T13:39:59-0600 60.000050 bro 0 0 >> 0.000% >> 2013-09-13T13:40:59-0600 60.000093 bro 0 0 >> 0.000% >> 2013-09-13T13:41:59-0600 60.000023 bro 0 0 >> 0.000% >> 2013-09-13T13:42:59-0600 60.000022 bro 0 0 >> 0.000% >> 2013-09-13T13:43:59-0600 60.000023 bro 0 0 >> 0.000% >> 2013-09-13T13:44:59-0600 60.000089 bro 0 0 >> 0.000% >> 2013-09-13T13:45:59-0600 60.000073 bro 0 0 >> 0.000% >> 2013-09-13T13:46:59-0600 60.000011 bro 0 0 >> 0.000% >> >> Bro entries: >> 2013-09-13T13:45:33-0600 dPCMEyJBiU7 x.x.x.x 47285 >> 50.18.192.250 80 tcp - - - - OTH >> - 0 C 0 0 0 0 (empty) >> 2013-09-13T13:45:33-0600 PFUbImVSSZ2 x.x.x.x 35306 >> 205.171.2.25 53 udp 63725 www.duckduckgo.com 1 >> C_INTERNET 1 A 0 NOERROR F F T >> T 0 duckduckgo.com,50.18.192.250,50.18.192.251 >> 900.000000,25.000000,25.000000 >> 2013-09-13T13:45:34-0600 8ZduhgTSjm6 x.x.x.x 37025 >> 205.171.2.25 53 udp 35309 duckduckgo.com 1 >> C_INTERNET 1 A 0 NOERROR F F T >> T 0 50.18.192.251,50.18.192.250 24.000000,24.000000 >> 2013-09-13T13:45:34-0600 dPCMEyJBiU7 x.x.x.x 47285 >> 50.18.192.250 80 active_connection_reuse - F bro >> >> >> Wget info: >> [13:45:20 gateway:~$] wget www.duckduckgo.com >> --2013-09-13 13:45:33-- http://www.duckduckgo.com/ Resolving >> www.duckduckgo.com (www.duckduckgo.com)... 50.18.192.250, >> 50.18.192.251 >> Connecting to www.duckduckgo.com >> (www.duckduckgo.com)|50.18.192.250|:80... connected. >> HTTP request sent, awaiting response... 301 Moved Permanently >> Location: https://duckduckgo.com/ [following] >> --2013-09-13 13:45:34-- https://duckduckgo.com/ Resolving >> duckduckgo.com (duckduckgo.com)... 50.18.192.251, >> 50.18.192.250 >> Connecting to duckduckgo.com (duckduckgo.com)|50.18.192.251|:443... >> connected. >> HTTP request sent, awaiting response... 200 OK >> Length: 8646 (8.4K) [text/html] >> Saving to: `index.html' >> >> 100%[=========================>] 8,646 --.-K/s in 0s >> >> 2013-09-13 13:45:34 (86.8 MB/s) - `index.html' saved [8646/8646] >> >> >> Tshark info: >> 2013-09-13 13:45:33.991079 x.x.x.x -> 50.18.192.250 TCP 76 47285 > 80 >> [SYN] Seq=0 Win=14520 Len=0 MSS=1452 SACK_PERM=1 TSval=147157135 >> TSecr=0 >> WS=16 >> 2013-09-13 13:45:34.035256 50.18.192.250 -> x.x.x.x TCP 76 80 > 47285 >> [SYN, ACK] Seq=0 Ack=1 Win=7240 Len=0 MSS=1460 SACK_PERM=1 >> TSval=23107600 TSecr=147157135 WS=1 >> 2013-09-13 13:45:34.035375 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >> [ACK] Seq=1 Ack=1 Win=14528 Len=0 TSval=147157146 TSecr=23107600 >> 2013-09-13 13:45:34.035595 x.x.x.x -> 50.18.192.250 HTTP 186 GET / >> HTTP/1.1 >> 2013-09-13 13:45:34.082121 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 >> [ACK] Seq=1 Ack=119 Win=7122 Len=0 TSval=23107612 TSecr=147157146 >> 2013-09-13 13:45:34.082132 50.18.192.250 -> x.x.x.x HTTP 503 HTTP/1.1 >> 301 Moved Permanently (text/html) >> 2013-09-13 13:45:34.082241 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >> [ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157158 TSecr=23107612 >> 2013-09-13 13:45:34.446981 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >> [FIN, ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157249 >> TSecr=23107612 >> 2013-09-13 13:45:34.492112 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 >> [FIN, ACK] Seq=436 Ack=120 Win=7121 Len=0 TSval=23107714 >> TSecr=147157249 >> 2013-09-13 13:45:34.492164 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >> [ACK] Seq=120 Ack=437 Win=15600 Len=0 TSval=147157260 TSecr=23107714 >> >> Not sure what to think...it's very strange. Thanks again. >> >> James >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From edthoma at sandia.gov Fri Sep 13 15:46:43 2013 From: edthoma at sandia.gov (Thomas, Eric D) Date: Fri, 13 Sep 2013 22:46:43 +0000 Subject: [Bro] Downside to using -b? Message-ID: >From what I can tell, not loading base/loading init-default.bro (using the -b option) significantly improves performance, particularly if you are not enabling a bunch of different kinds of analysis. Assuming my local.bro loads the base scripts it needs for processing, is there any reason why I wouldn't use -b? -- Eric Thomas edthoma at sandia.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130913/886bcce4/attachment.html From jlay at slave-tothe-box.net Fri Sep 13 16:51:26 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 13 Sep 2013 17:51:26 -0600 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> Message-ID: <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> WELLL this is gonna hurt my head. So?here's my setup: DSL modem (dummy mode) < -- > eth1/ppp0(Linuxbox)eth0 < -- > switch with everything else ext IP int IP For giggles I decided to see if there was any difference with requests originating from the linuxbox, compared to originating behind it. I tsharked at ppp0, here's what I have Curl from a Mac 2013-09-13 17:30:53.720063 x.x.x.x -> 23.63.227.171 TCP 80 51579 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1452 WS=16 TSval=962401899 TSecr=0 SACK_PERM=1 2013-09-13 17:30:53.781038 23.63.227.171 -> x.x.x.x TCP 76 80 > 51579 [SYN, ACK] Seq=0 Ack=1 Win=14480 Len=0 MSS=1460 SACK_PERM=1 TSval=923336834 TSecr=962401899 WS=2 2013-09-13 17:30:53.781706 x.x.x.x -> 23.63.227.171 TCP 68 51579 > 80 [ACK] Seq=1 Ack=1 Win=131760 Len=0 TSval=962401960 TSecr=923336834 2013-09-13 17:30:53.781917 x.x.x.x -> 23.63.227.171 HTTP 214 GET / HTTP/1.1 2013-09-13 17:30:53.844280 23.63.227.171 -> x.x.x.x TCP 68 80 > 51579 [ACK] Seq=1 Ack=147 Win=15552 Len=0 TSval=923336896 TSecr=962401960 2013-09-13 17:30:54.058292 23.63.227.171 -> x.x.x.x TCP 1508 [TCP segment of a reassembled PDU] 2013-09-13 17:30:54.059011 23.63.227.171 -> x.x.x.x TCP 1508 [TCP segment of a reassembled PDU] 2013-09-13 17:30:54.059543 x.x.x.x -> 23.63.227.171 TCP 68 51579 > 80 [ACK] Seq=147 Ack=2881 Win=129632 Len=0 TSval=962402235 TSecr=923337110 2013-09-13 17:30:54.059571 23.63.227.171 -> x.x.x.x TCP 1508 [TCP segment of a reassembled PDU] 2013-09-13 17:30:54.060385 23.63.227.171 -> x.x.x.x TCP 1508 [TCP segment of a reassembled PDU] 2013-09-13 17:30:54.060387 23.63.227.171 -> x.x.x.x HTTP 97 HTTP/1.1 200 OK (text/html) 2013-09-13 17:30:54.060889 x.x.x.x -> 23.63.227.171 TCP 68 51579 > 80 [ACK] Seq=147 Ack=5761 Win=129632 Len=0 TSval=962402236 TSecr=923337110 2013-09-13 17:30:54.060923 x.x.x.x -> 23.63.227.171 TCP 68 51579 > 80 [ACK] Seq=147 Ack=5790 Win=131040 Len=0 TSval=962402236 TSecr=923337110 2013-09-13 17:30:54.136000 x.x.x.x -> 23.63.227.171 TCP 68 51579 > 80 [FIN, ACK] Seq=147 Ack=5790 Win=131072 Len=0 TSval=962402311 TSecr=923337110 2013-09-13 17:30:54.196402 23.614 2013-09-13 17:30:54.196402 23.614 2013-09-13 17:30:54.196402 23.614 2013-09-13 17:30:54.19=923337249 TSecr=962402311 2013-09-13 17:30:54.197075 x.x.x.x -> 23.63.227.171 TCP 68 51579 > 80 [ACK] Seq=148 Ack=5791 Win=131072 Len=0 TSval=962402371 TSecr=923337249 Wget from the linuxbox 2013-09-13 17:30:56.769223 x.x.x.x -> 207.108.220.146 TCP 76 46641 > 80 [SYN] Seq=0 Win=14520 Len=0 MSS=1452 SACK_PERM=1 TSval=150537829 TSecr=0 WS=16 2013-09-13 17:30:56.800179 207.108.220.146 -> x.x.x.x TCP 76 80 > 46641 [SYN, ACK] Seq=0 Ack=1 Win=14480 Len=0 MSS=1460 SACK_PERM=1 TSval=811267839 TSecr=150537829 WS=2 2013-09-13 17:30:56.800271 x.x.x.x -> 207.108.220.146 TCP 68 46641 > 80 [ACK] Seq=1 Ack=1 Win=14528 Len=0 TSval=150537837 TSecr=811267839 2013-09-13 17:30:56.800559 x.x.x.x -> 207.108.220.146 HTTP 183 GET / HTTP/1.1 2013-09-13 17:30:56.833172 207.108.220.146 -> x.x.x.x TCP 68 80 > 46641 [ACK] Seq=1 Ack=116 Win=14480 Len=0 TSval=811267872 TSecr=150537837 2013-09-13 17:30:56.898798 207.108.220.146 -> x.x.x.x TCP 1508 [TCP segment of a reassembled PDU] 2013-09-13 17:30:56.898890 x.x.x.x -> 207.108.220.146 TCP 68 46641 > 80 [ACK] Seq=116 Ack=1441 Win=17408 Len=0 TSval=150537862 TSecr=811267937 2013-09-13 17:30:56.899540 207.108.220.146 -> x.x.x.x TCP 1508 [TCP segment of a reassembled PDU] 2013-09-13 17:30:56.899597 x.x.x.x -> 207.108.220.146 TCP 68 46641 > 80 [ACK] Seq=116 Ack=2881 Win=20288 Len=0 TSval=150537862 TSecr=811267937 2013-09-13 17:30:56.900295 207.108.220.146 -> x.x.x.x TCP 1508 [TCP segment of a reassembled PDU] 2013-09-13 17:30:56.900355 x.x.x.x -> 207.108.220.146 TCP 68 46641 > 80 [ACK] Seq=116 Ack=4321 Win=23168 Len=0 TSval=150537862 TSecr=811267937 2013-09-13 17:30:56.901041 207.108.220.146 -> x.x.x.x TCP 1508 [TCP segment of a reassembled PDU] 2013-09-13 17:30:56.901097 x.x.x.x -> 207.108.220.146 TCP 68 46641 > 80 [ACK] Seq=116 Ack=5761 Win=26048 Len=0 TSval=150537862 TSecr=811267937 2013-09-13 17:30:56.901043 207.108.220.146 -> x.x.x.x HTTP 97 HTTP/1.1 200 OK (text/html) 2013-09-13 17:30:56.901141 x.x.x.x -> 207.108.220.146 TCP 68 46641 > 80 [ACK] Seq=116 Ack=5790 Win=26048 Len=0 TSval=150537862 TSecr=811267937 2013-09-13 17:30:56.901950 x.x.x.x -> 207.108.220.146 TCP 68 46641 > 80 [FIN, ACK] Seq=116 Ack=5790 Win=26048 Len=0 TSval=150537863 TSecr=811267937 2013-09-13 17:30:56.933443 207.108.220.146 -> x.x.x.x TCP 68 80 > 46641 [FIN, ACK] Seq=5790 Ack=117 Win=14480 Len=0 TSva2013-09-13 17:30:56.933443 207.108.220.146 -> 71.39.117.84 TCP 68 80 > 46641 [FIN, ACK] Seq=5790 Ac > 80 [ACK] Seq=117 Ack=5791 Win=26048 Len=0 TSval=150537871 TSecr=811267973 Mac 2013-09-13T17:30:53-0600 BnQTbzFqpae x.x.x.x 51579 23.63.227.171 80 tcp http 0.476339 146 5789 SF - 0 ShADadFf 8 574 8 6213 (empty) 2013-09-13T17:30:53-0600 BnQTbzFqpae x.x.x.x 51579 23.63.227.171 80 1 GET www.spamcop.net / - curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8y zlib/1.2.5 0 5455 200 OK - - - (empty) - - - text/html - - Linuxbox 2013-09-13T17:30:56-0600 aDSEGHvanZ1 x.x.x.x 46641 207.108.220.146 80 tcp - - - - OTH - 0 C 0 0 0 0 (empty) 2013-09-13T17:30:56-0600 aDSEGHvanZ1 x.x.x.x 46641 207.108.220.146 80 active_connection_reuse - F bro When coming from the linuxbox, I get active_connection_reuse and no http logged. When routing(natting) internal traffic, it logs http and no active_connection_reuse. A puzzle to be sure. James On Sep 13, 2013, at 3:41 PM, "Schoenefeld, Keith P." wrote: > I'd point out that the conn.log file lines included in his email indicate that the received and sent bytes are 0, and the connection state is borked -- I believe this would prevent it from being detected as http traffic, and therefore not create an http.log file. > > -- KS > > -----Original Message----- > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of James Lay > Sent: Friday, September 13, 2013 4:37 PM > To: Keith Butler > Cc: > Subject: Re: [Bro] HTTP not being seen > > Hi Keith, > > Just ran that http.pcap...looked great in my http.log: > > #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file > #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file > 1320279566.452687 JIDwiHjbv85 192.168.2.76 52026 132.235.215.119 80 1 GET www.reddit.com / - Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 0 109978 200 OK - - - (empty) - - - text/html - - > 1320279566.831619 VKdyryoPlil 192.168.2.76 52030 72.21.211.173 80 1 GET e.thumbs.redditmedia.com /E-pbDbmiBclPkDaX.jpg http://www.reddit.com/ Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 0 2300 200 OK - - - (empty) - - - image/jpeg - - > > Thank you. > > James > > On Sep 13, 2013, at 15:27, Keith Butler wrote: > >> In line with Doug's suggestion, can you try a known good source of traffic? For example a packet trace from Bro Workshop: >> The first exercise on the following page has an http.pcap file: >> http://www.bro.org/bro-workshop-2011/exercises/logs/index.html >> >> Here is the direct link: >> http://www.bro.org/bro-workshop-2011/exercises/logs/http.pcap >> >> Try running: >> $ /path/to/your/bro -r http.pcap >> >> and see what happens? >> >> -kb >> >> >> On Sep 13, 2013, at 3:50 PM, James Lay wrote: >> >>> On 2013-09-13 13:31, Liam Randall wrote: >>>> Lets enable your capture loss and see what happens: >>>> >>>> Add the following to your local.bro; on security onion it will be >>>> located at /opt/bro/share/bro/site/ >>>> >>>> # count the ACKs, tell me the # and % I am missing @load >>>> misc/capture-loss.bro >>>> >>>> # By default capture-loss reports every 15 minutes, lets turn it up >>>> redef CaptureLoss::watch_interval = 1 min; >>>> >>>> Give it a couple of minutes and see what the log says under: >>>> >>>> /nsm/bro/logs/capture_loss.log >>>> >>>> You will see per worker statistics written every minute. >>>> >>>> Let us know. >>>> >>>> Thanks, >>>> >>>> Liam Randall >>> >>> >>> Thanks Liam...here's what I got: >>> #fields ts ts_delta peer gaps acks percent_lost >>> #types time interval string count count string >>> 2013-09-13T13:34:59-0600 60.000084 bro 0 16 >>> 0.000% >>> 2013-09-13T13:35:59-0600 60.000044 bro 0 0 >>> 0.000% >>> 2013-09-13T13:36:59-0600 60.000048 bro 0 14 >>> 0.000% >>> 2013-09-13T13:37:59-0600 60.000048 bro 1 1 >>> 100.000% >>> 2013-09-13T13:38:59-0600 60.000038 bro 0 0 >>> 0.000% >>> 2013-09-13T13:39:59-0600 60.000050 bro 0 0 >>> 0.000% >>> 2013-09-13T13:40:59-0600 60.000093 bro 0 0 >>> 0.000% >>> 2013-09-13T13:41:59-0600 60.000023 bro 0 0 >>> 0.000% >>> 2013-09-13T13:42:59-0600 60.000022 bro 0 0 >>> 0.000% >>> 2013-09-13T13:43:59-0600 60.000023 bro 0 0 >>> 0.000% >>> 2013-09-13T13:44:59-0600 60.000089 bro 0 0 >>> 0.000% >>> 2013-09-13T13:45:59-0600 60.000073 bro 0 0 >>> 0.000% >>> 2013-09-13T13:46:59-0600 60.000011 bro 0 0 >>> 0.000% >>> >>> Bro entries: >>> 2013-09-13T13:45:33-0600 dPCMEyJBiU7 x.x.x.x 47285 >>> 50.18.192.250 80 tcp - - - - OTH >>> - 0 C 0 0 0 0 (empty) >>> 2013-09-13T13:45:33-0600 PFUbImVSSZ2 x.x.x.x 35306 >>> 205.171.2.25 53 udp 63725 www.duckduckgo.com 1 >>> C_INTERNET 1 A 0 NOERROR F F T >>> T 0 duckduckgo.com,50.18.192.250,50.18.192.251 >>> 900.000000,25.000000,25.000000 >>> 2013-09-13T13:45:34-0600 8ZduhgTSjm6 x.x.x.x 37025 >>> 205.171.2.25 53 udp 35309 duckduckgo.com 1 >>> C_INTERNET 1 A 0 NOERROR F F T >>> T 0 50.18.192.251,50.18.192.250 24.000000,24.000000 >>> 2013-09-13T13:45:34-0600 dPCMEyJBiU7 x.x.x.x 47285 >>> 50.18.192.250 80 active_connection_reuse - F bro >>> >>> >>> Wget info: >>> [13:45:20 gateway:~$] wget www.duckduckgo.com >>> --2013-09-13 13:45:33-- http://www.duckduckgo.com/ Resolving >>> www.duckduckgo.com (www.duckduckgo.com)... 50.18.192.250, >>> 50.18.192.251 >>> Connecting to www.duckduckgo.com >>> (www.duckduckgo.com)|50.18.192.250|:80... connected. >>> HTTP request sent, awaiting response... 301 Moved Permanently >>> Location: https://duckduckgo.com/ [following] >>> --2013-09-13 13:45:34-- https://duckduckgo.com/ Resolving >>> duckduckgo.com (duckduckgo.com)... 50.18.192.251, >>> 50.18.192.250 >>> Connecting to duckduckgo.com (duckduckgo.com)|50.18.192.251|:443... >>> connected. >>> HTTP request sent, awaiting response... 200 OK >>> Length: 8646 (8.4K) [text/html] >>> Saving to: `index.html' >>> >>> 100%[=========================>] 8,646 --.-K/s in 0s >>> >>> 2013-09-13 13:45:34 (86.8 MB/s) - `index.html' saved [8646/8646] >>> >>> >>> Tshark info: >>> 2013-09-13 13:45:33.991079 x.x.x.x -> 50.18.192.250 TCP 76 47285 > 80 >>> [SYN] Seq=0 Win=14520 Len=0 MSS=1452 SACK_PERM=1 TSval=147157135 >>> TSecr=0 >>> WS=16 >>> 2013-09-13 13:45:34.035256 50.18.192.250 -> x.x.x.x TCP 76 80 > 47285 >>> [SYN, ACK] Seq=0 Ack=1 Win=7240 Len=0 MSS=1460 SACK_PERM=1 >>> TSval=23107600 TSecr=147157135 WS=1 >>> 2013-09-13 13:45:34.035375 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >>> [ACK] Seq=1 Ack=1 Win=14528 Len=0 TSval=147157146 TSecr=23107600 >>> 2013-09-13 13:45:34.035595 x.x.x.x -> 50.18.192.250 HTTP 186 GET / >>> HTTP/1.1 >>> 2013-09-13 13:45:34.082121 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 >>> [ACK] Seq=1 Ack=119 Win=7122 Len=0 TSval=23107612 TSecr=147157146 >>> 2013-09-13 13:45:34.082132 50.18.192.250 -> x.x.x.x HTTP 503 HTTP/1.1 >>> 301 Moved Permanently (text/html) >>> 2013-09-13 13:45:34.082241 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >>> [ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157158 TSecr=23107612 >>> 2013-09-13 13:45:34.446981 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >>> [FIN, ACK] Seq=119 Ack=436 Win=15600 Len=0 TSval=147157249 >>> TSecr=23107612 >>> 2013-09-13 13:45:34.492112 50.18.192.250 -> x.x.x.x TCP 68 80 > 47285 >>> [FIN, ACK] Seq=436 Ack=120 Win=7121 Len=0 TSval=23107714 >>> TSecr=147157249 >>> 2013-09-13 13:45:34.492164 x.x.x.x -> 50.18.192.250 TCP 68 47285 > 80 >>> [ACK] Seq=120 Ack=437 Win=15600 Len=0 TSval=147157260 TSecr=23107714 >>> >>> Not sure what to think...it's very strange. Thanks again. >>> >>> James >>> >>> _______________________________________________ >>> Bro mailing list >>> bro at bro-ids.org >>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130913/c3eaf445/attachment.bin From kebutler at gmail.com Sat Sep 14 02:04:01 2013 From: kebutler at gmail.com (Keith Butler) Date: Sat, 14 Sep 2013 05:04:01 -0400 Subject: [Bro] HTTP not being seen In-Reply-To: <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> References: <52334086.2000705@illinois.edu> <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> Message-ID: Maybe tell Bro to ignore checksums? http://www.bro.org/documentation/faq.html#why-isn-t-bro-producing-the-logs-i-expect-a-note-about-checksums Option 1 would probably entail: If you are using broctl, in your broctl.cfg file add the line (then install and restart in broctl)? broargs = --no-checksums On Sep 13, 2013, at 7:51 PM, James Lay wrote: > active_connection_reuse -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130914/ff0f4049/attachment.html From jlay at slave-tothe-box.net Sat Sep 14 05:24:06 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Sat, 14 Sep 2013 06:24:06 -0600 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> Message-ID: <8DB0A263-5B3A-4507-8F11-DEE88552BFB5@slave-tothe-box.net> On Sep 14, 2013, at 3:04 AM, Keith Butler wrote: > Maybe tell Bro to ignore checksums? > > http://www.bro.org/documentation/faq.html#why-isn-t-bro-producing-the-logs-i-expect-a-note-about-checksums > > Option 1 would probably entail: > If you are using broctl, in your broctl.cfg file add the line (then install and restart in broctl)? > broargs = --no-checksums > > > On Sep 13, 2013, at 7:51 PM, James Lay wrote: > >> active_connection_reuse > Hey hey! I think that was just the ticket. Thanks so much Keith! James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130914/0cb60aab/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130914/0cb60aab/attachment.bin From anthony.kasza at gmail.com Sat Sep 14 12:51:17 2013 From: anthony.kasza at gmail.com (anthony kasza) Date: Sat, 14 Sep 2013 12:51:17 -0700 Subject: [Bro] Intel Framework Extensions Message-ID: In an attempt to customize the intel framework I've run into a few issues and am looking for advice on how to proceed. I have two questions. Given the Intel::Type enum is not redefinable, what is the best way to add new types of indicators to the intel framework? I've managed to add DOMAIN_TLDs to the framework, but only by editing base/frameworks/intel/main. A rather specific case I've been considering is the best way to identify a set of unordered URL parameters. A chain of requires-signature might be a solution, but it seems inelegant. Although one may not necessarily care which domain, path, or file an HTTP request contains, it may become interesting given the presence of specific parameters. Example requests follow: hxxp://domain1.tld/path1/file1.asp?foo=1&bar=2 hxxp://domain2.tld/path2/file2.asp?foo=3&bar=4 hxxp://domain3.tld/path3/file3.asp?bar=6&foo=5 It would be nice to be able to include a set of strings in an intel.dat file. Does anyone have any ideas on how to extend the intel framework to support complex indicators? Patterns could be useful, too. -AK -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130914/030b9826/attachment.html From seth at icir.org Sun Sep 15 18:45:38 2013 From: seth at icir.org (Seth Hall) Date: Sun, 15 Sep 2013 21:45:38 -0400 Subject: [Bro] Intel Framework Extensions In-Reply-To: References: Message-ID: <73D44EAC-5A94-4A75-B5AA-056549B3E206@icir.org> On Sep 14, 2013, at 3:51 PM, anthony kasza wrote: > Given the Intel::Type enum is not redefinable, what is the best way to add new types of indicators to the intel framework? I've managed to add DOMAIN_TLDs to the framework, but only by editing base/frameworks/intel/main. Enums are implicitly redef-able. Have you tried it? > It would be nice to be able to include a set of strings in an intel.dat file. Does anyone have any ideas on how to extend the intel framework to support complex indicators? That's not possible through extensions yet. It's very possible that we'll add more capability for matching extensions later, but for now the intel framework is very minimal and simple. Keep in mind that I'm not saying you couldn't write a Bro script that does this, just that the intel framework is probably not what you're looking for right now. > Patterns could be useful, too. We've discussed this for a long time and it's something that we will approach in the future, but it likely won't be for full Bro patterns (regular expressions). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130915/509e6daa/attachment.bin From seth at icir.org Sun Sep 15 18:50:18 2013 From: seth at icir.org (Seth Hall) Date: Sun, 15 Sep 2013 21:50:18 -0400 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> Message-ID: On Sep 14, 2013, at 5:04 AM, Keith Butler wrote: > Option 1 would probably entail: > If you are using broctl, in your broctl.cfg file add the line (then install and restart in broctl)? > broargs = --no-checksums I want to chime in here quickly. Generally if you are running in BroControl it means that you are sniffing live traffic and probably in an environment where you want things to run correctly in which case you would *never* want to configure this option. If packet have bad checksums, they generally should be dropped. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130915/51c65f12/attachment.bin From seth at icir.org Sun Sep 15 18:56:21 2013 From: seth at icir.org (Seth Hall) Date: Sun, 15 Sep 2013 21:56:21 -0400 Subject: [Bro] Downside to using -b? In-Reply-To: References: Message-ID: <265723D6-A219-4126-AAE7-327E2210CD7F@icir.org> On Sep 13, 2013, at 6:46 PM, "Thomas, Eric D" wrote: > > From what I can tell, not loading base/loading init-default.bro (using the -b option) significantly improves performance, particularly if you are not enabling a bunch of different kinds of analysis. Assuming my local.bro loads the base scripts it needs for processing, is there any reason why I wouldn't use -b? Hi Eric :) There's no reason not to use -b if you actually don't want that stuff enabled. Generally speaking, the only thing that should be consuming processing time in the normal mode is the protocol analysis. Everything else feeds off of that so the rest of the code that gets loaded shouldn't actually be getting executed (for the most part).  We made the decision to enable so many things by default for the 2.0 release because we wanted Bro to be extremely easy to run (to shed the past reputation of Bro being difficult to run). My goal to make it easier to run than tcpdump and I think we achieved that (bro -r packets.pcap). The -b option was our way to leave the door open for more enterprising users to truly customize things as they wanted while still making Bro do a lot by default. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130915/d7799e21/attachment.bin From seth at icir.org Sun Sep 15 18:59:33 2013 From: seth at icir.org (Seth Hall) Date: Sun, 15 Sep 2013 21:59:33 -0400 Subject: [Bro] rescan_state()? In-Reply-To: References: Message-ID: <482A4FD9-6DD0-44F6-BD6B-C1EA47757F3F@icir.org> On Sep 13, 2013, at 4:12 PM, Mike Sconzo wrote: > # IP -> DHH -> proto -> [0] = number of connections [1] = bytes transferred > global ip_to_connection_info: table[addr] of table[int] of > table[string] of vector of count &persistent &synchronized; We're planning on removing the &persistent attribute before long too. It's not terribly safe to use if you want to keep data safe externally to the Bro process. Actually, we're hopefully going to be removing &synchronized too. Neither of those attributes give you the flexibility to define how to make things behave. If/when we remove them we'll be sure that equivalent and better functionality will be made available, I just wanted to make sure you're aware that your script will need to be updated in the future. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130915/aab3f763/attachment.bin From jlay at slave-tothe-box.net Mon Sep 16 05:26:46 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Mon, 16 Sep 2013 06:26:46 -0600 Subject: [Bro] HTTP not being seen In-Reply-To: References: <52334086.2000705@illinois.edu> <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> Message-ID: <003ACA3C-5E11-4D69-8323-E11123377D15@slave-tothe-box.net> On Sep 15, 2013, at 7:50 PM, Seth Hall wrote: > On Sep 14, 2013, at 5:04 AM, Keith Butler wrote: > >> Option 1 would probably entail: >> If you are using broctl, in your broctl.cfg file add the line (then install and restart in broctl)? >> broargs = --no-checksums > > I want to chime in here quickly. Generally if you are running in BroControl it means that you are sniffing live traffic and probably in an environment where you want things to run correctly in which case you would *never* want to configure this option. If packet have bad checksums, they generally should be dropped. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > I completely agree. In my case however, I believe it's a "feature" of PPPoE?those same packets captured on the physical eth1 NIC are flawless with zero issues. I tried having Bro listen to eth1, but I don't think it plays with the PPPoE. Thanks for the input Seth. James -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130916/d01734ce/attachment.bin From edthoma at sandia.gov Mon Sep 16 06:53:40 2013 From: edthoma at sandia.gov (Thomas, Eric D) Date: Mon, 16 Sep 2013 13:53:40 +0000 Subject: [Bro] [EXTERNAL] Re: Downside to using -b? In-Reply-To: <265723D6-A219-4126-AAE7-327E2210CD7F@icir.org> Message-ID: Terrific, thanks! -- Eric Thomas edthoma at sandia.gov On 9/15/13 6:56 PM, "Seth Hall" wrote: >On Sep 13, 2013, at 6:46 PM, "Thomas, Eric D" wrote: >> >> From what I can tell, not loading base/loading init-default.bro (using >>the -b option) significantly improves performance, particularly if you >>are not enabling a bunch of different kinds of analysis. Assuming my >>local.bro loads the base scripts it needs for processing, is there any >>reason why I wouldn't use -b? > >Hi Eric :) > >There's no reason not to use -b if you actually don't want that stuff >enabled. Generally speaking, the only thing that should be consuming >processing time in the normal mode is the protocol analysis. Everything >else feeds off of that so the rest of the code that gets loaded shouldn't >actually be getting executed (for the most part). > >We made the decision to enable so many things by default for the 2.0 >release because we wanted Bro to be extremely easy to run (to shed the >past reputation of Bro being difficult to run). My goal to make it >easier to run than tcpdump and I think we achieved that (bro -r >packets.pcap). The -b option was our way to leave the door open for more >enterprising users to truly customize things as they wanted while still >making Bro do a lot by default. > > .Seth > >-- >Seth Hall >International Computer Science Institute >(Bro) because everyone has a network >http://www.bro.org/ > > From anthony.kasza at gmail.com Mon Sep 16 07:26:24 2013 From: anthony.kasza at gmail.com (anthony kasza) Date: Mon, 16 Sep 2013 07:26:24 -0700 Subject: [Bro] Intel Framework Extensions In-Reply-To: <73D44EAC-5A94-4A75-B5AA-056549B3E206@icir.org> References: <73D44EAC-5A94-4A75-B5AA-056549B3E206@icir.org> Message-ID: Got it. Thanks for the info, Seth! On Sep 15, 2013 6:45 PM, "Seth Hall" wrote: > On Sep 14, 2013, at 3:51 PM, anthony kasza > wrote: > > > Given the Intel::Type enum is not redefinable, what is the best way to > add new types of indicators to the intel framework? I've managed to add > DOMAIN_TLDs to the framework, but only by editing > base/frameworks/intel/main. > > Enums are implicitly redef-able. Have you tried it? > > > It would be nice to be able to include a set of strings in an intel.dat > file. Does anyone have any ideas on how to extend the intel framework to > support complex indicators? > > That's not possible through extensions yet. It's very possible that we'll > add more capability for matching extensions later, but for now the intel > framework is very minimal and simple. > > Keep in mind that I'm not saying you couldn't write a Bro script that does > this, just that the intel framework is probably not what you're looking for > right now. > > > Patterns could be useful, too. > > > We've discussed this for a long time and it's something that we will > approach in the future, but it likely won't be for full Bro patterns > (regular expressions). > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130916/3cb863d5/attachment.html From seth at icir.org Mon Sep 16 08:20:05 2013 From: seth at icir.org (Seth Hall) Date: Mon, 16 Sep 2013 11:20:05 -0400 Subject: [Bro] HTTP not being seen In-Reply-To: <003ACA3C-5E11-4D69-8323-E11123377D15@slave-tothe-box.net> References: <52334086.2000705@illinois.edu> <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> <003ACA3C-5E11-4D69-8323-E11123377D15@slave-tothe-box.net> Message-ID: <2360282D-0203-4239-A315-C4CEE60B1BDA@icir.org> On Sep 16, 2013, at 8:26 AM, James Lay wrote: > I completely agree. In my case however, I believe it's a "feature" of PPPoE?those same packets captured on the physical eth1 NIC are flawless with zero issues. I tried having Bro listen to eth1, but I don't think it plays with the PPPoE. Thanks for the input Seth. Hm, I think I have a patch around somewhere that adds PPPoE support to Bro? yep, found it. topic/seth/pppoe I also went ahead and updated that branch to be only diffs against the current master branch. Let me know if you need any help if you decide to give it a try. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130916/6de3d91d/attachment.bin From jlay at slave-tothe-box.net Mon Sep 16 08:22:31 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Mon, 16 Sep 2013 09:22:31 -0600 Subject: [Bro] HTTP not being seen In-Reply-To: <2360282D-0203-4239-A315-C4CEE60B1BDA@icir.org> References: "<52334086.2000705@illinois.edu>" <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> <003ACA3C-5E11-4D69-8323-E11123377D15@slave-tothe-box.net> <2360282D-0203-4239-A315-C4CEE60B1BDA@icir.org> Message-ID: <6f81c5362668dc2cf28d25b96358f8f2@localhost> On 2013-09-16 09:20, Seth Hall wrote: > On Sep 16, 2013, at 8:26 AM, James Lay > wrote: > >> I completely agree. In my case however, I believe it's a "feature" >> of PPPoE?those same packets captured on the physical eth1 NIC are >> flawless with zero issues. I tried having Bro listen to eth1, but I >> don't think it plays with the PPPoE. Thanks for the input Seth. > > > Hm, I think I have a patch around somewhere that adds PPPoE support > to Bro? yep, found it. > topic/seth/pppoe > > I also went ahead and updated that branch to be only diffs against > the current master branch. Let me know if you need any help if you > decide to give it a try. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ Oh sweet thanks Seth...I'll give that a go today and report my results. James From hlin33 at illinois.edu Mon Sep 16 08:32:04 2013 From: hlin33 at illinois.edu (Hui Lin (Hugo) ) Date: Mon, 16 Sep 2013 10:32:04 -0500 Subject: [Bro] Modbus protocol event handler for Bro In-Reply-To: References: Message-ID: Hi Karl, The DNP3 analyzer is included in the master branch of the Bro 2.2. So you can directly git bro source code, compile and build it. ( http://www.bro.org/development/contribute.html). You are good to use it. I think that Bro 2.2-beta binary will be available for download later this week. Best Hugo On Mon, Sep 16, 2013 at 10:26 AM, Karl Kamin wrote: > Hugo, > > I am interested in your work with DNP3. What is the svn site for that? > > Karl > > > From: Michael Haney > Date: Friday, March 29, 2013 12:49 PM > To: Hugo , "Slagell, Adam J" , > "bro-blue at bro.org" > Cc: Jason Staggs , "bro at bro.org" > Subject: Re: [Bro] Modbus protocol event handler for Bro > > I appreciate you guys getting back to me, and I can also tell you that > there is other interest in this out there based on off-list responses. > > So, yes, Dynamic Protocol Detection (DPD) is the acronym that didn't come > to me last night. I'm relatively new to Bro, and unfortunately only have a > cursory knowledge of the SCADA protocols I need to know about. But I do > have access to about a terabyte of recorded pcaps of a mix of things on a > production network, but most of it is SCADA related and primarily in > Modbus, RockLink, and a couple other vendor versions of protocols. But as I > said, non-standard ports so port triggering is probably not what I'm > looking for (or we're going to have a list of about 5000 different ports to > trigger on). > > As I understand it, there are a couple of methods to use for handling > protocol anomalies and detecting malicious activities. One is to look at > the protocol specs and see if you can identify a packet as valid or invalid > for that protocol. Invalid packets show signs of fuzzing, > misconfiguration, or malicious activity. Then there is identifying the > "normal" traffic. Like matching one response to one request. If you get > unsolicited "responses" for example, it may be a replay attack, > misconfiguration, or other malicious activity. > > Then the third (final?) step is to look at the actual legitimate traffic > that matches up with the protocol and everything, but determine if it's a > malicious attempt to extract system information or sabotage critical > infrastructure, by an insider for example, or by a malicious outsider who > knows how to talk Modbus but is sending commands that are outside of normal > operation. Lots of work has been done in that area to see if we can > validate normal operation parameters and out-of-norm activities. The > problem is, we can't automate it on this dataset unless we can first > identify an arbitrary packet as being a modbus packet or not. > > Have I got that breakdown right? Is this similar to what others are > looking for? > > So I'm excited about being a brogrammer and I think this is as good a > place for me to jump in as any. But I'm also very glad to hear more about > the work that has already been done (and I don't have to take on such a > task alone from scratch). I'd like to offer myself as your beta > tester/guinea pig. But there will be some minor hurdles to overcome with > data sharing. Technically, our research grant has expired (ended in > December) and I need to get permission from the corporate entity to > continue to use their data for our testing, as well as carve out time from > work and school duties to focus on this. But assuming all that works out, > I look forward to helping out with this development effort. > > Regards, > Michael > > On Fri, Mar 29, 2013 at 9:08 AM, Hugo wrote: > >> >> >> On Fri, Mar 29, 2013 at 1:53 AM, Michael Haney wrote: >> >>> I'm reviewing this paper and the related code for DNP3: >>> http://csiir.ornl.gov/csiirw/12/BPAwards/csiirw8Submission7.pdf >>> >>> But I have a network I'm analyzing that has modbus over tcp and has >>> implemented things in a somewhat unorthodox way. They've used port >>> assignments as a means of categorizing subsets of systems, and a bit of >>> security by obscurity. So nothing is on the standard port 502. It's all >>> over the place on ranges of ports from 2100 to 9900. >>> >>> Enter Bro and it's acclaimed ability to recognize protocols not by port >>> number but by semantics of the payload. >>> >> >> Just FYI, Bro has three ways to activate a new analyzer. >> http://www.bro.org/development/dpd.html >> >> Analyzers can use one of three ways to be fed new connections: >> >> - Use a preconfigured set of ports, thus triggering on all >> connections using any of the registered ports. >> - Use content signatures, thus triggering on all connections that >> match the relevant signatures. >> - Hard-code to trigger on all connections, when signatures won?t cut >> it and the protocol uses arbitrary ports. This should be avoided whenever >> possible obviously. >> >> >> >>> >>> But has anyone done this for modbus yet? Anyone interested to use it if >>> I start working on it? (read: volunteer beta tester and guinea pig). >>> >> >> I believe that Modbus analyzer is already included in Bro master branch >> for a while now. So you can directly use it. You are more than welcome to >> use it. >> >> >>> >>> What about other ICS/SCADA protocols? >>> >> >> I am mainly responsible for the DNP3 analyzer. Due to some legacy issues, >> I spent a comparatively long period to include it in Bro. I actually >> finished almost everything just yesterday (still working on some comments >> to people who want to understand my code). If you want to use it, please >> let me know, I can direct you to download from one of the branch that we >> about to include in Master soon. >> >> >> Best, >> >> Hugo >> >> > -- Hui Lin PhD Candidate, Research Assistant Electrical and Computer Engineering Department University of Illinois at Urbana-Champaign -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130916/1e0dad0f/attachment.html From jlay at slave-tothe-box.net Mon Sep 16 08:53:45 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Mon, 16 Sep 2013 09:53:45 -0600 Subject: [Bro] Bro with PPPoE [was Re: HTTP not being seen] In-Reply-To: <2360282D-0203-4239-A315-C4CEE60B1BDA@icir.org> References: "<52334086.2000705@illinois.edu>" <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> <003ACA3C-5E11-4D69-8323-E11123377D15@slave-tothe-box.net> <2360282D-0203-4239-A315-C4CEE60B1BDA@icir.org> Message-ID: On 2013-09-16 09:20, Seth Hall wrote: > On Sep 16, 2013, at 8:26 AM, James Lay > wrote: > >> I completely agree. In my case however, I believe it's a "feature" >> of PPPoE?those same packets captured on the physical eth1 NIC are >> flawless with zero issues. I tried having Bro listen to eth1, but I >> don't think it plays with the PPPoE. Thanks for the input Seth. > > > Hm, I think I have a patch around somewhere that adds PPPoE support > to Bro? yep, found it. > topic/seth/pppoe > > I also went ahead and updated that branch to be only diffs against > the current master branch. Let me know if you need any help if you > decide to give it a try. > > .Seth Hi Seth, So...I've gone to: https://github.com/bro/bro/tree/topic/seth/pppoe and downloaded to a .zip file. Configure fails at: Build Directory : build Source Directory: /home/newbro/bro-topic-seth-pppoe -- The C compiler identification is GNU -- The CXX compiler identification is GNU -- Check for working C compiler: /usr/bin/gcc -- Check for working C compiler: /usr/bin/gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done CMake Error at CMakeLists.txt:3 (include): include could not find load file: cmake/CommonCMakeConfig.cmake CMake Error at CMakeLists.txt:43 (include): include could not find load file: FindRequiredPackage -- Found sed: /bin/sed CMake Error at CMakeLists.txt:55 (FindRequiredPackage): Unknown CMake command "FindRequiredPackage". -- Configuring incomplete, errors occurred! Something tells me I'm doing this wrong...I also cloned https://github.com/bro/bro.git and I see the same thing. Any help would be great..thank you. James From seth at icir.org Mon Sep 16 09:07:33 2013 From: seth at icir.org (Seth Hall) Date: Mon, 16 Sep 2013 12:07:33 -0400 Subject: [Bro] Bro with PPPoE [was Re: HTTP not being seen] In-Reply-To: References: "<52334086.2000705@illinois.edu>" <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> <003ACA3C-5E11-4D69-8323-E11123377D15@slave-tothe-box.net> <2360282D-0203-4239-A315-C4CEE60B1BDA@icir.org> Message-ID: <991225EE-7DBF-407F-A561-8ECF0FE01067@icir.org> On Sep 16, 2013, at 11:53 AM, James Lay wrote: > https://github.com/bro/bro/tree/topic/seth/pppoe > > and downloaded to a .zip file. Configure fails at: I wonder if we can turn off that downloading thingy since it doesn't seem to work with submodules.. Try this? git clone --recursive git://git.bro.org/bro git checkout topic/seth/pppoe ./configure make .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130916/f097ba3e/attachment.bin From jlay at slave-tothe-box.net Mon Sep 16 09:29:12 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Mon, 16 Sep 2013 10:29:12 -0600 Subject: [Bro] Bro with PPPoE [was Re: HTTP not being seen] In-Reply-To: <991225EE-7DBF-407F-A561-8ECF0FE01067@icir.org> References: "<52334086.2000705@illinois.edu>" <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> <003ACA3C-5E11-4D69-8323-E11123377D15@slave-tothe-box.net> <2360282D-0203-4239-A315-C4CEE60B1BDA@icir.org> <991225EE-7DBF-407F-A561-8ECF0FE01067@icir.org> Message-ID: <8aa1d80f0821a940b27b93dfe1250357@localhost> On 2013-09-16 10:07, Seth Hall wrote: > On Sep 16, 2013, at 11:53 AM, James Lay > wrote: >> https://github.com/bro/bro/tree/topic/seth/pppoe >> >> and downloaded to a .zip file. Configure fails at: > > > I wonder if we can turn off that downloading thingy since it doesn't > seem to work with submodules.. > > Try this? > > git clone --recursive git://git.bro.org/bro > git checkout topic/seth/pppoe > ./configure > make > > .Seth Got it built, so it's a start. However I'm greeted with: ==== stderr.log fatal error in /usr/local/bro/share/bro/policy/frameworks/software/vulnerable.bro, line 41: BroType::AsRecordType (table/record) (set[record { min:record { major:count; minor:count; minor2:count; minor3:count; addl:string; }; max:record { major:count; minor:count; minor2:count; minor3:count; addl:string; }; }]) my exact process was: git clone --recursive git://git.bro.org/bro cd bro git checkout topic/seth/pppoe ./configure make sudo make install sudo broctl install start Thanks Seth. James From seth at icir.org Mon Sep 16 09:52:07 2013 From: seth at icir.org (Seth Hall) Date: Mon, 16 Sep 2013 12:52:07 -0400 Subject: [Bro] Bro with PPPoE [was Re: HTTP not being seen] In-Reply-To: <8aa1d80f0821a940b27b93dfe1250357@localhost> References: "<52334086.2000705@illinois.edu>" <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> <003ACA3C-5E11-4D69-8323-E11123377D15@slave-tothe-box.net> <2360282D-0203-4239-A315-C4CEE60B1BDA@icir.org> <991225EE-7DBF-407F-A561-8ECF0FE01067@icir.org> <8aa1d80f0821a940b27b93dfe1250357@localhost> Message-ID: On Sep 16, 2013, at 12:29 PM, James Lay wrote: > /usr/local/bro/share/bro/policy/frameworks/software/vulnerable.bro, line > 41: BroType::AsRecordType (table/record) (set[record { min:record { > major:count; minor:count; minor2:count; minor3:count; addl:string; }; > max:record { major:count; minor:count; minor2:count; minor3:count; > addl:string; }; }]) Your local.bro is probably using the old style for defining vulnerable software. You could just comment that out for now. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130916/f837ddb1/attachment.bin From jlay at slave-tothe-box.net Mon Sep 16 10:08:08 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Mon, 16 Sep 2013 11:08:08 -0600 Subject: [Bro] Bro with PPPoE [was Re: HTTP not being seen] In-Reply-To: References: "<52334086.2000705@illinois.edu>" <3D984197-7489-418E-83C8-3F6705DB3EED@gmail.com> <5FB7160E-C30D-49D5-AEBF-3C17887F2BB3@slave-tothe-box.net> <003ACA3C-5E11-4D69-8323-E11123377D15@slave-tothe-box.net> <2360282D-0203-4239-A315-C4CEE60B1BDA@icir.org> <991225EE-7DBF-407F-A561-8ECF0FE01067@icir.org> <8aa1d80f0821a940b27b93dfe1250357@localhost> Message-ID: On 2013-09-16 10:52, Seth Hall wrote: > On Sep 16, 2013, at 12:29 PM, James Lay > wrote: > >> /usr/local/bro/share/bro/policy/frameworks/software/vulnerable.bro, >> line >> 41: BroType::AsRecordType (table/record) (set[record { min:record { >> major:count; minor:count; minor2:count; minor3:count; addl:string; >> }; >> max:record { major:count; minor:count; minor2:count; minor3:count; >> addl:string; }; }]) > > > Your local.bro is probably using the old style for defining > vulnerable software. You could just comment that out for now. > > .Seth Thanks Seth...that did the trick. Additionally I had to comment out: @load protocols/http/detect-MHR redef CaptureLoss::watch_interval = 1 min; error in /usr/local/bro/share/bro/policy/protocols/http/detect-MHR.bro, line 22: no such field in record (HTTP::rec?$md5) error in /usr/local/bro/share/bro/policy/protocols/http/detect-MHR.bro, line 24: no such field in record (HTTP::rec$md5) error in /usr/local/bro/share/bro/policy/protocols/http/detect-MHR.bro, line 31: no such field in record (HTTP::rec$md5) error in /usr/local/bro/spool/installed-scripts-do-not-touch/site/local.bro, line 76: "redef" used but not previously defined (CaptureLoss::watch_interval) I saw just one of these: #types time string addr port addr port string string bool string 1379351003.903370 - - - - - non_ip_packet_in_pppoe_encapsulation - F bro so it's working good...this REALLY helps me out when my ISP flakes out and ppp0 drops/reconnects. Thanks Seth...also is there a spot that I can read about the "new" style for defines? Thanks again. James From david at mandelberg.org Mon Sep 16 15:49:50 2013 From: david at mandelberg.org (David Mandelberg) Date: Mon, 16 Sep 2013 18:49:50 -0400 Subject: [Bro] =?utf-8?q?network=5Ftime=28=29_on_a_non-worker_node?= Message-ID: <655801b78f34ecc1267bf6138d4253d8@mail.mandelberg.org> Hi, According to the documentation, network_time() "returns the timestamp of the last packet processed." So what does it do on a manager or proxy node? Is there loose synchronization of the timestamp? Does it return the wall-clock time? Is it undefined? -- David Eric Mandelberg / dseomn http://david.mandelberg.org/ From seth at icir.org Mon Sep 16 18:25:31 2013 From: seth at icir.org (Seth Hall) Date: Mon, 16 Sep 2013 21:25:31 -0400 Subject: [Bro] network_time() on a non-worker node In-Reply-To: <655801b78f34ecc1267bf6138d4253d8@mail.mandelberg.org> References: <655801b78f34ecc1267bf6138d4253d8@mail.mandelberg.org> Message-ID: <0EA90BB4-BBA4-4B90-AB02-529EC19AABE4@icir.org> On Sep 16, 2013, at 6:49 PM, David Mandelberg wrote: > According to the documentation, network_time() "returns the timestamp > of the last packet processed." So what does it do on a manager or proxy > node? When there is no traffic source it reverts to being driven by the wall clock. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130916/1fa02af2/attachment.bin From david at mandelberg.org Mon Sep 16 19:16:27 2013 From: david at mandelberg.org (David Mandelberg) Date: Mon, 16 Sep 2013 22:16:27 -0400 Subject: [Bro] network_time() on a non-worker node In-Reply-To: <0EA90BB4-BBA4-4B90-AB02-529EC19AABE4@icir.org> References: <655801b78f34ecc1267bf6138d4253d8@mail.mandelberg.org> <0EA90BB4-BBA4-4B90-AB02-529EC19AABE4@icir.org> Message-ID: <5237BB7B.6030304@mandelberg.org> On 09/16/2013 09:25 PM, Seth Hall wrote: > On Sep 16, 2013, at 6:49 PM, David Mandelberg wrote: > >> According to the documentation, network_time() "returns the timestamp >> of the last packet processed." So what does it do on a manager or proxy >> node? > > > When there is no traffic source it reverts to being driven by the wall clock. Thanks! -- David Eric Mandelberg / dseomn http://david.mandelberg.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130916/8364e6ed/attachment.bin From benson.mathews at gmail.com Tue Sep 17 13:55:57 2013 From: benson.mathews at gmail.com (Benson Mathews) Date: Tue, 17 Sep 2013 16:55:57 -0400 Subject: [Bro] SSH heuristic In-Reply-To: References: <52334086.2000705@illinois.edu> Message-ID: Makes sense. Thanks for the response! On Fri, Sep 13, 2013 at 1:47 PM, Alex Waher wrote: > Right, if the ssh connection's response bytes are above this threshold > it'll log the session as having a successful login. If you have a large > pre-auth login banner (usually for compliance reasons) it will very likely > report as a false positive. > > > On Fri, Sep 13, 2013 at 9:42 AM, Oehlert, Samuel wrote: > >> For the first part, that "5725" is the number of response bytes. The >> threshold for what is considered successful is: const >> authentication_data_size, which can be found in: >> /bro/share/bro/base/protocols/ssh and I believe it's default value was >> either 5000 or 5500 on 2.0. Though if you want to redef this you'll want to >> do it elsewhere such as your local.bro. Seth spent lots of time figuring >> out the best threshold, but it is still only a guess based on bytes, so >> there will be false positives. We find that many of them just around 5000 >> bytes were false positives. >> >> I don't know about your second part off the top of my head. >> >> -Sam >> >> >> >> >> On 9/13/13 10:28 AM, Benson Mathews wrote: >> >> I had a question about the SSH analyzer and how it determines a >> successful connection. I have a bro notification for a successful SSH login >> to a system on my network for a connection originating from outside US. Log >> entry: >> conn.log 2013-09-01 10:04:31 KIDipWlFWSi y.y.y.y 40014 >> x.x.x.x 22 tcp ssh 147.140508 1160 2377 S3 >> F 0 ShAadDf 21 2664 48 8221 >> ssh.log 2013-09-01 10:04:33 KIDipWlFWSi y.y.y.y 40014 >> x.x.x.x 22 success INBOUND SSH-2.0-libssh-0.2 >> SSH-2.0-OpenSSH_5.9 5725 CN - - - - >> >> The log entry is dated on the 1st and my client side logs have rolled >> over, so I can't valid this with syslog on the client. >> >> I wanted to check if there's a chance this could be a false positive. >> Also, the switch providing the feed itself is dropping packets and I see >> alerts for CaptureLoss::Too_Much_Loss and PacketFilter::Dropped_Packets. >> >> Would this loss cause Bro to misinterpret a brute forcing/scan attempt to >> a successful login? >> >> I'm running Bro 2.0. >> >> >> _______________________________________________ >> Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> >> >> >> -- >> Sam Oehlert >> Security Engineer >> NCSAsoehlert at illinois.edu(217)300-1076 >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130917/552e3853/attachment.html From itsecderek at gmail.com Wed Sep 18 12:35:04 2013 From: itsecderek at gmail.com (Derek Banks) Date: Wed, 18 Sep 2013 15:35:04 -0400 Subject: [Bro] question about emailing hits from the intel framework Message-ID: I am still learning Bro, so I apologize if this is a noob question. Is there a way to get Bro to email hits from the intel framework? I have Bro emailing me, and am able to get test domains to fire and populate the intel.log. I tried: redef Notice::emailed_types += { Intel::DOMAIN, }; But I suppose that the intel hits are outside of the notice framework? my BroFu is not strong enough to figure this out myself. Regards, Derek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130918/8fceecce/attachment.html From hhoffman at ip-solutions.net Fri Sep 20 05:58:30 2013 From: hhoffman at ip-solutions.net (Harry Hoffman) Date: Fri, 20 Sep 2013 08:58:30 -0400 Subject: [Bro] Summary Reports and service listing Message-ID: <523C4676.5010801@ip-solutions.net> Hi All, The summary reports that are emailed hourly contain service listings (e.g. port 80 HTTP). Are there processors that match the service to the port based upon packets seen or is this just based off of /etc/services or the like? I ask as syslog is being noted on port 514 but not being noted as syslog. Cheers, Harry From jlay at slave-tothe-box.net Fri Sep 20 06:13:42 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 20 Sep 2013 07:13:42 -0600 Subject: [Bro] broctl and filtering Message-ID: <1E937ADE-6DD8-4392-A8FD-636E93C6B1D3@slave-tothe-box.net> Hey all?trying to pass this: broargs = --filter not ip6 a few different ways, but I'm not having much luck: broargs = --filter not ip6 broargs = --filter 'not ip6' broargs = --filter "not ip6" ==== stderr.log error: can't open ip6 or ==== stderr.log /usr/local/bro/share/broctl/scripts/broctl-config.sh: line 67: ip6: command not found error: can't open ip6 ==== .cmdline -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto --filter not ip6 Any hints on how to pass the filter the right way? Thanks all. James -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130920/036f0c85/attachment.bin From vladg at cmu.edu Fri Sep 20 06:39:55 2013 From: vladg at cmu.edu (Vlad Grigorescu) Date: Fri, 20 Sep 2013 13:39:55 +0000 Subject: [Bro] Summary Reports and service listing In-Reply-To: <961_1379682781_r8KDD06l007080_523C4676.5010801@ip-solutions.net> References: <961_1379682781_r8KDD06l007080_523C4676.5010801@ip-solutions.net> Message-ID: <1202BE242E080642B0CD0AD0A03E8552E35E09@PGH-MSGMB-04.andrew.ad.cmu.edu> The service field in conn.log (and in those summary reports) is the result of Bro's dynamic protocol detection (DPD). One of Bro's protocol analyzers has confirmed that the traffic seen is that protocol. If you look in conn.log, you should be able to see if the 514 traffic is TCP or UDP. Bro doesn't have a TCP syslog analyzer right now. If it's UDP, something is sending malformed syslog. --Vlad On Sep 20, 2013, at 8:58 AM, Harry Hoffman wrote: > Hi All, > > The summary reports that are emailed hourly contain service listings > (e.g. port 80 HTTP). > > Are there processors that match the service to the port based upon > packets seen or is this just based off of /etc/services or the like? > > I ask as syslog is being noted on port 514 but not being noted as syslog. > > Cheers, > Harry > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From hhoffman at ip-solutions.net Fri Sep 20 06:48:45 2013 From: hhoffman at ip-solutions.net (Harry Hoffman) Date: Fri, 20 Sep 2013 09:48:45 -0400 Subject: [Bro] Summary Reports and service listing In-Reply-To: <1202BE242E080642B0CD0AD0A03E8552E35E09@PGH-MSGMB-04.andrew.ad.cmu.edu> References: <961_1379682781_r8KDD06l007080_523C4676.5010801@ip-solutions.net> <1202BE242E080642B0CD0AD0A03E8552E35E09@PGH-MSGMB-04.andrew.ad.cmu.edu> Message-ID: <523C523D.7080905@ip-solutions.net> Hey Vlad, It's udp and the traffic is syslog. I have rsyslog listening on my network and traffic is making it from the network devices to my syslog server so I don't think it's malformed. Bro even has the syslog files in its log directory which is why I thought it odd that things weren't being reported correctly. Thoughts on where to look next? Cheers, Harry On 09/20/2013 09:39 AM, Vlad Grigorescu wrote: > The service field in conn.log (and in those summary reports) is the result of Bro's dynamic protocol detection (DPD). One of Bro's protocol analyzers has confirmed that the traffic seen is that protocol. > > If you look in conn.log, you should be able to see if the 514 traffic is TCP or UDP. Bro doesn't have a TCP syslog analyzer right now. If it's UDP, something is sending malformed syslog. > > --Vlad > > On Sep 20, 2013, at 8:58 AM, Harry Hoffman wrote: > >> Hi All, >> >> The summary reports that are emailed hourly contain service listings >> (e.g. port 80 HTTP). >> >> Are there processors that match the service to the port based upon >> packets seen or is this just based off of /etc/services or the like? >> >> I ask as syslog is being noted on port 514 but not being noted as syslog. >> >> Cheers, >> Harry >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > From dnthayer at illinois.edu Fri Sep 20 13:39:29 2013 From: dnthayer at illinois.edu (Daniel Thayer) Date: Fri, 20 Sep 2013 15:39:29 -0500 Subject: [Bro] broctl and filtering In-Reply-To: <1E937ADE-6DD8-4392-A8FD-636E93C6B1D3@slave-tothe-box.net> References: <1E937ADE-6DD8-4392-A8FD-636E93C6B1D3@slave-tothe-box.net> Message-ID: <523CB281.7090808@illinois.edu> On 09/20/2013 08:13 AM, James Lay wrote: > Hey all?trying to pass this: > > broargs = --filter not ip6 > > a few different ways, but I'm not having much luck: > > broargs = --filter not ip6 > broargs = --filter 'not ip6' > broargs = --filter "not ip6" > > ==== stderr.log > error: can't open ip6 > > or > > ==== stderr.log > /usr/local/bro/share/broctl/scripts/broctl-config.sh: line 67: ip6: command not found > error: can't open ip6 > > ==== .cmdline > -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local -p bro local.bro broctl broctl/standalone broctl/auto --filter not ip6 > > Any hints on how to pass the filter the right way? Thanks all. > > James You need to quote the value of the --filter option like this: broargs = --filter 'not ip6' There was a bug that was preventing this from working. If you look in /share/broctl/scripts ( is your Bro install prefix), you can patch the following scripts: --- check-config.orig +++ check-config @@ -25,7 +25,7 @@ export PATH=${bindir}:${scriptsdir}:$PATH echo $@ >.cmdline touch .checking -${bro} $@ +${bro} "$@" rc=$? if [ "$rc" == "0" ] && [ "$print_scripts" == "1" ]; then --- helpers/start.orig +++ helpers/start @@ -9,7 +9,7 @@ shift rm -f .pid -nohup ${scriptsdir}/run-bro $@ >stdout.log 2>stderr.log & +nohup ${scriptsdir}/run-bro "$@" >stdout.log 2>stderr.log & while [ ! -s .pid ]; do sleep 1 --- run-bro.orig +++ run-bro @@ -71,9 +71,9 @@ if [ -n "${pin_command}" -a $pin_cpu -ge 0 ]; then fi fi - nohup ${pin_command} $pin_cpu $mybro $@ & + nohup ${pin_command} $pin_cpu $mybro "$@" & else - nohup $mybro $@ & + nohup $mybro "$@" & fi child=$! --- run-bro-on-trace.orig +++ run-bro-on-trace @@ -20,5 +20,5 @@ cd $cwd echo $@ >.cmdline touch .testing -${bro} -r $trace $@ +${bro} -r $trace "$@" From jlay at slave-tothe-box.net Fri Sep 20 14:12:53 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 20 Sep 2013 15:12:53 -0600 Subject: [Bro] broctl and filtering In-Reply-To: <523CB281.7090808@illinois.edu> References: <1E937ADE-6DD8-4392-A8FD-636E93C6B1D3@slave-tothe-box.net> <523CB281.7090808@illinois.edu> Message-ID: On 2013-09-20 14:39, Daniel Thayer wrote: > On 09/20/2013 08:13 AM, James Lay wrote: >> Hey all?trying to pass this: >> >> broargs = --filter not ip6 >> >> a few different ways, but I'm not having much luck: >> >> broargs = --filter not ip6 >> broargs = --filter 'not ip6' >> broargs = --filter "not ip6" >> >> ==== stderr.log >> error: can't open ip6 >> >> or >> >> ==== stderr.log >> /usr/local/bro/share/broctl/scripts/broctl-config.sh: line 67: ip6: >> command not found >> error: can't open ip6 >> >> ==== .cmdline >> -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local >> -p bro local.bro broctl broctl/standalone broctl/auto --filter not ip6 >> >> Any hints on how to pass the filter the right way? Thanks all. >> >> James > > You need to quote the value of the --filter option like this: > > broargs = --filter 'not ip6' > > There was a bug that was preventing this from working. > If you look in /share/broctl/scripts ( > is your Bro install prefix), you can patch the following > scripts: > Wow thanks a TON Daniel....patched up and good to go :) James From JAzoff at albany.edu Fri Sep 20 14:19:41 2013 From: JAzoff at albany.edu (Justin Azoff) Date: Fri, 20 Sep 2013 17:19:41 -0400 Subject: [Bro] broctl and filtering In-Reply-To: <1E937ADE-6DD8-4392-A8FD-636E93C6B1D3@slave-tothe-box.net> References: <1E937ADE-6DD8-4392-A8FD-636E93C6B1D3@slave-tothe-box.net> Message-ID: <20130920211941.GN4043@datacomm.albany.edu> On Fri, Sep 20, 2013 at 07:13:42AM -0600, James Lay wrote: > Hey all?trying to pass this: > > broargs = --filter not ip6 > Any hints on how to pass the filter the right way? Thanks all. in your local.bro, something like this: redef PacketFilter::all_packets = F; # don't capture all packets redef capture_filters = [[ "all"] = "not ip6"]; -- -- Justin Azoff -- Network Security & Performance Analyst From jlay at slave-tothe-box.net Fri Sep 20 14:23:29 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Fri, 20 Sep 2013 15:23:29 -0600 Subject: [Bro] broctl and filtering In-Reply-To: <20130920211941.GN4043@datacomm.albany.edu> References: <1E937ADE-6DD8-4392-A8FD-636E93C6B1D3@slave-tothe-box.net> <20130920211941.GN4043@datacomm.albany.edu> Message-ID: <08ec13dae023b256d29435565b2ab49b@localhost> On 2013-09-20 15:19, Justin Azoff wrote: > On Fri, Sep 20, 2013 at 07:13:42AM -0600, James Lay wrote: >> Hey all?trying to pass this: >> >> broargs = --filter not ip6 >> Any hints on how to pass the filter the right way? Thanks all. > > in your local.bro, something like this: > > redef PacketFilter::all_packets = F; # don't capture all packets > redef capture_filters = [[ "all"] = "not ip6"]; Thanks Justin...I had not thought of that. James From r.fulton at auckland.ac.nz Sun Sep 22 15:15:44 2013 From: r.fulton at auckland.ac.nz (Russell Fulton) Date: Mon, 23 Sep 2013 10:15:44 +1200 Subject: [Bro] bro won't start -- I've broken something... Message-ID: HI folks I ran out of disk last week and after cleaning up I find that bro won't start. At first "broctl start" would hang after the 'starting manager?' you had to kill the processes. I then reinstalled bro (keeping my config) and now I get: sensors at secmontst01:~$ sudo broctl start starting manager ... manager terminated immediately after starting; check output with "diag" so I did that: sensors at secmontst01:~$ sudo broctl diag [manager] No gdb installed. ==== No reporter.log ==== stderr.log error in /opt/bro/share/bro/base/frameworks/cluster/__load__.bro, line 16: can't open cluster-layout ==== stdout.log unlimited unlimited unlimited ==== .cmdline -U .status -p broctl -p broctl-live -p local -p manager local.bro broctl base/frameworks/cluster local-manager.bro broctl/auto ==== .env_vars PATH=/opt/bro/bin:/opt/bro/share/broctl/scripts:/sbin:/bin:/usr/sbin:/usr/bin BROPATH=/var/opt/bro/spool/installed-scripts-do-not-touch/site::/var/opt/bro/spool/installed-scripts-do-not-touch/auto:/opt/bro/share/bro:/opt/bro/share/bro/policy:/opt/bro/share/bro/site CLUSTER_NODE=manager broctl.cfg: ## Global BroControl configuration file. # Recipient address for all emails send out by Bro and BroControl. MailTo = r.fulton at auckland.ac.nz # Site-specific policy script to load. Bro will look for this in # $PREFIX/share/bro/site. A default local.bro comes preinstalled # and can be customized as desired. SitePolicyStandalone = local.bro # Location of other configuration files that can be used to customize # BroControl operation (e.g. local networks, nodes). CfgDir = /opt/bro/etc # Location of the spool directory where files and data that are currently being # written are stored. SpoolDir = /home/sensors/data/test1/bro-spool # Location of the log directory. This is longer term storage for rotated logs. LogDir = /home/sensors/data/test1/bro-logs # Rotation interval in seconds for log files on manager/standalone node. LogRotationInterval = 3600 # Expiration interval for log files in LogDir. Files older than this many days # will be deleted upon running "broctl cron". # LogExpireInterval = 30 # Lower threshold for space available on the disk that holds SpoolDir. If less # space is available, BroControl starts sending out warning emails. MinDiskSpace = 5 # Logs debug information into spool/debug.log. Debug = 1 network.cfg: List of local networks in CIDR notation, optionally followed by a # descriptive tag. # For example, "10.0.0.0/8" or "fe80::/64" are valid prefixes. 172.24.0.0/16 UoA staff wireless 172.23.0.0/16 UoA student wireless 130.216.0.0/16 UoA wired 10.2.0.0/16 UoA resnet node.cfg: [manager] type=manager host=130.216.5.218 # [proxy-1] type=proxy host=130.216.5.218 # [worker-1] type=worker lb_method=pf_ring host=130.216.5.218 interface=eth2 [worker-2] type=worker lb_method=pf_ring host=130.216.5.218 interface=eth2 [worker-3] type=worker lb_method=pf_ring host=130.216.5.218 interface=eth2 [worker-4] type=worker lb_method=pf_ring host=130.216.5.218 interface=eth2 From r.fulton at auckland.ac.nz Sun Sep 22 15:41:00 2013 From: r.fulton at auckland.ac.nz (Russell Fulton) Date: Mon, 23 Sep 2013 10:41:00 +1200 Subject: [Bro] bro won't start -- I've broken something... In-Reply-To: References: Message-ID: THanks Justin! Doh! It had to be something simple! Time I retired ;) Russell On 23/09/2013, at 10:31 AM, "Azoff, Justin" wrote: > Could be simple... Have you ran broctl install? > > I would do a > > broctl stop > broctl cleanup --all > broctl install > broctl check > broctl start > > For good measure > > Russell Fulton wrote: > > > HI folks > > I ran out of disk last week and after cleaning up I find that bro won't start. At first "broctl start" would hang after the 'starting manager?' you had to kill the processes. > > I then reinstalled bro (keeping my config) and now I get: > > sensors at secmontst01:~$ sudo broctl start > starting manager ... > manager terminated immediately after starting; check output with "diag" > > so I did that: > > sensors at secmontst01:~$ sudo broctl diag > [manager] > No gdb installed. > > ==== No reporter.log > > ==== stderr.log > error in /opt/bro/share/bro/base/frameworks/cluster/__load__.bro, line 16: can't open cluster-layout > > ==== stdout.log > unlimited > unlimited > unlimited > > ==== .cmdline > -U .status -p broctl -p broctl-live -p local -p manager local.bro broctl base/frameworks/cluster local-manager.bro broctl/auto > > ==== .env_vars > PATH=/opt/bro/bin:/opt/bro/share/broctl/scripts:/sbin:/bin:/usr/sbin:/usr/bin > BROPATH=/var/opt/bro/spool/installed-scripts-do-not-touch/site::/var/opt/bro/spool/installed-scripts-do-not-touch/auto:/opt/bro/share/bro:/opt/bro/share/bro/policy:/opt/bro/share/bro/site > CLUSTER_NODE=manager > > broctl.cfg: > > ## Global BroControl configuration file. > > # Recipient address for all emails send out by Bro and BroControl. > MailTo = r.fulton at auckland.ac.nz > > # Site-specific policy script to load. Bro will look for this in > # $PREFIX/share/bro/site. A default local.bro comes preinstalled > # and can be customized as desired. > SitePolicyStandalone = local.bro > > # Location of other configuration files that can be used to customize > # BroControl operation (e.g. local networks, nodes). > CfgDir = /opt/bro/etc > > # Location of the spool directory where files and data that are currently being > # written are stored. > SpoolDir = /home/sensors/data/test1/bro-spool > > # Location of the log directory. This is longer term storage for rotated logs. > LogDir = /home/sensors/data/test1/bro-logs > > # Rotation interval in seconds for log files on manager/standalone node. > LogRotationInterval = 3600 > > # Expiration interval for log files in LogDir. Files older than this many days > # will be deleted upon running "broctl cron". > # LogExpireInterval = 30 > > # Lower threshold for space available on the disk that holds SpoolDir. If less > # space is available, BroControl starts sending out warning emails. > MinDiskSpace = 5 > > # Logs debug information into spool/debug.log. > Debug = 1 > > network.cfg: > > List of local networks in CIDR notation, optionally followed by a > # descriptive tag. > # For example, "10.0.0.0/8" or "fe80::/64" are valid prefixes. > > > 172.24.0.0/16 UoA staff wireless > > 172.23.0.0/16 UoA student wireless > > 130.216.0.0/16 UoA wired > > 10.2.0.0/16 UoA resnet > > node.cfg: > > > [manager] > type=manager > host=130.216.5.218 > # > [proxy-1] > type=proxy > host=130.216.5.218 > # > > [worker-1] > type=worker > lb_method=pf_ring > host=130.216.5.218 > interface=eth2 > > [worker-2] > type=worker > lb_method=pf_ring > host=130.216.5.218 > interface=eth2 > > [worker-3] > type=worker > lb_method=pf_ring > host=130.216.5.218 > interface=eth2 > > [worker-4] > type=worker > lb_method=pf_ring > host=130.216.5.218 > interface=eth2 > > > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > From r.fulton at auckland.ac.nz Sun Sep 22 21:14:02 2013 From: r.fulton at auckland.ac.nz (Russell Fulton) Date: Mon, 23 Sep 2013 16:14:02 +1200 Subject: [Bro] trying to puppetise bro -- status Message-ID: <63979469-7651-464C-B7D4-6F044CB65B32@auckland.ac.nz> The bald question is "Does broctl status return information via exit code?" I had a quick look at the code and could not figure it out. Background: I use puppet to maintain the configurations of my sensors, puppet will update config changes and restart jobs as appropriate. It also expects to check the status of services and will start them if they are not running. It expect a "status command" to return code ah la http://refspecs.linux-foundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/iniscrptact.html Russell From dnthayer at illinois.edu Sun Sep 22 22:30:53 2013 From: dnthayer at illinois.edu (Daniel Thayer) Date: Mon, 23 Sep 2013 00:30:53 -0500 Subject: [Bro] trying to puppetise bro -- status In-Reply-To: <63979469-7651-464C-B7D4-6F044CB65B32@auckland.ac.nz> References: <63979469-7651-464C-B7D4-6F044CB65B32@auckland.ac.nz> Message-ID: <523FD20D.7060609@illinois.edu> On 09/22/2013 11:14 PM, Russell Fulton wrote: > The bald question is "Does broctl status return information via exit code?" I had a quick look at the code and could not figure it out. > > Background: > > I use puppet to maintain the configurations of my sensors, puppet will update config changes and restart jobs as appropriate. It also expects to check the status of services and will start them if they are not running. It expect a "status command" to return code ah la http://refspecs.linux-foundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/iniscrptact.html > > Russell If you're using the Bro 2.1 release, then the answer is "no". This feature was added very recently (it's now in git master). The "broctl status" command will return 0 if all bro nodes are running, and non-zero otherwise. From dn1nj4 at gmail.com Mon Sep 23 02:29:35 2013 From: dn1nj4 at gmail.com (. .) Date: Mon, 23 Sep 2013 05:29:35 -0400 Subject: [Bro] smtp.log empty for long-running MTA capture Message-ID: Hi list, Is there a way to tell bro to generate log events (specifically smtp.log) from partial sessions? I have a system that periodically feeds packet capture files through bro in order to generate its log data. I recently discovered that much of my smtp traffic was not showing up in the smtp.log. The segment in question is doing long-running bulk email transfers, resulting in the capture file seldom having SYN or FIN flagged packets, only PUSH and ACK flags. (This is due to the capture file rotation time being shorter than the MTA sessions). Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130923/14ab778c/attachment.html From pachinko.tw at gmail.com Mon Sep 23 06:51:10 2013 From: pachinko.tw at gmail.com (Po-Ching Lin) Date: Mon, 23 Sep 2013 21:51:10 +0800 Subject: [Bro] log-append.bro Message-ID: <5240474E.7060706@gmail.com> Hi, I would like to append the logs in the current run to existing logs. There was log-append.bro which can change the default setting in earlier Bro. How could it be done in Bro 2.1? open_log_file is defined in init-bare.bro, which seems not loaded in init-default.bro, so changing open to open_for_append in its definition does not work. I run bro directly from the command line as "bro -r xxx.pcap". Any suggestion is appreciated. Po-Ching From martin at gta.ufrj.br Mon Sep 23 12:38:23 2013 From: martin at gta.ufrj.br (martin) Date: Mon, 23 Sep 2013 16:38:23 -0300 Subject: [Bro] signature match script Message-ID: <524098AF.7040207@gta.ufrj.br> Hi guys, I'm new on Bro scripts. I'm traying to test the most easiest signature match script. I have read the manual and some examples, but it is not working. So I tried the following: on signatura.sig: signature my-first-sig { ip-proto == tcp dst-port == 12345 payload /.*hello/ event "Signature Found" } on test_sig.bro (was added on local.bro): @load base/frameworks/notice @load base/frameworks/signatures/main @load-sigs ./signatura.sig #in the same directory event signature_match(state: signature_state, msg: string, data: string) { print fmt("Match!!! %s",msg); print fmt("%s",data); } event tcp_packet (c: connection, is_orig: bool, flags: string, seq: count, ack: count, len: count, payload: string) { print fmt ("%s",payload); } So I create a socket to connect the host and send messages just to test it. When i run the program bro -i eth0 local # the interface is the correct one I can see the payload of the tcp packets (the messages I'm sending) but I don't have any type of signature.log not even the print Im doing. What could be happening? Thanks -- Martin From martin at gta.ufrj.br Mon Sep 23 16:46:31 2013 From: martin at gta.ufrj.br (martin) Date: Mon, 23 Sep 2013 20:46:31 -0300 Subject: [Bro] signature match script In-Reply-To: <524098AF.7040207@gta.ufrj.br> References: <524098AF.7040207@gta.ufrj.br> Message-ID: <5240D2D7.4030208@gta.ufrj.br> Hi guys, I'm new on Bro scripts. I'm traying to test the most easiest signature match script. I have read the manual and some examples, but it is not working. So I tried the following: on signatura.sig: signature my-first-sig { ip-proto == tcp dst-port == 12345 payload /.*hello/ event "Signature Found" } on test_sig.bro (was added on local.bro): @load base/frameworks/notice @load base/frameworks/signatures/main @load-sigs ./signatura.sig #in the same directory event signature_match(state: signature_state, msg: string, data: string) { print fmt("Match!!! %s",msg); print fmt("%s",data); } event tcp_packet (c: connection, is_orig: bool, flags: string, seq: count, ack: count, len: count, payload: string) { print fmt ("%s",payload); } So I create a socket to connect the host and send messages just to test it. When i run the program bro -i eth0 local # the interface is the correct one I can see the payload of the tcp packets (the messages I'm sending) but I don't have any type of signature.log not even the print Im doing. What could be happening? Thanks -- Martin From sconzo at visiblerisk.com Mon Sep 23 18:20:09 2013 From: sconzo at visiblerisk.com (Mike Sconzo) Date: Mon, 23 Sep 2013 20:20:09 -0500 Subject: [Bro] using broccoli to send events to bro Message-ID: I've got a .bro file that looks like: module A; global f: file = open("wtf.txt"); redef Communication::nodes += { ["test"] = [$host = 127.0.0.1, $events = /test1/], }; event test1(a: string) { print f, "got here"; } and a python script that looks like: #! /usr/bin/python from broccoli import * bc = Connection("127.0.0.1:47761") bc.send("test1", "aaaaaaaaaaaaaaaa") bc.processInput(); print "done" I get the following log messages: 1379985413.067179 manager child - - - info [#10005/127.0.0.1:34609] accepted clear connection 1379985413.068412 manager parent - - - info [#10005/127.0.0.1:34609] added peer 1379985413.068412 manager parent - - - info [#10005/127.0.0.1:34609] peer connected 1379985413.068412 manager parent - - - info [#10005/127.0.0.1:34609] phase: version 1379985413.068412 manager script - - - info connection established 1379985413.068412 manager script - - - info requesting events matching /^?(test1)$?/ 1379985413.068412 manager script - - - info accepting state 1379985413.069943 manager parent - - - info [#10005/127.0.0.1:34609] phase: handshake 1379985413.270825 manager parent - - - info [#10005/127.0.0.1:34609] peer does not support 64bit PIDs; using compatibility mode 1379985413.270825 manager parent - - - info [#10005/127.0.0.1:34609] peer is a Broccoli 1379985413.270825 manager parent - - - info [#10005/127.0.0.1:34609] phase: running 1379985413.270825 manager script - - - info connection closed 1379985413.272093 manager parent - - - info [#10005/127.0.0.1:34609] peer disconnected 1379985413.273243 manager child - - - info [#10005/127.0.0.1:34609] connection closed 1379985413.851595 worker-1 child - - - info selects=3100000 canwrites=0 timeouts=3098508 1379985416.921436 manager child - - - info selects=3200000 canwrites=0 timeouts=3198460 1379985411.837037 proxy-1 child - - - info selects=3100000 canwrites=0 timeouts=30999 But nothing gets persisted to the "wtf.txt" file. I'm sure (I hope) I'm missing something super easy. Any ideas? Thanks! -- cat ~/.bash_history > documentation.txt From ppowell at 21ct.com Mon Sep 23 18:40:37 2013 From: ppowell at 21ct.com (Patrick Powell) Date: Mon, 23 Sep 2013 20:40:37 -0500 Subject: [Bro] signature match script In-Reply-To: <5240D2D7.4030208@gta.ufrj.br> Message-ID: Martin, The .sig file looks fine. I don't load those in local.bro, I load them in /base/frameworks/signatures/main.bro. I have my sigs in that directory as well. Patrick Powell Security Analyst 21CT, Inc. Direct: 512.682.4754 Cell: 210.587.9255 Direct Fax: 512.682.4749 On 9/23/13 6:46 PM, "martin" wrote: > > > >Hi guys, I'm new on Bro scripts. I'm traying to test the most easiest >signature match script. I have read the manual and some examples, but it >is not working. > >So I tried the following: >on signatura.sig: > >signature my-first-sig { >ip-proto == tcp >dst-port == 12345 >payload /.*hello/ >event "Signature Found" >} > >on test_sig.bro (was added on local.bro): > >@load base/frameworks/notice >@load base/frameworks/signatures/main >@load-sigs ./signatura.sig #in the same directory > >event signature_match(state: signature_state, msg: string, data: string) >{ >print fmt("Match!!! %s",msg); >print fmt("%s",data); >} > >event tcp_packet (c: connection, is_orig: bool, flags: string, seq: >count, ack: count, len: count, payload: string) >{ >print fmt ("%s",payload); >} > >So I create a socket to connect the host and send messages just to test >it. >When i run the program >bro -i eth0 local # the interface is the correct one >I can see the payload of the tcp packets (the messages I'm sending) but >I don't have any type of signature.log not even the print Im doing. >What could be happening? >Thanks > >-- Martin > > > >_______________________________________________ >Bro mailing list >bro at bro-ids.org >http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3799 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130923/846865c9/attachment.bin From martin at gta.ufrj.br Tue Sep 24 05:58:55 2013 From: martin at gta.ufrj.br (martin) Date: Tue, 24 Sep 2013 09:58:55 -0300 Subject: [Bro] signature match script In-Reply-To: References: Message-ID: <52418C8F.90606@gta.ufrj.br> IT must be something else, I tryied what you said Patrick but is still not working. May be something in configuration I should switch on? On 09/23/2013 10:40 PM, Patrick Powell wrote: > Martin, > The .sig file looks fine. I don't load those in local.bro, I load them in > /base/frameworks/signatures/main.bro. I have my sigs in that > directory as well. > > Patrick Powell > Security Analyst > 21CT, Inc. > Direct: 512.682.4754 > Cell: 210.587.9255 > Direct Fax: 512.682.4749 > > > > > On 9/23/13 6:46 PM, "martin" wrote: > >> >> >> Hi guys, I'm new on Bro scripts. I'm traying to test the most easiest >> signature match script. I have read the manual and some examples, but it >> is not working. >> >> So I tried the following: >> on signatura.sig: >> >> signature my-first-sig { >> ip-proto == tcp >> dst-port == 12345 >> payload /.*hello/ >> event "Signature Found" >> } >> >> on test_sig.bro (was added on local.bro): >> >> @load base/frameworks/notice >> @load base/frameworks/signatures/main >> @load-sigs ./signatura.sig #in the same directory >> >> event signature_match(state: signature_state, msg: string, data: string) >> { >> print fmt("Match!!! %s",msg); >> print fmt("%s",data); >> } >> >> event tcp_packet (c: connection, is_orig: bool, flags: string, seq: >> count, ack: count, len: count, payload: string) >> { >> print fmt ("%s",payload); >> } >> >> So I create a socket to connect the host and send messages just to test >> it. >> When i run the program >> bro -i eth0 local # the interface is the correct one >> I can see the payload of the tcp packets (the messages I'm sending) but >> I don't have any type of signature.log not even the print Im doing. >> What could be happening? >> Thanks >> >> -- Martin >> >> >> >> _______________________________________________ >> Bro mailing list >> bro at bro-ids.org >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- ------------------------------------------------------------------------ *Eng. Martin Andreoni * /Estudante de Mestrado/ /Grupo de Teleinform?tica e Automa??o (GTA)/ /Programa de Engenharia El?trica (PEE)/ /Universidade Federal do Rio de Janeiro (UFRJ)/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130924/7176fb0c/attachment.html From jsiwek at illinois.edu Tue Sep 24 07:44:14 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Tue, 24 Sep 2013 14:44:14 +0000 Subject: [Bro] using broccoli to send events to bro In-Reply-To: References: Message-ID: <299E36E8-1F80-4337-98A4-AD375C31A980@illinois.edu> On Sep 23, 2013, at 8:20 PM, Mike Sconzo wrote: > #! /usr/bin/python > > from broccoli import * > > bc = Connection("127.0.0.1:47761") > bc.send("test1", "aaaaaaaaaaaaaaaa") > bc.processInput(); > print "done" processInput() doesn't guarantee that any events are actually processed. It returns True if the send-queue is non-empty so calling it in a loop to make sure the event is sent is probably what you want: while bc.processInput(): pass # or sleep or do other stuff - Jon From sconzo at visiblerisk.com Tue Sep 24 08:12:49 2013 From: sconzo at visiblerisk.com (Mike Sconzo) Date: Tue, 24 Sep 2013 10:12:49 -0500 Subject: [Bro] using broccoli to send events to bro In-Reply-To: <299E36E8-1F80-4337-98A4-AD375C31A980@illinois.edu> References: <299E36E8-1F80-4337-98A4-AD375C31A980@illinois.edu> Message-ID: Thanks that's good to know. I changed it to while bc.processInput(): sleep(2) and it still seems to exit immediately. Any additional thoughts? I'm pretty lost on this one. I'm using 2.1 with the broccoli.py included with it. Thanks again. On Tue, Sep 24, 2013 at 9:44 AM, Siwek, Jonathan Luke wrote: > > On Sep 23, 2013, at 8:20 PM, Mike Sconzo wrote: > >> #! /usr/bin/python >> >> from broccoli import * >> >> bc = Connection("127.0.0.1:47761") >> bc.send("test1", "aaaaaaaaaaaaaaaa") >> bc.processInput(); >> print "done" > > processInput() doesn't guarantee that any events are actually processed. It returns True if the send-queue is non-empty so calling it in a loop to make sure the event is sent is probably what you want: > > while bc.processInput(): > pass # or sleep or do other stuff > > - Jon -- cat ~/.bash_history > documentation.txt From jsiwek at illinois.edu Tue Sep 24 09:35:46 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Tue, 24 Sep 2013 16:35:46 +0000 Subject: [Bro] using broccoli to send events to bro In-Reply-To: References: <299E36E8-1F80-4337-98A4-AD375C31A980@illinois.edu> Message-ID: <644A28E9-7595-4C44-B6E3-5457367AB875@illinois.edu> > Any additional thoughts? I'm pretty lost on this one. I'm using 2.1 > with the broccoli.py included with it. If output to the "wtf.txt" file is buffered, you probably aren't going to see anything in there right away. Maybe not even until you terminate the bro process since there's so little data. You can put a regular print statement to stdout in the event handler in your bro script to verify you actually get events, but nothing has yet been written to disk. You could also have your python script send a whole bunch of events and hope you actually cause output to be flushed. - Jon From sconzo at visiblerisk.com Tue Sep 24 11:07:46 2013 From: sconzo at visiblerisk.com (Mike Sconzo) Date: Tue, 24 Sep 2013 13:07:46 -0500 Subject: [Bro] using broccoli to send events to bro In-Reply-To: <644A28E9-7595-4C44-B6E3-5457367AB875@illinois.edu> References: <299E36E8-1F80-4337-98A4-AD375C31A980@illinois.edu> <644A28E9-7595-4C44-B6E3-5457367AB875@illinois.edu> Message-ID: -1 for me. It was a buffering problem (sending 1000 events worked quite well). Thanks for the assistance, I feel "special". -=Mike On Tue, Sep 24, 2013 at 11:35 AM, Siwek, Jonathan Luke wrote: >> Any additional thoughts? I'm pretty lost on this one. I'm using 2.1 >> with the broccoli.py included with it. > > If output to the "wtf.txt" file is buffered, you probably aren't going to see anything in there right away. Maybe not even until you terminate the bro process since there's so little data. You can put a regular print statement to stdout in the event handler in your bro script to verify you actually get events, but nothing has yet been written to disk. You could also have your python script send a whole bunch of events and hope you actually cause output to be flushed. > > - Jon -- cat ~/.bash_history > documentation.txt From robin at icir.org Tue Sep 24 13:11:36 2013 From: robin at icir.org (Robin Sommer) Date: Tue, 24 Sep 2013 13:11:36 -0700 Subject: [Bro] Bro 2.2 Beta available Message-ID: <20130924201136.GT78116@icir.org> We have released a public beta version of Bro 2.2. See the blog posting for more: http://blog.bro.org/2013/09/bro-22-beta-available.html Please help us test so that we can work out any kinks that might be left. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From robin at icir.org Tue Sep 24 13:36:17 2013 From: robin at icir.org (Robin Sommer) Date: Tue, 24 Sep 2013 13:36:17 -0700 Subject: [Bro] using broccoli to send events to bro In-Reply-To: References: <299E36E8-1F80-4337-98A4-AD375C31A980@illinois.edu> Message-ID: <20130924203617.GU78116@icir.org> On Tue, Sep 24, 2013 at 10:12 -0500, Mike Sconzo wrote: > while bc.processInput(): > sleep(2) Actually I'm not sure if processInput() returns true if there's something in *send* queue (as opposed to the receive queue). There might still be a race condition there. Does it work with an infite loop: while True: bc.processInput(): sleep(1) Often the most reliable way to get such stuff working is sending an acknowledge event back and only terminating once that has been received. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From rjenkins at rmjconsulting.net Tue Sep 24 13:40:21 2013 From: rjenkins at rmjconsulting.net (Ron Jenkins) Date: Tue, 24 Sep 2013 20:40:21 +0000 Subject: [Bro] Bro 2.2 Beta available In-Reply-To: <20130924201136.GT78116@icir.org> References: <20130924201136.GT78116@icir.org> Message-ID: Very exciting! Thank you! Ron Jenkins (SnortCP, VCP (3/4), MCNE, CNE6, MCP,CCNA) RMJ Consulting, LLC. "Bringing Companies and Solutions Together" Owner / Senior Architect 11715 Bricksome Ave STE B-7 Baton Rouge, LA 70816 Toll: 855-448-5214 Direct. 225-448-5214 Fax. 225-448-5324 Cell. 225-931-1632 Email. rjenkins at rmjconsulting.net Web. http://www.rmjconsulting.net Log Siphon. http://www.logsiphon.com Linkedin. http://www.linkedin.com/profile/view?id=28564151&trk=tab_pro -----Original Message----- From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Robin Sommer Sent: Tuesday, September 24, 2013 3:12 PM To: bro at bro.org Subject: [Bro] Bro 2.2 Beta available We have released a public beta version of Bro 2.2. See the blog posting for more: http://blog.bro.org/2013/09/bro-22-beta-available.html Please help us test so that we can work out any kinks that might be left. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin _______________________________________________ Bro mailing list bro at bro-ids.org http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130924/39769e96/attachment.html From michal at rsbac.org Tue Sep 24 13:59:19 2013 From: michal at rsbac.org (Michal Purzynski) Date: Tue, 24 Sep 2013 22:59:19 +0200 Subject: [Bro] Bro 2.2 Beta available In-Reply-To: References: <20130924201136.GT78116@icir.org> Message-ID: <5241FD27.7020107@rsbac.org> Now that's a news of the month :) Does it contain the fix for a HUGE memory leak that I've seen a few weeks ago in the git version? On 9/24/13 10:40 PM, Ron Jenkins wrote: > > > -----Original Message----- > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of > Robin Sommer > Sent: Tuesday, September 24, 2013 3:12 PM > To: bro at bro.org > Subject: [Bro] Bro 2.2 Beta available > > We have released a public beta version of Bro 2.2. See the blog > posting for more: > > http://blog.bro.org/2013/09/bro-22-beta-available.html > > Please help us test so that we can work out any kinks that might be left. > > Robin > > -- > > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > > > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130924/7c02ef89/attachment.html From robin at icir.org Tue Sep 24 14:01:18 2013 From: robin at icir.org (Robin Sommer) Date: Tue, 24 Sep 2013 14:01:18 -0700 Subject: [Bro] log-append.bro In-Reply-To: <5240474E.7060706@gmail.com> References: <5240474E.7060706@gmail.com> Message-ID: <20130924210118.GW78116@icir.org> On Mon, Sep 23, 2013 at 21:51 +0800, Po-Ching Lin wrote: > I would like to append the logs in the current run to existing logs. Bro 2.x doesn't support that anymore as the logs no longer use the open*() functions but go through the internal logging system. You could work around it by definining a post-processor which takes the log file and copies/appends it to somewhere else. Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From liam at broala.com Tue Sep 24 14:26:02 2013 From: liam at broala.com (Liam Randall) Date: Tue, 24 Sep 2013 17:26:02 -0400 Subject: [Bro] Bro 2.2 Beta available In-Reply-To: <5241FD27.7020107@rsbac.org> References: <20130924201136.GT78116@icir.org> <5241FD27.7020107@rsbac.org> Message-ID: Yes. Most of the large sites now seem to be stable. A couple were new, a couple were old an only uncovered by new features. If you are still seeing problems please report them on list. Thanks, Liam Randall On Tue, Sep 24, 2013 at 4:59 PM, Michal Purzynski wrote: > Now that's a news of the month :) > > Does it contain the fix for a HUGE memory leak that I've seen a few weeks > ago in the git version? > > > On 9/24/13 10:40 PM, Ron Jenkins wrote: > > **** > > ** ** > > ** ** > > -----Original Message----- > From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] > On Behalf Of Robin Sommer > Sent: Tuesday, September 24, 2013 3:12 PM > To: bro at bro.org > Subject: [Bro] Bro 2.2 Beta available > > ** ** > > We have released a public beta version of Bro 2.2. See the blog posting > for more:**** > > ** ** > > http://blog.bro.org/2013/09/bro-22-beta-available.html**** > > ** ** > > Please help us test so that we can work out any kinks that might be left.* > *** > > ** ** > > Robin **** > > ** ** > > -- **** > > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org**** > > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin**** > > _______________________________________________**** > > Bro mailing list**** > > bro at bro-ids.org**** > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro**** > > > _______________________________________________ > Bro mailing listbro at bro-ids.orghttp://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Liam Randall Managing Partner 510-281-0760 www.Broala.com >From the creators of Bro -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130924/f29da1f8/attachment.html From robin at icir.org Tue Sep 24 14:28:06 2013 From: robin at icir.org (Robin Sommer) Date: Tue, 24 Sep 2013 14:28:06 -0700 Subject: [Bro] Bro 2.2 Beta available In-Reply-To: <5241FD27.7020107@rsbac.org> References: <20130924201136.GT78116@icir.org> <5241FD27.7020107@rsbac.org> Message-ID: <20130924212806.GZ78116@icir.org> On Tue, Sep 24, 2013 at 22:59 +0200, Michal Purzynski wrote: > Does it contain the fix for a HUGE memory leak that I've seen a few > weeks ago in the git version? >From what we hear memory seems stable now [1]. If you see something else, please let us know. Robin [1] The one possible exception is loading policy/misc/profiling.bro. It's unclear right now if there's still a leak with that script but it's mainly for debugging anyways and not used by default. -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jsiwek at illinois.edu Tue Sep 24 15:37:00 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Tue, 24 Sep 2013 22:37:00 +0000 Subject: [Bro] using broccoli to send events to bro In-Reply-To: <20130924203617.GU78116@icir.org> References: <299E36E8-1F80-4337-98A4-AD375C31A980@illinois.edu> <20130924203617.GU78116@icir.org> Message-ID: <1398C6B3-B187-4070-B74D-93D11C07F60C@illinois.edu> > Actually I'm not sure if processInput() returns true if there's > something in *send* queue (as opposed to the receive queue). If it doesn't, there's a couple code comments in broccoli/bindings that need fixing :) - Jon From anthony.kasza at gmail.com Tue Sep 24 21:19:36 2013 From: anthony.kasza at gmail.com (anthony kasza) Date: Tue, 24 Sep 2013 21:19:36 -0700 Subject: [Bro] Third Party Scripts Message-ID: Hi All, In an effort to assist other in locating example Bro scripts, I've cataloged all the Bro scripts I could find on a single page. < http://supbrosup.blogspot.com/2013/09/third-party-scripts-and-example-code.html > It's nice to know what all is out there. If I missed anything or if anyone has code they would like included in the list, please send me links. -AK -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130924/262763ca/attachment.html From jlay at slave-tothe-box.net Wed Sep 25 03:48:42 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 25 Sep 2013 04:48:42 -0600 Subject: [Bro] Third Party Scripts In-Reply-To: References: Message-ID: On Sep 24, 2013, at 10:19 PM, anthony kasza wrote: > Hi All, > > In an effort to assist other in locating example Bro scripts, I've cataloged all the Bro scripts I could find on a single page. > > > > It's nice to know what all is out there. If I missed anything or if anyone has code they would like included in the list, please send me links. > > -AK > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro Nicely done..thanks AK. James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130925/b2e1be52/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130925/b2e1be52/attachment.bin From seth at icir.org Wed Sep 25 06:17:54 2013 From: seth at icir.org (Seth Hall) Date: Wed, 25 Sep 2013 09:17:54 -0400 Subject: [Bro] Third Party Scripts In-Reply-To: References: Message-ID: <13F5F404-501B-4041-A7AB-20B66989013A@icir.org> On Sep 25, 2013, at 12:19 AM, anthony kasza wrote: > In an effort to assist other in locating example Bro scripts, I've cataloged all the Bro scripts I could find on a single page. Nice! Your list might be a very nice start to a list like this on the bro.org site somewhere. Thanks! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130925/80908830/attachment.bin From sconzo at visiblerisk.com Wed Sep 25 07:10:08 2013 From: sconzo at visiblerisk.com (Mike Sconzo) Date: Wed, 25 Sep 2013 09:10:08 -0500 Subject: [Bro] using broccoli to send events to bro In-Reply-To: <1398C6B3-B187-4070-B74D-93D11C07F60C@illinois.edu> References: <299E36E8-1F80-4337-98A4-AD375C31A980@illinois.edu> <20130924203617.GU78116@icir.org> <1398C6B3-B187-4070-B74D-93D11C07F60C@illinois.edu> Message-ID: >> Actually I'm not sure if processInput() returns true if there's >> something in *send* queue (as opposed to the receive queue). >If it doesn't, there's a couple code comments in broccoli/bindings that need fixing :) That was my confusion exactly. Regardless it seems to be working, I'm less concerned with termination of the python script and I'm entirely ok with waiting a while. I just wasn't getting any output and was confused. Thanks again. On Tue, Sep 24, 2013 at 5:37 PM, Siwek, Jonathan Luke wrote: > >> Actually I'm not sure if processInput() returns true if there's >> something in *send* queue (as opposed to the receive queue). > > If it doesn't, there's a couple code comments in broccoli/bindings that need fixing :) > > - Jon -- cat ~/.bash_history > documentation.txt From jlay at slave-tothe-box.net Wed Sep 25 07:22:19 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 25 Sep 2013 08:22:19 -0600 Subject: [Bro] Couple script questions Message-ID: <83768b2abec16f1f15332765de15014f@localhost> Stemming from the previous posting, I'd like to try the below scripts: https://github.com/sethhall/ssn-exposure https://github.com/sethhall/credit-card-exposure From the ssn-exposure I see the install info: cd /share/bro/site/ git clone git://github.com/sethhall/ssn-exposure.git echo "@load ssn-exposure" >> local.bro I'm assuming this is the same for credit-card-exposure as well? Also, I see: Detect US Social Security Numbers with Bro. This script only works with Bro 2.1. Is this still the case? I'm running 2.2 beta...thanks for any assistance. James From seth at icir.org Wed Sep 25 08:35:59 2013 From: seth at icir.org (Seth Hall) Date: Wed, 25 Sep 2013 11:35:59 -0400 Subject: [Bro] Couple script questions In-Reply-To: <83768b2abec16f1f15332765de15014f@localhost> References: <83768b2abec16f1f15332765de15014f@localhost> Message-ID: <53CBD217-DD29-4711-9284-8281239C3632@icir.org> On Sep 25, 2013, at 10:22 AM, James Lay wrote: > cd /share/bro/site/ > git clone git://github.com/sethhall/ssn-exposure.git > echo "@load ssn-exposure" >> local.bro Make sure you also follow the configuration example. That script won't work without it. https://github.com/sethhall/ssn-exposure > I'm assuming this is the same for credit-card-exposure as well? Huh, I was sure I added a README to that repository but I guess not. Yes, it's pretty much the same installation though. > Is this still the case? I'm running 2.2 beta...thanks for any > assistance. I just updated it, it should work with 2.2 too. :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130925/0569f0b4/attachment.bin From seth at icir.org Wed Sep 25 08:37:55 2013 From: seth at icir.org (Seth Hall) Date: Wed, 25 Sep 2013 11:37:55 -0400 Subject: [Bro] Couple script questions In-Reply-To: <83768b2abec16f1f15332765de15014f@localhost> References: <83768b2abec16f1f15332765de15014f@localhost> Message-ID: <3C3F0BE1-2D69-428C-9D34-86280E1A3F16@icir.org> On Sep 25, 2013, at 10:22 AM, James Lay wrote: > Is this still the case? I'm running 2.2 beta...thanks for any > assistance. Hrm, now that I go back and look at those scripts, I'm not sure they will work as they are. I'll try to come back around soon and fix them. Maybe I'll actually add some tests! :) .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130925/68e36699/attachment.bin From jlay at slave-tothe-box.net Wed Sep 25 09:00:53 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Wed, 25 Sep 2013 10:00:53 -0600 Subject: [Bro] Couple script questions In-Reply-To: <3C3F0BE1-2D69-428C-9D34-86280E1A3F16@icir.org> References: <83768b2abec16f1f15332765de15014f@localhost> <3C3F0BE1-2D69-428C-9D34-86280E1A3F16@icir.org> Message-ID: On 2013-09-25 09:37, Seth Hall wrote: > On Sep 25, 2013, at 10:22 AM, James Lay > wrote: > >> Is this still the case? I'm running 2.2 beta...thanks for any >> assistance. > > > Hrm, now that I go back and look at those scripts, I'm not sure they > will work as they are. I'll try to come back around soon and fix > them. Maybe I'll actually add some tests! :) > > .Seth Thanks Seth. And for those that may need it: [$state="Alabama", $low=416, $high=424], [$state="Alaska", $low=574, $high=574], [$state="Arizona1", $low=526, $high=527], [$state="Arizona2", $low=600, $high=601], [$state="Arizona3", $low=764, $high=765], [$state="Arkansas1", $low=429, $high=432], [$state="Arkansas2", $low=676, $high=679], [$state="California1", $low=545, $high=573], [$state="California2", $low=602, $high=626], [$state="Colorado1", $low=521, $high=524], [$state="Colorado2", $low=650, $high=653], [$state="Connecticut", $low=040, $high=049], [$state="Delaware", $low=221, $high=222], [$state="District of Columbia", $low=577, $high=579], [$state="Florida1", $low=261, $high=267], [$state="Florida2", $low=589, $high=595], [$state="Florida3", $low=765, $high=772], [$state="Georgia1", $low=252, $high=260], [$state="Georgia2", $low=667, $high=675], [$state="Guam American Samoa Philippines", $low=586, $high=586], [$state="Hawaii1", $low=575, $high=576], [$state="Hawaii2", $low=750, $high=751], [$state="Idaho", $low=518, $high=519], [$state="Illinois", $low=318, $high=361], [$state="Indiana", $low=303, $high=317], [$state="Iowa", $low=478, $high=485], [$state="Kansas", $low=508, $high=515], [$state="Kentucky", $low=400, $high=407], [$state="Louisiana1", $low=433, $high=439], [$state="Louisiana2", $low=659, $high=665], [$state="Maine", $low=004, $high=007], [$state="Maryland", $low=212, $high=220], [$state="Massachusetts", $low=010, $high=034], [$state="Michigan", $low=362, $high=386], [$state="Minnesota", $low=468, $high=477], [$state="Mississippi1", $low=425, $high=428], [$state="Mississippi2", $low=587, $high=588], [$state="Mississippi3", $low=752, $high=755], [$state="Missouri", $low=486, $high=500], [$state="Montana", $low=516, $high=517], [$state="Nebraska", $low=505, $high=508], [$state="Nevada", $low=530, $high=680], [$state="New Hampshire", $low=001, $high=003], [$state="New Jersey", $low=135, $high=158], [$state="New Mexico1", $low=525, $high=585], [$state="New Mexico2", $low=648, $high=649], [$state="New York", $low=050, $high=134], [$state="North Carolina1", $low=237, $high=246], [$state="North Carolina2", $low=681, $high=690], [$state="North Dakota", $low=501, $high=502], [$state="Ohio", $low=268, $high=302], [$state="Oklahoma", $low=440, $high=448], [$state="Oregon", $low=540, $high=544], [$state="Pennsylvania", $low=159, $high=211], [$state="Puerto Rico", $low=596, $high=599], [$state="Rhode Island", $low=035, $high=039], [$state="South Carolina1", $low=247, $high=251], [$state="South Carolina2", $low=654, $high=658], [$state="South Dakota", $low=503, $high=504], [$state="Tennessee1", $low=408, $high=415], [$state="Tennessee2", $low=756, $high=763], [$state="Texas1", $low=449, $high=467], [$state="Texas2", $low=627, $high=647], [$state="Utah", $low=528, $high=529], [$state="Vermont", $low=008, $high=009], [$state="Virginia1", $low=223, $high=231], [$state="Virginia2", $low=691, $high=699], [$state="Virgin Islands", $low=580, $high=580], [$state="Washington", $low=531, $high=539], [$state="West Virginia", $low=233, $high=236], [$state="West Virginia North Carolina", $low=232, $high=232], [$state="Wisconsin", $low=387, $high=399], [$state="Wyoming", $low=520, $high=520], I made a leap of faith and assumed that states with only one prefix low and high were assigned the same number. James From vladg at cmu.edu Wed Sep 25 09:22:12 2013 From: vladg at cmu.edu (Vlad Grigorescu) Date: Wed, 25 Sep 2013 16:22:12 +0000 Subject: [Bro] Third Party Scripts In-Reply-To: <15851_1380083253_r8P4RVGq015995_CAEZw2bxJYZ-Mb7OCO0QtzN1c6pmK6j0CN5D2ZsT4RtjmEu6-tw@mail.gmail.com> References: <15851_1380083253_r8P4RVGq015995_CAEZw2bxJYZ-Mb7OCO0QtzN1c6pmK6j0CN5D2ZsT4RtjmEu6-tw@mail.gmail.com> Message-ID: <1202BE242E080642B0CD0AD0A03E8552E53B45@PGH-MSGMB-03.andrew.ad.cmu.edu> Awesome. Just FYI, you can also search GitHub for repos containing Bro code: https://github.com/search?q=bro&type=Repositories&ref=advsearch&l=Bro --Vlad On Sep 25, 2013, at 12:19 AM, anthony kasza wrote: > Hi All, > > In an effort to assist other in locating example Bro scripts, I've cataloged all the Bro scripts I could find on a single page. > > > > It's nice to know what all is out there. If I missed anything or if anyone has code they would like included in the list, please send me links. > > -AK > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro From michal at rsbac.org Wed Sep 25 14:22:45 2013 From: michal at rsbac.org (Michal Purzynski) Date: Wed, 25 Sep 2013 23:22:45 +0200 Subject: [Bro] error when generating documentation Message-ID: <52435425.4060500@rsbac.org> Hello, that's on the 2.2-beta on OS X make doc results in the following error Failed to document '@load base/protocols/ssl' in file: policy^protocols^ssl^expiring-certs.rst error in /Users/mpurzynski/Downloads/bro-2.2-beta/scripts/policy/protocols/ssl/expiring-certs.bro, line 5: can't open /Users/mpurzynski/Downloads/bro-2.2-beta/scripts/base/protocols/ssl/__load__.bro make[4]: *** [doc/rest_output/test-all-policy.rst] Error 1 make[3]: *** [doc/scripts/CMakeFiles/restdoc.dir/all] Error 2 make[2]: *** [doc/CMakeFiles/doc.dir/rule] Error 2 make[1]: *** [doc] Error 2 make: *** [doc] Error 2 From jsiwek at illinois.edu Wed Sep 25 15:02:06 2013 From: jsiwek at illinois.edu (Siwek, Jonathan Luke) Date: Wed, 25 Sep 2013 22:02:06 +0000 Subject: [Bro] error when generating documentation In-Reply-To: <52435425.4060500@rsbac.org> References: <52435425.4060500@rsbac.org> Message-ID: On Sep 25, 2013, at 4:22 PM, Michal Purzynski wrote: > Hello, that's on the 2.2-beta on OS X > > make doc results in the following error > > Failed to document '@load base/protocols/ssl' in file: > policy^protocols^ssl^expiring-certs.rst > error in > /Users/mpurzynski/Downloads/bro-2.2-beta/scripts/policy/protocols/ssl/expiring-certs.bro, > line 5: can't open > /Users/mpurzynski/Downloads/bro-2.2-beta/scripts/base/protocols/ssl/__load__.bro Try raising `ulimit -n`, the shell limit for max number of open files (and I've already got a pending change to improve the error message for this case). - Jon From anthony.kasza at gmail.com Wed Sep 25 17:42:59 2013 From: anthony.kasza at gmail.com (anthony kasza) Date: Wed, 25 Sep 2013 17:42:59 -0700 Subject: [Bro] Third Party Scripts In-Reply-To: <1202BE242E080642B0CD0AD0A03E8552E53B45@PGH-MSGMB-03.andrew.ad.cmu.edu> References: <15851_1380083253_r8P4RVGq015995_CAEZw2bxJYZ-Mb7OCO0QtzN1c6pmK6j0CN5D2ZsT4RtjmEu6-tw@mail.gmail.com> <1202BE242E080642B0CD0AD0A03E8552E53B45@PGH-MSGMB-03.andrew.ad.cmu.edu> Message-ID: Nice search foo, Vlad. Thanks. I'll add the repos I missed tonight. I find it comical how many different projects are named "bro-scripts". If anyone comes across new Bro code or code not on github, please post here and I'll update the page. -AK On Sep 25, 2013 9:22 AM, "Vlad Grigorescu" wrote: > Awesome. Just FYI, you can also search GitHub for repos containing Bro > code: > > https://github.com/search?q=bro&type=Repositories&ref=advsearch&l=Bro > > --Vlad > > > On Sep 25, 2013, at 12:19 AM, anthony kasza > wrote: > > > Hi All, > > > > In an effort to assist other in locating example Bro scripts, I've > cataloged all the Bro scripts I could find on a single page. > > > > < > http://supbrosup.blogspot.com/2013/09/third-party-scripts-and-example-code.html > > > > > > It's nice to know what all is out there. If I missed anything or if > anyone has code they would like included in the list, please send me links. > > > > -AK > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130925/7f1941c4/attachment.html From seth at icir.org Thu Sep 26 05:39:29 2013 From: seth at icir.org (Seth Hall) Date: Thu, 26 Sep 2013 08:39:29 -0400 Subject: [Bro] Third Party Scripts In-Reply-To: References: <15851_1380083253_r8P4RVGq015995_CAEZw2bxJYZ-Mb7OCO0QtzN1c6pmK6j0CN5D2ZsT4RtjmEu6-tw@mail.gmail.com> <1202BE242E080642B0CD0AD0A03E8552E53B45@PGH-MSGMB-03.andrew.ad.cmu.edu> Message-ID: On Sep 25, 2013, at 8:42 PM, anthony kasza wrote: > Nice search foo, Vlad. Thanks. I'll add the repos I missed tonight. I find it comical how many different projects are named "bro-scripts". Yeah, I figure we aren't too far off from seeing projects named similarly to projects in other languages where they have random made up names. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130926/7e972b75/attachment.bin From edthoma at sandia.gov Thu Sep 26 15:13:21 2013 From: edthoma at sandia.gov (Thomas, Eric D) Date: Thu, 26 Sep 2013 22:13:21 +0000 Subject: [Bro] Multiple interfaces on 2.2-beta-4 Message-ID: I'm not sure this is a bug, which is why I didn't post it on the tracker. In 2.1 one could do this in node.cfg: interface=eth2 -i eth3 And bro would open both interfaces. In 2.2-beta-4 specifying two -i's while manually running bro works, but trying to do the above with the "interface" specification doesn't, giving this error: fatal error: /usr/local/bro/bin/bro: problem with interface eth2 -i eth3 - pcap_open_live: eth2 -i eth3: No such device exists (SIOCGIFHWADDR: No such device) Looks like it's not parsing things the same way. In 2.2 will there be a new way to specify multiple interfaces? -- Eric Thomas edthoma at sandia.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130926/656634e1/attachment.html From r.fulton at auckland.ac.nz Thu Sep 26 16:25:41 2013 From: r.fulton at auckland.ac.nz (Russell Fulton) Date: Fri, 27 Sep 2013 11:25:41 +1200 Subject: [Bro] any interest in crash reports? Message-ID: I had a worker crash and then the manager ? is anyone interested in reports? Russell From robin at icir.org Thu Sep 26 16:44:43 2013 From: robin at icir.org (Robin Sommer) Date: Thu, 26 Sep 2013 16:44:43 -0700 Subject: [Bro] any interest in crash reports? In-Reply-To: References: Message-ID: <20130926234443.GW48087@icir.org> Yes, please forward the mails to the address mentioned in there. Thanks, Robin On Fri, Sep 27, 2013 at 11:25 +1200, Russell Fulton wrote: > I had a worker crash and then the manager ? is anyone interested in reports? > > Russell > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jlay at slave-tothe-box.net Thu Sep 26 17:06:03 2013 From: jlay at slave-tothe-box.net (James Lay) Date: Thu, 26 Sep 2013 18:06:03 -0600 Subject: [Bro] Multiple interfaces on 2.2-beta-4 In-Reply-To: References: Message-ID: On Sep 26, 2013, at 16:13, Thomas, Eric D wrote: > I'm not sure this is a bug, which is why I didn't post it on the tracker. In 2.1 one could do this in node.cfg: > > interface=eth2 -i eth3 > > And bro would open both interfaces. In 2.2-beta-4 specifying two -i's while manually running bro works, but trying to do the above with the "interface" specification doesn't, giving this error: > > fatal error: /usr/local/bro/bin/bro: problem with interface eth2 -i eth3 - pcap_open_live: eth2 -i eth3: No such device exists (SIOCGIFHWADDR: No such device) > > Looks like it's not parsing things the same way. In 2.2 will there be a new way to specify multiple interfaces? > -- > Eric Thomas > edthoma at sandia.gov > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro If you?re using broctl add: broargs = -i eth3 to your broctl.cfg James -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130926/f8fa4abc/attachment.html From seth at icir.org Thu Sep 26 19:27:29 2013 From: seth at icir.org (Seth Hall) Date: Thu, 26 Sep 2013 22:27:29 -0400 Subject: [Bro] Multiple interfaces on 2.2-beta-4 In-Reply-To: References: Message-ID: On Sep 26, 2013, at 6:13 PM, "Thomas, Eric D" wrote: > I'm not sure this is a bug, which is why I didn't post it on the tracker. In 2.1 one could do this in node.cfg: Heh, the previous behavior was technically a bug. :) I guess I don't really know what to say, sniffing multiple interfaces was never something we actually supported when you run Bro with broctl and we continue not to support it. Generally we recommend merging multiple streams of traffic upstream of where Bro receives the packets. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130926/56642eee/attachment.bin From JAzoff at albany.edu Thu Sep 26 19:37:53 2013 From: JAzoff at albany.edu (Justin Azoff) Date: Thu, 26 Sep 2013 22:37:53 -0400 Subject: [Bro] Multiple interfaces on 2.2-beta-4 In-Reply-To: References: Message-ID: <20130927023753.GL3250@datacomm.albany.edu> On Thu, Sep 26, 2013 at 10:27:29PM -0400, Seth Hall wrote: > I guess I don't really know what to say, sniffing multiple interfaces was never something we actually supported when you run Bro with broctl and we continue not to support it. Generally we recommend merging multiple streams of traffic upstream of where Bro receives the packets. What about with something like: [worker-1] type=worker host=localhost interface=eth0 [worker-2] type=worker host=localhost interface=eth1 as long as those aren't half streams from a tap, that should work, right? -- -- Justin Azoff -- Network Security & Performance Analyst From jessebowling at gmail.com Thu Sep 26 19:55:21 2013 From: jessebowling at gmail.com (Jesse Bowling) Date: Thu, 26 Sep 2013 22:55:21 -0400 Subject: [Bro] Multiple interfaces on 2.2-beta-4 In-Reply-To: <20130927023753.GL3250@datacomm.albany.edu> References: <20130927023753.GL3250@datacomm.albany.edu> Message-ID: I'll also bring up the case of using PF_RING with Bro...In that case it seems the way to do it is to use the PF_RING specific interface notation: interface=p2p1\;p2p2\;p2p3\;p2p4 and then let the workers split up the stream as required. I seem to remember also having to modify a bash script in Bro (and having written to the list about it) in order to get this notation passed through properly...I suppose it's time to see if this new version requires the same tweaking. :) Cheers, Jesse On Thu, Sep 26, 2013 at 10:37 PM, Justin Azoff wrote: > On Thu, Sep 26, 2013 at 10:27:29PM -0400, Seth Hall wrote: > > I guess I don't really know what to say, sniffing multiple interfaces > was never something we actually supported when you run Bro with broctl and > we continue not to support it. Generally we recommend merging multiple > streams of traffic upstream of where Bro receives the packets. > > What about with something like: > > [worker-1] > type=worker > host=localhost > interface=eth0 > > [worker-2] > type=worker > host=localhost > interface=eth1 > > as long as those aren't half streams from a tap, that should work, > right? > > -- > -- Justin Azoff > -- Network Security & Performance Analyst > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Jesse Bowling -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130926/da8a43cd/attachment.html From edthoma at sandia.gov Fri Sep 27 06:55:21 2013 From: edthoma at sandia.gov (Thomas, Eric D) Date: Fri, 27 Sep 2013 13:55:21 +0000 Subject: [Bro] [EXTERNAL] Re: Multiple interfaces on 2.2-beta-4 In-Reply-To: <20130927023753.GL3250@datacomm.albany.edu> Message-ID: Thanks all for the replies. I'll try them all, starting with the easiest. For the record, the interfaces are both half streams, so I don't think the cluster method will work. -- Eric Thomas edthoma at sandia.gov On 9/26/13 7:37 PM, "Justin Azoff" wrote: >On Thu, Sep 26, 2013 at 10:27:29PM -0400, Seth Hall wrote: >> I guess I don't really know what to say, sniffing multiple interfaces >>was never something we actually supported when you run Bro with broctl >>and we continue not to support it. Generally we recommend merging >>multiple streams of traffic upstream of where Bro receives the packets. > >What about with something like: > >[worker-1] >type=worker >host=localhost >interface=eth0 > >[worker-2] >type=worker >host=localhost >interface=eth1 > >as long as those aren't half streams from a tap, that should work, >right? > >-- >-- Justin Azoff >-- Network Security & Performance Analyst From phatbuckett at gmail.com Fri Sep 27 23:14:50 2013 From: phatbuckett at gmail.com (Darren Spruell) Date: Fri, 27 Sep 2013 23:14:50 -0700 Subject: [Bro] [EXTERNAL] Re: Multiple interfaces on 2.2-beta-4 In-Reply-To: References: <20130927023753.GL3250@datacomm.albany.edu> Message-ID: What about bonding/trunking interfaces on the host? DS On Fri, Sep 27, 2013 at 6:55 AM, Thomas, Eric D wrote: > Thanks all for the replies. I'll try them all, starting with the easiest. > For the record, the interfaces are both half streams, so I don't think the > cluster method will work. > > -- > Eric Thomas > > edthoma at sandia.gov > > > > > On 9/26/13 7:37 PM, "Justin Azoff" wrote: > >>On Thu, Sep 26, 2013 at 10:27:29PM -0400, Seth Hall wrote: >>> I guess I don't really know what to say, sniffing multiple interfaces >>>was never something we actually supported when you run Bro with broctl >>>and we continue not to support it. Generally we recommend merging >>>multiple streams of traffic upstream of where Bro receives the packets. >> >>What about with something like: >> >>[worker-1] >>type=worker >>host=localhost >>interface=eth0 >> >>[worker-2] >>type=worker >>host=localhost >>interface=eth1 >> >>as long as those aren't half streams from a tap, that should work, >>right? >> >>-- >>-- Justin Azoff >>-- Network Security & Performance Analyst > > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro -- Darren Spruell phatbuckett at gmail.com From anthony.kasza at gmail.com Fri Sep 27 23:31:11 2013 From: anthony.kasza at gmail.com (anthony kasza) Date: Fri, 27 Sep 2013 23:31:11 -0700 Subject: [Bro] [EXTERNAL] Re: Multiple interfaces on 2.2-beta-4 In-Reply-To: References: <20130927023753.GL3250@datacomm.albany.edu> Message-ID: This makes me wonder if anyone has tried using linux containers to build cluster nodes... Anyone? -AK On Sep 27, 2013 11:18 PM, "Darren Spruell" wrote: > What about bonding/trunking interfaces on the host? > > DS > > On Fri, Sep 27, 2013 at 6:55 AM, Thomas, Eric D > wrote: > > Thanks all for the replies. I'll try them all, starting with the easiest. > > For the record, the interfaces are both half streams, so I don't think > the > > cluster method will work. > > > > -- > > Eric Thomas > > > > edthoma at sandia.gov > > > > > > > > > > On 9/26/13 7:37 PM, "Justin Azoff" wrote: > > > >>On Thu, Sep 26, 2013 at 10:27:29PM -0400, Seth Hall wrote: > >>> I guess I don't really know what to say, sniffing multiple interfaces > >>>was never something we actually supported when you run Bro with broctl > >>>and we continue not to support it. Generally we recommend merging > >>>multiple streams of traffic upstream of where Bro receives the packets. > >> > >>What about with something like: > >> > >>[worker-1] > >>type=worker > >>host=localhost > >>interface=eth0 > >> > >>[worker-2] > >>type=worker > >>host=localhost > >>interface=eth1 > >> > >>as long as those aren't half streams from a tap, that should work, > >>right? > >> > >>-- > >>-- Justin Azoff > >>-- Network Security & Performance Analyst > > > > > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > -- > Darren Spruell > phatbuckett at gmail.com > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130927/07b1830a/attachment.html From panaman at geekempire.com Sat Sep 28 02:19:34 2013 From: panaman at geekempire.com (Panaman) Date: Sat, 28 Sep 2013 05:19:34 -0400 Subject: [Bro] Bond Message-ID: I bond my interfaces together and have bro listen on the bond On Friday, September 27, 2013, wrote: > Send Bro mailing list submissions to > bro at bro.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > or, via email, send a message with subject or body 'help' to > bro-request at bro.org > > You can reach the person managing the list at > bro-owner at bro.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Bro digest..." > > > Today's Topics: > > 1. Re: [EXTERNAL] Re: Multiple interfaces on 2.2-beta-4 > (Thomas, Eric D) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 27 Sep 2013 13:55:21 +0000 > From: "Thomas, Eric D" > > Subject: Re: [Bro] [EXTERNAL] Re: Multiple interfaces on 2.2-beta-4 > To: Justin Azoff > > Cc: "bro at bro.org " > > Message-ID: > > Content-Type: text/plain; charset="us-ascii" > > Thanks all for the replies. I'll try them all, starting with the easiest. > For the record, the interfaces are both half streams, so I don't think the > cluster method will work. > > -- > Eric Thomas > > edthoma at sandia.gov > > > > > On 9/26/13 7:37 PM, "Justin Azoff" > > wrote: > > >On Thu, Sep 26, 2013 at 10:27:29PM -0400, Seth Hall wrote: > >> I guess I don't really know what to say, sniffing multiple interfaces > >>was never something we actually supported when you run Bro with broctl > >>and we continue not to support it. Generally we recommend merging > >>multiple streams of traffic upstream of where Bro receives the packets. > > > >What about with something like: > > > >[worker-1] > >type=worker > >host=localhost > >interface=eth0 > > > >[worker-2] > >type=worker > >host=localhost > >interface=eth1 > > > >as long as those aren't half streams from a tap, that should work, > >right? > > > >-- > >-- Justin Azoff > >-- Network Security & Performance Analyst > > > > > ------------------------------ > > _______________________________________________ > Bro mailing list > Bro at bro.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > End of Bro Digest, Vol 89, Issue 34 > *********************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130928/af482b10/attachment.html From la_arshadi at yahoo.com Sat Sep 28 06:07:45 2013 From: la_arshadi at yahoo.com (Laleh Arshadi) Date: Sat, 28 Sep 2013 06:07:45 -0700 (PDT) Subject: [Bro] TCP connection summaries Message-ID: <1380373665.21251.YahooMailNeo@web140604.mail.bf1.yahoo.com> Hi all ? I see that you can get a connection summary log?of an offline pcap traffic file?by running bro with a simple command line as: bro -r traffic_file_name I have tested this command and it works well. But I am only interested in TCP connection summaries so I tried: bro -r traffic_file_name tcp But I get an error indication 'tcp' as unkown. What have I missed here? ? Regards L. Arshadi -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130928/3b6c2bd4/attachment.html From jonschipp at gmail.com Sat Sep 28 08:00:47 2013 From: jonschipp at gmail.com (Jon Schipp) Date: Sat, 28 Sep 2013 10:00:47 -0500 Subject: [Bro] TCP connection summaries In-Reply-To: <1380373665.21251.YahooMailNeo@web140604.mail.bf1.yahoo.com> References: <1380373665.21251.YahooMailNeo@web140604.mail.bf1.yahoo.com> Message-ID: To pass a BPF try ``-f tcp'' On Sat, Sep 28, 2013 at 8:07 AM, Laleh Arshadi wrote: > Hi all > > I see that you can get a connection summary log of an offline pcap traffic > file by running bro with a simple command line as: > bro -r traffic_file_name > I have tested this command and it works well. But I am only interested in > TCP connection summaries so I tried: > bro -r traffic_file_name tcp > But I get an error indication 'tcp' as unkown. What have I missed here? > > Regards > L. Arshadi > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Jon Schipp, jonschipp.com, sickbits.net -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130928/34529e80/attachment.html From jdness at mac.com Sat Sep 28 21:51:15 2013 From: jdness at mac.com (Jonathan Ness) Date: Sun, 29 Sep 2013 04:51:15 +0000 (UTC) Subject: [Bro] Trouble building Bro 2.2 on ARMv6 (Raspberry Pi / Wheezy) Message-ID: Hi, I have been using Bro 2.1 on my Raspberry Pi device. It worked mostly out of the box. (There was a small configuration change due to RPI's missing realtime clock but otherwise, worked fine.) However, Bro 2.2 does not build on the same device. It builds fine on the same configuration of Linux (Debian Wheezy) under Hyper-V on x64 so I suspect the ARM aspect of the RPI. I haven't used cmake before so really I'm not even sure how to debug the build issue but here's stdout: [ 8%] [BIFCL] Processing top-k.bif usage: bifcl [-p] *.bif make[3]: *** [scripts/base/bif/top-k.bif.bro] Error 1 make[3]: Leaving directory `/home/pi/bro/build' make[2]: *** [src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/all] Error 2 make[2]: Leaving directory `/home/pi/bro/build' make[1]: *** [all] Error 2 make[1]: Leaving directory `/home/pi/bro/build' make: *** [all] Error 2 I assume it's a problem with bifcl so I tried replacing the Bro 2.2 build tree bifcl with the one from the Bro 2.1 build tree but that didn't work either. (different error when doing that: "Error: cannot open file: -s") Maybe it's an option mismatch...? Has anyone gotten Bro 2.2 to build on ARMv6? Any tricks / tweaks needed? Thanks! Jonathan P.S. The new work on file carving is fantastic! Enables all kinds of interesting new scenarios! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130929/53870a13/attachment.html From anthony.kasza at gmail.com Sun Sep 29 09:31:54 2013 From: anthony.kasza at gmail.com (anthony kasza) Date: Sun, 29 Sep 2013 09:31:54 -0700 Subject: [Bro] Trouble building Bro 2.2 on ARMv6 (Raspberry Pi / Wheezy) In-Reply-To: References: Message-ID: It's not just you. I tried compiling on the RPI about a week ago and encountered the exact same issue. I tried cross compiling for ARM from my laptop but didn't get very far. -AK On Sep 28, 2013 9:54 PM, "Jonathan Ness" wrote: > Hi, I have been using Bro 2.1 on my Raspberry Pi device. It worked mostly > out of the box. (There was a small configuration change due to RPI's > missing realtime clock but otherwise, worked fine.) However, Bro 2.2 does > not build on the same device. It builds fine on the same configuration of > Linux (Debian Wheezy) under Hyper-V on x64 so I suspect the ARM aspect of > the RPI. I haven't used cmake before so really I'm not even sure how to > debug the build issue but here's stdout: > > > [ 8%] [BIFCL] Processing top-k.bif > usage: bifcl [-p] *.bif > make[3]: *** [scripts/base/bif/top-k.bif.bro] Error 1 > make[3]: Leaving directory `/home/pi/bro/build' > make[2]: *** > [src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/all] > Error 2 > make[2]: Leaving directory `/home/pi/bro/build' > make[1]: *** [all] Error 2 > make[1]: Leaving directory `/home/pi/bro/build' > make: *** [all] Error 2 > > > I assume it's a problem with bifcl so I tried replacing the Bro 2.2 build > tree bifcl with the one from the Bro 2.1 build tree but that didn't work > either. (different error when doing that: "Error: cannot open file: -s") > Maybe it's an option mismatch...? > > Has anyone gotten Bro 2.2 to build on ARMv6? Any tricks / tweaks needed? > Thanks! > > Jonathan > > P.S. The new work on file carving is fantastic! Enables all kinds of > interesting new scenarios! > > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130929/25855ddf/attachment.html From robin at icir.org Sun Sep 29 19:48:19 2013 From: robin at icir.org (Robin Sommer) Date: Sun, 29 Sep 2013 19:48:19 -0700 Subject: [Bro] Trouble building Bro 2.2 on ARMv6 (Raspberry Pi / Wheezy) In-Reply-To: References: Message-ID: <20130930024819.GB13422@icir.org> On Sun, Sep 29, 2013 at 04:51 +0000, you wrote: > [ 8%] [BIFCL] Processing top-k.bif > usage: bifcl [-p] *.bif How does that part look like with "make VERBOSE=1"? Robin -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From anthony.kasza at gmail.com Sun Sep 29 21:17:02 2013 From: anthony.kasza at gmail.com (anthony kasza) Date: Sun, 29 Sep 2013 21:17:02 -0700 Subject: [Bro] Trouble building Bro 2.2 on ARMv6 (Raspberry Pi / Wheezy) In-Reply-To: <20130930024819.GB13422@icir.org> References: <20130930024819.GB13422@icir.org> Message-ID: Sorry for the bad formatting Linking CXX executable bifcl cd /opt/bro_src/build/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/bifcl.dir/link.txt --verbose=1 /usr/bin/c++ -Wall -Wno-unused -O2 -g CMakeFiles/bifcl.dir/bif_parse.cc.o CMakeFiles/bifcl.dir/bif_lex.cc.o CMakeFiles/bifcl.dir/bif_arg.cc.o CMakeFiles/bifcl.dir/module_util.cc.o -o bifcl -rdynamic make[3]: Leaving directory `/opt/bro_src/build' /usr/bin/cmake -E cmake_progress_report /opt/bro_src/build/CMakeFiles 8 [ 8%] Built target bifcl make -f src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build.make src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend make[3]: Entering directory `/opt/bro_src/build' cd /opt/bro_src/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /opt/bro_src /opt/bro_src/src/probabilistic /opt/bro_src/build /opt/bro_src/build/src/probabilistic /opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/DependInfo.cmake --color= Dependee "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/DependInfo.cmake" is newer than depender "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend.internal". Dependee "/opt/bro_src/build/src/probabilistic/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend.internal". Scanning dependencies of target bif-alt-probabilistic-top-k.bif make[3]: Leaving directory `/opt/bro_src/build' make -f src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build.make src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build make[3]: Entering directory `/opt/bro_src/build' /usr/bin/cmake -E cmake_progress_report /opt/bro_src/build/CMakeFiles [ 8%] [BIFCL] Processing top-k.bif cd /opt/bro_src/build/src/probabilistic && ../bifcl -s /opt/bro_src/src/probabilistic/top-k.bif || ( rm -f /opt/bro_src/build/scripts/base/bif/top-k.bif.bro top-k.bif.h top-k.bif.cc top-k.bif.init.cc && exit 1 ) usage: bifcl [-p] *.bif make[3]: *** [scripts/base/bif/top-k.bif.bro] Error 1 make[3]: Leaving directory `/opt/bro_src/build' make[2]: *** [src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/all] Error 2 make[2]: Leaving directory `/opt/bro_src/build' make[1]: *** [all] Error 2 make[1]: Leaving directory `/opt/bro_src/build' make: *** [all] Error 2 On Sun, Sep 29, 2013 at 7:48 PM, Robin Sommer wrote: > > > On Sun, Sep 29, 2013 at 04:51 +0000, you wrote: > > > [ 8%] [BIFCL] Processing top-k.bif > > usage: bifcl [-p] *.bif > > How does that part look like with "make VERBOSE=1"? > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130929/2077a277/attachment.html From oguzyarimtepe at gmail.com Mon Sep 30 01:59:39 2013 From: oguzyarimtepe at gmail.com (=?UTF-8?B?T8SfdXogWWFyxLFtdGVwZQ==?=) Date: Mon, 30 Sep 2013 11:59:39 +0300 Subject: [Bro] suggestion request Message-ID: Hi, For the ones dealing with machine learning KDD Cup 99 dataset is used widely for testing the algorithm proposed. ( http://kdd.ics.uci.edu/databases/kddcup99/task.html) The data set is created with some features defined at Table 1, Table 2 and Table 3. Now i would like to test my algorithm with the real data so i will collect traffic and convert it to KDD Cup 99 format. I am searching a method. How can i gather or calculate the properties at Table2? The properties are number of ``hot'' indicators: meaning hidden directory creation number of failed login attempts 1 if successfully logged in; 0 otherwise number of ``compromised''continuous 1 if root shell is obtained; 0 otherwise 1 if ``su root'' command attempted; 0 otherwise number of ``root'' accesses number of file creation operations number of shell prompts number of operations on access control files number of outbound commands in an ftp session So it seems payload analysis is required? Anybody who had experience with such thing or any suggestion? I will be listening a mirrored port and saving the traffic data to db. Can Bro Time Machine help me on this issue? Cheers. -- O?uz Yar?mtepe http://about.me/oguzy -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130930/9138dab3/attachment.html From robin at icir.org Mon Sep 30 05:17:47 2013 From: robin at icir.org (Robin Sommer) Date: Mon, 30 Sep 2013 05:17:47 -0700 Subject: [Bro] Trouble building Bro 2.2 on ARMv6 (Raspberry Pi / Wheezy) In-Reply-To: References: <20130930024819.GB13422@icir.org> Message-ID: <20130930121747.GE57016@icir.org> Hmm ... No immediate idea. Does this work from the top-level directory? # ./build/src/bifcl -s src/probabilistic/top-k.bif # ls top-k* top-k.bif.bro top-k.bif.cc top-k.bif.h top-k.bif.init.cc (Note that bifcl's usage message isn't up to date, it does now in fact support an -s option.) Is this a fresh checkout/build, or on top of an older build/install? Robin On Sun, Sep 29, 2013 at 21:17 -0700, you wrote: > Sorry for the bad formatting > > Linking CXX executable bifcl > cd /opt/bro_src/build/src && /usr/bin/cmake -E cmake_link_script > CMakeFiles/bifcl.dir/link.txt --verbose=1 > /usr/bin/c++ -Wall -Wno-unused -O2 -g > CMakeFiles/bifcl.dir/bif_parse.cc.o CMakeFiles/bifcl.dir/bif_lex.cc.o > CMakeFiles/bifcl.dir/bif_arg.cc.o CMakeFiles/bifcl.dir/module_util.cc.o -o > bifcl -rdynamic > make[3]: Leaving directory `/opt/bro_src/build' > /usr/bin/cmake -E cmake_progress_report /opt/bro_src/build/CMakeFiles 8 > [ 8%] Built target bifcl > make -f > src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build.make > src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend > make[3]: Entering directory `/opt/bro_src/build' > cd /opt/bro_src/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" > /opt/bro_src /opt/bro_src/src/probabilistic /opt/bro_src/build > /opt/bro_src/build/src/probabilistic > /opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/DependInfo.cmake > --color= > Dependee > "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/DependInfo.cmake" > is newer than depender > "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend.internal". > Dependee > "/opt/bro_src/build/src/probabilistic/CMakeFiles/CMakeDirectoryInformation.cmake" > is newer than depender > "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend.internal". > Scanning dependencies of target bif-alt-probabilistic-top-k.bif > make[3]: Leaving directory `/opt/bro_src/build' > make -f > src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build.make > src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build > make[3]: Entering directory `/opt/bro_src/build' > /usr/bin/cmake -E cmake_progress_report /opt/bro_src/build/CMakeFiles > [ 8%] [BIFCL] Processing top-k.bif > cd /opt/bro_src/build/src/probabilistic && ../bifcl -s > /opt/bro_src/src/probabilistic/top-k.bif || ( rm -f > /opt/bro_src/build/scripts/base/bif/top-k.bif.bro top-k.bif.h top-k.bif.cc > top-k.bif.init.cc && exit 1 ) > usage: bifcl [-p] *.bif > make[3]: *** [scripts/base/bif/top-k.bif.bro] Error 1 > make[3]: Leaving directory `/opt/bro_src/build' > make[2]: *** > [src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/all] > Error 2 > make[2]: Leaving directory `/opt/bro_src/build' > make[1]: *** [all] Error 2 > make[1]: Leaving directory `/opt/bro_src/build' > make: *** [all] Error 2 > > > > On Sun, Sep 29, 2013 at 7:48 PM, Robin Sommer wrote: > > > > > > > On Sun, Sep 29, 2013 at 04:51 +0000, you wrote: > > > > > [ 8%] [BIFCL] Processing top-k.bif > > > usage: bifcl [-p] *.bif > > > > How does that part look like with "make VERBOSE=1"? > > > > Robin > > > > -- > > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > > _______________________________________________ > > Bro mailing list > > bro at bro-ids.org > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin From jdness at mac.com Mon Sep 30 10:59:46 2013 From: jdness at mac.com (Jonathan Ness) Date: Mon, 30 Sep 2013 17:59:46 +0000 (UTC) Subject: [Bro] Trouble building Bro 2.2 on ARMv6 (Raspberry Pi / Wheezy) In-Reply-To: <20130930121747.GE57016@icir.org> Message-ID: <6d228d26-d14f-4973-97d9-2d33bdc3189f@me.com> For me, it was a fresh checkout/build. I will try that exact command when I get home tonight. I remember trying a couple different combinations from the build/src directory and not being successful. Not sure I tried exactly that one from exactly there though. On Sep 30, 2013, at 05:17 AM, Robin Sommer wrote: Hmm ... No immediate idea. Does this work from the top-level directory? # ./build/src/bifcl -s src/probabilistic/top-k.bif # ls top-k* top-k.bif.bro top-k.bif.cc top-k.bif.h top-k.bif.init.cc (Note that bifcl's usage message isn't up to date, it does now in fact support an -s option.) Is this a fresh checkout/build, or on top of an older build/install? Robin On Sun, Sep 29, 2013 at 21:17 -0700, you wrote: Sorry for the bad formatting Linking CXX executable bifcl cd /opt/bro_src/build/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/bifcl.dir/link.txt --verbose=1 /usr/bin/c++ -Wall -Wno-unused -O2 -g CMakeFiles/bifcl.dir/bif_parse.cc.o CMakeFiles/bifcl.dir/bif_lex.cc.o CMakeFiles/bifcl.dir/bif_arg.cc.o CMakeFiles/bifcl.dir/module_util.cc.o -o bifcl -rdynamic make[3]: Leaving directory `/opt/bro_src/build' /usr/bin/cmake -E cmake_progress_report /opt/bro_src/build/CMakeFiles 8 [ 8%] Built target bifcl make -f src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build.make src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend make[3]: Entering directory `/opt/bro_src/build' cd /opt/bro_src/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /opt/bro_src /opt/bro_src/src/probabilistic /opt/bro_src/build /opt/bro_src/build/src/probabilistic /opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/DependInfo.cmake --color= Dependee "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/DependInfo.cmake" is newer than depender "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend.internal". Dependee "/opt/bro_src/build/src/probabilistic/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend.internal". Scanning dependencies of target bif-alt-probabilistic-top-k.bif make[3]: Leaving directory `/opt/bro_src/build' make -f src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build.make src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build make[3]: Entering directory `/opt/bro_src/build' /usr/bin/cmake -E cmake_progress_report /opt/bro_src/build/CMakeFiles [ 8%] [BIFCL] Processing top-k.bif cd /opt/bro_src/build/src/probabilistic && ../bifcl -s /opt/bro_src/src/probabilistic/top-k.bif || ( rm -f /opt/bro_src/build/scripts/base/bif/top-k.bif.bro top-k.bif.h top-k.bif.cc top-k.bif.init.cc && exit 1 ) usage: bifcl [-p] *.bif make[3]: *** [scripts/base/bif/top-k.bif.bro] Error 1 make[3]: Leaving directory `/opt/bro_src/build' make[2]: *** [src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/all] Error 2 make[2]: Leaving directory `/opt/bro_src/build' make[1]: *** [all] Error 2 make[1]: Leaving directory `/opt/bro_src/build' make: *** [all] Error 2 On Sun, Sep 29, 2013 at 7:48 PM, Robin Sommer wrote: > > > On Sun, Sep 29, 2013 at 04:51 +0000, you wrote: > > > [ 8%] [BIFCL] Processing top-k.bif > > usage: bifcl [-p] *.bif > > How does that part look like with "make VERBOSE=1"? > > Robin > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > _______________________________________________ > Bro mailing list > bro at bro-ids.org > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > -- Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130930/3f16eff7/attachment.html From edthoma at sandia.gov Mon Sep 30 12:08:18 2013 From: edthoma at sandia.gov (Thomas, Eric D) Date: Mon, 30 Sep 2013 19:08:18 +0000 Subject: [Bro] Log::add_filter with mime_type or filename predicate Message-ID: I'm looking for the new way (in 2.2) for filtering HTTP::LOG logging based upon mime_type or filename. It seems with the new file analysis framework the filename and mime_type of an HTTP connection are set in HTTP::Info in base/protocols/http/entities.bro inside the file_over_new_connection event. However I'm thinking at this point that that event is triggered only AFTER the HTTP::LOG filter predicates are processed, since all of the new entities fields in the HTTP::Info record are "" when printed from the predicate function. Here is a possibly helpful code snippet that goes inside bro_init() (Excuse the formatting, not much I can do.) Log::add_filter(HTTP::LOG, [$name = "http-executables", $path = "http_exe", $pred(rec: HTTP::Info) = { print "file:", rec; return 1==1; }, # This line was in the predicate function, but it no longer works # return rec?$mime_type && rec$mime_type == "application/x-dosexec"; }, $include=set("ts","id.orig_h","id.orig_p","id.resp_h","id.resp_p","method" ,"host","uri","referrer","user_agent","request_body_len","response_body_len ","status_code","info_msg","contenttype","filename","mime_type") ]); Thoughts? -- Eric Thomas edthoma at sandia.gov From seth at icir.org Mon Sep 30 13:40:01 2013 From: seth at icir.org (Seth Hall) Date: Mon, 30 Sep 2013 16:40:01 -0400 Subject: [Bro] Log::add_filter with mime_type or filename predicate In-Reply-To: References: Message-ID: On Sep 30, 2013, at 3:08 PM, "Thomas, Eric D" wrote: > # This line was in the predicate function, but it no longer works > # return rec?$mime_type && rec$mime_type == "application/x-dosexec"; }, return rec?$resp_mime_types && "application/x-dosexec" in rec$resp_mime_types; .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130930/1786024f/attachment.bin From edthoma at sandia.gov Mon Sep 30 13:44:07 2013 From: edthoma at sandia.gov (Thomas, Eric D) Date: Mon, 30 Sep 2013 20:44:07 +0000 Subject: [Bro] [EXTERNAL] Re: Log::add_filter with mime_type or filename predicate In-Reply-To: Message-ID: resp_mime_types is also uninitialized: file: , [ts=1380560274.291225, uid=CtYbny3SoceMiawke6, id=[orig_h=X.X.X.X, orig_p=43457/tcp, resp_h=74.125.239.123, resp_p=80/tcp], trans_depth=1, method=GET, host=s0.2mdn.net, uri=/viewad/910797/pixel.gif, referrer=http://www.kbb.com/used-cars/, user_agent=Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0, request_body_len=0, response_body_len=0, status_code=, status_msg=, info_code=, info_msg=, filename=, tags={ }, username=, password=, capture_password=F, range_request=F, orig_fuids=, orig_mime_types=, resp_fuids=, resp_mime_types=, current_entity=, orig_mime_depth=1, resp_mime_depth=0, contenttype=] -- Eric Thomas edthoma at sandia.gov On 9/30/13 1:40 PM, "Seth Hall" wrote: > >On Sep 30, 2013, at 3:08 PM, "Thomas, Eric D" wrote: > >> # This line was in the predicate function, but it no longer works >> # return rec?$mime_type && rec$mime_type == "application/x-dosexec"; }, > > >return rec?$resp_mime_types && "application/x-dosexec" in >rec$resp_mime_types; > > .Seth > >-- >Seth Hall >International Computer Science Institute >(Bro) because everyone has a network >http://www.bro.org/ > From seth at icir.org Mon Sep 30 13:55:02 2013 From: seth at icir.org (Seth Hall) Date: Mon, 30 Sep 2013 16:55:02 -0400 Subject: [Bro] [EXTERNAL] Log::add_filter with mime_type or filename predicate In-Reply-To: References: Message-ID: On Sep 30, 2013, at 4:44 PM, "Thomas, Eric D" wrote: > resp_mime_types is also uninitialized: Your traffic is messed up. You aren't seeing both sides of that connection. Perhaps this is related to your questions about sniffing multiple interfaces? .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130930/65f028dd/attachment.bin From edthoma at sandia.gov Mon Sep 30 14:06:02 2013 From: edthoma at sandia.gov (Thomas, Eric D) Date: Mon, 30 Sep 2013 21:06:02 +0000 Subject: [Bro] [EXTERNAL] Log::add_filter with mime_type or filename predicate In-Reply-To: Message-ID: Yep, that was it. Good catch, thanks! -- Eric Thomas edthoma at sandia.gov On 9/30/13 1:55 PM, "Seth Hall" wrote: > >On Sep 30, 2013, at 4:44 PM, "Thomas, Eric D" wrote: > >> resp_mime_types is also uninitialized: > > >Your traffic is messed up. You aren't seeing both sides of that >connection. Perhaps this is related to your questions about sniffing >multiple interfaces? > > .Seth > >-- >Seth Hall >International Computer Science Institute >(Bro) because everyone has a network >http://www.bro.org/ > From anthony.kasza at gmail.com Mon Sep 30 20:15:57 2013 From: anthony.kasza at gmail.com (anthony kasza) Date: Mon, 30 Sep 2013 20:15:57 -0700 Subject: [Bro] Trouble building Bro 2.2 on ARMv6 (Raspberry Pi / Wheezy) In-Reply-To: <6d228d26-d14f-4973-97d9-2d33bdc3189f@me.com> References: <20130930121747.GE57016@icir.org> <6d228d26-d14f-4973-97d9-2d33bdc3189f@me.com> Message-ID: Fresh install here, too. Command output below. root at raspberrypi:/opt/bro_src# ./build/src/bifcl -s src/probabilistic/top-k.bif usage: bifcl [-p] *.bif root at raspberrypi:/opt/bro_src# find ./ -iname top-k* ./src/probabilistic/top-k.bif I checked the dependency versions from the RPi's repos and they are equal to or newer than a vanilla Debian 6 system (which Bro builds fine on). -AK On Mon, Sep 30, 2013 at 10:59 AM, Jonathan Ness wrote: > For me, it was a fresh checkout/build. I will try that exact command when > I get home tonight. I remember trying a couple different combinations from > the build/src directory and not being successful. Not sure I tried exactly > that one from exactly there though. > > On Sep 30, 2013, at 05:17 AM, Robin Sommer wrote: > > Hmm ... No immediate idea. Does this work from the top-level > directory? > > # ./build/src/bifcl -s src/probabilistic/top-k.bif > # ls top-k* > top-k.bif.bro top-k.bif.cc top-k.bif.h top-k.bif.init.cc > > (Note that bifcl's usage message isn't up to date, it does now in fact > support an -s option.) > > Is this a fresh checkout/build, or on top of an older build/install? > > Robin > > > On Sun, Sep 29, 2013 at 21:17 -0700, you wrote: > > Sorry for the bad formatting > > Linking CXX executable bifcl > > cd /opt/bro_src/build/src && /usr/bin/cmake -E cmake_link_script > > CMakeFiles/bifcl.dir/link.txt --verbose=1 > > /usr/bin/c++ -Wall -Wno-unused -O2 -g > > CMakeFiles/bifcl.dir/bif_parse.cc.o CMakeFiles/bifcl.dir/bif_lex.cc.o > > CMakeFiles/bifcl.dir/bif_arg.cc.o CMakeFiles/bifcl.dir/module_util.cc.o -o > > bifcl -rdynamic > > make[3]: Leaving directory `/opt/bro_src/build' > > /usr/bin/cmake -E cmake_progress_report /opt/bro_src/build/CMakeFiles 8 > > [ 8%] Built target bifcl > > make -f > > src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build.make > > src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend > > make[3]: Entering directory `/opt/bro_src/build' > > cd /opt/bro_src/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" > > /opt/bro_src /opt/bro_src/src/probabilistic /opt/bro_src/build > > /opt/bro_src/build/src/probabilistic > > > /opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/DependInfo.cmake > > --color= > > Dependee > > > "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/DependInfo.cmake" > > is newer than depender > > > "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend.internal". > > Dependee > > > "/opt/bro_src/build/src/probabilistic/CMakeFiles/CMakeDirectoryInformation.cmake" > > is newer than depender > > > "/opt/bro_src/build/src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/depend.internal". > > Scanning dependencies of target bif-alt-probabilistic-top-k.bif > > make[3]: Leaving directory `/opt/bro_src/build' > > make -f > > src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build.make > > src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/build > > make[3]: Entering directory `/opt/bro_src/build' > > /usr/bin/cmake -E cmake_progress_report /opt/bro_src/build/CMakeFiles > > [ 8%] [BIFCL] Processing top-k.bif > > cd /opt/bro_src/build/src/probabilistic && ../bifcl -s > > /opt/bro_src/src/probabilistic/top-k.bif || ( rm -f > > /opt/bro_src/build/scripts/base/bif/top-k.bif.bro top-k.bif.h top-k.bif.cc > > top-k.bif.init.cc && exit 1 ) > > usage: bifcl [-p] *.bif > > make[3]: *** [scripts/base/bif/top-k.bif.bro] Error 1 > > make[3]: Leaving directory `/opt/bro_src/build' > > make[2]: *** > > [src/probabilistic/CMakeFiles/bif-alt-probabilistic-top-k.bif.dir/all] > > Error 2 > > make[2]: Leaving directory `/opt/bro_src/build' > > make[1]: *** [all] Error 2 > > make[1]: Leaving directory `/opt/bro_src/build' > > make: *** [all] Error 2 > > On Sun, Sep 29, 2013 at 7:48 PM, Robin Sommer wrote: > > > > > > > > > On Sun, Sep 29, 2013 at 04:51 +0000, you wrote: > > > > > > > [ 8%] [BIFCL] Processing top-k.bif > > > > usage: bifcl [-p] *.bif > > > > > > How does that part look like with "make VERBOSE=1"? > > > > > > Robin > > > > > > -- > > > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > > > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > > > _______________________________________________ > > > Bro mailing list > > > bro at bro-ids.org > > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro > > > > > > > -- > Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org > ICSI/LBNL * Fax +1 (510) 666-2956 * www.icir.org/robin > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20130930/af209286/attachment.html