From scampbell at lbl.gov Mon Oct 6 17:18:02 2003 From: scampbell at lbl.gov (scott campbell) Date: Mon, 06 Oct 2003 17:18:02 -0700 Subject: basic bro health monitor Message-ID: <3F82063A.1080808@lbl.gov> I discovered this weekend that it would be nice to be made aware of many of the following situations: high cpu load, large memory footprint, unusually large connections/minute recorded, or a large number of drops/minute activated in a running bro. Having bro alert on this information can give us a better idea of when unusual (as in hostile) things are happening to the network. This is a basic outline for a monitor script - it is a bit rough about the edges, but it seems to do the job. Making additions to the script should be trivial. scott -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: monitor.bro Url: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20031006/16d475de/attachment.ksh -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 252 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20031006/16d475de/attachment.bin From anton at netForensics.com Wed Oct 8 11:54:39 2003 From: anton at netForensics.com (Anton Chuvakin, Ph.D.) Date: Wed, 8 Oct 2003 14:54:39 -0400 (EDT) Subject: turing this off In-Reply-To: References: <3F79D345.6090007@lbl.gov> Message-ID: All, Can somebody suggest how do I turn off the generation of this: 'WeirdActivity ** non_IPv4_packet' I am getting loads of those in my honeynet. I looked ALL over the weird.bro, but I can't find anything even remotely similar to this.... I admit, maybe its my slow day :-) Help is deeply appreciated. Best, -- Anton Chuvakin, Ph.D., GCIA, GCIH Senior Security Analyst Product Management Group netForensics - http://www.netForensics.com 732-393-6071 From vern at icir.org Wed Oct 8 16:07:23 2003 From: vern at icir.org (Vern Paxson) Date: Wed, 08 Oct 2003 16:07:23 -0700 Subject: turing this off In-Reply-To: Your message of Wed, 08 Oct 2003 14:54:39 EDT. Message-ID: <200310082307.h98N7NrP030989@jaguar.icir.org> > Can somebody suggest how do I turn off the generation of this: > > 'WeirdActivity ** non_IPv4_packet' > > I am getting loads of those in my honeynet. This should do the trick: redef weird_action += { ["non_IPv4_packet"] = WEIRD_IGNORE }; - Vern From vern at icir.org Tue Oct 21 15:59:47 2003 From: vern at icir.org (Vern Paxson) Date: Tue, 21 Oct 2003 15:59:47 -0700 Subject: new bro "CURRENT" release - 0.8a48 Message-ID: <200310212259.h9LMxlvt031623@jaguar.icir.org> An updated "CURRENT" version of Bro is now available from ftp://ftp.ee.lbl.gov/bro-pub-0.8-current.tar.gz I've appended the changes between it and the last "CURRENT" version (0.8a37). Vern -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 0.8a48 Tue Oct 21 15:56:13 PDT 2003 - There is now a mechanism in place for multiple Bro's to communicate with one another via sockets (Robin Sommer). *This is quite experimental at this point* and may have significant bugs and/or need significant enhancements. By loading listen-clear.bro or listen-ssl.bro, an instance of Bro starts to listen on a TCP port. The first of these listens for unencrypted connections and the second for connections encrypted via OpenSSL. To connect to a listening Bro, you load remote-clear.bro or remote-ssl.bro. For this connection, you specify which events you want to receive by giving a regular expression (e.g. "/http_*/" for all HTTP events), although only those events for which you have defined a local handler are actually requested. Communication is uni-directional in the sense that for a certain connection the events go only from Bro-A to Bro-B but not from B to A (you could set up a second connection for the other direction, though). The OpenSSL certificate used to authorize remote Bro's is specified in the script variable "ssl_ca_certificate" and the private key and certificate for the local Bro via "ssl_private_key". If Bro A connects to Bro B, by default it sends over its capture_filter. But Bro B uses it only if it has loaded remote-pcap.bro. This is the beginning of structuring inter-Bro trust mechanisms. It is done via two new events, remote_capture_filter(ip: addr, p: port, filter: string) and send_capture_filter(ip: addr, p: port, s: string) : bool. The implementation forks a second process which does the socket communication, so that the main process should not be affected too much. The two processes communicate via a pipe. You can call is_remote_event() to determine whether the event currently being handled was originated by a remote Bro (if T) or the local Bro (if F). If a connection with a remote Bro terminates (for whatever reason), Bro may try to reconnect automatically. A new function, get_event_source(), returns a record event_source describing the source that raised the last event. See doc/ssl.txt for an explanation of how to create the keys/certificates. - A fledgling Gnutella analyzer has been contributed (Mark Allman). It generates the following events: event gnutella_text_msg(c: connection, orig: bool, headers: string) event gnutella_binary_msg(c: connection, orig: bool, msg_type: count, ttl: count, hops: count, msg_len: count, payload: string, payload_len: count, trunc: bool, complete: bool) event gnutella_partial_binary_msg(c: connection, orig: bool, msg: string, len: count) event gnutella_establish(c: connection) event gnutella_not_establish(c: connection) event gnutella_http_notify(c: connection) - Bro now supports a secondary channel for acquiring packets (Chema Gonzalez). You access it by redef'ing the new global "secondary_filters", adding table[string] of event(filter: string, pkt: pkt_hdr). The string specifies a tcpdump filter; anytime a packet matches the filter (including packets that would *not* otherwise be matched by capture_filter), then the given event handler is invoked. For example, redef secondary_filters += { ["tcp[13] & 7 != 0"] = rst_syn_fin_flag, } will invoke rst_syn_fin_flag() anytime a TCP packet is seen for which the SYN/FIN/RST bits are non-zero. The event handler will be passed the string "tcp[13] & 7 != 0" (so it can tell which of possibly multiple filters matched) and a pkt_hdr value, which includes the IP header and, if present, the TCP, UDP or ICMP header. Another example, and what motivated the addition, is: redef secondary_filters += { ["ip[10:2] & 0xffc == 0x398"] = sampled_1_in_1024_packet, } which will invoke sampled_1_in_1024_packet() any time the given 10 bits in the IP checksum match the pattern 0x398. If the checksum field is uniformly distributed then this roughly corresponds to 1-in-1024 random sampling. (Chema has also developed BPF additions to support true random sampling.) See policy/secondary-filter.bro for examples. - Bro now does a much better job of keeping track of how much memory has been allocated for different structures (Robin Sommer). This includes more accurate computations for global_size(). In addition, if you redef "statistics_interval" to be a non-zero time interval, then with that periodicity a summary of memory usage (including memory used by event engine components) is dumped to the file "statistics_file". In addition, at this point a "do_statistics" event is generated. You can also call the new built-in statistics_update() to generate memory statistics on demand. The above structure is likely to change in the future. statistics_interval will probably go away, to be replaced by either explicit calls to statistics_update() (which you can do on a timer if you like by using "schedule"), or by a built-in function that returns a record of all the statistics, that you can then format however you want. - A major memory leak in HTTP analysis has been fixed (Ruoming Pang). - New attributes &rotate = and &postprocessor = can be associated with a file variable in order to specify how often the file should be rotated to a new filename on disk, and, when rotation occurs, the name of a shell program to run on the now-older version as a postprocessor (Robin Sommer). - Similarly, log_postprocessor and log_rotate_interval specify the default values for files. Unless redef'd, these themselves default to the empty string (= no postprocessing) and 0.0 seconds (no rotation). (Robin Sommer) - A new attribute, &encrypt, applies to variables of "file" type. It specifies that the version on disk should be encrypted, using either the key specified as the value of the attribute, or, if no value is specified, using the value of the new script variable log_encryption_key. The key is an OpenSSL public key; it's used to then embed a Blowfish session key. (Robin Sommer) A new utility, aux/bdcat/bdcat ("Bro decrypt cat") can be used to decrypt the files. - The internal structure of TCP analysis has been significantly altered. Previously, TCP_Endpoint tracked endpoint state and TCP_EndpointContents (derived from it) did stream reassembly. These have now been separated; TCP_Endpoint still tracks endpoint state, but TCP_EndointContents has become TCP_Contents, and is only loosely coupled with TCP_Endpoint. The reason for this change is to lay the groundwork for (1) applying an application analyzer to a connection after several packets for the connection have already been read, and (2) applying *multiple* application analyzers to a single connection. - Bro now supports the universal hashing if USE_UHASH is defined (Ruoming Pang). Universal hashing is a lighter-weight alternative to MD5 hashing that retains the property of being very difficult for an attacker to guess. It comes in two flavors, a 32-bit version (which you get with USE_UHASH) and a faster 16-bit version (which you get if you also define USE_NH). Bro will likely switch to using these by default in the near future, as their performance gain over MD5 is significant. - New built-in functions srand() and rand() provide access to the usual C-style random number seeding & generation (Chema Gonzalez). - You can now specify server/client addresses to leave in the clear in IP address anonymization (via the built-in variables preserve_orig_addr and preserve_resp_addr). Correspondingly, the built-in function for IP anonymization now takes a parameter to specify the type of the address (orig, resp, or other), instead of the method of anonymization (seq, random, prefix-preserving). (Ruoming Pang) - Trace anonymization now has prelminary support for handling TCP options via the new event "tcp_option" (Ruoming Pang). It is only invoked if skip_event_tcp_option is F (it defaults to T). - A new event, tcp_segment, is similar to the event "packet_content" but provides more information: is_orig (directionality), sequence number, and acknowledgement number (Ruoming Pang). - ./configure finds OpenSSL if it's in some standard location. Otherwise, you may specify it --with-openssl=. If OpenSSL is not available, Bro compiles cleanly without and gives warnings if a script tries use SSL. (Robin Sommer) - The internal links in manual/entire.html have been fixed so they now work (Chema Gonzalez). - A new policy script, blaster.bro, detects instances of the W32.Blaster worm (Robin Sommer). - Signature files (for input to the signature engine) now reside in policy/sigs/*.sig. This directory is now on the default $BROPATH. - sig.ex.ssl-worm.bro and sig.ex.web-rules.bro have been updated to reflect changes in keywords (Robin Sommer). They've been renamed to ex.ssl-worm.sig and ex.web-rules.sig and reside in policy/sigs/, per the above. - The module facility has been changed to have its scope limited to the current file plus any additional files that are automatically processed based on its name plus $BRO_PREFIXES. - As an experiment, ftp.bro has been modified to a style that includes using "module FTP". Likely other policy scripts will be converted in the near future, and their variable names changed accordingly (e.g., "ftp_hot_files" will become "FTP::hot_files"). - The new "match" expression has been modified to allow any yield type rather than just strings. It is likely to change significantly again soon. - Iterating over multi-dimensional tables/sets now works (Robin Sommer). For example: const remote_peers_ssl : table[addr, port] of Peer &redef; [...] for ( [ip, p] in remote_peers_ssl ) connect_ssl(ip, p, remote_peers_ssl[ip, p]$retry); - Checkpointing of persistent state on SIGHUP now happens via bro.init (Robin Sommer). Not tested. - fmt() now supports %x for hexadecimal formatting (Chema Gonzalez). - alert.bro logs the source for remote alerts; by redefining the new "event_source_description: string" locally, you can specify how a Bro is to refered to on the remote side. (Robin Sommer) - software.bro now tracks HTTP clients, too (Robin Sommer). This will be extended in the near future. - Some FreeBSD 5.X porting tweaks (Sergey Osokin). From anton at netForensics.com Wed Oct 22 06:52:28 2003 From: anton at netForensics.com (Anton Chuvakin, Ph.D.) Date: Wed, 22 Oct 2003 09:52:28 -0400 (EDT) Subject: new bro "CURRENT" release - 0.8a48 COMPILATION In-Reply-To: <200310212259.h9LMxlvt031623@jaguar.icir.org> References: <200310212259.h9LMxlvt031623@jaguar.icir.org> Message-ID: All, Trying to compile 0.48 -> getting weird errors. system: Linux RH 9 Intel A. Default =========== 1. ./configure results in: everything seems OK, it says ------------------------ ... config.status: config.h is unchanged Compiling Bro with OpenSSL support: Yes ------------------------ 2. make results in: errors ------------------------ /usr/include/openssl/kssl.h:134: 'krb5_enctype' is used as a type, but is not defined as a type. /usr/include/openssl/kssl.h:136: syntax error before `*' token /usr/include/openssl/kssl.h:149: parse error before `*' token /usr/include/openssl/kssl.h:153: parse error before `*' token /usr/include/openssl/kssl.h:155: parse error before `*' token /usr/include/openssl/kssl.h:157: parse error before `*' token /usr/include/openssl/kssl.h:159: parse error before `*' token /usr/include/openssl/kssl.h:161: `krb5_context' was not declared in this scope /usr/include/openssl/kssl.h:161: parse error before `,' token /usr/include/openssl/kssl.h:162: `krb5_context' was not declared in this scope /usr/include/openssl/kssl.h:162: parse error before `,' token /usr/include/openssl/kssl.h:165: `krb5_timestamp' was not declared in this scope /usr/include/openssl/kssl.h:165: parse error before `,' token /usr/include/openssl/kssl.h:167: parse error before `*' token /usr/include/openssl/kssl.h:169: `krb5_enctype' was not declared in this scope /usr/include/openssl/kssl.h:169: parse error before `,' token make: *** [main.o] Error 1 ------------------------ B. Seeing the above and trying to recover by disabling ssl ========================================================== 1. ./configure --without-openssl results in: everything seems OK, it says ------------------------ ... config.status: config.h is unchanged Compiling Bro with OpenSSL support: Yes (no) ------------------------ 2. make results in: same errors! ------------------------ /usr/include/openssl/kssl.h:134: 'krb5_enctype' is used as a type, but is not defined as a type. /usr/include/openssl/kssl.h:136: syntax error before `*' token /usr/include/openssl/kssl.h:149: parse error before `*' token /usr/include/openssl/kssl.h:153: parse error before `*' token /usr/include/openssl/kssl.h:155: parse error before `*' token /usr/include/openssl/kssl.h:157: parse error before `*' token /usr/include/openssl/kssl.h:159: parse error before `*' token /usr/include/openssl/kssl.h:161: `krb5_context' was not declared in this scope /usr/include/openssl/kssl.h:161: parse error before `,' token /usr/include/openssl/kssl.h:162: `krb5_context' was not declared in this scope /usr/include/openssl/kssl.h:162: parse error before `,' token /usr/include/openssl/kssl.h:165: `krb5_timestamp' was not declared in this scope /usr/include/openssl/kssl.h:165: parse error before `,' token /usr/include/openssl/kssl.h:167: parse error before `*' token /usr/include/openssl/kssl.h:169: `krb5_enctype' was not declared in this scope /usr/include/openssl/kssl.h:169: parse error before `,' token make: *** [main.o] Error 1 ------------------------ Ideas? -- Anton Chuvakin, Ph.D., GCIA, GCIH Senior Security Analyst Product Management Group netForensics - http://www.netForensics.com 732-393-6071 From robin at icir.org Wed Oct 22 10:16:44 2003 From: robin at icir.org (Robin Sommer) Date: Wed, 22 Oct 2003 19:16:44 +0200 Subject: new bro "CURRENT" release - 0.8a48 COMPILATION In-Reply-To: References: <200310212259.h9LMxlvt031623@jaguar.icir.org> Message-ID: <20031022171644.GA10201@net.informatik.tu-muenchen.de> On Wed, Oct 22, 2003 at 09:52 -0400, Anton Chuvakin, Ph.D. wrote: > system: Linux RH 9 Intel Which versions of OpenSSL and gcc are you using? I tried the combinations gcc 2.95.4/OpenSSL 0.9.6c, 2.95.4/0.9.7c and 3.2/0.9.7b. > 1. ./configure --without-openssl Unfortunately, --without-openssl doesn't really work currently. But there doesn't seem to be an easy fix while still keeping the current behaviour of using OpenSSL even if --with-openssl is not given (but my autoconf knowledge is quite sketchy, so perhaps there is). Robin -- Robin Sommer * Room 01.08.055 * www.net.in.tum.de TU Munich * Phone (089) 289-18006 * sommer at in.tum.de -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20031022/9b364aa6/attachment.bin From pande at usc.edu Wed Oct 22 11:03:34 2003 From: pande at usc.edu (mohit pande) Date: Wed, 22 Oct 2003 12:03:34 -0600 Subject: Paper on Algorithmic Complexity Attacks Message-ID: <2d44c62d2134.2d21342d44c6@usc.edu> Hi Scott, I am a student at USC and am giving a presentation on your paper for a course I have taken here. I wanted to know what was the fix (patch) for the Bro IDS. Any idea on what was done to fix the problem. Was the library provided by you used or some other solution. Also, is Perl 5.8.1 vulnerable? If not then what was the fix over the previous versions. I would appreciate your reply on the above. Regards, mohit University of Southern California 2632 Ellendale Pl Apt #118 Los Angeles, CA - 90007 Tel: (323) 737 2099 From scrosby at cs.rice.edu Thu Oct 23 02:26:48 2003 From: scrosby at cs.rice.edu (Scott A Crosby) Date: 23 Oct 2003 04:26:48 -0500 Subject: Paper on Algorithmic Complexity Attacks In-Reply-To: <2d44c62d2134.2d21342d44c6@usc.edu> References: <2d44c62d2134.2d21342d44c6@usc.edu> Message-ID: On Wed, 22 Oct 2003 12:03:34 -0600, mohit pande writes: > Hi Scott, > > I am a student at USC and am giving a presentation on your paper for > a course I have taken here. > I wanted to know what was the fix (patch) for the Bro IDS. Any idea > on what was done to fix the problem. Was the library provided by you > used or some other solution. AFAIK, They have chosen to use a hybrid scheme that uses NH hash construction (see UHASH) for small inputs and a cryptographic hash for large inputs. > Also, is Perl 5.8.1 vulnerable? If not then what was the fix over > the previous versions. No, they fixed Perl 5.8.0. Perl 5.8.x uses a variant of Jenkin's hash (a construction that uses two shifts, 2 adds and an XOR per byte). The difference with 5.8.1 over 5.8.0 was to seed the initial state randomly. In Perl 5.8.2, this will be partially reverted for binary compatibility reasons. It will detect a long chain during insertion and only switch to a randomized hash function if it detects itself under attack. The linux kernel has also been fixed to use a keyed variant of jenkin's hash. Scott From rpang at CS.Princeton.EDU Fri Oct 24 22:27:58 2003 From: rpang at CS.Princeton.EDU (Ruoming Pang) Date: Sat, 25 Oct 2003 01:27:58 -0400 (EDT) Subject: new bro "CURRENT" release - 0.8a48 COMPILATION In-Reply-To: References: <200310212259.h9LMxlvt031623@jaguar.icir.org> Message-ID: I met the same problem with RH9. The short answer is "krb5.h" is in /usr/kerberos/include and therefore is not found in standard include path. More details are discussed at: http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=82 You can either add /usr/kerberos/include to the include path, or apply the following patch and run autoconf before running ./configure: *** bro-pub-0.8a48/configure.ac 2003-10-16 14:17:12.000000000 -0400 --- bro/configure.ac 2003-10-25 01:07:07.000000000 -0400 *************** *** 62,67 **** --- 62,73 ---- with_openssl="No" ) + if test "$with_openssl" != "No"; then + openssl_incl=`pkg-config --cflags openssl` + echo "OpenSSL CFLAGS: ${openssl_incl}" + V_INCLS="${V_INCLS} ${openssl_incl}" + fi + case "$target_os" in linux*) -Ruoming On Wed, 22 Oct 2003, Anton Chuvakin, Ph.D. wrote: > All, > > Trying to compile 0.48 -> getting weird errors. > > system: Linux RH 9 Intel > > A. Default > =========== > > 1. ./configure > results in: everything seems OK, it says > ------------------------ > ... > config.status: config.h is unchanged > > Compiling Bro with OpenSSL support: Yes > ------------------------ > 2. make > results in: errors > ------------------------ > /usr/include/openssl/kssl.h:134: 'krb5_enctype' is used as a type, but is > not > defined as a type. > /usr/include/openssl/kssl.h:136: syntax error before `*' token > /usr/include/openssl/kssl.h:149: parse error before `*' token > /usr/include/openssl/kssl.h:153: parse error before `*' token > /usr/include/openssl/kssl.h:155: parse error before `*' token > /usr/include/openssl/kssl.h:157: parse error before `*' token > /usr/include/openssl/kssl.h:159: parse error before `*' token > /usr/include/openssl/kssl.h:161: `krb5_context' was not declared in this > scope > /usr/include/openssl/kssl.h:161: parse error before `,' token > /usr/include/openssl/kssl.h:162: `krb5_context' was not declared in this > scope > /usr/include/openssl/kssl.h:162: parse error before `,' token > /usr/include/openssl/kssl.h:165: `krb5_timestamp' was not declared in this > scope > /usr/include/openssl/kssl.h:165: parse error before `,' token > /usr/include/openssl/kssl.h:167: parse error before `*' token > /usr/include/openssl/kssl.h:169: `krb5_enctype' was not declared in this > scope > /usr/include/openssl/kssl.h:169: parse error before `,' token > make: *** [main.o] Error 1 > ------------------------ > > B. Seeing the above and trying to recover by disabling ssl > ========================================================== > > 1. ./configure --without-openssl > results in: > everything seems OK, it says > ------------------------ > ... > config.status: config.h is unchanged > > Compiling Bro with OpenSSL support: Yes (no) > ------------------------ > > 2. make > results in: same errors! > ------------------------ > /usr/include/openssl/kssl.h:134: 'krb5_enctype' is used as a type, but is > not > defined as a type. > /usr/include/openssl/kssl.h:136: syntax error before `*' token > /usr/include/openssl/kssl.h:149: parse error before `*' token > /usr/include/openssl/kssl.h:153: parse error before `*' token > /usr/include/openssl/kssl.h:155: parse error before `*' token > /usr/include/openssl/kssl.h:157: parse error before `*' token > /usr/include/openssl/kssl.h:159: parse error before `*' token > /usr/include/openssl/kssl.h:161: `krb5_context' was not declared in this > scope > /usr/include/openssl/kssl.h:161: parse error before `,' token > /usr/include/openssl/kssl.h:162: `krb5_context' was not declared in this > scope > /usr/include/openssl/kssl.h:162: parse error before `,' token > /usr/include/openssl/kssl.h:165: `krb5_timestamp' was not declared in this > scope > /usr/include/openssl/kssl.h:165: parse error before `,' token > /usr/include/openssl/kssl.h:167: parse error before `*' token > /usr/include/openssl/kssl.h:169: `krb5_enctype' was not declared in this > scope > /usr/include/openssl/kssl.h:169: parse error before `,' token > make: *** [main.o] Error 1 > ------------------------ > > Ideas? > > > -- > Anton Chuvakin, Ph.D., GCIA, GCIH > Senior Security Analyst > Product Management Group > netForensics - http://www.netForensics.com > 732-393-6071 >