[Bro] new Bro CURRENT release (0.9a8)
Vern Paxson
vern at icir.org
Wed Feb 16 17:09:34 PST 2005
A new CURRENT release, 0.9a8, is now available from:
ftp://ftp.ee.lbl.gov/bro-0.9-current.tar.gz
This release includes a large number of changes and bug fixes (appended).
Vern
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0.9a8
- aux/rst/ contains the source for the "rst" tool used by Bro (via the
policy script function terminate_connection() in conn.bro) to tear
down established connections by forging RST packets.
- Bro's main event loop has been reworked (Robin Sommer). This should
(1) not cause any visible differences in most cases, (2) improve
performance in some cases, (3) fixed problems running Bro without
a network input (but still receiving asynchronous input from remote
event sources). There are some more changes coming to this soon.
- Passive OS fingerprinting has been added, based on Michal Zalewski's
"p0f" tool (Holger Dreger). Currently, it's limited to fingerprinting
clients based on the initial SYNs they send. To use it, define
an event handler:
OS_version_found(c: connection, host: addr, OS: OS_version)
OS_version is a record containing a string $genre (e.g., "Solaris"),
a string $detail (e.g., "2.0.27"), a count $dist (hop-count distance
from monitor to host), and $match_type, which specifies via an
enumerated type whether the match was direct from a signature,
generic to the genre, or "fuzzy".
The match is done against a passive fingerprinting signature file,
which is specified by the variable passive_fingerprint_file.
It defaults to "sigs/p0fsyn", which is found using $BROPATH
and has an "osf" suffix added.
You can restrict the matching to only be performed for hosts from
particular subnets by adding those subnets to the variable
generate_OS_version_event. If it's empty (default), then all subnets
are analyzed.
Note, the passive fingerprinting should be integrated with the
version-tracking in software.bro, but this hasn't been done yet.
- Support for IPv6 has been repaired and brought up to date. Note, however,
that inter-Bro communication currently only works over IPv4.
- Signature-matching is now off by default in brolite.bro. If you want
to use it, define use_signatures = T prior to @load'ing it.
- Notices are now tied to their corresponding connections (Scott Campbell).
- New backdoor detectors for IRC, SMTP, Gaobot (Scott Campbell).
- Signature matches now have a connection associated with them (Scott Campbell).
- Bro scripts that set initial timers (via "schedule" statements in a
bro_init handler) but don't have any source of network input (trace
files or live interfaces) now execute in real-time, with network_time
set to the current time, rather than having their timers expire immediately.
- Default timeouts have been added to tables in trw.bro and http.bro, which
have been found operationally to potentially grow very large (Scott Campbell).
- The new policy script large-conns.bro can be included in order to
track the size of TCP connections (each direction is referred to
as a "flow") using a secondary packet filter (Chema Gonzalez).
This method is completely separate from Bro's usual size accounting,
and offers the advantages that it tracks sizes even for connections
that don't terminate (or for which Bro misses their establishment)
and for connections with sizes > 4 GB.
The interface is via the function:
function estimate_flow_size_and_remove(cid: conn_id, orig: bool)
If $orig=T, then an estimate of the size of the forward (originator)
direction is returned. If $orig=F, then the reverse (responder) direction
is returned. In both cases, what's returned is a "flow_size_est" record,
which includes a flag, $have_est, indicating whether there was any
estimate formed. If $have_est is T, then the record also includes
an estimate in bounded by $lower (lower bound) and $upper (upper bound).
The estimate also includes $num_inconsistent, which, if > 0, means that
the estimates came from sequence numbers that were inconsistent, and
thus something is wrong - perhaps packet drops by the secondary filter).
Finally, calling the function causes the flow's record to be deleted.
- An RSH analyzer has been contributed by Manu (ManuX at rstack.org).
It generates rsh_request and rsh_reply events, and the following notices:
DifferentRSH_Usernames
Client and server username differ.
FailedRSH_Authentication
Attempt to authenticate via RSH failed.
InteractiveRSH
The RSH session appears to be interactive (multiple
lines of user commands).
SensitiveRSH_Input
SensitiveRSH_Output
RSH client input or server output match input_trouble/
full_input_trouble or output_trouble/full_output_trouble.
- The new notice action NOTICE_EMAIL indicates that in addition to
logging an alarm, it should also be sent out as email (Scott Campbell).
By default, email is only sent if Bro is running on live traffic;
you can override this via redef'ing the script variable mail_notification.
Mail is sent using the script specified by the mail_script variable
(default: "mail_script.sh", which is now included in the distribution,
but at present is not installed), which must be in $PATH. The mail
is sent to the username specified in mail_dest (default: the local
"bro" user, though you can change this to name at domain).
Note that specifying email as a separate notice action may change
in the future, to instead be an attribute that's associated with
other notice actions. For example, it may make sense to want
to specify both NOTICE_ALARM_PER_CONN and NOTICE_EMAIL; currently,
however, you can't do this.
- A similar notice action NOTICE_PAGE does the same thing as NOTICE_EMAIL
except it send the mail to mail_page_dest (Scott Campbell).
- You can now use the attribute &rotate_size for file objects to
specify the maximum file size in bytes (Robin Sommer). If the limit
is reached, the file is rotated similiarly as is already done with
&rotate (which, for consistency, has been renamed to &rotate_interval).
For both &rotate_size and &rotate_interval, when they trigger they
now generate events (rotate_size and rotate_interval, respectively;
each takes the file as the sole argument) rather than invoking
&postprocessor, which has been removed.
There's also a new variable log_rotate_size to set a global size maximum.
Related to log rotation are the following new built-in functions:
rotate_file(f: file) closes the file, moves it to a temporary
name, and opens a new one. It returns the new "rotate_info"
record, which gives the temporary name and the open/close times.
rotate_file_by_name(s: string): similar, but call by the name
of the file rather than a Bro script value. This is needed
because some files are not represented by file objects but need
to be rotated nevertheless (most importantly, the tcpdump save
file and the dump files for dump_current_packet()). This function
rotates the file with the given name.
Finally, you can load the new policy script rotate-logs.bro to
get default behavior of rotating all log files every hour.
- The new "@unload <script>" directive specifies that future @load's of
<script> should be skipped. This is useful for overriding analyzers
loaded by scripts that pull in a bunch of analysis. For example,
@unload ntp
@load mt
would load all of the "mt" analyzers *except* ntp.bro.
- The new built-in function get_file_name(f: file): string returns
the filename associated with a file (John McNicholas).
- The new built-in function get_contents_file(id: conn_id, direction: count)
returns the contents file (set using set_contents_file()) for the given
direction (John McNicholas).
- The new built-ins time_to_double() and double_to_time() convert between
double values and time values (Robin Sommer). The new built-in floor()
returns the floor of a double value; this returned value is also a double.
Thus, floor(-3.4) returns -4.0.
- Support for sending packets between Bro's (Robin Sommer).
- Bro now has a geneal mechanism internal for traversing policy scripts
(Umesh Shankar). Various script analyses can be specified using the
new -z flag.
Currently, the one supported form of analysis is "-z notice", which
prints all of the different types of notices that the script you've
loaded can generate. For example, "bro -z notice ftp" will generate:
Found NOTICE: BackscatterSeen
Found NOTICE: FTP_PrivPort
Found NOTICE: FTP_BadPort
Found NOTICE: PortScan
Found NOTICE: FTP_ExcessiveFilename
Found NOTICE: ScanSummary
Found NOTICE: AddressDropped
Found NOTICE: DroppedPackets
Found NOTICE: SensitiveConnection
Found NOTICE: FTP_UnexpectedConn
Found NOTICE: SSH_Overflow
Found NOTICE: FTP_Sensitive
Found NOTICE: TerminatingConnection
Found NOTICE: PasswordGuessing
Found NOTICE: AddressDropIgnored
Found NOTICE: AddressScan
- The signature rule language now supports an "active" keyword,
which can be set to "true" or "false", with the latter turning
off the rule (Roger Winslow). If set to false the signature will
not be loaded into the rule matcher, otherwise it is.
- The signature rule language now supports meta data of the form
".MMM<whitespace>XXX", where MMM is arbitrary text which makes up the name
of the meta data option and where XXX is arbitrary text up to the end
of the current line (Roger Winslow). The intent is that some forms of
meta-data will be regularized/standardized in the future - information
such as date modifed, category/class, weighting, etc. For now, it
provides a way to annotate rules with nominally more structure than just
using comments (though it is currently treated the same, i.e., everything
is ignored).
- The following meta data option names are now reserved: .version, .revision,
.version-date, .revision-date, .date-created, .location
- The new enumerated type "transport_proto" is used to specify different
types of transport protocols: "tcp", "udp", "icmp", and "unknown_transport".
Associated with it are new built-in functions: is_udp_prot(), is_icmp_port(),
get_conn_transport_proto, and get_port_transport_proto. The latter two
map a given connection and a given "port" value to their corresponding
transport_proto value.
- A bunch of tuning (regular expressions for sensitive login sessions,
scan detection thresholds, forbidden/hot usernames and filenames,
sensitive URIs, "weird" actions) have been incorporated from
operational configurations.
- Serious bugs in managing large numbers of files fixed.
- A serious bug with negative DNS TTL settings (and, more generally,
with negative timer values) fixed.
- The traditional connection logging format is no longer supported.
- The SMTP analyzer's state machine processing has been modified to
correctly deal with clients that (incorrectly) pipeline their commands
(Ruoming Pang).
- A bug fixed in detecting SMTP relays for connections w/o message bodies
(Ruoming Pang).
- A bunch of bugs in recording connection summaries for UDP flows
have been fixed.
- A new script module, passwords.bro, generates PasswordExposed notices
for activity (currently just rlogin/telnet logins) that expose passwords.
- A new script module, file-flush.bro, can be loaded to cause all log
files to be flushed every file_flush_interval (default: 10) seconds.
This is handy if you like to watch the files in real time.
- Zone transfers now generate a ZoneTransfer notice, unless the host
making the request is in DNS::zone_transfers_okay.
- Bro's DNS cache (generated using -P and accessed using -F) is
now kept in the .state/ subdirectory rather than in the user's
home directory (Roger Winslow).
- Some changes to remote propagation of events/values and detection of
state inconsistencies (Robin Sommer).
- A fix for avoiding delays on low-volume links for some systems for
which it can take a long time to fill up the pcap buffer, and pcap doesn't
return partial buffers (Robin Sommer).
- A bug in table expiration timers has been fixed (Robin Sommer).
- A bug in comparing subnets has been fixed.
- A bug in using a non-constant value for a &write_expire attribute
has been fixed.
- A bug in using CONTENTS_BOTH for writing reassembled streams to
files has been fixed (John McNicholas).
- A subtle but potentially damaging bug in fragment reassembly has
been fixed.
- A bug with using local variables of vector types has been fixed.
- A bug with comparing strings has been fixed.
- Bro no longer generates the RST_with_data "weird", as with modern
stacks it's no longer any sort of strange occurrence.
- Related to this, the signature rule matcher no longer matches
against the payload of RST packets. (Note, this is an imcompatibility
with Snort.)
- Portmapper mappings are now written in the connection log in
alphabetical order.
- The event engine variable frag_timeout now defaults to 5 minutes if you
load frag.bro, and is accessed via redef rather than by defining the
global directly.
- The interval that signatures.bro waits for until generating a signature
summary can now be set using the new script variable sig_summary_interval,
and a bug in generating the summaries has been fixed (Robin Sommer).
- The new script peer-status.bro generates periodic "update" events regarding
a remote peer's status (Robin Sommer). These take the form:
type peer_status: record {
res: bro_resources;
stats: net_stats;
current_time: time;
cpu: double; # average CPU load since last update
default_filter: string; # default capture filter
};
- The bro_resources record returned by resource_usage() now includes
three additional fields, $version (the version of Bro), $debug
(T if Bro was compiled with debugging information), and $start_time
(the time Bro began executing - clock time, not network time).
- The new built-in function same_object(o1: any, o2: any): bool
returns true if its arguments refer to the same object, false
otherwise. This can be useful for comparing tables, for example
in calls to table element expiration functions.
- The new built-in function bro_is_terminating(): bool returns true if
Bro is done reading from its network input source(s) and is now
in its final termination cleanup (Robin Sommer).
- A new built-in strftime() formats a timestamp, returning a string
(Robin Sommer).
- A new built-in file_size() returns the size of the file with a
given name (Robin Sommer - note: *not* a Bro file value).
- A potential deadlock with inter-Bro communication has been fixed
(Robin Sommer).
- Bro now always forks a copy of itself when executing, as this
can save considerable memory when using inter-Bro communication
(Robin Sommer).
- The Bro interconnection protocol now includes explicit handshaking
during session establishment to mark that a peer is ready (Robin Sommer).
Implementing this includes a change in the wire protocol that is
incompatible with the protocol used in the past.
- The TCP inactivity timer is now started whenever a connection
transitions from a pre-establishment state (including "inactive")
to some sort of established state (Robin Sommer). Prior to this
fix, connections for which a proper SYN handshake was not seen would
not be timed out as inactive.
- The --disable-openssl configure option has been removed; now
the only option is --with-openssl, and --with-openssl=no disables
use of OpenSSL (Gregor Maier).
- A bug in invoking &expire_func functions has been fixed (Robin Sommer).
- A bug in logfile rotation has been fixed (Robin Sommer).
- A bug in recognizing negative floating point values has been fixed.
- worm.bro now suppresses the default signature action for worms
it knows about, since it generates events for them (Robin Sommer).
The list of worms detected via signatures now includes Bagle-BC.
- Signatures for known worms are now skipped when doing signature
summaries and scan detection, if worm.bro is loaded (Robin Sommer).
- request_remote_events and request_remote_sync now implicitly
do set_accept_state, too.
- Better error handling for SSL connections (Robin Sommer).
- Bug fixed which caused diagnostic messages to be lost when using
inter-Bro communication (Robin Sommer).
- gcc 3.4 portability fixes (Brian Lindauer).
- Solaris portability fixes (Robin Sommer).
- The Bro distribution now includes and uses its own version of libpcap
for portability reasons (Jason Lee).
- Some minor bug fixes to handling of tcpdump save files (Robin Sommer).
- Detection added for a (now quite old) SSHv1 overflow attack.
- A bug in skipping processing of connections for large chunks of
data has been fixd (Chema Gonzalez).
- Some memory leaks fixed (Robin Sommer).
- fmt()'s "%d" format now accepts values of enum types.
More information about the Bro
mailing list