new Bro release available - 0.7a175
Vern Paxson
vern at icir.org
Fri Aug 30 00:30:30 PDT 2002
A new Bro alpha release is now available from:
ftp://ftp.ee.lbl.gov/.vp-bro-pub-0.7a175.tar.gz
It includes a considerable number of changes and new features (some quite
experimental and likely to change somewhat in the near future). I've
appended the relevant entries from the CHANGES file.
Vern
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
v0.7a174 Thu Aug 29 21:14:34 PDT 2002
- bro -s <file> reads in a *signature* file to search for regular expressions
in packet payloads or TCP byte streams (written by Robin Sommer).
See policy/rules.bro for an example of a policy script for processing
the matches.
Note that this feature is experimental, and will be evolving in the
near future.
- The python script "snort2bro" reads in Snort signatures and translates
them into Bro signature rules, suitable for processing using -s.
An example of its operation is seen by running
python snort2bro < sig.ex.web-rules.snort
which, after reading in sig.ex.classification.config and
sig.ex.reference.config, generates the output given in
sig.ex.web-rules.bro, which is suitable to use as input to
bro -s.
- bro -d invokes a gdb-like debugger (written by Umesh Shankar). You can
set breakpoints and watchpoints, examine tracebacks, print Bro expressions,
and the like. Type "help" for on-line help.
- bro -t <tracefile> turns on tracing of the policy script execution,
written to the given file.
- Bro now includes an SMTP analyzer, which includes processing MIME
message bodies (written by Ruoming Pang). See smtp.bro and mime.bro
for related policy scripts. smtp.bro includes several experimental
techniques for detecting mail relaying.
- You can now define enumerated types, such as
type scanner_type: enum {
SCANNER_STEALTH, SCANNER_HIGH_SPEED, SCANNER_AMBIGUOUS,
};
Enumerated types can be compared for equality with one another, and used
as table indices, but cannot be converted to/from integers.
- bro -A <file> invokes an experimental, general trace transformation/
anonymization framework (written by Ruoming Pang) which writes a modified
tcpdump trace file from the input (which can be the network or another
trace file) with potentially extensive modifications to the recorded
packets.
Transformers are built from .rw files (currently, {finger,ftp,ident,smtp}.rw),
which are processed by the utility "rwcl" to generate both event engine
analyzer components and rewriter policy scripts (for example, after
configuring and building Bro, you'll find the scripts
policy/{finger,ftp,ident,smtp}.rw.bro).
See policy/smtp-rewriter.bro for an example of a policy script that
performs transformation/anonymization.
- New built-ins:
split(s: string, p: pattern): string_array;
takes a string and splits it into pieces at each occurrence of
the regular expression pattern p. (The functionality is like
that in awk.) It returns a string_array, which is a table[count]
of string that is indexed starting at 1, giving the different
pieces.
For example,
split("foobar", /o/)
returns a 3-element table, for which [1] is the string "f",
[2] is the string "" (empty), and [3] is the string "bar".
split1(s: string, p: pattern): string_array;
split1() does the same thing as split(), but only performs splitting
at the first occurrence, so it returns either a one-element table
(if the pattern doesn't appear in the string) or a two-element
table. split1("foobar", /o/) returns a 2-element table for which
[1] is "f" and [2] is "obar".
md5_hash(s: string): string
returns (in human-readable form) the MD5 hash of a given string.
So, for example,
md5_hash("foobar")
yields "3858f62230ac3c915f300c664312c63f".
to_addr(s: string): addr
takes a string representing an address in "dotted quad" format
and returns the correponding "addr" value.
set_buf(f: file, buffered: bool)
sets the given file to have its writes buffered or unbuffered
depending on the value of "buffered". It does not return a value.
connection_exists: function(c: conn_id): bool
returns T if the given connection identifier corresponds to a
currently instantiated connection (one for which the event engine
has state), F otherwise.
lookup_connection(c: conn_id): connection
returns the "connection" record associated with the given
connection identifier, or a fatal run-time error if there
isn't one.
set_inactivity_timeout(c: conn_id, timeout: interval): interval
sets the inactivity timeout for the given connection to the
given interval, returning the old interval.
If the interval is non-zero, then when no packets have been
processed for a connection after that much time has elapsed,
the connection is deleted, and an "inactivity_timeout" event
generated.
get_matcher_stats(): matcher_stats
used for gathering statistics about the signature matcher
rewriting_trace(): bool
returns T if -A was specified (anonymize/rewrite a trace),
F otherwise.
- New events:
connection_state_remove(c: connection);
Invoked when the event engine has removed the connection from
its state.
connection_SYN_packet(c: connection, pkt: SYN_packet);
Invoked for each SYN/SYN-ACK packet.
connection_timeout(c: connection);
Invoked when the event engine times out a connection - for
example, because the originator sent a SYN that was never
answered, so the connection was never established.
connection_reused: event(c: connection);
Invoked when the event engine decides that a new SYN for
an existing connection reflects a reuse of the connection
four-tuple, rather than belonging to the existing connection.
- New globals:
const ignore_checksums = F &redef;
If true, then the event engine does not verify checksums (and
hence will not discard packets with bad checksums).
const tcp_deliver_undelivered = F &redef;
If true, then when the event engine closes a connection, if
that connection has a chunk of data not yet delivered to its
analyzer (which only happens if the data is above a sequence
hole, indicating either a packet filter glitch or a protocol
botch), then the undelivered data will at that point be delivered
to the connection's analyzer.
const tcp_reassembler_ports_orig: set[port] = {} &redef;
const tcp_reassembler_ports_resp: set[port] = {} &redef;
Sets of ports for which, if a connection has the corresponding
originator/responder port, then the event engine will reassemble
the byte stream of the connection.
Normally, the event engine reassembles byte streams for any
connection for which there's an analyzer, and otherwise doesn't.
These variables can be used to force reassembly for the originator
or responder side (respectively) of connections for which there
isn't otherwise an analyzer. This is useful when doing signature
matching on reassembled byte streams, for protocols that are
not otherwise analyzed by the event engine.
const table_expire_interval = 1 min &redef;
How often to check table entries to see whether they've expired
(see &read_expire, etc., below).
const requires_trace_commitment = F;
If true, then when rewriting/anonymizing traces, nothing will
actually be written to the edited trace file unless you call:
rewrite_commit_trace(c: connection, commit: bool, future: bool)
If "future" is true, then future rewritten packets will be
automatically commited; otherwise, writing them to the trace
file requires another explicit rewrite_commit_trace() call.
const inactivity_timeout = 0 secs &redef;
As noted above, when a connection becomes inactive, time it out
after this interval. If 0 secs, then don't time it out.
- An SSH analyzer extracts SSH client/server version information. See
ssh.bro for the related policy script.
- There's now a (very) simple TFTP analyzer available in tftp.bro.
- You can now set the global "frag_timeout" to an interval which controls
how long fragments are kept before discarding them (contributed by Ashley
Thomas). If you don't set the global, or set it to 0.0 sec, then fragments
are kept around indefinitely.
- An implementation of an experimental anti-evasion technique, "active
mapping", has been written by Umesh Shankar. It is not yet ready for
general use, and isn't compiled in unless -DACTIVE_MAPPING.
- Four new attributes can now be associated with tables (implemented
by Robin Sommer): &read_expire, &write_expire, and &create_expire
will delete table entries after a given interval has elapsed since
the table entry was last read, written, or created. For example:
global a: table[addr] of count &default=0 &create_expire = 5 sec;
will delete each entry added to it 5 seconds after the entry was added,
regardless of subsequent read/write activity to the element.
&expire_func allows you to associate a function with the table such that
whenever an entry expires, the function is invoked. It's passed the
value of the table entry (not the index - perhaps this should be changed),
and returns an "interval" value. If the interval is <= 0.0 seconds, then
the table entry is immediately deleted. Otherwise, it is deleted after
the given interval has elapsed.
- When listing multiple attributes, you no longer separate them with
commas. For example, if you used to have:
global a: table[string] of count &default=0, &redef;
you now need to use:
global a: table[string] of count &default=0 &redef;
- You can now construct records using
[$field1 = <expression>, $field2 = <expression>, ...]
Such record values can be assigned to other records providing that the
target value's type includes all of the fields (same name and type)
present in the record value, and that any missing fields have the
&optional or &default attribute (see next item).
You can also include a record value inside the record constructor, and
all of its fields will be included in the constructed record value.
- Record fields can now be annotated with &optional, to indicate
that the field needn't be present, or &default, which indicates
a default value to provide if the field is missing.
- You can query as to whether a record has a value for a given field
using the new "?$" operator. So for example:
type my_rec: record {
num: count &default = 0;
msg: string; # mandatory, since no &optional/&default
};
global r: my_rec;
r = [$msg = "hello"];
print r?$num, r?$msg, r$num;
will print "F, T, 0," because even though 'r' has a default value
for $num (which shows up when printing r$num), that field is missing,
hence r?$num is F.
- An experimental scheme has been added (by Umesh Shankar) for managing
general attributes associated either with all values ("global attributes")
or particular particular values. This scheme is likely to change in
the near future, and hence isn't explained here further.
- The DNS analysis now includes ADDL and AUTH records, and much richer
policy script analysis (per policy/dns.bro).
- You can now "redef" a function or event handler to override its
previous definition. For a function, this looks like:
redef log_hook = function(msg: string): bool
{
...
}
For an event handler, it's just the usual definition preceded by "redef.
For example,
redef event ack_above_hole(c: connection) { }
would replace the default ack_above_hole handler with one that does nothing.
- HTTP server and HTTP proxy backdoor detectors have been added,
generating http_signature_found and http_proxy_signature_found,
respectively (contributed by Ruoming Pang).
- A KaZaA backdoor detector has been added, which generates
kazaa_signature_found for likely KaZaA connections.
- The new policy scripts flag-irc.bro and flag-warez.bro provide
hooks for defining site policies for detecting IRC and access
to warez.
- portmapper.bro now tracks the services it sees, and the names are
used in connection summaries rather than generic services like port-656.
- bro -C (or redef'ing the "ignore_checksums" global to T) instructs
Bro to ignore any checksum errors and go ahead and analyze such packets.
- The (trivial) policy script print-globals.bro dumps out all of the policy
script global variables and the amount of memory they consume.
- The policy script code-red.bro has been renamed worm.bro and generalized
to detect Nimda as well as Code Red 1 & 2.
- A bunch of additional default sensitive URIs have been added to http.bro.
http.bro also now doesn't report worm-related URIs.
- A bunch of less common portnames were removed from port-names.bro.
- Empty regular expressions are now allowed.
- The finger_request event now has a third parameter, the additional
text after the username.
- More systematic handling of NULs and CRLF by the event engine.
- Hex escape sequences now must have exactly two hexadecimal characters.
- FYI - work has begun on significantly altering the way that policy
scripts generate alerts.
- Work has begun (by Robin Sommer) on a general framework for tracking
client/server versions. See software.bro.
- Work has begun on a NETBIOS analyzer (see NetbiosSSN.cc). Contributions
(e.g., finishing it :-) welcome.
- Work has begun on migrating the internals to process IPv6 in addition
to IPv4.
- A number of bug fixes, leaks, and memory allocation lint tweaks.
More information about the Bro
mailing list