[Bro] new Bro CURRENT release (0.9a7)

[Vern Paxson]

A new CURRENT release, 0.9a7, is now available from:

	ftp://ftp.ee.lbl.gov/bro-0.9-current.tar.gz

This release includes a number of changes (appended), with one major one
that will break backwards compatibility being the renaming of "log" to
"alarm" and "alert" to "notice".

This is a fairly stable snapshot prior to another release forthcoming soon
that will have a number of further changes, including some new log file
formats.

		Vern


-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


0.9a7 Mon Nov  1 13:21:05 PST 2004

- New terminology:

	o We've found that the term "log" has been too overloaded,
	  sometimes meaning "something to record for audit purposes" and
	  other times meaning "something worthy of getting the operator's
	  attention right now, for example via syslog".

	  We are now using "log" to only refer to the first of these,
	  and refer to the second as "alarm".

	o We've found that "alert" (and the ALERT() function, etc) is
	  likewise confusing.  Some expect it to mean something alarm-worthy
	  (to use the new name from the previous item), while others
	  expect it to mean the output from a sensor, which might not
	  be worth getting the operator's attention.

	  We are now using "notice" to refer to what had previously
	  been called "alert".  So, for example, rather than call ALERT()
	  you call NOTICE() to enter something into the I've-seen-something-
	  maybe-it's-worth-an-alarm framework.  The first field associated
	  with such a call was $alert, which is now $note.  These notifications
	  are logged to notice.$BRO_LOG_SUFFIX rather than "alert."...

  These changes show up in many places in the policy scripts.  Some
  globals log_XXX are now alarm_XXX or notice_XXX.  However, uses of
  "log" that refer to "something to record for audit purposes" remain
  using that name.

- Bro now uses the "automake" suite of tools for its configuration
  (Jason Lee).  This includes major reworking of its Makefile's
  and configuration/installation scripts.

- Bro now flags event handlers that are never invoked (Umesh Shankar).
  This catches typos like:

	event bro_initt() { ... }

- The scripts directory now includes scripts for generating nightly
  reports (Roger Winslow).  Note that these are not presently a supported
  part of the public Bro distribution, but will become so in the
  not-too-distant future.

- policy/brolite.bro contains a first version of the "Bro Lite" configuration
  (Scott Campbell, Roger Winslow, et al).  This also is not presently
  supported for the public Bro distribution, but will become so.

- Major overhaul of the internal serialization framework (Robin Sommer).
  Some more changes are forthcoming, though not on this scale.

- Changed packet capture/drop statistics to (1) explicitly track received
  packets, (2) work better under Linux, (3) report link counts when available
  (Robin Sommer).  The net_stats record field "interface_drops" has been
  renamed "pkts_link".  It reports the number of packets captured by the
  NIC (if available), rather than the number dropped by the NIC (which never
  actually worked, anyway).

- The DNS analysis has been extended to deal with TSIG and to better deal
  with EDNS0 records, and its overall structure reworked somewhat
  (Scott Campbell & Roger Winslow).  The scripts in dns.bro are now
  inside "module DNS" scope.

- Improved logging of ICMP flows (Scott Campbell).  This includes the
  addition of a "len" field in icmp_conn.  Some significant additional
  changes/improvements to ICMP processing will be coming very soon.

- The Bro README and "quick start" documentation has been updated
  (Brian Tierney and Jim Rothfuss).

- Some significant state-holding problems in the presence of packet filtering
  or packet drops have been fixed.

- You can now instruct the signature engine to constrain the number of
  regular expressions it groups together into a single matcher using the
  new script variable "sig_max_group_size" (Robin Sommer).  This can result
  in significant memory savings for large sets of signatures.  It can
  increase the CPU processing required; however, in our testing so far
  this does not appear to be the case.

- The signature engine now provides better location information in its
  error messages (Robin Sommer).

- statistics.bro has been renamed profiling.bro.  Along with this change,
  the following script variables have been renamed:

	do_statistics => do_profiling
	statistics_file => profiling_file
	statistics_interval => profiling_interval

- A new script, stats.bro, can be included to generate light-weight
  running statistics on memory and CPU use.

- A new script, print-sig-states.bro, can be included to generate
  periodic dumps of signature-matching statistics (Robin Sommer).

- The connect_clear()/connect_ssl() built-ins have been replaced
  by a single connect() function (Robin Sommer).

- Remote Bro's (more generally, remote event sources) are now represented
  in policy scripts using the new "event_peer" type rather than as an
  addr/port pair, and get_event_source() has been renamed get_event_peer()
  (Robin Sommer).  The new function get_local_event_peer() returns an
  event_peer corresponding to the local Bro.

- The new script remote-print.bro can be used to print all events
  received from remote sources (Robin Sommer).

- When you call send_state(), an event finished_send_state() is now
  generated when all of the state has been sent (Robin Sommer).

- If you define a handler for the new print_hook() event, then it
  will be invoked every time a "print" statement executes (Robin Sommer).

- The -g flag no longer takes a directory as an argument, but always
  uses the .state directory (Robin Sommer).

- The new -I flag prints the value of a given identifier and exits
  (Robin Sommer).  This flag may go away in the future.  For now,
  its role is that it prints timestamps of things like table elements,
  which are no longer printed by a regular "print" statement.

- If a connection proceeds as SYN ->, <- RST, RST ->, this is now
  reported as a regular rejected ("REJ") connection, rather than
  one reset by the originator ("RSTO").

- The TCP analyzer now attempts to detect connections for which
  one side's packets are being filtered out.  It does this using
  two C++ variables (not presently accessible at the script level),

	const int max_initial_window = 4096;
	const int max_above_hole_without_any_acks = 4096;

  If more than max_initial_window data has been sent by one side and no
  ACKs have been seen by the other side, then it's assumed that no ACKs
  will ever show up, and the buffer for the data should be reclaimed
  immediately after it's reassembled.  max_above_hole_without_any_acks
  plays a similar role, but for the case when measurement drops have
  lead to Bro holding data for a sender for which it will never see
  an ACK (which would be an "ACK above hole"), so it should give up
  on buffering it.

- A portability bug in Bro's parser code has been fixed (Jason Lee).

- STARTTLS for SMTP now causes the SMTP session to be ignored, rather
  than generating a slew of error messages because the session becomes
  unparseable.

- Bro's "watchdog" handler is now less prone to crashing due to
  using non-reentrant library calls (Jason Lee).

- Better error message when reading an input file encounters an error.

- adtrace now prints its usage when invoked without the required
  arguments (Jason Lee).

- A bug in using recursive script functions has been fixed.

- A bunch of whitespace/code layout tweaks.