[Bro] bro bug? - unreported packet loss

[Siwek, Jonathan Luke]

On Jan 20, 2014, at 3:23 PM, David Gugelmann <david.gugelmann at tik.ee.ethz.ch> wrote:

> sorry for bothering you again. As I posted two weeks ago (see below), it
> seems to me that bro sometimes does not report packet loss in TCP
> connections (missed_bytes in conn.log is 0 even though there are packets
> missing).
> 
> You can find the corresponding pcap files and additional information here:
> http://people.ee.ethz.ch/~gugdavid/bro_missed_bytes.zip
> 
> Am I missing something or did anybody encounter something similar?

You did seem to find situations where Bro doesn’t accurately report missed content.  Generally, I think it’s situations where packets are missed towards the end of connections, but specifically your example pcaps show slightly different special cases in the TCP reassembler…

example1.pcap: a gap is seen at the same time as a FIN from an endpoint and no content has been reassembled yet.  Code in TCP_Reassembler.cc skips reporting that on the chance the pcap trace if pre-filtered.  Associated comments:

 		// We could be running on a SYN/FIN/RST-filtered trace - don't
		// complain about data missing at the end of the connection.
		//
		// ### However, note that the preceding test is not a precise
		// one for filtered traces, and may fail, for example, when
		// the SYN packet carries data.
		//
		// Skip the undelivered part without reporting to the endpoint.

example2.pcap: a gap is seen, but the connection state is no longer established at both ends (a FIN has been seen from one).  Associated comments:

 			// Only report on content gaps for connections that
			// are in a cleanly established state.  In other
			// states, these can arise falsely due to things
			// like sequence number mismatches in RSTs, or
			// unseen previous packets in partial connections.
			// The one opportunity we lose here is on clean FIN
			// handshakes, but Oh Well.

The second case has a workaround by redefining the “report_gaps_for_partial” [1] option to true at risk of unreliable gap reporting for partial connections.  There’s no workaround for the first case, but I think adding a similar redef-able option for explicitly indicating whether a filtered trace is used would be better than Bro just making a guess.

Would you create a bug report for this at bro-tracker.atlassian.net ?

- Jon

[1] http://www.bro.org/sphinx/scripts/base/init-bare.html#id-report_gaps_for_partial