[Zeek] R: tcp partial connections

[Palumbo Mauro]

Hi Jon,
   Thanks for your reply. As a follow up to my previous question here, I am analyzing zeek processing network traffic which is quite messy, lots of retransmissions, duplicate acks, etc. and a weird log reporting a lot of potential issues. We are looking into fixing these issues. In this context, tcp conns are detected as partials quite often by Zeek and several analyzers do not process correctly the related traffic.

As I would like to get a better understanding of what is going on, I examined a single tcp+http connection in more details. Zeek starts processing it correctly at first, there is the tcp handshake and the first GET/POST are logged correctly. Then Zeek stops processing the http traffic. By debugging, I noticed that in the middle of the conn, zeek calls again the ctor of the tcp analyzer and and as a consequence resets "is_partial" and other variables. Again from debugging it seems this happens because of the timer TCPConnectionExpireTimer is dispatched at a certain point. 

Is this what you would expect in the middle of a conn? It doesn't seem to me that there is a long inactivity on this conn.

Best wishes,
Mauro

-----Messaggio originale-----
Da: Jon Siwek [mailto:jsiwek at corelight.com] 
Inviato: giovedì 28 novembre 2019 17:06
A: Palumbo Mauro <mauro.palumbo at aizoon.it>
Cc: zeek at zeek.org
Oggetto: Re: [Zeek] tcp partial connections

On Thu, Nov 28, 2019 at 7:06 AM Palumbo Mauro <mauro.palumbo at aizoon.it> wrote:

>         if ( TCP() && TCP()->IsPartial() )
>                 return;
>
> This is true for example for the HTTP, SSH, SSL analyzers and more. My understanding is that this is to prevent app layer analyzers or scripts relying on them from breaking down or missing some information when processing packets with possible missing bytes.

Mre related to the "breaking down" part: current protocol parsers don't have any type of "re-synchronization" mechanism so particularly if we miss the TCP handshake and assume we may be starting in the middle of the app-layer protocol stream (or else have a content gap), the parser won't know what to do with the incoming data and so the
IsPartial() checks just exit early, before attempting to parse further.

> How much reliable is this check TCP()->IsPartial()  for partial tcp sessions in the tcp analyzer?

Should be reliable in detecting the problematic scenario AFAIK, but in the case where just the TCP handshake packets are missing and not any segment data, analyzers that exit early like that are skipping streams they actually should be able to parse.

- Jon