[Bro] [Non-DoD Source] Re: File Extraction Gaps

[Jon Siwek]

Would be most helpful to get a pcap, scripts, and command that can
reproduce what you see.  Or else steps one could follow to try to
reproduce a pcap that may exhibit the same problem.  Otherwise, only
guess I have is to check for anything unusual at the TCP-layer -- the
file analysis/extraction is dependent on the TCP reassembly process,
so if the sequence of events at the TCP level lead Bro to believe it
missed part of the TCP stream, that also manifests as a gap event in
any associated file analysis.  You could go as far as looking at the
contents/ordering of things in a tcp_packet event handler as a sanity
check, though there may also be residuals in weird.log that you could
simply check (I don't recall particular names to look for off the top
of my head).

- Jon

On Tue, May 29, 2018 at 12:22 PM, Weasel, Gary W Jr CIV DISA RE (US)
<gary.w.weasel2.civ at mail.mil> wrote:
> So I just tested running bro in command line mode (i.e. not using broctl), fed it my usual policy files and dumped to pcap.
>
> The extracted_files folder showed the same story, lots of file gaps, all different hashes for the same file.
>
> When I loaded up the pcap in wireshark and extracted all the files, all files hash correctly.
>
> So this tells me, at some point Bro is getting all the data.  Something is just messing up for some reason when it comes to the file analysis and/or file extraction modules.
>
> -----Original Message-----
> From: Jon Siwek <jsiwek at corelight.com>
> Sent: Tuesday, May 29, 2018 12:52 PM
> To: Weasel, Gary W Jr CIV DISA RE (US) <gary.w.weasel2.civ at mail.mil>
> Cc: bro at bro.org
> Subject: [Non-DoD Source] Re: [Bro] File Extraction Gaps
>
> On Tue, May 29, 2018 at 9:55 AM, Weasel, Gary W Jr CIV DISA RE (US)
> <gary.w.weasel2.civ at mail.mil> wrote:
>
>> In my example, I am curling an exe to a server, where that traffic is spanned to my Bro sensor (the exe in question is 1 MB in size).  If I curl repeatedly, Bro sees all the files, but the number of file gap events varies wildly (anywhere from 2 or 3 to over 100).  The part that gets me, if I tcpdump alongside Bro, and pull the files out of pcap, they're all intact and hash correctly, so I know I'm getting all the packets on wire.  Bro and PF_RING report 0 packet loss.
>
> Seems either Bro behaves differently in offline vs. live usage (not
> what I'd expect to see in this case, though I can't rule it out for
> sure) or it's not actually seeing the same packet input in the live
> deployment as it was in offline usage.  Maybe to pursue whether the
> later is true you could have Bro write out the packets that it
> actually saw with the `-w <pcap file>` command line flag and examine
> the resulting pcap to see if it looks like you expect.
>
> - Jon
>