[Bro] File Extraction Gaps

[Weasel, Gary W Jr CIV DISA RE (US)]

Hey Bro List,

So I seem to be running into a problem with file extraction (or perhaps just file analysis in general).  I have a basic extraction script running pulling out EXEs that are seen coming across HTTP and for some reason, there are consistently a large number of file gaps in the file it sees.  I have a custom log outputting the fuid for any file_gap event (https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html#id-file_gap), and I seem to get a wildly varying number of file gap events for a given file.

In my example, I am curling an exe to a server, where that traffic is spanned to my Bro sensor (the exe in question is 1 MB in size).  If I curl repeatedly, Bro sees all the files, but the number of file gap events varies wildly (anywhere from 2 or 3 to over 100).  The part that gets me, if I tcpdump alongside Bro, and pull the files out of pcap, they're all intact and hash correctly, so I know I'm getting all the packets on wire.  Bro and PF_RING report 0 packet loss.

Does anyone have anything that could shed light on what's going on here?

Thanks,
- Gary
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5577 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180529/e3be3b4c/attachment.bin