[Bro] Inconsistent file size during extraction
Seth Hall
seth at corelight.com
Thu Feb 1 19:49:28 PST 2018
Yep, I was going to comment that that's probably the issue, but I'll
give some more details on why things may end up that way.
"total_bytes" - is for when the size of the file is known by some
secondary mechanism, like the file size being transmitted as part of a
protocol or a file being read off disk.
"seen_bytes" - represents the number of actual bytes of data that
were passed into the file analysis framework.
This is another case where small packet loss issues can have outsized
effects because the following bytes can't be reassembled into the file
correctly and you don't get anymore data.
Also, nice to see on the mailing list again Josh!
.Seth
On 1 Feb 2018, at 22:07, Josh Liburdi wrote:
> Seems that this particular connection may be affected by tapping
> issues.
>
> On Thu, Feb 1, 2018 at 4:13 PM, Josh Liburdi
> <liburdi.joshua at gmail.com>
> wrote:
>
>> Hi all,
>>
>> I'm seeing instances where files are being extracted inconsistently
>> with
>> what is reported in files.log. Here is a redacted example:
>>
>> files.log:
>> #fields ts fuid tx_hosts rx_hosts conn_uids source depth analyzers
>> mime_type filename duration local_orig is_orig *seen_bytes*
>> *total_bytes*
>> missing_bytes overflow_bytes timedout parent_fuid md5 sha1 sha256
>> extracted extracted_cutoff extracted_size
>> #types time string set[addr] set[addr] set[string] string count
>> set[string] string string interval bool bool count count count count
>> bool
>> string string string string string bool count
>> 1517528771.042220 Fz2Z2m3zwQcc3gqDS3 x.x.x.x x.x.x.x
>> CpaGD227W0Cy2BA1Tf
>> HTTP 0 EXTRACT application/vnd.openxmlformats-officedocument.
>> spreadsheetml.sheet 0.258350 - F *219414* *12977556* 0 0 F - - - -
>> extract-1517528771.04222-HTTP-Fz2Z2m3zwQcc3gqDS3 F -
>>
>> File on disk:
>> *219414* Feb 1 16:04
>> extract-1517528771.04222-HTTP-Fz2Z2m3zwQcc3gqDS3
>>
>> The file on disk is the same size as the amount of bytes sent to the
>> file
>> analyzer (seen_bytes field) -- it should be the same size as the
>> total_bytes field. I've seen this happen many times (though,
>> relatively
>> speaking, it is rare).
>>
>> Any thoughts on this behavior? I'm seeing this on Bro 2.5.1.
>>
>> Thanks,
>> Josh
>>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
--
Seth Hall * Corelight, Inc * www.corelight.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20180201/98b496a3/attachment-0001.html
More information about the Bro
mailing list