[Bro] Hashing incomplete files

[Josh Liburdi]

Hey everyone,

Hopefully anyone who has looked at or worked on the hashing component of
the file analysis framework can help out with my request. I have a need for
Bro to hash all files, including incomplete ones. I looked at the file
hashing source code and making Bro hash incomplete files seemed straight
forward (comment out the lines that break file hashing if there is an
undelivered chunk), but I'm getting an odd result: the hashes reported by
Bro for incomplete files are not the same hashes as what is extracted by
Bro.

For example, here's a files.log entry for an incomplete file with hashing
enabled:

1493035575.544634 Fb19KI1OvvCjlT49eg 1.2.3.4 1.2.3.4 CKcFdN2BuVOe1wiFB HTTP
0 EXTRACT,SHA1,MD5 - - 0.036770 - F 32221 59247 27026 0 T -
62f2c17b427ab54f9a8e30f384ba2a5e 6cba20d301dde6d7cbc4f41c689c1ecd108d7bef -
extract-1493035575.544634-HTTP-Fb19KI1OvvCjlT49eg

Here is the MD5 hash as reported by the file system:

f0d987adb1015a05aabfcbade38751b1
 extract-1493035575.544634-HTTP-Fb19KI1OvvCjlT49eg

Any thoughts on why these hashes don't match? I'm guessing that enabling
this functionality isn't as simple as not breaking the hashing function
when an undelivered chunk is found.

Thanks,
Josh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170425/a9337df6/attachment.html