[Bro] Extract files based on magic number using Bro 2.2

Marius Portaas Haugen mariusph at ifi.uio.no
Sun Jan 26 13:23:39 PST 2014

Hi Jon,

thanks a lot for taking the time to answer my question!
Comparing the the bof_buffer works like a charm! Again, thanks!

- Marius

On 2014-01-24 16:21, Siwek, Jonathan Luke wrote:
> On Jan 24, 2014, at 6:44 AM, Marius Portaas Haugen 
> <mariusph at ifi.uio.no> wrote:
>> I'm just wondering; Is it possible to extract files based solely on
>> their magic number using Bro 2.2
>> In Bro 2.1, it was possible to extract files just by comparing the
>> magic number
>> with the first X bytes. I used the script provided here, with great
>> success:
>> http://scrapbook.zscaler.com/2012/05/bro-script-to-extract-artifacts-from.html
>> However, in Bro 2.2, thigs seem to have changed. Most examples and 
>> docs
>> now only
>> seem to use the MIME-type to determine if a file will be extracted or
>> not, e.g. here:
>> http://www.bro.org/sphinx-git/frameworks/file-analysis.html
> You can try handling the “file_new” event, compare f$bof_buffer
> (Beginning Of File Buffer) to whatever magic you want, and then add
> the file extraction analyzer to f if it matches (similar to the
> examples in that webpage you cite, except using f$bof_buffer as the
> condition instead of f$mime_type).
> The “file_new” event is network protocol agnostic so if it’s
> important to only extract stuff over HTTP, check the value of f$source
> to find the protocol over which it’s transferred.
>> I also see that there har been included some sort of "magic number
>> database"(/bro/share/bro/magic/), but I find little
>> documentation on what its role is in regards of file extraction, as
>> well as the formatting that is being used.
> The magic database is used by libmagic (the library which implements
> [1]) to determine the value of f$mime_type.  See the magic(5) man page
> [2] for how magic files are formatted.
> - Jon
> [1] http://www.darwinsys.com/file/
> [2] http://linux.die.net/man/5/magic

More information about the Bro mailing list