[Bro-Dev] [JIRA] (BIT-1143) Investigate replacing libmagic w/ signatures for file identificaiton

[Robin Sommer]

On Thu, Feb 20, 2014 at 16:59 -0600, you wrote:

> In that case, what's gained from Bro having it's own file magic
> signatures instead of just using libmagic by itself?

I was thinking better control over the matching, but I guess there's
not really that much to gain in addition.

> If Bro did completely switch to its own magic sigs, I think we have to
> do a best effort approach to porting all the current MIME magics.

Can this be (semi-)automated, i.e., converting the magic mime db into
Bro regular expressions?

Also, we should investigate performance: Bro's signature engine
doesn't have a reputation for being the fastest in the world. :) Hard
to predict how it performs compared to libmagic; but then I also don't
know if it mattered much if the file type detection got slower.

One more caveat, something I actually didn't think about so far: the
signature engine has some depenedencies on connection state, not sure
if using files as the analysis units goes without pain.

> Tests for everything would be nice, but I don't think a test per MIME
> is a requirement for now.

Agreed.

So if we can basically keep detecting all the MIME types we currently
find, without hurting performance in a significant way, I'm fine fully
switching.