[Bro-Dev] [JIRA] (BIT-1238) High false-positive for application/x-tar signature

Brian O'Berry (JIRA) jira at bro-tracker.atlassian.net
Sat Oct 11 08:42:07 PDT 2014


    [ https://bro-tracker.atlassian.net/browse/BIT-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18304#comment-18304 ] 

Brian O'Berry edited comment on BIT-1238 at 10/11/14 10:41 AM:
---------------------------------------------------------------

After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7}[[:space:]]?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include:
* don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code}
* don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code}
* change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}}
* change the possible digits for mode, uid, gid to octal -- change {{\[:digit:\]}} to {{0-7}}
* move the space and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end
* change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group
* make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly

UPDATE: corrected the 4th bullet, and changed the space in the pattern to {{\[\[:space:]]}} to avoid possible errors if someone's editor converts the blank to a tab


was (Author: boberry):
After looking at [http://www.gnu.org/software/tar/manual/html_node/Standard.html], experimenting some more with regex101.com, and running {code}perl -ne '/[[:print:]\x00]{100}(?:[0-7]{1,7}[[:space:]]?\x00){3}/ and print "yup\n";'{code} on various tar files, the pcre shown in the perl command seems like a better pattern for the {{file-tar}} signature. The changes from the original pcre include:
* don't capture the name field group -- remove the parenthesis from {code}([[:print:]\x00]{100}){code}
* don't capture the mode, uid, gid group -- remove the outer parenthesis from {code}(([[:digit:]\x00\x20]){8}){code}
* change the group pattern for mode, uid, gid to be non-capturing -- prefix the enclosed pattern with {{?:}}
* change the possible digits for mode, uid, gid to octal -- change {{\[:digit:\]}} to {{0-7}}
* move the space ({{\x20}} and null ({{\x00}}) bytes for mode, uid, gid outside the group pattern so they match only at the end
* change the count value on the mode, uid, gid group from 8 to range 1-7 to account for moving the optional space and required null byte outside the group
* make the trailing space optional for the mode, uid, gid group , so tar files created by both gtar and bsdtar match correctly

UPDATE: corrected the 4th bullet, and changed the space in the pattern to {{\[\[:space:]]}} to avoid possible errors if someone's editor converts the blank to a tab

> High false-positive for application/x-tar signature
> ---------------------------------------------------
>
>                 Key: BIT-1238
>                 URL: https://bro-tracker.atlassian.net/browse/BIT-1238
>             Project: Bro Issue Tracker
>          Issue Type: Problem
>          Components: Bro
>    Affects Versions: 2.3
>            Reporter: Brian O'Berry
>              Labels: file, mime, signature
>
> The following signature in base/frameworks/files/magic/general.sig frequently triggers on text files in our environment, and includes a strength value higher than GNU and POSIX tar signatures in libmagic.sig.
> {code}
> signature file-tar {
>     file-magic /([[:print:]\x00]){100}(([[:digit:]\x00\x20]){8}){3}/
>     file-mime "application/x-tar", 150
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4-OD-05-009#64003)


More information about the bro-dev mailing list