[Bro] Renaming carved files
Michael Cochran
macochran0 at gmail.com
Wed Mar 2 11:25:26 PST 2016
Derek,
This is nearly spot on. Here's what I have in main.bro from the git link
you provided that almost works, but is missing some sort of syntax, as it's
giving me errors. If I comment out the If/else statement f$info$filename
gives me the content-disposition extracted filename from the protocol. But
I need a check placed in line to see if f$info$filename is empty, it's
empty it should go ahead and try to figure out a mime-type extension. Very
close, and it's probably something very obvious I'm looking over.
@load ./file-extensions
module FileExtraction;
export {
## Path to store files
const path: string = "" &redef;
## Hook to include files in extraction
global extract: hook(f: fa_file, meta: fa_metadata);
## Hook to exclude files from extraction
global ignore: hook(f: fa_file, meta: fa_metadata);
}
event file_sniff(f: fa_file, meta: fa_metadata)
{
if ( meta?$mime_type && !hook FileExtraction::extract(f, meta) )
{
if ( !hook FileExtraction::ignore(f, meta) )
return;
if ( meta$mime_type in mime_to_ext )
local fext = mime_to_ext[meta$mime_type];
else
fext = split_string(meta$mime_type, /\//)[1];
if ( f$info$filename != "" )
local fname = cat("%s%s-%s", path, f$source,
f$info$filename);
else
local fname = cat("%s%s-%s.%s", path, f$source,
f$id, fext);
Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
[$extract_filename=fname]);
}
}
error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro,
line 26 and
/opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line 28:
already defined (FileExtraction::fname)
error in /opt/bro/share/bro/base/frameworks/files/./main.bro, lines 18-28
and /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line
30: incompatible record types (Files::AnalyzerArgs and
[$extract_filename=FileExtraction::fname])
error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro,
line 30 and /opt/bro/share/bro/base/frameworks/files/./main.bro, lines
18-28: type mismatch ([$extract_filename=FileExtraction::fname] and
Files::AnalyzerArgs)
error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro,
lines 29-30: argument type mismatch in function call
(Files::add_analyzer(FileExtraction::f, Files::ANALYZER_EXTRACT,
[$extract_filename=FileExtraction::fname]))
warning in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro,
line 30: expression value ignored (Files::add_analyzer(FileExtraction::f,
Files::ANALYZER_EXTRACT, [$extract_filename=FileExtraction::fname]))
On Wed, Mar 2, 2016 at 10:51 AM, Derek Ditch <derek.ditch at gmail.com> wrote:
> Michael,
>
> I haven’t tested this other than validate syntax, but I think the logic
> you’re looking for is below. You of course have to add in the dynamic
> extension mapping and maybe make the outputdir configurable w/ an export {}
> block. Basically, you have to check to see if the filename is set. I would
> caution you, that there are many instances where it is not set, however. If
> you’re looking for a more robust file extraction strategy, I would
> recommend [1]. There’s some additional overhead in moving files around, but
> it allows you to store files by hash once extraction is complete. This
> should greatly reduce your disk usage and processing overhead of any follow
> on processing.
>
>
> event file_sniff(f: fa_file, meta: fa_metadata)
> {
> local fname = "";
> local outputdir = "/data/bro/extracted_files/";
> local ext = ".out";
>
> # .. logic here to generate ext (with starting .) and outputdir (with
> ending /)
> if ( f?$info && f$info?$filename )
> fname = cat(outputdir, f$info$filename, ext);
> else
> fname = cat(outputdir, f$source, f$id, ext);
>
> Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
> [$extract_filename=fname]);
> }
>
> [1] https://github.com/hosom/bro-file-extraction
> —
> Derek Ditch
> derek.ditch at gmail.com
> GPG: 0x2543A3B5
>
> > On 02Mar 2016, at 09:15, Michael Cochran <macochran0 at gmail.com> wrote:
> >
> > So the problem I'm running into with this extraction script is here
> (I've already got a script that handles the extracted metadata mime types):
> >
> > local fname = fmt("/nsm/bro/extracted/%s-%s.%s", f$source, f$id, ext);
> >
> > I don't need f$source or f$id in the filename. What I'm searching for is
> being generated here in main.bro. I just need a way to grab this
> information and add it to the extract.bro script to rename extracted file.
> >
> >
> https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html#type-Files::Info
> > Files::Info
> > filename: string &log &optional
> > A filename for the file if one is available from the source for the
> file. These will frequently come from “Content-Disposition” headers in
> network protocols
> >
> > The logic (forgive my terrible syntax) should be along the lines of
> > if f$filename is not empty,
> > local fname = fmt(outputdir, f$filename, ext);
> > else
> > local fname = fmt("outputdir", f$source, f$id, ext);
> >
> >
> >
> > On Tue, Mar 1, 2016 at 2:18 PM, Daniel Guerra <daniel.guerra69 at gmail.com>
> wrote:
> >
> >
> https://github.com/Security-Onion-Solutions/securityonion-bro-scripts/blob/master/file-extraction/extract.bro
> >
> >
> >> On 01 Mar 2016, at 18:35, Michael Cochran <macochran0 at gmail.com> wrote:
> >>
> >> I'm trying to find a simple way to rename a carved file back to it's
> original file name using bro-script rather than having bash try to rip it
> out of the files.log file. I have seen the mime type analyzers on git that
> re-add the extension based on known mime types, but I'd rather be able to
> immediately identify the original file name as it came across the wire. I
> don't need the unique session identifier because by the time I'm using bro
> file analysis I already have the individual session pcap isolated.
> >>
> >> I'm guessing there should be a way to capture the files.log table data
> in broscript, match the unique file identifier then rename the file with
> that filename string from files.log.
> >> _______________________________________________
> >> Bro mailing list
> >> bro at bro-ids.org
> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> >
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160302/8fa1292e/attachment.html
More information about the Bro
mailing list