[Bro] Renaming carved files
Michael Cochran
macochran0 at gmail.com
Wed Mar 2 11:40:45 PST 2016
Disregard last, the correct answer was to not go off on my own and try to
use an != "" Also used fmt instead of cat, and removed the unnecessary
local statement. Thank you to everyone that lent a hand in this.
The correct script (which now works...)
@load ./file-extensions
module FileExtraction;
export {
## Path to store files
const path: string = "" &redef;
## Hook to include files in extraction
global extract: hook(f: fa_file, meta: fa_metadata);
## Hook to exclude files from extraction
global ignore: hook(f: fa_file, meta: fa_metadata);
}
event file_sniff(f: fa_file, meta: fa_metadata)
{
if ( meta?$mime_type && !hook FileExtraction::extract(f, meta) )
{
if ( !hook FileExtraction::ignore(f, meta) )
return;
if ( meta$mime_type in mime_to_ext )
local fext = mime_to_ext[meta$mime_type];
else
fext = split_string(meta$mime_type, /\//)[1];
if ( f?$info && f$info?$filename )
local fname = fmt("%s%s-%s", path, f$source,
f$info$filename);
else
fname = fmt("%s%s-%s.%s", path, f$source, f$id,
fext);
Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
[$extract_filename=fname]);
}
}
On Wed, Mar 2, 2016 at 2:25 PM, Michael Cochran <macochran0 at gmail.com>
wrote:
> Derek,
>
> This is nearly spot on. Here's what I have in main.bro from the git link
> you provided that almost works, but is missing some sort of syntax, as it's
> giving me errors. If I comment out the If/else statement f$info$filename
> gives me the content-disposition extracted filename from the protocol. But
> I need a check placed in line to see if f$info$filename is empty, it's
> empty it should go ahead and try to figure out a mime-type extension. Very
> close, and it's probably something very obvious I'm looking over.
>
>
> @load ./file-extensions
>
> module FileExtraction;
>
> export {
> ## Path to store files
> const path: string = "" &redef;
> ## Hook to include files in extraction
> global extract: hook(f: fa_file, meta: fa_metadata);
> ## Hook to exclude files from extraction
> global ignore: hook(f: fa_file, meta: fa_metadata);
> }
>
> event file_sniff(f: fa_file, meta: fa_metadata)
> {
> if ( meta?$mime_type && !hook FileExtraction::extract(f, meta) )
> {
> if ( !hook FileExtraction::ignore(f, meta) )
> return;
> if ( meta$mime_type in mime_to_ext )
> local fext = mime_to_ext[meta$mime_type];
> else
> fext = split_string(meta$mime_type, /\//)[1];
>
> if ( f$info$filename != "" )
> local fname = cat("%s%s-%s", path, f$source,
> f$info$filename);
> else
> local fname = cat("%s%s-%s.%s", path, f$source,
> f$id, fext);
> Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
> [$extract_filename=fname]);
> }
> }
>
>
>
> error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro,
> line 26 and
> /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line 28:
> already defined (FileExtraction::fname)
> error in /opt/bro/share/bro/base/frameworks/files/./main.bro, lines 18-28
> and /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line
> 30: incompatible record types (Files::AnalyzerArgs and
> [$extract_filename=FileExtraction::fname])
> error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro,
> line 30 and /opt/bro/share/bro/base/frameworks/files/./main.bro, lines
> 18-28: type mismatch ([$extract_filename=FileExtraction::fname] and
> Files::AnalyzerArgs)
> error in /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro,
> lines 29-30: argument type mismatch in function call
> (Files::add_analyzer(FileExtraction::f, Files::ANALYZER_EXTRACT,
> [$extract_filename=FileExtraction::fname]))
> warning in
> /opt/bro/share/bro/site/file-extraction/plugins/./.././main.bro, line 30:
> expression value ignored (Files::add_analyzer(FileExtraction::f,
> Files::ANALYZER_EXTRACT, [$extract_filename=FileExtraction::fname]))
>
> On Wed, Mar 2, 2016 at 10:51 AM, Derek Ditch <derek.ditch at gmail.com>
> wrote:
>
>> Michael,
>>
>> I haven’t tested this other than validate syntax, but I think the logic
>> you’re looking for is below. You of course have to add in the dynamic
>> extension mapping and maybe make the outputdir configurable w/ an export {}
>> block. Basically, you have to check to see if the filename is set. I would
>> caution you, that there are many instances where it is not set, however. If
>> you’re looking for a more robust file extraction strategy, I would
>> recommend [1]. There’s some additional overhead in moving files around, but
>> it allows you to store files by hash once extraction is complete. This
>> should greatly reduce your disk usage and processing overhead of any follow
>> on processing.
>>
>>
>> event file_sniff(f: fa_file, meta: fa_metadata)
>> {
>> local fname = "";
>> local outputdir = "/data/bro/extracted_files/";
>> local ext = ".out";
>>
>> # .. logic here to generate ext (with starting .) and outputdir (with
>> ending /)
>> if ( f?$info && f$info?$filename )
>> fname = cat(outputdir, f$info$filename, ext);
>> else
>> fname = cat(outputdir, f$source, f$id, ext);
>>
>> Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
>> [$extract_filename=fname]);
>> }
>>
>> [1] https://github.com/hosom/bro-file-extraction
>> —
>> Derek Ditch
>> derek.ditch at gmail.com
>> GPG: 0x2543A3B5
>>
>> > On 02Mar 2016, at 09:15, Michael Cochran <macochran0 at gmail.com> wrote:
>> >
>> > So the problem I'm running into with this extraction script is here
>> (I've already got a script that handles the extracted metadata mime types):
>> >
>> > local fname = fmt("/nsm/bro/extracted/%s-%s.%s", f$source, f$id, ext);
>> >
>> > I don't need f$source or f$id in the filename. What I'm searching for
>> is being generated here in main.bro. I just need a way to grab this
>> information and add it to the extract.bro script to rename extracted file.
>> >
>> >
>> https://www.bro.org/sphinx-git/scripts/base/frameworks/files/main.bro.html#type-Files::Info
>> > Files::Info
>> > filename: string &log &optional
>> > A filename for the file if one is available from the source for the
>> file. These will frequently come from “Content-Disposition” headers in
>> network protocols
>> >
>> > The logic (forgive my terrible syntax) should be along the lines of
>> > if f$filename is not empty,
>> > local fname = fmt(outputdir, f$filename, ext);
>> > else
>> > local fname = fmt("outputdir", f$source, f$id, ext);
>> >
>> >
>> >
>> > On Tue, Mar 1, 2016 at 2:18 PM, Daniel Guerra <
>> daniel.guerra69 at gmail.com> wrote:
>> >
>> >
>> https://github.com/Security-Onion-Solutions/securityonion-bro-scripts/blob/master/file-extraction/extract.bro
>> >
>> >
>> >> On 01 Mar 2016, at 18:35, Michael Cochran <macochran0 at gmail.com>
>> wrote:
>> >>
>> >> I'm trying to find a simple way to rename a carved file back to it's
>> original file name using bro-script rather than having bash try to rip it
>> out of the files.log file. I have seen the mime type analyzers on git that
>> re-add the extension based on known mime types, but I'd rather be able to
>> immediately identify the original file name as it came across the wire. I
>> don't need the unique session identifier because by the time I'm using bro
>> file analysis I already have the individual session pcap isolated.
>> >>
>> >> I'm guessing there should be a way to capture the files.log table data
>> in broscript, match the unique file identifier then rename the file with
>> that filename string from files.log.
>> >> _______________________________________________
>> >> Bro mailing list
>> >> bro at bro-ids.org
>> >> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> >
>> >
>> > _______________________________________________
>> > Bro mailing list
>> > bro at bro-ids.org
>> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20160302/ab724d8c/attachment-0001.html
More information about the Bro
mailing list