<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style></head><body style="word-wrap:break-word"><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">Justin,</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">Thank you. I peeled the egg off my face and updated the github code accordingly.</div> <br> <div id="bloop_sign_1509651040289586176" class="bloop_sign"><div style="font-family:Helvetica;font-size:12px">However, I have run into an additional interesting tidbit if I use event file_sniff to attach an analyzer or Files::register_for_mime_types, neither will generate a files.log entry when I am not running a PCAP from the command line. So any kind of normal network processing and/or playing a pcap over a listening interface via tcpreplay will cause the analyzer to fire properly.</div><div style="font-family:Helvetica;font-size:12px"><br></div><div style="font-family:Helvetica;font-size:12px">However, if I attach the EXTRACT analyzer, all processing goes as expected. What nuance could I be missing here? The plugin effectively initializes like the existing file analysis analyzers, save it’s a plugin. Is there a hard and fast requirement that ignorer for the file analysis framework to work properly the file has to be explicitly extracted? Most additional analysis, I would assume not use disk resources to extract them and, instead, observe what I need and move on.</div><div style="font-family:Helvetica;font-size:12px"><br></div><div style="font-family:Helvetica;font-size:12px">Any insight from anyone would be greatly appreciated.</div><div style="font-family:Helvetica;font-size:12px"><br></div><div style="font-family:Helvetica;font-size:12px">Thank you,</div><div style="font-family:Helvetica;font-size:12px"><br></div><div style="font-family:Helvetica;font-size:12px">Aaron</div></div> <br><p class="airmail_on">On October 13, 2017 at 11:32:46 AM, Azoff, Justin S (<a href="mailto:jazoff@illinois.edu">jazoff@illinois.edu</a>) wrote:</p> <blockquote type="cite" class="clean_bq"><span><div><div></div><div>
<br>> On Oct 13, 2017, at 11:01 AM, Aaron Eppert <<a href="mailto:aaron.eppert@packetsled.com">aaron.eppert@packetsled.com</a>> wrote:
<br>>
<br>> Justin,
<br>>
<br>> Indeed, cutting new territory is always interesting. As for the code,
<br>>
<br>> <a href="https://github.com/aeppert/test_file_analyzer">https://github.com/aeppert/test_file_analyzer</a>
<br>>
<br>>
<br>> File I am using for this case:
<br>> <a href="https://www.bro.org/static/exchange-2013/faf-exercise.pcap">https://www.bro.org/static/exchange-2013/faf-exercise.pcap</a>
<br>>
<br>> `bro -C -r faf-exercise.pcap` after building and installing the plugin.
<br>>
<br>> My suspicion is it’s either unbelievably trivial and I keep missing it because I am the only one staring at it, or it’s a rather deep rabbit hole.
<br>>
<br>> Aaron
<br>
<br>Thanks for putting that together.. now I see what you mean. Building the plugin with ASAN confirms it is trying to access uninitialized memory:
<br>
<br>
<br> $ /usr/local/bro/bin/bro -C -r faf-exercise.pcap
<br>TEST::Finalize total_len = 65960
<br>BUFFER
<br>00 ea 09 00 50 61 00 00 80 eb 09 00 50 61 00 00
<br>=================================================================
<br>==93650==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000bf5d08 at pc 0x00010b19d39b bp 0x7fff57829e10 sp 0x7fff57829e08
<br>READ of size 1 at 0x603000bf5d08 thread T0
<br> #0 0x10b19d39a in print_bytes(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned char const*, unsigned long, bool) TEST.cc:21
<br> #1 0x10b19e43b in file_analysis::TEST::Finalize() TEST.cc:87
<br>
<br>...
<br>
<br>
<br>
<br>The problem is this line:
<br>
<br> bufv->push_back(data);
<br>
<br>That's only pushing the first char of the buffer onto the vector, not the entire buffer.
<br>
<br>If you print out bufv->size() you'll see that it is not what it should be.
<br>
<br>If you apply this change it will run without crashing and I believe give the expected output:
<br>
<br>diff --git a/src/TEST.cc b/src/TEST.cc
<br>index 8d78ef2..56d0a83 100644
<br>--- a/src/TEST.cc
<br>+++ b/src/TEST.cc
<br>@@ -56,7 +56,7 @@ bool TEST::DeliverStream(const u_char* data, uint64 len)
<br> }
<br>
<br> if ( total_len < TEST_MAX_BUFFER) {
<br>- bufv->push_back(data);
<br>+ print_bytes(std::cout, "BUFFER", data, len);
<br> total_len += len;
<br> }
<br>
<br>@@ -84,7 +84,7 @@ void TEST::Finalize()
<br>
<br> //auto pos = std::find(bufv->begin(), bufv->end(), (unsigned char *)"Exif");
<br> //std::cout << "Offset = " << std::distance( bufv->begin(), pos ) << std::endl;
<br>- print_bytes(std::cout, "BUFFER", (const u_char *)&bufv[0], total_len);
<br>+ //print_bytes(std::cout, "BUFFER", (const u_char *)&bufv[0], total_len);
<br>
<br> val_list* vl = new val_list();
<br> vl->append(GetFile()->GetVal()->Ref());
<br>
<br>I don't know off the top of my head the right way to extend a c++ vector by a c buffer, but doing so should fix things.
<br>
<br>—
<br>Justin Azoff
<br>
<br></div></div></span></blockquote></body></html>