[Bro] http analyzer and de-obfuscating the payload

Peter Erickson redlamb19 at gmail.com
Tue Oct 12 20:30:06 PDT 2010

On Tue Oct 12 21:17:36 2010, Seth Hall <seth at icir.org> wrote:
>> 2. When processing events, i.e. http_message_done, is it possible to
>> access the entire assembled stream without writing it to disk first?
> No.  Generally when doing stream analysis with Bro you have two   
> options.  The best, if your analysis method allows it is to do the   
> analysis in a streaming fashion with chunks of data as they become   
> available.  If your analysis method needs random access to the data,  
>  then you are probably best off writing to disk and kicking off an   
> external process (from within Bro) once the stream is completed and   
> the file is closed.  The output of that analysis could then feed   
> back into Bro using Broccoli.

I didn't think of using broccoli to feed it back into the system. I'll  
have to reconsider my current setup to see if that makes sense. It  
works now without it, but there is definitely a benefit of having  
additional information within bro's log files.

> You typically don't want to try storing large streams in memory   
> because it would be far too easy to use all available memory and   
> crash Bro.  Of course, if you are running Bro on tracefiles instead   
> of live network interfaces that may not be a concern.

All the analysis that I have been (and will be doing) is with  
tracefiles on a machine that is not connected to a network. I figured  
that there were chances that I could run out of memory, but was hoping  
that the memory would be released once the connection was terminated.  
I did not think about using a table of strings to keep the data...  
guess I was thinking too deep.

>> 3. Along the same lines as #2, is the assembled stream available for
>> connections that are not http?
> It depends on the protocol and the analyzer.  If you search through   
> the event.bif.bro file for "_data", that will point out analyzer   
> events which likely are sending a stream of data.  The analyzers   
> which currently have _data events are: HTTP, SMTP, POP3, and MIME.    
> Unfortunately some of the other obvious ones like SMB and NFS don't   
> currently have _data events.  We accept patches though if you'd like  
>  to add support for that. :)

I figured that you would accept patches. It has been awhile since I've  
used C++, but hoping it will come back to me. I have spent a lot of  
time looking at the source code to better understand how bro works. I  
would love to see RDP and SSL decryption, but I know that those aren't  
easy tasks... doesnt mean I wont try eventually.

> Is there a protocol or set of protocols in particular that you'd   
> like to see supported with _data events?

I haven't seen anything yet, but I'm sure that I'll come across  
something eventually.

Thanks for all the help.

More information about the Bro mailing list