[Bro] http analyzer and de-obfuscating the payload
seth at icir.org
Tue Oct 12 19:17:36 PDT 2010
On Oct 12, 2010, at 7:38 PM, Peter Erickson wrote:
> Is it possible to extract http items using the binpac analyzer or am I
> better off sticking with the hand-written one?
Binpac analyzers are preferred when writing new analyzers, but some of the binpac analyzers are not at feature parity with their handwritten counterparts (HTTP is the primary problem in this regard). For now, I recommend not using the --enable-binpac flag when doing HTTP analysis.
> 2. When processing events, i.e. http_message_done, is it possible to
> access the entire assembled stream without writing it to disk first?
No. Generally when doing stream analysis with Bro you have two options. The best, if your analysis method allows it is to do the analysis in a streaming fashion with chunks of data as they become available. If your analysis method needs random access to the data, then you are probably best off writing to disk and kicking off an external process (from within Bro) once the stream is completed and the file is closed. The output of that analysis could then feed back into Bro using Broccoli.
You typically don't want to try storing large streams in memory because it would be far too easy to use all available memory and crash Bro. Of course, if you are running Bro on tracefiles instead of live network interfaces that may not be a concern.
> 3. Along the same lines as #2, is the assembled stream available for
> connections that are not http?
It depends on the protocol and the analyzer. If you search through the event.bif.bro file for "_data", that will point out analyzer events which likely are sending a stream of data. The analyzers which currently have _data events are: HTTP, SMTP, POP3, and MIME. Unfortunately some of the other obvious ones like SMB and NFS don't currently have _data events. We accept patches though if you'd like to add support for that. :)
Is there a protocol or set of protocols in particular that you'd like to see supported with _data events?
More information about the Bro