[Bro] http analyzer and de-obfuscating the payload

[Peter Erickson]

While writing a few policies to track an extremely basic malware
"protocol" that sits on top of HTTP, I ran into a few questions that I
haven't been able to find answers for.

1. Are binpac analyzers preferred over the hand-written one? From what
I can tell, which may be wrong, the http binpac analyzer does not send a
http_entity_data event so using http-extract-items is not possible. Is
it possible to extract http items using the binpac analyzer or am I
better off sticking with the hand-written one?

2. When processing events, i.e. http_message_done, is it possible to
access the entire assembled stream without writing it to disk first? I
have some malware traffic that I would like to analyze with bro, but the
data is obfuscated within the http data section using layers of xor,
compression, and encryption techniques. Ideally, I would use bro to
de-obfuscate the streams and provide additional info in the log files
instead of using python scripts after running bro.  I have no problems
writing the bifs (I've already created an xor one), but want to make
sure the info is available if I do write them. 

3. Along the same lines as #2, is the assembled stream available for
connections that are not http?

Any help is appreciated. Thanks in advance.