[Bro] help regarding using bro on application-level byte stream

[Robin Sommer]

On Wed, Apr 23, 2008 at 06:29 -0700, you wrote:

> capabilities). I do not have access to the corresponding TCP byte stream /
> IP byte stream, but I do have the TCP state information required
> (source/dest addr, source/dest port). I am wondering how to have Bro process
> these packets.

Uh, that's a tricky situation!

> 1. Cook up fake link-layer, TCP,IP headers, and feed Bro via a FIFO.

That seems to be the easiest option for an implementation as you
wouldn't need to dive into Bro but could write the conversion
completely externally. Also, with tools like tcpdump etc. you could
quickly see if things look like they're supposed to. However, I'm
not sure I fully understand in which format your input is in
exactly, so not sure how easy it would be to turn it into fake
packets (e.g., is it already reassembeled or still packetized?). 

> 2. Use Brocolli to send really low-level events (events being "so and so
> bytes seen on so and so conn").

Won't really work because Bro doesn't have any events which are so
low-level. All its events are coming out of the packet/payload
analysis, they aren't any which provide input for it. (You could add
some of your on to feed your data into Bro protocol processing via
Broccoli but that wouldn't be too different from faking packets as
in (1).)

> 3. Use the Bro source code directly, and somehow instantiate an analyzer
> directly on the byte-stream. Any state needed (such as connection endpoints)
> have to be cooked up.

That's an interesting thought. I don't have an immediate opinion on
how difficult this would be. My guess is that you'd quickly be
running into lots of subtle problems with lacking the state you need
to keep the analysis going and which is hard to cook up. That said,
if you're game to dive into Bro's internals for such a solution, you
could just give it a try. However, I wouldn't spend too much time on
it if it turns out to get problematic (and again at lot of this
depends on how *exactly* your input looks like). 

One other thought: which applications are you interested in? If it's
only a few and there happen to be binpac analyzers for them, you
could write a standalone program feeding your data into these binpac
analyzers.

Final note: you mention that you want to rewrite the content: I'm
not very familiar with that part of Bro but I'm guessing it also has
quite a few dependencies on having packets as input.

Robin

-- 
Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org 
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org