[Bro] Data truncated in signature_match and misc questions

[Robin Sommer]

(I think I lost track which of these are already solved (if any) so
I'll just repsonse briefly. Please ask again for the points which
aren't yet clear. Also, when having several independent questions,
it's usually easier to keep track if you mail them to list
separately.)

On Thu, May 29, 2008 at 19:10 -0700, Lorenzo Cavallaro wrote:

>       Stream reassembly is very important for me, but I suppose Bro
>       takes care of it when matching against signatures.

That's right. 

>       that's where things got weird (I would have preferred to leverage
>       on signature_match events than instead of digging into the
>       policy).

(Not sure what you're referring to here.)

>       [+] signature_match(sig-AAAA) called
>       payload length: 153
>       payload (first 400 bytes): HTTP/1.1 404 Not Found^M^JDate: Wed, 28
>       May 2008 22:07:28 GMT^M^JServer: Apache^M^JContent-Length:
>       270^M^JConnection: close^M^JContent-Type: text/html...
> 
>       I don't see any AAAA in there... even if that's the payload which

Where in the packet stream is the AAAA (according to tcpdump)? 

Generally, the data passed to the signature_match event is the last
chunk of bytes which triggered the signature match. This might
include only parts of the text which matches the pattern and in some
cases even none at all (when other conditions are involved as well;
I wouldn't expect this with your signature). 

As Vern wrote, it's easiest to track down the specifics of what
you're seeing if you could send a small trace file along with the
signature. 

>       The point is that I'd like to extract any matching pattern from the
>       payload which triggered the signature. Once the pattern is extracted
>       I'd have to iterate over each element of the string do something.

That's not really possible without building further script-level
infrastructure yourself. The problem here is that Bro does not
buffer the connection's payload internally so when you get a
signature match, you don't have access to any earlier data. You'd
need to do this buffering yourself but it depends on the specifics
of your application whether that it feasible. 

>       to check whether the metacharacters {} worked properly or not.  It
>       turned out they are ok here (signatures) but they don't work, for
>       instance, with gsub.

Again, a trace and sample script would be good which demonstrates
the gsub problem.

>    2. Does tcp_contents reassembles flows (I don't think so)? I'd use

It does (though see the options dpd_* in policy/bro.init for
specifics of when Bro reassembles streams).  

>    3. I'm not able to see packets that are generated by the same host
>       Bro is running on. Is this a normal behavior (performance tuning)?
>       If so, is there a way to disable it just for testing purposes?

That's as OS issue. Iirc, you indeed don't see packets generated
locally on some OSs, though I don't remember the details here. You
can check with tcpdump whether libpcap applications like Bro see the
packets. 

Robin

-- 
Robin Sommer * Phone +1 (510) 666-2886 * robin at icir.org 
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org