[Bro-Dev] Decapsulating "payload" tunnels

[Slagell, Adam J]

I can't say that I like this idea of conflating tunneling and proxies, which IMO are two very different things. Sometimes abstracting and refactoring makes sense, especially when it can get us two things for less work. I don't think that is the case here, though. It seems to confuse things further and force us to make uncomfortable hacks. 

Also, I'm not seeing how this would work in the default case of someone tapping at their border, between the proxy and the target http server. I don't think there is enough information there without access to the internal state of the proxy server. If you were tapping both internal and WAN traffic in different spots as we do, you would have the information maybe. However, I would rather do something in scriptland on the cluster that would correlate connections between internal hosts and the proxy and connections from the proxy to external web servers.

If tunnels and proxies were more semantically similar, I think I would be more onboard. But right now I think we should separate them and just work on handling AYIYA, 6to4 and Torredo tunnels well. In any case, I don't want to make 2.1 depend on solving these problems for proxies. They are a lower priority in my mind, and I think we want to avoid rushing on how we handle proxies.

:Adam

On Apr 25, 2012, at 8:54 AM, Seth Hall wrote:

> Jon and I have been working on the 2.1 tunnel decapsulation recently and we encountered some major architectural questions.  We seem to have the groundwork laid for doing IP encapsulation tunnels (AYIYA, Teredo, 6to4), but I want to support tunnels like SOCKS and HTTP CONNECT which are essentially session payload tunnels since they are tunneling reassembled TCP streams.
> 
> This brings up a problem if we want to create logs that are useful forensically because right now any connection to a SOCKS proxy looks like the client is sending all the traffic to the proxy.  The HTTP logs will show the client doing HTTP requests to the proxy even though the proxy is really sending them onward to other hosts.  In environments with pervasive proxying, this makes the logs much less useful.
> 
> Robin, Jon, and I discussed this for a while yesterday and we came up with a proposal where we would extract the payload from the proxy connection and mock up IP headers for the Sessions::DoNextPacket method which looks like the client connecting to the host it's requesting to actually talk to.  We would need to extend the DoNextPacket method to provide a short circuit for skipping the TCP reassembly and analysis since it would be reassembled payload bytes immediately after the fake IP header.  This would result in two connections showing up in conn.log when there was *really* 
> 
> There is one other niggle in this.  It seems that most proxy protocols (SOCKS and HTTP at least) support requesting a proxy connection by name instead of IP address.  I fully expect to be beat up over this, but I think it would be great to be able to support doing a lookup to create the fake ip header.  
> 
> I'm sure we'll end up sticking configuration options all over the place to turn things off and we'll definitely figure out a good set of things to turn on by default.
> 
> Does anyone have reservations with this design?  It definitely seems nasty on some levels and Robin pointed out yesterday that it would probably be much better to pass data around with abstracted metadata instead of packets, but packets are what we deal with internally for now so that's what we would have to fake without doing a major redesign.
> 
> Robin, Jon: please follow up if there are any points that I didn't make clear enough. :)
> 
> .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro-ids.org/
> 
> 
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro-ids.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
> 

------

Adam J. Slagell, CISO, CISSP
Chief Information Security Officer
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info
217.244.8965

"Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure."