[Xorp-hackers] XRL IPC questions

[Orion Hodson]

Dave

You are on the right lines.  I've attempt to clarify a few points  
below as I am responsible for this portion of the code and the lousy  
documentation.

1. How do processes interact with the Finder?

The Finder is the resolver process.  There is one Finder per XORP  
router instantiation.  The address and port of the Finder are  
configurable (via XRL_FINDER_SERVER_ADDRESS and  
XRL_FINDER_SERVER_PORT environment variables).  The default is  
127.0.0.1/19999 and no DNS resolution is done in the default case  
since a router can't depend on DNS to function.

When a process starts it connects to the Finder and informs it of all  
the XRLs it supports and how they resolve into something the XRL  
library can invoke - a transport type, an transport specific address,  
and a string identifying the method.  For each XRL registered the  
Finder provides the client with a strong name for the method  
supplied.  The registering will only respond to invocations that use  
the  transport, transport address, and the strong name.  The strong  
name is one piece in making network attacks harder, otherwise it'd be  
trivial to craft XRLs remotely and direct them at a XORP router.

A processes connection to the Finder exists for the lifetime of the  
process.  When a process is running and wants to dispatch an XRL it  
first consults it's cache to see if it knows how the XRL resolves  
already.  If it doesn't it queries the Finder to get the resolution  
and caches successful results.  If the caller doesn't have an  
available transport endpoint for the transport protocol with the  
specified address, it creates one, otherwise it simply uses the  
endpoint it already has to send the XRL to the dispatching process.

Arguments are marshalled in XRLs according to the transport type.   
Each argument is represented by an XrlAtom.  For the most commonly  
used transport, simple TCP (stcp), marshalling consists of rendering  
the atom into a host-independent binary format.  Library support  
exists for rendering to ASCII.  In the examples, we present the ASCII  
rendering since it's easier to follow.

When the dispatching process gets the XRL it does an reverse lookup  
from the strong-name to the internal name and then maps that to a  
method call.  In the current implementation, all XRL dispatches  
result in an acknowledgement that contains an error code and dispatch  
results.  Dispatch results are just a list of XrlAtoms marshalled  
according to the transport scheme.

2. How does a process register with the Finder?

The basic process is hopefully outlined in above.

One interesting point to note is that the Finder itself is an XRL  
dispatcher just like other XORP processes.  When a process makes a  
request to the Finder, such as registering an XRL, it sends the  
request as an XRL to the Finder and the finder responds in the usual  
manner - an error code plus result in the form of a list of XrlAtoms.

The transport for the Finder is handled as a special case as trying  
to do otherwise made portions of the code hard to reason about.  The  
protocol used to transport XRLs to the Finder is simply "finder" and  
translates into a TCP connection to the known Finder host and port.

There is no strong-name mapping with Finder XRLs.  The method name  
sent over the wire corresponds to code incable on the finder.   
Otherwise we'd have to deal with how to resolve Finder XRLs.

The first incarnation of the Finder used a totally disjoint  
communication mechanism, but this went against the goal of  
extensibility.  Having the Finder export XRL interfaces makes it  
easier to extend.  We added the Finder Event Notifier interface  
pretty trivially once we'd gotten the basic process-to-Finder  
communication over XRLs.

3. How many outstanding XRL requests are allowed by a process at one  
time?

The library imposes no limit.  Individual protocol families can set  
their own limits.  The UDP transport fails with and  
XrlError::SEND_FAILED if it can't allocate memory to buffer the  
message state.  The TCP transport has no limit coded.  The inproc  
family calls synchronously in-process and has no limits.

This is obviously an area that could be improved.

4. How do Finder notifications work?

The Finder supports the Finder Event Notifier interface.  Clients  
that are interested in receiving birth and death of processes (or  
classes of process) can register their interest through the Finder  
Event Nofifier interface (xorp/xrl/interfaces/ 
finder_event_notifier.xif.  The clients implement the Finder Event  
Observer interface which the Finder uses to notify the clients of the  
birth and death events.

The Finder is aware of all process births because XORP processes must  
connect to it to register their XRLs and resolve XRLs to other  
processes in the system.  The Finder gets notified explicitly of  
clean shutdown and detects crashes and forced quits through the  
socket going bad.

When a process registers its interest in a particular event, it  
specifies the protocol and address of where the notification to be  
sent, ie the process-finder channel is not overloaded to carry this  
information.  This keeps the send-receive semantics the same as other  
protocols - channels carry a request to target and the answer back  
again, we don't support sends in both directions over a channel.

5. Why don't transports support bi-directional requests and responses?

This was a design decision to keep the code as simple as possible.   
Making the code bi-directional isn't a huge delta in complexity, but  
requires some tweaks to the resolution process.  If we wanted to do  
this we'd need to export the notion of identity when resolving XRLs.   
The identity would be associated with each XrlRouter in a process.   
There might be more than one instance of a process running on a  
router and a single process may have multiple XrlRouters which share  
the same address space but are logically separate.  Addressing this  
would reduce the number of open sockets and make some small  
performance improvements, but would need a determined effort to get  
working.

6. What about Finder security?

By default the Finder only accepts connections from the IP addresses  
associated with the host it is running on, ie no external  
connections.  Additional addresses can be added at the command-line.

By dumping packets on the localhost (or local network for a  
distributed XORP router) a hacker could determine the resolutions of  
XRLs and send malicious messages to the XORP processes.  Adding  
encryption to the Finder protocol would raise the bar to this attack.

The Finder XRL transport currently uses a naive packet encoding and  
it is possible to crash the finder by connecting to it and sending it  
junk.  This needs to be addressed and should happen at the same time  
encryption goes in since it touches the same code paths.


I hope this helps clarify the Finder and XRLs.  Let me know if any of  
this is opaque or inadequately described.
Kind Regards
- Orion


On Mar 9, 2006, at 1:26 PM, Dave Roberts wrote:

> I have been looking at the XRL mechanism lately and had some  
> questions about how it works. I have read both the XRL  
> documentation on the XORP site as well as looked at the code, but a  
> bit of confirmation from those who know the code in detail would be  
> of assistance. Pointers to docs (RTFM) or particular code files  
> that should be investigated also appreciated.
>
> 1. I'm a bit unclear in how processes interact with the finder to  
> invoke XRLs. From what I have read, I think this is the process,  
> but any comments or corrections would be much appreciated. The  
> following assumes the STCP XRL transport. Let's also assume the  
> XRLs used in the "XORP Inter-Process Communication Library  
> Overview" document on the XORP site.
>    a. First, a client process makes a TCP connection to 127.0.0.1  
> (does it actually connect to "localhost", going through DNS  
> resolution on that, or just to the address directly) on port 19999  
> (Finder's well known port?).
>    b. Client sends a request packet to the Finder. The packet  
> containss the XRL. Say the XRL is "finder://fea/fti/0.1/add_route? 
> net:ipv4net=10.0.0.1/8&gateway:ipv4=192.150.187.1".
>    c. The Finder does the resolution and returns a response packet  
> to the client over the connection. The response packet contains the  
> same sequence number as in the request.
>    d. The response data is itself another complete encoded XRL,  
> this time resolved. Per the documentation, this would be something  
> like: "stcp://192.150.1.5:1992/fti/0.1/add_route? 
> net:ipv4net=10.0.0.1&gateway:ipv4=192.150.1.1".
>    e. The client parses this XRL and extracts the addressing  
> information.
>    f. The client opens a TCP connection (because of the "stcp"  
> scheme in the XRL) to 192.150.1.5 (won't this typically be  
> 127.0.0.1?), port 1992.
>    g. The client sends a request packet to the server (in this case  
> the FEA process). The request contains the encoded, resolved XRL.
>    f. The server sends back a response packet. This packet again  
> has matched sequence numbers to the request.
>    g. I'm not sure what the response contains? In this case, it  
> would seem that no response data is really required, though I  
> haven't looked at the add_route API to know for sure. If data is  
> required, is it returned in some sort of XRL form, or just as raw  
> XRL atoms?
>
> 2. How does a process register with the Finder? I know that it  
> probably opens a connection to the Finder and sends it one or more  
> XRLs that define the address and port on which it is listening, but  
> I'm not sure what those are. Are those XRLs sent on a special  
> connection to the finder, or just over the same one on which  
> resolution happens? Do XRLs invoked on the Finder itself go through  
> the two-hop resolution process?
>
> 3. How many outstanding XRL requests are allowed by a process at  
> one time? The IPC library seems to have some limits, but it isn't  
> clear what happens when those are exceeded.
>
> 4. The IPC Lib Overview document at one point says: "In addition to  
> handle XRL registrations and resolutions, the Finder is capable of  
> notifying XRL Targets about events it is aware of, like the birth  
> and death of other XRL Targets. The Finder exposes an XRL interface  
> for this purpose and is able to invoke XRLs on XRL Targets to  
> perform the notĩcation. A special tunneling mechanism exists in  
> the communication protocol used to communicate with the Finder for  
> this purpose. The details of this communication will be expanded  
> upon later on [XXX in a later edit to this document]."
>
> Are there any more documents for this? Can somebody give an  
> overview of how this works?
>
> Thanks,
>
> -- Dave
> _______________________________________________
> Xorp-hackers mailing list
> Xorp-hackers@icir.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/xorp-hackers