[Xorp-hackers] XRL Timing

Bruce M. Simpson bms at ICSI.Berkeley.EDU
Tue Feb 10 10:45:57 PST 2009


Victor Faion wrote:
>
> Is there a way to set the order in which the XRLs called from
> config.boot are called? I just want to insure some of the state is set
> before performing further functions. So for example I can have the
> callback of the last XRL call these functions, but is there a way to
> guarantee which XRL will return last?
>   

    I can't speak for the config.boot mechanism at the moment in detail.

    Off the top of my head, the following would explain why we generally 
implement such config operations from the Router Manager to use single 
XRLs.    If you are implementing such things in your own code, there are 
things to be aware of.

    You can't rely on the order of execution of multiple XRLs fired off 
in the same flow-of-control. XRLs are implemented to be completely 
asynchronous. The API provides no such guarantee, although you may see 
effects in implementation where they *are* processed in the order in 
which you queue them.

    You can't rely on this behaviour, as the mechanism for delivering 
the XRL may well change to use "true" asynchronous I/O primitives in future.
    For example, during the Windows port, we ruled out the use of I/O 
completion ports as they were too intrusive to the whole XRL event 
framework, although as AIO matures and evolves within other systems (it 
still very much has a role and NTOSKRNL.EXE is built on that 
foundation), that may change.

    If you need a guarantee of XRLs being processed in a known order in 
the time domain, then you need to re-enter the EventLoop every time you 
send the XRL and run that loop to completion.
    You can do this in-line in the same flow of control, by simply 
re-entering EventLoop::run() in a while loop until the XRL callback 
routine is dispatched.

    It stems from the design trade-off we made in using explicit 
co-routines rather than threads.
    In the XORP architecture, the XRLs are effectively the only 
synchronization point between XORP processes; we rely on the 
serialization mechanisms inherent in the base operating system.
    Most of the time, that serialization ends up happening in the host's 
socket buffers, as the XRL layer will try to use TCP sockets by default 
as the IPC mechanism.

    An example of code which sends multiple XRLs in the same flow of 
control would be RIP's socket initialization.
    The OLSR implementation did this too, before the semantics of "give 
me a broadcast-capable IPv4 socket bound to an interface" were pushed 
into the FEA.

    Because some of the operations to construct such a socket are 
mutually dependent, and platform specific, adding a single XRL to the 
FEA process was more expedient, and avoids the "ping-pong" effect of 
XRLs being fired between the OLSR and FEA process during OLSR startup, 
which can be unnecessarily time consuming, as well as difficult to read 
and debug.

    If you look at RIP's socket initialization, upon which this part of 
the OLSR code was based, you will see a "waterfall" style flow of 
control, where each successive XRL triggers another to be sent.
    It is tricky to read for folk not familiar with it!

    If you need to dispatch events of your own whilst XRLs are in 
flight, without limiting the flow of control to the scope in which you 
send the XRLs, you may need to construct state machines around those events.

    [i.e. we didn't roll UML collaboration diagrams for this, we were 
busy building a protocol stack and all the developers were sharing the 
same semantic map of how the system worked :-) now that XORP is going 
out to the wider world, we need to explain why these things exist.]

cheers
BMS



More information about the Xorp-hackers mailing list