[Xorp-hackers] XRL Timing
Bruce M. Simpson
bms at ICSI.Berkeley.EDU
Tue Feb 10 10:45:57 PST 2009
Victor Faion wrote:
>
> Is there a way to set the order in which the XRLs called from
> config.boot are called? I just want to insure some of the state is set
> before performing further functions. So for example I can have the
> callback of the last XRL call these functions, but is there a way to
> guarantee which XRL will return last?
>
I can't speak for the config.boot mechanism at the moment in detail.
Off the top of my head, the following would explain why we generally
implement such config operations from the Router Manager to use single
XRLs. If you are implementing such things in your own code, there are
things to be aware of.
You can't rely on the order of execution of multiple XRLs fired off
in the same flow-of-control. XRLs are implemented to be completely
asynchronous. The API provides no such guarantee, although you may see
effects in implementation where they *are* processed in the order in
which you queue them.
You can't rely on this behaviour, as the mechanism for delivering
the XRL may well change to use "true" asynchronous I/O primitives in future.
For example, during the Windows port, we ruled out the use of I/O
completion ports as they were too intrusive to the whole XRL event
framework, although as AIO matures and evolves within other systems (it
still very much has a role and NTOSKRNL.EXE is built on that
foundation), that may change.
If you need a guarantee of XRLs being processed in a known order in
the time domain, then you need to re-enter the EventLoop every time you
send the XRL and run that loop to completion.
You can do this in-line in the same flow of control, by simply
re-entering EventLoop::run() in a while loop until the XRL callback
routine is dispatched.
It stems from the design trade-off we made in using explicit
co-routines rather than threads.
In the XORP architecture, the XRLs are effectively the only
synchronization point between XORP processes; we rely on the
serialization mechanisms inherent in the base operating system.
Most of the time, that serialization ends up happening in the host's
socket buffers, as the XRL layer will try to use TCP sockets by default
as the IPC mechanism.
An example of code which sends multiple XRLs in the same flow of
control would be RIP's socket initialization.
The OLSR implementation did this too, before the semantics of "give
me a broadcast-capable IPv4 socket bound to an interface" were pushed
into the FEA.
Because some of the operations to construct such a socket are
mutually dependent, and platform specific, adding a single XRL to the
FEA process was more expedient, and avoids the "ping-pong" effect of
XRLs being fired between the OLSR and FEA process during OLSR startup,
which can be unnecessarily time consuming, as well as difficult to read
and debug.
If you look at RIP's socket initialization, upon which this part of
the OLSR code was based, you will see a "waterfall" style flow of
control, where each successive XRL triggers another to be sent.
It is tricky to read for folk not familiar with it!
If you need to dispatch events of your own whilst XRLs are in
flight, without limiting the flow of control to the scope in which you
send the XRLs, you may need to construct state machines around those events.
[i.e. we didn't roll UML collaboration diagrams for this, we were
busy building a protocol stack and all the developers were sharing the
same semantic map of how the system worked :-) now that XORP is going
out to the wider world, we need to explain why these things exist.]
cheers
BMS
More information about the Xorp-hackers
mailing list