[Xorp-hackers] RFC: Use one socket per interface for receiving packets in the FEA.

Pavlin Radoslavov pavlin at ICSI.Berkeley.EDU
Wed Mar 5 22:32:11 PST 2008


Ben,

Now that you have (I presume) a working solution, can you get some
numbers about the performance increase you can get with one socket
per interface.
I agree that once you have a large number of interfaces and large
number of virtual XORP instances, the number of unnecessary packet
delivery increases as O(V*I), but I still would like to see what the
actual CPU savings are.

An even more interesting question would be to test those numbers
with and without the pif_index->vif mapping optimization.
Based on your profiling that indicates that the pif_index search
uses lots of CPU, with the pif_index->vif mapping in place, I
wouldn't be surprised if the CPU savings from the one socket per
interface solution will be reduced.


Anyway, for the rest of the email I will assume that the savings are
large enough to justify the extra modifications/complexity.

It seems that your code will work only if the system supports
SO_BINDTODEVICE (i.e., only Linux) which bothers me quite a bit.
The alternative (OS-independent) solution would be to open a socket
per IP address per interface. The argument for doing something like
this is that typically the number of interfaces (both physical and
virtual like tunnels) and the number of IP addresses have same order
of magnitude (though I'd be interested to hear real-world examples
where this is not the case).

Another issue I see is with handling the special multicast routing
socket (it must have protocol type IGMP) and the handling of the
regular IGMP socket for IGMP control traffic.
On system like Linux, if you open two IGMP sockets and use one of
them as the special multicast routing socket and the other one for
regular IGMP control traffic, certain IGMP messages won't arrive on
the regular IGMP socket. This is the reason that the MFEA has extra
logic for handling the situation so a single IGMP socket is used for
both purposes.
However, if we have multiple IGMP sockets (one of them for multicast
routing purpose and the rest of them using SO_BINDTODEVICE to bind
to a specific interface), then I don't know whether we will still
have problems with the delivery of IGMP control traffic.
This is something that requires careful testing to find the answer.

It seems that some of your changes might step over some of Bruce's
OLSR related changes, so from this aspect it also requires careful
coordination.

Said that, I think it will be premature to just take your patch and
commit it now, because it will create more problems than it solves.

However I don't want those changes to be lost in email.
Hence, could you create a Bugzilla entry and add your patch to it so
it can be easily located later.
Also, please add two versions of your patch: one vs the current tree
(i.e., the patch as you sent it to the list), and another one that
contains only the socket-related delta, because there are changes in
your patch that are for some earlier unrelated issues.

Thanks,
Pavlin



More information about the Xorp-hackers mailing list