[Xorp-users] Question on supporting multiple routing tables.

[Kristian Larsson]

Pavlin Radoslavov wrote:
> Ben Greear <greearb at candelatech.com> wrote:
> 
>>>> I'm sure you'll want to be able to bind the socket to a local IP, but
>>>> if you want to leave out the SO_BINDTODEVICE I can test it and see
>>>> if it works.  I can add the SO_BINDTODEVICE if needed and send you a patch.
>>>>     
>>> Currently, we don't bind to the local IP (we do but in certain cases
>>> only). I believe even if you bind to a local IP you cannot really
>>> force the unicast IP packet to exit the system on the particular
>>> interface. Anyway, I might be wrong here, so please let me know if
>>> you find that bind()-ing only gives us the desired behavior.
>>>   
>> If you set up the routing tables and rules correctly, then binding to a 
>> local IP
>> is probably sufficient.  If you are certain that you want the pkt to 
>> leave by a certain
>> interface, then I don't think it can ever hurt to bind to that local IP, 
>> but just in case,
>> it could also be a config option...
>>> BTW, what protocols are you planning to run? Without SO_BINDTODEVICE
>>> we might have to use different solution for each type of
>>> sockets/packets: raw IP packets, TCP, UDP.
>>> FYI, the I/O system-specific stuff is inside fea/data_plane/io,
>>> though io_tcpudp_socket.cc itself uses the xorp/libcomm wrapper
>>> library.
>>> BGP only doesn't use the FEA (yet) and does its own TCP connection
>>> (inside bgp/socket.{hh,cc}).
>>>
>>> Also, there could be some gotchas with RIP's UDP socket, but lets
>>> address first the protocols you are actually going to use.
>>>   
>> At a minimum, I want to support OSPF.  However, I'd like to have options to
>> do other protocols as well.  In my own experience, binding UDP is very 
>> similar
>> to binding TCP, but if you want some sample code I can post it.  I'm not 
>> sure
>> about raw IP packets.
>>
>> Also, for my application, it will always be running on Linux, so I can 
>> depend on
>> SO_BINDTODEVICE being available...
> 
> I played a bit with SO_BINDTODEVICE, and here is what I found.
> For the record, I am using Gentoo 2006.1 with kernel 2.6.20.1.
> 
> In my test I opened an UDP socket, then used
> setsockopt(SO_BINDTODEVICE) to bind the socket to a specific
> interface (eth1), and then used sendto() to transmit a single UDP packet.
> At the end I am attaching the test program in case someone wants to play
> with it.
> 
> * If the destination address belongs to the same network interface
>   that is used with SO_BINDTODEVICE (eth1), then the transmitted packets
>   are sent over the loopback interface (lo) instead of the external
>   (physical) interface (eth1).

I believe some of your problems might arise from that IP addresses per 
say don't belong to interfaces in Linux. I'm no real expert here, I'd 
rather invite Martin Josefsson, a friend and NetFilter developer, to the 
discussion but with limited time.. you know :)
Linux uses the host-based addressing solution approach, so an IP address 
  merely belongs to the system. Routing is used to determine which 
interface is to be used for outgoing packets and so forth. I can't 
really go into details here for I simply don't know, but I believe this 
is what creates your "problem" - it is merely how the system is 
implemented. "It's a feature, not a bug" ;)

Maybe someone with a better view of how Linux and other OSes handle this 
can give us an overview or perhaps a tip on where to read up on the 
subject. I know that this host-based vs interface-based addressing is 
discussed on the netfilter team lists something like once every half 
year or so ;)

> * If the destination address belongs to the same subnet as the
>   network interface that is used with SO_BINDTODEVICE (eth1), then
>   an ARP request is sent first by the kernel (as we would expect).
>   If the ARP is resolved, then the UDP packet should follow.
> 
> * For all other (i.e., remote) destination addresses or IP
>   addresses that belong to some other interfaces of that host, an
>   ARP request is sent first by the kernel for the destination
>   address. If the ARP request is not answered (as it would be the
>   case for a remote destination unless somebody else is acting as a
>   proxy), then the UDP transmission will fail.
> 
>>From the above observations, the interesting behavior (at least for
> me) is that SO_BINDTODEVICE can be used to force a packet with
> destination address that belongs to some other interface of that
> host (e.g., eth0) to be transmitted over the specified interface
> (eth1). Without SO_BINDTODEVICE such packets are transmitted over
> the loopback interface (lo).
> 
> I continued the testing by connecting eth0 directly with eth1, and
> then used SO_BINDTODEVICE to see whether the UDP packet will be
> actually sent out of interface eth1 to eth0's IP address.
> It turned out that the preceding ARP request out of eth1 is never
> answered by the eth0 interface (probably the kernel recognizes that
> the origin of the ARP is that host itself so the ARP reply is
> suppressed).
> However, after playing a bit I was able to add the ARP entry by
> hand, but it was a bit tricky:
> 
> 1. Added the ARP entry, where xx:xx:xx:xx:xx:xx is the Ethernet
> address of eth0. Note that it must be done before configuring
> the IP address on eth0:
> arp -s 10.0.0.3 xx:xx:xx:xx:xx:xx
> 
> 2. Configure the IP address on eth0:
> ip addr add 10.0.0.3/8 dev eth0
> 
> Finally after that I was able to see the UDP packet going out on
> eth1 with destination address the IP address on eth0.
> 
> In conclusion, if you are running 2+ virtual routing instances each
> of them with separate forwarding tables in the kernel and a
> allocated corresponding subset of the physical interfaces, then it
> is possible to use SO_BINDTODEVICE to make sure that unicast packets
> transmit by one virtual router are transmit over the physical link
> to another virtual router, but it is tricky (see the above ARP
> hack).
> 
> However, if we don't care that the unicast packets between the
> virtual instances are transmitted over the loopback interface, then
> we don't need SO_BINDTODEVICE: just bind(2)-ing to the outgoing
> interface's IP address will be sufficient to guarantee the IP source
> address is set to the desired value.
> 
> Regards,
> Pavlin
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Xorp-users mailing list
> Xorp-users at xorp.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/xorp-users