From vfaion at gmail.com Mon Jun 8 10:23:28 2009 From: vfaion at gmail.com (Victor Faion) Date: Mon, 8 Jun 2009 18:23:28 +0100 Subject: [Xorp-hackers] TCP sockets and packet splitting Message-ID: <38f1dbe80906081023g61a0e1eahb313e82375349360@mail.gmail.com> Hello, I've been using XORP's socket library through the XrlSocket4V0p1Client interface. I've noticed that when sending large packets sometimes XORP will split them up into smaller ones and then on the receiving end I have to piece them together manually. Is this the expected behaviour of the FEA or just a configuration mismatch between my interface's MTU and XORP? Also when sending the same packet it is not always split the same way. On the sending end, the FEA accepts the whole packet without complaining so I expected it would piece it together on the receiving end. Is the programmer suposed to deal with these fragmentation issues? Cheers, Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/xorp-hackers/attachments/20090608/e8a1e705/attachment.html From bms at incunabulum.net Mon Jun 8 21:54:57 2009 From: bms at incunabulum.net (Bruce Simpson) Date: Tue, 09 Jun 2009 05:54:57 +0100 Subject: [Xorp-hackers] TCP sockets and packet splitting In-Reply-To: <38f1dbe80906081023g61a0e1eahb313e82375349360@mail.gmail.com> References: <38f1dbe80906081023g61a0e1eahb313e82375349360@mail.gmail.com> Message-ID: <4A2DEB21.8010202@incunabulum.net> Hi Victor, Thanks for your question about the socket4.xif interface. I'm assuming that you are using XRL encapsulated in TCP over the loopback interface, as these are the default settings for the libxipc library, and you are sending UDP packets (raw IP uses a different XRL interface, and host based TCP would perform MTU discovery for you). Normally the use of TCP as the IPC transport for XRL shouldn't have any issues, other than the issues which already exist with TCP to do with head-of-line blocking, and those are endemic to the TCP protocol anyway. With most TCP/IP implementations, the MTU of the loopback interface will be 65535 bytes. As you probably know already, the segment size used by a TCP session is going to be negotiated upfront during the 3 way handshake. Victor Faion wrote: > Hello, > > I've been using XORP's socket library through the XrlSocket4V0p1Client > interface. I've noticed that when sending large packets sometimes XORP > will split them up into smaller ones and then on the receiving end I > have to piece them together manually. Is this the expected behaviour > of the FEA or just a configuration mismatch between my interface's MTU > and XORP? Also when sending the same packet it is not always split the > same way. On the sending end, the FEA accepts the whole packet without > complaining so I expected it would piece it together on the receiving > end. Is the programmer suposed to deal with these fragmentation issues? The socket4.xif interface will *not* perform any MTU based fragmentation for your sending interface to the best of my knowledge. I believe you still need to use the IfMgrXrlMirror classes to get information about the interface you are trying to use, i.e. the MTU. This is pretty analogous to using a host's TCP/IP stack directly using the BSD sockets API -- you would still need to know the MTU anyway, even if you were writing a simple UDP/IP application, and even in that instance, a host stack would not perform Path MTU Discovery for you. I had to look though. This code is *not* going to coalesce the buffers used by UDP transmission as this may break message boundaries: http://cvsweb.xorp.org/cgi-bin/cvsweb.cgi/xorp/fea/data_plane/io/io_tcpudp_socket.cc?rev=1.27 ...see function IoTcpUdpSocket::send(). In summary, I believe the behaviour you are seeing is expected, and that you need to ensure the packets you are trying to send fit in the interface's MTU. thanks, BMS From bms at incunabulum.net Tue Jun 9 00:31:47 2009 From: bms at incunabulum.net (Bruce Simpson) Date: Tue, 09 Jun 2009 08:31:47 +0100 Subject: [Xorp-hackers] TCP sockets and packet splitting In-Reply-To: <4A2DEB21.8010202@incunabulum.net> References: <38f1dbe80906081023g61a0e1eahb313e82375349360@mail.gmail.com> <4A2DEB21.8010202@incunabulum.net> Message-ID: <4A2E0FE3.6040500@incunabulum.net> Bruce Simpson wrote: > With most TCP/IP implementations, the MTU of the loopback interface > will be 65535 bytes. As you probably know already, the segment size > used by a TCP session is going to be negotiated upfront during the 3 > way handshake. Sorry, I meant to say 16384 bytes here (at least, that's the default in FreeBSD). It can usually be tweaked upwards. In any event, whilst the MTU of the loopback interface will affect XORP XRL IPC transport, the loopback MTU is *usually* sufficiently large not to cause problems with the socket4.xif, unless you are attempting to send IPv6 jumbograms. cheers, BMS From vfaion at gmail.com Tue Jun 9 05:18:04 2009 From: vfaion at gmail.com (Victor Faion) Date: Tue, 9 Jun 2009 13:18:04 +0100 Subject: [Xorp-hackers] TCP sockets and packet splitting In-Reply-To: <4A2E0FE3.6040500@incunabulum.net> References: <38f1dbe80906081023g61a0e1eahb313e82375349360@mail.gmail.com> <4A2DEB21.8010202@incunabulum.net> <4A2E0FE3.6040500@incunabulum.net> Message-ID: <38f1dbe80906090518s290d5923ue77ff74f1175ab1e@mail.gmail.com> On Tue, Jun 9, 2009 at 08:31, Bruce Simpson wrote: > Bruce Simpson wrote: > >> With most TCP/IP implementations, the MTU of the loopback interface will >> be 65535 bytes. As you probably know already, the segment size used by a TCP >> session is going to be negotiated upfront during the 3 way handshake. >> > > Sorry, I meant to say 16384 bytes here (at least, that's the default in > FreeBSD). It can usually be tweaked upwards. > > In any event, whilst the MTU of the loopback interface will affect XORP XRL > IPC transport, the loopback MTU is *usually* sufficiently large not to cause > problems with the socket4.xif, unless you are attempting to send IPv6 > jumbograms. > > cheers, > BMS > Thanks for the reply, I was using TCP sockets with IPv4 packets, that's why I thought XORP would handle the merging, but I saw the comment in IoTcpUdpSocket::send(): // We don't coalesce for TCP as well, but this could be changed in the // future if it improves performance. I think the socket library should indicate to the programmer when it's going to split the packet being sent. Cheers, Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/xorp-hackers/attachments/20090609/c9487823/attachment.html From bms at incunabulum.net Tue Jun 9 05:46:23 2009 From: bms at incunabulum.net (Bruce Simpson) Date: Tue, 09 Jun 2009 13:46:23 +0100 Subject: [Xorp-hackers] TCP sockets and packet splitting In-Reply-To: <38f1dbe80906090518s290d5923ue77ff74f1175ab1e@mail.gmail.com> References: <38f1dbe80906081023g61a0e1eahb313e82375349360@mail.gmail.com> <4A2DEB21.8010202@incunabulum.net> <4A2E0FE3.6040500@incunabulum.net> <38f1dbe80906090518s290d5923ue77ff74f1175ab1e@mail.gmail.com> Message-ID: <4A2E599F.7090605@incunabulum.net> Victor Faion wrote: > ... > Thanks for the reply, I was using TCP sockets with IPv4 packets, > that's why I thought XORP would handle the merging, Hang on, need to disambiguate: are you are sending raw IPv4 datagrams (this does not use the socket4.xif interface), or are you sending TCP stream traffic (this does use the socket4.xif interface) ? I assume here you mean: sending TCP stream traffic via the socket4.xif XRL interface, with the default XRL library settings for IPC encapsulation, between your application and the XORP FEA (i.e. TCP over the loopback interface). > but I saw the comment in IoTcpUdpSocket::send(): > // We don't coalesce for TCP as well, but this could be changed in the > > // future if it improves performance. > > > I think the socket library should indicate to the programmer when it's > going to split the packet being sent. Actually there is no guarantee in the socket4.xif API for preserving message boundaries in TCP, as it is a stream protocol. The behaviour of splitting TCP writes across segments is constrained by the host's TCP implementation, as well as the underlying network conditions. It sounds like you want something similar in principle to Linux's TCP_CORK socket option: "If set, don't send out partial frames. All queued partial frames are sent when the option is cleared again." See here for constructive bashing of TCP_CORK: http://www.baus.net/on-tcp_cork Normally the behaviour of most TCPs is to attempt to send the payload you provide right away in one segment, and set the PUSH flag on the segment. However TCP does not necessarily guarantee to preserve message boundaries in this way -- it is, after all, a stream protocol, not a sequenced message protocol, and is subject to retransmissions, sliding window effects, head-of-line blocking, Nagle algorithm, etc. As an example, HTTP/1.0 is an exception to this, as it will shut down one half of the full-duplex connection after the request is sent, this serves as its message boundary. The use of Selective Acknowledgements (SACK) can mean that multiple segments are in flight out-of-order. In summary: It would be difficult for socket4.xif to provide any API guarantee about splitting TCP writes, as it depends on the host's network stack. thanks, BMS From schintan at gmail.com Tue Jun 9 09:45:14 2009 From: schintan at gmail.com (Chintan Shah) Date: Tue, 9 Jun 2009 12:45:14 -0400 Subject: [Xorp-hackers] Getting Mrt and Mfc information from PIM Message-ID: Hello, We are implementing a separate process in xorp for multicast fast repair. For this process, I need the Mrt and Mfc information exactly as it is in the PIM process. I also needed the Mrib information but that was much easier to get because of the route redistribution mechanism in the RIB process. >From what I see, there are about 55 events that can change the state of the Mrt table. This would be very difficult to track in a separate process without any redistribution mechanism. I wanted to ask if there is any centralized mechanism in the PIM process to make the changes in the Mrt table that I could follow and use to inform my process whenever there is a change in the table entry. Or if there can be some other alternative. Any help would be greatly appreciated. Thanks, Chintan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/xorp-hackers/attachments/20090609/6e07433a/attachment.html From bms at incunabulum.net Tue Jun 9 23:25:16 2009 From: bms at incunabulum.net (Bruce Simpson) Date: Wed, 10 Jun 2009 07:25:16 +0100 Subject: [Xorp-hackers] Getting Mrt and Mfc information from PIM In-Reply-To: References: Message-ID: <4A2F51CC.20505@incunabulum.net> Chintan Shah wrote: > Hello, > > We are implementing a separate process in xorp for multicast fast repair. > For this process, I need the Mrt and Mfc information exactly as it is > in the > PIM process. I also needed the Mrib information but that was much > easier to get > because of the route redistribution mechanism in the RIB process. > From what I see, there are about 55 events that can change the > state of the Mrt table. This would be very difficult to track in a > separate process without > any redistribution mechanism. OK, so you know the MRIB is actually not the multicast routing table, but rather the unicast routes used by PIM's uRPF checks. So far, so good. > I wanted to ask if there is any centralized mechanism in the > PIM process to make the changes in the Mrt table that I could follow > and use to inform > my process whenever there is a change in the table entry. Or if there > can be some other > alternative. It's a difficult one. At the moment, the MRT library holds the state for the Multicast Forwarding Cache (MFC) as used by PIM. Both PIM and the FEA process link against this library, although the FEA mostly just uses static definitions and include files from libmrt, without actually using the abstractions from the library. I think you've run into a barrier which other contributors have also, and that is that a lot of the intelligence about the MFC is actually centralized in the PIM-SM process. There is currently no observer idiom in the mfea_client.xif for the MFC *itself* and you would probably need to modify the pim.xif XRL interface to supply some of the coupling you need to PIM. What hooks exist there at XRL are mostly for implementing the management interface for the PIM-SM process. In practice this isn't really an issue, unless you actually need to modify multicast routing behaviour beyond PIM-SM v2 as implemented in XORP at the moment. I did play with the idea of adding a more generic event mechanism to XRL last year to support SNMP traps, but there wasn't interest in it. XRL itself is utterly tied to C++. AMQP could offer an alternative to XRL, however, some of the more popular implementations depend on having an Erlang bytecode interpreter to provide the broker service -- and XRL came into being to solve certain bootstrapping problems to do with the situation where you implement an IP router with the IPC using IP itself. thanks, BMS From jonathan at scotttechnology.net Tue Jun 16 18:12:35 2009 From: jonathan at scotttechnology.net (Jonathan Creasy) Date: Tue, 16 Jun 2009 20:12:35 -0500 Subject: [Xorp-hackers] hello Message-ID: <494fef30906161812n1874ddffm679ba69620365f83@mail.gmail.com> Not having read any archives, I have no idea if anything like this has been brought up. I have been kicking around a project of a load balancer with good SIP support and also a "router like" interface for a SIP proxy like Kamailio. Your architecture looks like adding a module, or process, to handle modifying the configuration and operation of a Kamailio process running on a box along side Xorp would be a pretty elegant way to do that. Is there interest in adding such functionality to your system? I'm not sure I'm up for the task (I'm a weak C++ programmer) but myself and a couple of other guys might be interested in taking on that as a project but I just thought I'd query this group and introduce myself first. I look forward to working with you. -Jonathan From bms at incunabulum.net Thu Jun 18 02:27:56 2009 From: bms at incunabulum.net (Bruce Simpson) Date: Thu, 18 Jun 2009 10:27:56 +0100 Subject: [Xorp-hackers] hello In-Reply-To: <494fef30906161812n1874ddffm679ba69620365f83@mail.gmail.com> References: <494fef30906161812n1874ddffm679ba69620365f83@mail.gmail.com> Message-ID: <4A3A089C.6000307@incunabulum.net> Hi Jonathan, Welcome to the list... Jonathan Creasy wrote: > Not having read any archives, I have no idea if anything like this has > been brought up. I have been kicking around a project of a load > balancer with good SIP support and also a "router like" interface for > a SIP proxy like Kamailio. Your architecture looks like adding a > module, or process, to handle modifying the configuration and > operation of a Kamailio process running on a box along side Xorp would > be a pretty elegant way to do that. > VRRP was added in the 1.6 community release, that should help with the load balancing story we hope... There is no reason why the XORP code base couldn't support VOIP as an application, but, the framework probably needs more thought, see below. > Is there interest in adding such functionality to your system? I'm not > sure I'm up for the task (I'm a weak C++ programmer) but myself and a > couple of other guys might be interested in taking on that as a > project but I just thought I'd query this group and introduce myself > first. > I'm kicking about some ideas about how to improve the XORP situation with regards to software componentry right now with JT Conklin. One of the major limitations of XRL is that it's tied to the XORP code base, and tied to C++ because of how event callbacks work. So we are evaluating new ideas, i.e. Facebook Thrift, AMQP, and CORBA. Component messages are also perhaps not the right IPC mechanism in all situations, particularly where massive amounts of structured data are involved -- i.e. a full BGP routing table. AMQP is particularly interesting as an RPC transport, because of how it makes it possible to build service clusters. There is an effort, txAMQP, which implements this within the Python 'Twisted' framework. If you've got thoughts about this, please share! However, the answer is not going to materialize right away... it's hectic here. cheers, BMS From greearb at candelatech.com Thu Jun 18 06:17:59 2009 From: greearb at candelatech.com (Ben Greear) Date: Thu, 18 Jun 2009 06:17:59 -0700 Subject: [Xorp-hackers] hello In-Reply-To: <4A3A089C.6000307@incunabulum.net> References: <494fef30906161812n1874ddffm679ba69620365f83@mail.gmail.com> <4A3A089C.6000307@incunabulum.net> Message-ID: <4A3A3E87.2010107@candelatech.com> Bruce Simpson wrote: > Hi Jonathan, > > Welcome to the list... > > Jonathan Creasy wrote: > >> Not having read any archives, I have no idea if anything like this has >> been brought up. I have been kicking around a project of a load >> balancer with good SIP support and also a "router like" interface for >> a SIP proxy like Kamailio. Your architecture looks like adding a >> module, or process, to handle modifying the configuration and >> operation of a Kamailio process running on a box along side Xorp would >> be a pretty elegant way to do that. >> >> > > VRRP was added in the 1.6 community release, that should help with the > load balancing story we hope... > > There is no reason why the XORP code base couldn't support VOIP as an > application, but, the framework probably needs more thought, see below. > > >> Is there interest in adding such functionality to your system? I'm not >> sure I'm up for the task (I'm a weak C++ programmer) but myself and a >> couple of other guys might be interested in taking on that as a >> project but I just thought I'd query this group and introduce myself >> first. >> >> > > I'm kicking about some ideas about how to improve the XORP situation > with regards to software componentry right now with JT Conklin. > > One of the major limitations of XRL is that it's tied to the XORP code > base, and tied to C++ because of how event callbacks work. > > So we are evaluating new ideas, i.e. Facebook Thrift, AMQP, and CORBA. > Component messages are also perhaps not the right IPC mechanism in all > situations, particularly where massive amounts of structured data are > involved -- i.e. a full BGP routing table. > Please don't add something that is even more bloated and cumbersome than XRL! It's easy to wrap c code in c++, and that seems to be enough code language support to me (I like Java, but no reason to write router code in java when the project already supports C++). As for SIP, why would you want to tie sip into xorp anyway? Seems you could run an external sip proxy beside xorp if you wanted? Thanks, Ben -- Ben Greear Candela Technologies Inc http://www.candelatech.com From jonathan at scotttechnology.net Thu Jun 18 11:37:02 2009 From: jonathan at scotttechnology.net (Jonathan Creasy) Date: Thu, 18 Jun 2009 13:37:02 -0500 Subject: [Xorp-hackers] hello In-Reply-To: <4A3A3E87.2010107@candelatech.com> References: <494fef30906161812n1874ddffm679ba69620365f83@mail.gmail.com> <4A3A089C.6000307@incunabulum.net> <4A3A3E87.2010107@candelatech.com> Message-ID: <494fef30906181137l4383e8dfl26497592f0d2d20e@mail.gmail.com> On Thu, Jun 18, 2009 at 8:17 AM, Ben Greear wrote: > Bruce Simpson wrote: > >> Hi Jonathan, >> >> Welcome to the list... >> >> Jonathan Creasy wrote: >> >> >>> Not having read any archives, I have no idea if anything like this has >>> been brought up. I have been kicking around a project of a load >>> balancer with good SIP support and also a "router like" interface for >>> a SIP proxy like Kamailio. Your architecture looks like adding a >>> module, or process, to handle modifying the configuration and >>> operation of a Kamailio process running on a box along side Xorp would >>> be a pretty elegant way to do that. >>> >>> >> >> VRRP was added in the 1.6 community release, that should help with the >> load balancing story we hope... >> >> There is no reason why the XORP code base couldn't support VOIP as an >> application, but, the framework probably needs more thought, see below. >> >> >> >>> Is there interest in adding such functionality to your system? I'm not >>> sure I'm up for the task (I'm a weak C++ programmer) but myself and a >>> couple of other guys might be interested in taking on that as a >>> project but I just thought I'd query this group and introduce myself >>> first. >>> >>> >> >> I'm kicking about some ideas about how to improve the XORP situation with >> regards to software componentry right now with JT Conklin. >> >> One of the major limitations of XRL is that it's tied to the XORP code >> base, and tied to C++ because of how event callbacks work. >> So we are evaluating new ideas, i.e. Facebook Thrift, AMQP, and CORBA. >> Component messages are also perhaps not the right IPC mechanism in all >> situations, particularly where massive amounts of structured data are >> involved -- i.e. a full BGP routing table. >> >> > Please don't add something that is even more bloated and cumbersome than > XRL! It's easy to wrap c code in c++, and that > seems to be enough code language support to me (I like Java, but no reason > to write router code in java > when the project already supports C++). > > As for SIP, why would you want to tie sip into xorp anyway? Seems you > could run an external sip > proxy beside xorp if you wanted? > > Thanks, > Ben > We could run an external sip proxy. I'm trying to make something simple for our Network Engineers to operate. We'd like to explore a device doing OSPF and SIP load balancing in one unit to put out in each POP nationwide. By making the underlying SIP functionality handled by Kamailio but making the configuration process of Kamailio easier (and integrated in with the networking config) we can reduce the time it takes for a random guy to come up to speed in the NOC and also make it easier to support and standardize our configurations. At this time I was not planning to add anything. It seemed like the XRL was capable of handling the tasks. -Jonathan > > -- > Ben Greear Candela Technologies Inc > http://www.candelatech.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ICSI.Berkeley.EDU/pipermail/xorp-hackers/attachments/20090618/e50bc263/attachment.html From bms at incunabulum.net Thu Jun 18 11:55:30 2009 From: bms at incunabulum.net (Bruce Simpson) Date: Thu, 18 Jun 2009 19:55:30 +0100 Subject: [Xorp-hackers] hello In-Reply-To: <4A3A3E87.2010107@candelatech.com> References: <494fef30906161812n1874ddffm679ba69620365f83@mail.gmail.com> <4A3A089C.6000307@incunabulum.net> <4A3A3E87.2010107@candelatech.com> Message-ID: <4A3A8DA2.8030308@incunabulum.net> Hi Ben, Ben Greear wrote: > Please don't add something that is even more bloated and cumbersome > than XRL! It's easy to wrap c code in c++, and that > seems to be enough code language support to me (I like Java, but no > reason to write router code in java > when the project already supports C++). This is why Thrift+AMQP is going to get a lot of sweet personal attention from me. I'm not so sure CORBA is the right way to go, but it has its appeal for some. I think improvements in the componentry are needed. XRL has known limitations. Wielding C++ *is* a professional speciality -- and it's unreasonable to expect everyone to do it, when protocols can be realized in languages which are easier to pick up and use. Using Java is not my intention, but if someone else wants to, why not? There are some very low footprint Java VMs out there. > > As for SIP, why would you want to tie sip into xorp anyway? Seems you > could run an external sip > proxy beside xorp if you wanted? I'm excited by the possibility of XORP becoming a platform for VoIP applications. There are certainly cases for it out there. thanks, BMS From jonathan at scotttechnology.net Thu Jun 18 21:07:49 2009 From: jonathan at scotttechnology.net (Jonathan Creasy) Date: Thu, 18 Jun 2009 23:07:49 -0500 Subject: [Xorp-hackers] hello In-Reply-To: <4A3A8DA2.8030308@incunabulum.net> References: <494fef30906161812n1874ddffm679ba69620365f83@mail.gmail.com> <4A3A089C.6000307@incunabulum.net> <4A3A3E87.2010107@candelatech.com> <4A3A8DA2.8030308@incunabulum.net> Message-ID: <1245384469.15694.7.camel@jonathan-laptop> On Thu, 2009-06-18 at 19:55 +0100, Bruce Simpson wrote: > Hi Ben, > > Ben Greear wrote: > > Please don't add something that is even more bloated and cumbersome > > than XRL! It's easy to wrap c code in c++, and that > > seems to be enough code language support to me (I like Java, but no > > reason to write router code in java > > when the project already supports C++). > > This is why Thrift+AMQP is going to get a lot of sweet personal > attention from me. > I'm not so sure CORBA is the right way to go, but it has its appeal for > some. > > I think improvements in the componentry are needed. XRL has known > limitations. Wielding C++ *is* a professional speciality -- and it's > unreasonable to expect everyone to do it, when protocols can be realized > in languages which are easier to pick up and use. > > Using Java is not my intention, but if someone else wants to, why not? > There are some very low footprint Java VMs out there. > I'm probably showing some naivety here as I don't know what Thrift+AMQP are other than what a 10 second google search revealed. Being able to use a named pipe to send pre-defined commands and get responses would allow a great many languages to interact with your project. I'm still getting a feel for your architecture and reading the docs so it may not fit in real well at this time. I could connect to the named pipe and send a "REGISTER" command which would allow me to announce my applications presence. I could then send a "CONFIG" command which would include all of the commands that I have available to be called and what events should be sent to me when those commands are entered. My application could then sit and wait for events to be sent and process them appropriately. There could be some standard events which would cause an application to dump properly formatted status data. In a VOIP context I might be returning the list of available call routes for the LCR module in Kamailio. Just some thoughts, it may be outside the scope of your project and too much of a detour from network routing but call routing can be dealt with quite similarly. -Jonathan From bms at incunabulum.net Fri Jun 19 02:04:28 2009 From: bms at incunabulum.net (Bruce Simpson) Date: Fri, 19 Jun 2009 10:04:28 +0100 Subject: [Xorp-hackers] hello In-Reply-To: <1245384469.15694.7.camel@jonathan-laptop> References: <494fef30906161812n1874ddffm679ba69620365f83@mail.gmail.com> <4A3A089C.6000307@incunabulum.net> <4A3A3E87.2010107@candelatech.com> <4A3A8DA2.8030308@incunabulum.net> <1245384469.15694.7.camel@jonathan-laptop> Message-ID: <4A3B549C.3020905@incunabulum.net> Jonathan Creasy wrote: > I'm probably showing some naivety here as I don't know what Thrift+AMQP > are other than what a 10 second google search revealed. Thrift is what Facebook did instead of CORBA (object RPC mechanism). AMQP is a messaging protocol. The two together can scale RPC across large and/or dense and/or highly distributed service clusters. This is compelling for XORP's use in network simulation and virtualization. Thrift's efficiency is competitive against Google's 'Protocol Buffers'. Its attraction comes from having stronger language binding / IDL support. > > I could connect to the named pipe and send a "REGISTER" command which > would allow me to announce my applications presence. I could then send a > "CONFIG" command which would include all of the commands that I have > available to be called and what events should be sent to me when those > commands are entered. My application could then sit and wait for events > to be sent and process them appropriately. There could be some standard > events which would cause an application to dump properly formatted > status data. In a VOIP context I might be returning the list of > available call routes for the LCR module in Kamailio. > Protocol events might want a more standardized mechanism. I'll dig up what I wrote up for XORP, Inc. last year. > Just some thoughts, it may be outside the scope of your project and too > much of a detour from network routing but call routing can be dealt with > quite similarly. You'd be surprised the places where 'hunt groups' and 'LCR' are needed in an Internet architecture, but aren't readily available realized concepts... telecomms background is very useful indeed, even if you might have gotten it infiltrating an Ericsson AXE-10 during your teenage years. cheers BMS From jtc at acorntoolworks.com Fri Jun 19 06:34:18 2009 From: jtc at acorntoolworks.com (J.T. Conklin) Date: Fri, 19 Jun 2009 06:34:18 -0700 Subject: [Xorp-hackers] hello In-Reply-To: <4A3A3E87.2010107@candelatech.com> (Ben Greear's message of "Thu, 18 Jun 2009 06:17:59 -0700") References: <494fef30906161812n1874ddffm679ba69620365f83@mail.gmail.com> <4A3A089C.6000307@incunabulum.net> <4A3A3E87.2010107@candelatech.com> Message-ID: <87vdmscpph.fsf@orac.acorntoolworks.com> Ben Greear writes: >> One of the major limitations of XRL is that it's tied to the XORP code >> base, and tied to C++ because of how event callbacks work. >> >> So we are evaluating new ideas, i.e. Facebook Thrift, AMQP, and CORBA. >> Component messages are also perhaps not the right IPC mechanism in all >> situations, particularly where massive amounts of structured data are >> involved -- i.e. a full BGP routing table. >> > Please don't add something that is even more bloated and cumbersome > than XRL! It's easy to wrap c code in c++, and that seems to be > enough code language support to me (I like Java, but no reason to > write router code in java when the project already supports C++). One of the reasons I'm interested in investigating other distributed middleware is because I believe XRL is too bloated and cumbersome. libxipc, plus the generated XRL interface and target code represent a significant proportion of the static memory footprint. This might be justified if we required something uniquely available in XRL, but it doesn't seem to offer anything that cannot be found in other off-the- shelf middleware packages. And if we can use one of those, less time needs be spent on maintaining our own. After all, people don't run XORP for its innovations in distributed middleware. Right now Bruce and I have making ourselves familiar with the systems out there. We haven't discussed it yet, but I suspect the next steps will be to make a list of requirements and to do some experimentation and prototyping. We will do this openly on the -hackers list, and I encourage everyone who has a interest to participate in the discussion. Whatever is ultimately done with XORP's IPC mechanism, the process will likely enough time that investment in the existing XRL code is worthwhile. I have several changes that should result in significant footprint reductions that I hope to get in as soon as the public repo reopens. --jtc -- J.T. Conklin From greearb at candelatech.com Wed Jun 24 15:35:58 2009 From: greearb at candelatech.com (Ben Greear) Date: Wed, 24 Jun 2009 15:35:58 -0700 Subject: [Xorp-hackers] Compile problems for Xorp on Fedora 11. Message-ID: <4A42AA4E.5040204@candelatech.com> It seems that F11's g++ no longer supports #ident, and there are some const char* warnings. I have fixed this in my code tree, and have a patch if anyone is interested, but likely my changes won't apply against the latest internal xorp tree, and the changes (comment out #ident, mostly) are trivial anyway... If someone wants them, I'll be happy to post the patch. Thanks, Ben -- Ben Greear Candela Technologies Inc http://www.candelatech.com From bms at incunabulum.net Thu Jun 25 09:51:51 2009 From: bms at incunabulum.net (Bruce Simpson) Date: Thu, 25 Jun 2009 17:51:51 +0100 Subject: [Xorp-hackers] Compile problems for Xorp on Fedora 11. In-Reply-To: <4A42AA4E.5040204@candelatech.com> References: <4A42AA4E.5040204@candelatech.com> Message-ID: <4A43AB27.5090506@incunabulum.net> Hi Ben, What is radically different about Fedora's shipping GCC compiler that requires these changes -- have they moved to a radically newer version? Pardon my ignorance, I don't use Fedora. The #ident pragma is useful as it's a compiler mandated way of embedding object IDs in which can then be retrieved using the 'ident' command from binutils (or compatible toolchains). It even works with MinGW in Windows. Whilst #ident is not part of the current ISO C++ standard, it has been generally supported by most UNIX C++ compilers for years. So this strikes me more as a regression on Fedora's part. Can we be sure that this isn't due to a change in Fedora's shipped gcc spec file? thanks, BMS From greearb at candelatech.com Thu Jun 25 10:20:51 2009 From: greearb at candelatech.com (Ben Greear) Date: Thu, 25 Jun 2009 10:20:51 -0700 Subject: [Xorp-hackers] Compile problems for Xorp on Fedora 11. In-Reply-To: <4A43AB27.5090506@incunabulum.net> References: <4A42AA4E.5040204@candelatech.com> <4A43AB27.5090506@incunabulum.net> Message-ID: <4A43B1F3.6000008@candelatech.com> On 06/25/2009 09:51 AM, Bruce Simpson wrote: > Hi Ben, > > What is radically different about Fedora's shipping GCC compiler that > requires these changes -- have they moved to a radically newer version? > Pardon my ignorance, I don't use Fedora. > > The #ident pragma is useful as it's a compiler mandated way of embedding > object IDs in which can then be retrieved using the 'ident' command from > binutils (or compatible toolchains). It even works with MinGW in Windows. What would you use this for? > Whilst #ident is not part of the current ISO C++ standard, it has been > generally supported by most UNIX C++ compilers for years. So this > strikes me more as a regression on Fedora's part. > > Can we be sure that this isn't due to a change in Fedora's shipped gcc > spec file? [greearb at ben-dt2 git]$ gcc --version gcc (GCC) 4.4.0 20090506 (Red Hat 4.4.0-4) Copyright (C) 2009 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. I've no particular idea why it complains. It seems to be only a warning, but I think xorp is using -Werr or something like that. Seemed easier to comment out all the idents instead of digging further as I wasn't aware of any useful features derived from #ident. Ben > > thanks, > BMS -- Ben Greear Candela Technologies Inc http://www.candelatech.com From bms at incunabulum.net Thu Jun 25 10:42:45 2009 From: bms at incunabulum.net (Bruce Simpson) Date: Thu, 25 Jun 2009 18:42:45 +0100 Subject: [Xorp-hackers] Compile problems for Xorp on Fedora 11. In-Reply-To: <4A43B1F3.6000008@candelatech.com> References: <4A42AA4E.5040204@candelatech.com> <4A43AB27.5090506@incunabulum.net> <4A43B1F3.6000008@candelatech.com> Message-ID: <4A43B715.20500@incunabulum.net> Ben Greear wrote: >> >> The #ident pragma is useful as it's a compiler mandated way of embedding >> object IDs in which can then be retrieved using the 'ident' command from >> binutils (or compatible toolchains). It even works with MinGW in >> Windows. > > What would you use this for? Traceability -- knowing exactly which compiled objects went into a binary from a build. Typically more useful in production builds, not as useful for debug builds, where this information is redundantly coded anyway. > >> Whilst #ident is not part of the current ISO C++ standard, it has been >> generally supported by most UNIX C++ compilers for years. So this >> strikes me more as a regression on Fedora's part. >> >> Can we be sure that this isn't due to a change in Fedora's shipped gcc >> spec file? > > [greearb at ben-dt2 git]$ gcc --version > gcc (GCC) 4.4.0 20090506 (Red Hat 4.4.0-4) > Copyright (C) 2009 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. There > is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR > PURPOSE. What about -dumpspec ? > > I've no particular idea why it complains. It seems to be only a > warning, but > I think xorp is using -Werr or something like that. Seemed easier to > comment > out all the idents instead of digging further as I wasn't aware of any > useful > features derived from #ident. Which warning(s) are specifically triggered? thans, BMS From greearb at candelatech.com Thu Jun 25 11:51:46 2009 From: greearb at candelatech.com (Ben Greear) Date: Thu, 25 Jun 2009 11:51:46 -0700 Subject: [Xorp-hackers] Compile problems for Xorp on Fedora 11. In-Reply-To: <4A43B715.20500@incunabulum.net> References: <4A42AA4E.5040204@candelatech.com> <4A43AB27.5090506@incunabulum.net> <4A43B1F3.6000008@candelatech.com> <4A43B715.20500@incunabulum.net> Message-ID: <4A43C742.8060008@candelatech.com> On 06/25/2009 10:42 AM, Bruce Simpson wrote: > Ben Greear wrote: >>> >>> The #ident pragma is useful as it's a compiler mandated way of embedding >>> object IDs in which can then be retrieved using the 'ident' command from >>> binutils (or compatible toolchains). It even works with MinGW in >>> Windows. >> >> What would you use this for? > > Traceability -- knowing exactly which compiled objects went into a > binary from a build. > > Typically more useful in production builds, not as useful for debug > builds, where this information is redundantly coded anyway. If you moved to something like 'git' for the repository, you could just use the HEAD tag or similar and encode that once in the binary somewhere. Only with cvs would you need to know the version for every file. >>> Whilst #ident is not part of the current ISO C++ standard, it has been >>> generally supported by most UNIX C++ compilers for years. So this >>> strikes me more as a regression on Fedora's part. >>> >>> Can we be sure that this isn't due to a change in Fedora's shipped gcc >>> spec file? >> >> [greearb at ben-dt2 git]$ gcc --version >> gcc (GCC) 4.4.0 20090506 (Red Hat 4.4.0-4) >> Copyright (C) 2009 Free Software Foundation, Inc. >> This is free software; see the source for copying conditions. There is NO >> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR >> PURPOSE. > > What about -dumpspec ? [greearb at ben-dt2 xorp]$ gcc -dumpspecs *asm: %{v:-V} %{Qy:} %{!Qn:-Qy} %{n} %{T} %{Ym,*} %{Yd,*} %{Wa,*:%*} %{m32:--32} %{m64:--64} %{!mno-sse2avx:%{mavx:-msse2avx}} %{msse2avx:%{!mavx:-msse2avx}} *asm_debug: %{!g0:%{gstabs*:--gstabs}%{!gstabs*:%{g*:--gdwarf2}}} %{fdebug-prefix-map=*:--debug-prefix-map %*} *asm_final: *asm_options: %{--target-help:%:print-asm-header()} %a %Y %{c:%W{o*}%{!o*:-o %w%b%O}}%{!c:-o %d%w%u%O} *invoke_as: %{!S:-o %|.s | as %(asm_options) %|.s %A } *cpp: %{posix:-D_POSIX_SOURCE} %{pthread:-D_REENTRANT} *cpp_options: %(cpp_unique_options) %1 %{m*} %{std*&ansi&trigraphs} %{W*&pedantic*} %{w} %{f*} %{g*:%{!g0:%{g*} %{!fno-working-directory:-fworking-directory}}} %{O*} %{undef} %{save-temps:-fpch-preprocess} *cpp_debug_options: %{d*} *cpp_unique_options: %{C|CC:%{!E:%eGCC does not support -C or -CC without -E}} %{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %{I*&F*} %{P} %I %{MD:-MD %{!o:%b.d}%{o*:%.d%*}} %{MMD:-MMD %{!o:%b.d}%{o*:%.d%*}} %{M} %{MM} %{MF*} %{MG} %{MP} %{MQ*} %{MT*} %{!E:%{!M:%{!MM:%{!MT:%{!MQ:%{MD|MMD:%{o*:-MQ %*}}}}}}} %{remap} %{g3|ggdb3|gstabs3|gcoff3|gxcoff3|gvms3:-dD} %{H} %C %{D*&U*&A*} %{i*} %Z %i %{fmudflap:-D_MUDFLAP -include mf-runtime.h} %{fmudflapth:-D_MUDFLAP -D_MUDFLAPTH -include mf-runtime.h} %{E|M|MM:%W{o*}} *trad_capable_cpp: cc1 -E %{traditional|ftraditional|traditional-cpp:-traditional-cpp} *cc1: %(cc1_cpu) %{profile:-p} *cc1_options: %{pg:%{fomit-frame-pointer:%e-pg and -fomit-frame-pointer are incompatible}} %1 %{!Q:-quiet} -dumpbase %B %{d*} %{m*} %{a*} %{c|S:%{o*:-auxbase-strip %*}%{!o*:-auxbase %b}}%{!c:%{!S:-auxbase %b}} %{g*} %{O*} %{W*&pedantic*} %{w} %{std*&ansi&trigraphs} %{v:-version} %{pg:-p} %{p} %{f*} %{undef} %{Qn:-fno-ident} %{--help:--help} %{--target-help:--target-help} %{--help=*:--help=%(VALUE)} %{!fsyntax-only:%{S:%W{o*}%{!o*:-o %b.s}}} %{fsyntax-only:-o %j} %{-param*} %{fmudflap|fmudflapth:-fno-builtin -fno-merge-constants} %{coverage:-fprofile-arcs -ftest-coverage} *cc1plus: *link_gcc_c_sequence: %{static:--start-group} %G %L %{static:--end-group}%{!static:%G} *link_ssp: %{fstack-protector:} *endfile: %{ffast-math|funsafe-math-optimizations:crtfastmath.o%s} %{mpc32:crtprec32.o%s} %{mpc64:crtprec64.o%s} %{mpc80:crtprec80.o%s} %{shared|pie:crtendS.o%s;:crtend.o%s} crtn.o%s *link: %{!static:--eh-frame-hdr} %{!r:--build-id} %{!m32:-m elf_x86_64} %{m32:-m elf_i386} --hash-style=gnu %{shared:-shared} %{!shared: %{!static: %{rdynamic:-export-dynamic} %{m32:%{!dynamic-linker:-dynamic-linker %{muclibc:%{mglibc:%e-mglibc and -muclibc used together}/lib/ld-uClibc.so.0;:/lib/ld-linux.so.2}}} %{!m32:%{!dynamic-linker:-dynamic-linker %{muclibc:%{mglibc:%e-mglibc and -muclibc used together}/lib/ld64-uClibc.so.0;:/lib64/ld-linux-x86-64.so.2}}}} %{static:-static}} *lib: %{pthread:-lpthread} %{shared:-lc} %{!shared:%{mieee-fp:-lieee} %{profile:-lc_p}%{!profile:-lc}} *mfwrap: %{static: %{fmudflap|fmudflapth: --wrap=malloc --wrap=free --wrap=calloc --wrap=realloc --wrap=mmap --wrap=munmap --wrap=alloca} %{fmudflapth: --wrap=pthread_create}} %{fmudflap|fmudflapth: --wrap=main} *mflib: %{fmudflap|fmudflapth: -export-dynamic} *link_gomp: *libgcc: %{static|static-libgcc:-lgcc -lgcc_eh}%{!static:%{!static-libgcc:%{!shared-libgcc:-lgcc --as-needed -lgcc_s --no-as-needed}%{shared-libgcc:-lgcc_s%{!shared: -lgcc}}}} *startfile: %{!shared: %{pg|p|profile:gcrt1.o%s;pie:Scrt1.o%s;:crt1.o%s}} crti.o%s %{static:crtbeginT.o%s;shared|pie:crtbeginS.o%s;:crtbegin.o%s} *switches_need_spaces: *cross_compile: 0 *version: 4.4.0 *multilib: . !m64 !m32;64:../lib64 m64 !m32;32:../lib !m64 m32; *multilib_defaults: m64 *multilib_extra: *multilib_matches: m64 m64;m32 m32; *multilib_exclusions: *multilib_options: m64/m32 *linker: collect2 *link_libgcc: %D *md_exec_prefix: *md_startfile_prefix: *md_startfile_prefix_1: *startfile_prefix_spec: *sysroot_spec: --sysroot=%R *sysroot_suffix_spec: *sysroot_hdrs_suffix_spec: *cc1_cpu: %{mcpu=*:-mtune=%* %n`-mcpu=' is deprecated. Use `-mtune=' or '-march=' instead. } % >> >> I've no particular idea why it complains. It seems to be only a >> warning, but >> I think xorp is using -Werr or something like that. Seemed easier to >> comment >> out all the idents instead of digging further as I wasn't aware of any >> useful >> features derived from #ident. > > Which warning(s) are specifically triggered? For instance: [greearb at ben-dt2 xorp]$ make make all-recursive make[1]: Entering directory `/home/greearb/git/xorp' Making all in libxorp make[2]: Entering directory `/home/greearb/git/xorp/libxorp' make all-am make[3]: Entering directory `/home/greearb/git/xorp/libxorp' /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -I.. -g -Werror -W -Wall -Wwrite-strings -Wbad-function-cast -Wmissing-prototypes -Wcast-qual -Wmissing-declarations -Wpointer-arith -Wcast-align -Wstrict-prototypes -Wnested-externs -pipe -MT debug.lo -MD -MP -MF .deps/debug.Tpo -c -o debug.lo debug.c gcc -DHAVE_CONFIG_H -I. -I.. -I.. -g -Werror -W -Wall -Wwrite-strings -Wbad-function-cast -Wmissing-prototypes -Wcast-qual -Wmissing-declarations -Wpointer-arith -Wcast-align -Wstrict-prototypes -Wnested-externs -pipe -MT debug.lo -MD -MP -MF .deps/debug.Tpo -c debug.c -o debug.o debug.c:23:2: error: #ident is a deprecated GCC extension make[3]: *** [debug.lo] Error 1 make[3]: Leaving directory `/home/greearb/git/xorp/libxorp' make[2]: *** [all] Error 2 make[2]: Leaving directory `/home/greearb/git/xorp/libxorp' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/greearb/git/xorp' make: *** [all] Error 2 [greearb at ben-dt2 xorp]$ Thanks, Ben > > > thans, > BMS -- Ben Greear Candela Technologies Inc http://www.candelatech.com From bms at incunabulum.net Thu Jun 25 12:43:02 2009 From: bms at incunabulum.net (Bruce Simpson) Date: Thu, 25 Jun 2009 20:43:02 +0100 Subject: [Xorp-hackers] Compile problems for Xorp on Fedora 11. In-Reply-To: <4A43C742.8060008@candelatech.com> References: <4A42AA4E.5040204@candelatech.com> <4A43AB27.5090506@incunabulum.net> <4A43B1F3.6000008@candelatech.com> <4A43B715.20500@incunabulum.net> <4A43C742.8060008@candelatech.com> Message-ID: <4A43D346.20703@incunabulum.net> Ben Greear wrote: > ... > If you moved to something like 'git' for the repository, you could just > use the HEAD tag or similar and encode that once in the binary > somewhere. Only with cvs would you need to know the version > for every file. I think we're comparing apples with oranges here -- regardless of whether CVS, git, SVN, or whatever revision control tool is in use, the traceability concept supported by the #ident pragma doesn't necessarily mean a version tag; it is just a string and could in fact be anything. In practice, we embed the CVS $Id$ tag in the #ident pragma. Aside from that, the plan is to move to SVN. Having said that, a more portable syntax is probably "#pragma ident". At the end of the day, the ident pragma's job is probably better done by a linker map, and it might just be better to get rid of them entirely as you point out. I couldn't find anything in the spec file which would obviously help or hinder. XORP's strict warning flags are probably turning the warning into a fatal error. What is more disturbing, however, is that this change wasn't documented in the GCC 4.4 change log:- http://gcc.gnu.org/gcc-4.4/changes.html In any event, we can't do anything about this until the community sources are moved to SourceForge SVN. I am still waiting for clearance. thanks, BMS From greearb at candelatech.com Tue Jun 30 14:59:53 2009 From: greearb at candelatech.com (Ben Greear) Date: Tue, 30 Jun 2009 14:59:53 -0700 Subject: [Xorp-hackers] Fix for PIM task list hang Message-ID: <4A4A8AD9.2020107@candelatech.com> On some error conditions related to interface removal, the PIM callbacks would not handle the next task, and so nothing would ever look at the task queue again, effectively hanging the multicast routing daemon. The attached patch fixes it. I'll open a bugzilla entry as well. Thanks, Ben -- Ben Greear Candela Technologies Inc http://www.candelatech.com -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: xorp_pim_task_hang.patch Url: http://mailman.ICSI.Berkeley.EDU/pipermail/xorp-hackers/attachments/20090630/0ad82398/attachment.ksh