From bms@spc.org Wed Nov 3 00:36:09 2004 From: bms@spc.org (Bruce M Simpson) Date: Tue, 2 Nov 2004 16:36:09 -0800 Subject: [Xorp-hackers] namespace {..} vs static at file scope Message-ID: <20041103003609.GB32562@dhcp120.icir.org> Whilst reading some notes regarding the C++ standard at the weekend: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndeepc/html/deep04062000.asp %%% For reasons it leaves unstated -- but which I suspect mostly involve template difficulties -- the C++ standard considers internal linkage a deprecated feature. In standardese, "deprecated" describes an ostensibly obsolete feature that may be removed from a future standard revision. Fortunately, the standard both giveth and taketh away: For the features it deems deprecated, the standard typically provides more modern alternatives. In the case of internal linkage, the recommended alternative is the unnamed namespace. %%% This is more C++-like than C-like, but if 'static' meaning 'file scope and internal linkage only' is taken away, things could get interesting. It would be interesting to see what GCC's algorithm is, and to take note of it. BMS From bms@spc.org Wed Nov 3 03:00:23 2004 From: bms@spc.org (Bruce M Simpson) Date: Tue, 2 Nov 2004 19:00:23 -0800 Subject: [Xorp-hackers] [PROPOSAL] Add 'virtual Vifs' to the FEA Message-ID: <20041103030023.GD32562@dhcp120.icir.org> After looking over the RIB code today, and much discussion with Pavlin, I propose that we should implement a 'soft Vif'. This is an interface which never actually gets pushed to the underlying ifconfig layer. What's a soft Vif? ------------------ It's a Vif which exists in the FEA, but never, ever gets pushed to the underlying kernel. Why would we need such a thing? - We now represent discard routes within the RIB now as those whose Vif field points to a discard interface. - Such an interface has to exist within the FEA, but may not necessarily have a counterpart within the host's interface list. - Platforms where this is the case, at least for a 'discard interface': - Linux implements discards by adding routes of type RTN_BLACKHOLE to its FIB. - *BSD implements discards by adding routes whose RTF_BLACKHOLE flag is set; the gateway field must point to a loopback interface. - FreeBSD 5.x has its own discard interface disc(4), and does not need such hacks; but is backwards-compatible with the traditional BSD semantics, and doing it the old fashioned way may actually perform better as it can be picked up on the fast-forwarding path. - Unfortunately the administrator must explicitly configure an instance of disc(4), usually with an incantation along the lines of the following: 'ifconfig disc0 plumb'. - Click uses a Discard element. Then the FtiConfig code for Linux and *BSD need only notice that the vifname specifies a Vif with the is_discard() flag set, and do the appropriate blackhole fandango. The Ifconfig code would see that is_virtual() has been set on the platform discard interface, and would not try to push it down to the kernel. For other platforms with a 'real' software blackhole interface, we can take advantage of that by not setting is_virtual. So it seems to me that adding such things would protect the administrator from having to know how the host platform implements blackholes within the forwarding layer, and protect the RIB code from having to deal with discards-implemented-as-interfaces as a special case. It is worth nothing that Zebra kludges such a thing by hardcoding an interface name Null0, as does Vendor C. Rationale --------- I don't see an easy way of supporting discards without considerably hacking the RIB to do it, and we are still faced with the problem of pushing the discard routes to the FEA. Treating discards as simple interface routes sidesteps the problem, and would at least contain the ugliness to the FEA itself and avoid layering violations. The Vif field in IPRouteEntry cannot, for most intents and purposes, be NULL. Adding the necessary special cases to the RIB looked to be much more trouble than it's worth; we already have a notion of a 'connected route', i.e. a route where the Vif is known and used. A DiscardNextHop implies that the Vif/vifname is ignored for most intents and purposes. Therefore, the use of DiscardNextHop only seems to be useful if it isn't possible to assign an address to the interface used to blackhole packets, for whatever reason. So I'd like to regard DiscardNextHop as being a special case. Supporting unnumbered interfaces (a la Vendor C) is a desirable feature, but for the purposes of implementing discard routes, it is easier to consider it as a separate problem to be addressed at a later date. Thoughts/input requested... but time is wearing on and I'd like to implement this so that it's done. BMS From rafael.guimaraes@ac.upc.es Wed Nov 3 16:31:29 2004 From: rafael.guimaraes@ac.upc.es (Rafael Paoliello Guimaraes) Date: Wed, 03 Nov 2004 17:31:29 +0100 Subject: [Xorp-hackers] Migrating Routing Process to XORP Message-ID: <418907E1.1070908@ac.upc.es> Hello, I am trying to migrate a routing process to XORP but I am having some problems on understanding somethings. I have already mapped in the routing process code everywhere it uses sockets to send/receive control packets and the first thing I want to do is to migrate this functionality to XORP. But I confess that I am a bit lost. At first I though that I should use the XrlSocketServer class for this, but looking at the RIP example code I found no reference to this class. So, what how should I do it? Maybe it could be easier to understand from an example. The original code is based on a "select", i.e., it creates a socket server for listening the interface and an infinite loop keeps monitoring the arrival of packets (through a select) so that they can be processed. How can I convert this kind of construction to go through XORP? Cheers, Rafael From eju2002@sina.com Thu Nov 4 02:50:33 2004 From: eju2002@sina.com (=?gb2312?B?uMrBwQ==?=) Date: Thu, 4 Nov 2004 10:50:33 +0800 Subject: [Xorp-hackers] I am a new user... Message-ID: <200411040251.iA42pM9l003394@wyvern.icir.org> respected hackers: I am a new user of the XORP, master of router architect in NUDT(National University of Defense Technology in China ). Because of my reseach, FORCES's implement, our Pro. want us use XORP & Click to build a experimental environment. we have a team of several person, each of us have devided into CE and FE's implement. My job is the CE's implement, we choose XORP, for its eXtensible, open and it claimed that he will add the FORCES. If you can tell me , how can I start the reading of this source code. thank for all of you that help us and your advice! Forrest 　　　　　　　　eju2002@sina.com 　　　　　　　　　　2004-11-04 From pavlin@icir.org Thu Nov 4 21:09:06 2004 From: pavlin@icir.org (Pavlin Radoslavov) Date: Thu, 04 Nov 2004 13:09:06 -0800 Subject: [Xorp-hackers] Migrating Routing Process to XORP In-Reply-To: Message from Rafael Paoliello Guimaraes of "Wed, 03 Nov 2004 17:31:29 +0100." <418907E1.1070908@ac.upc.es> Message-ID: <200411042109.iA4L96wK050191@possum.icir.org> Rafael, > I am trying to migrate a routing process to XORP but I am having some > problems on understanding somethings. I have already mapped in the > routing process code everywhere it uses sockets to send/receive control > packets and the first thing I want to do is to migrate this > functionality to XORP. But I confess that I am a bit lost. At first I > though that I should use the XrlSocketServer class for this, but looking > at the RIP example code I found no reference to this class. So, what how > should I do it? Currently, you can use the XrlSocketServer for TCP and UDP sockets only. To send and receive packets, set socket options, etc, you have to use the xrl/interfaces/socket{4,6}.xif and xrl/interfaces/socket{4,6}_user.xif XRL interfaces to use the FEA For example, one of the XRLs inside socket4.xif is "udp_open_and_bind" to create a bound UDP socket. Hence, if you search the rip directory for "udp_open_and_bind" you will find how it is used. Similarly, the "recv_event" XRL in file socket4_user.xif is used to receive a packet on a socket, so you can search for that string as well to find-out how to receive packets via XRLs. > Maybe it could be easier to understand from an example. The original > code is based on a "select", i.e., it creates a socket server for > listening the interface and an infinite loop keeps monitoring the > arrival of packets (through a select) so that they can be processed. How > can I convert this kind of construction to go through XORP? You would have to use the XORP EventLoop class, and then the sending and receiving of XRLs you need to send/receive your data packets and everything else is handled automatically. You can have a look in function static_routes_main() inside file static_routes/xorp_static_routes.cc for a very simple example how to do that. Regards, Pavlin From atanu@ICSI.Berkeley.EDU Thu Nov 4 22:43:46 2004 From: atanu@ICSI.Berkeley.EDU (Atanu Ghosh) Date: Thu, 04 Nov 2004 14:43:46 -0800 Subject: [Xorp-hackers] I am a new user... In-Reply-To: Message from "=?gb2312?B?uMrBwQ==?=" of "Thu, 04 Nov 2004 10:50:33 +0800." <200411040251.iA42pM9l003394@wyvern.icir.org> Message-ID: <48940.1099608226@tigger.icir.org> Hi, Currently XORP can not use Click for forwarding. We are hoping that by the end of the month XORP and Click we be re-integrated. In terms of documentation is a good place to start. Atanu. >>>>> "eju2002" == eju2002 writes: eju2002> respected hackers: I am a new user of the XORP, master of eju2002> router architect in NUDT(National University of Defense eju2002> Technology in China ). Because of my reseach, FORCES's eju2002> implement, our Pro. want us use XORP & Click to build a eju2002> experimental environment. we have a team of several eju2002> person, each of us have devided into CE and FE's eju2002> implement. My job is the CE's implement, we choose XORP, eju2002> for its eXtensible, open and it claimed that he will add eju2002> the FORCES. If you can tell me , how can I start the eju2002> reading of this source code. thank for all of you that eju2002> help us and your advice! eju2002> Forrest eju2002> 　　　　　　　　eju2002@sina.com eju2002> 　　　　　　　　　　2004-11-04 eju2002> _______________________________________________ eju2002> Xorp-hackers mailing list Xorp-hackers@icir.org eju2002> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/xorp-hackers From bms@spc.org Fri Nov 5 04:30:09 2004 From: bms@spc.org (Bruce M Simpson) Date: Thu, 4 Nov 2004 20:30:09 -0800 Subject: [Xorp-hackers] Blackhole routes In-Reply-To: <20041025212951.GF706@empiric.icir.org> References: <20041025212951.GF706@empiric.icir.org> Message-ID: <20041105043009.GC746@empiric.icir.org> Blackhole routing support has now been committed to CVS. BMS From bms@spc.org Fri Nov 5 04:30:14 2004 From: bms@spc.org (Bruce M Simpson) Date: Thu, 4 Nov 2004 20:30:14 -0800 Subject: [Xorp-hackers] On-demand routing and XORP. In-Reply-To: <20041027023552.GB779@empiric.icir.org> References: <20041027023552.GB779@empiric.icir.org> Message-ID: <20041105043014.GD746@empiric.icir.org> The hooks for on-demand routing notifications have now been committed to CVS. Platform support for this feature will be forthcoming in future commits. BMS From bms@spc.org Fri Nov 5 22:08:52 2004 From: bms@spc.org (Bruce M Simpson) Date: Fri, 5 Nov 2004 14:08:52 -0800 Subject: [Xorp-hackers] [PATCH] FIB upcall/event support, please review Message-ID: <20041105220852.GB734@empiric.icir.org> --WfZ7S8PLGjBY9Voh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline [Cc: to possibly interested parties] Here is the next piece of forwarding table notification support. Because this messes with the innards of XORP on FreeBSD, I could do with some review and additional testing before this goes in. This is needed in order to support ad-hoc/on-demand routing protocols. It's currently targeted only at BSD, but could easily be adapted to Linux. The last part missing is the wiring-up of these upcall notifications to the XrlFtiTransactionManager, by way of the FibTableObserver and FtiConfigTableObserver::propagate_fib_changes(); that will come next. Then, routing protocols implemented as XORP processes will be able to receive route resolve/miss upcalls, if they implement the XRL interface 'fea_fib_client' and ask for them. Regards, BMS --WfZ7S8PLGjBY9Voh Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="fea-rtm-upcalls1.diff" In the routing socket implementation of the Fti subsystem, change the parse_buffer_rtm() method in classes FtiConfigEntryGet and FtiConfigTableGet to accept an enum, RtmParseType, which the caller uses to convey which subset of BSD routing socket messages it is interested in. Tidy up the methods afterwards. Update callers to use the new function signature. Because the methods compile to stubs, don't wrap the enum declaration with #ifdef HAVE_ROUTING_SOCKETS. IfConfigGet never needs to see this, so don't touch it. Index: fticonfig.hh =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig.hh,v retrieving revision 1.23 diff -u -p -r1.23 fticonfig.hh --- fticonfig.hh 5 Nov 2004 00:47:43 -0000 1.23 +++ fticonfig.hh 5 Nov 2004 22:02:20 -0000 @@ -25,6 +25,15 @@ #include "libxorp/trie.hh" #include "fte.hh" + +// Knobs for parse_buffer_rtm() in routing socket FtiConfig implementation +// to denote what caller is interested in. +enum FtiRtmParseType { + CHANGES, // RTM_ADD, RTM_CHANGE, RTM_DELETE, RTM_GET. + GET, // RTM_GET only. + UPCALLS // RTM_MISS and RTM_RESOLVE. +}; + #include "fticonfig_entry_get.hh" #include "fticonfig_entry_set.hh" #include "fticonfig_entry_observer.hh" Index: fticonfig_entry_get.hh =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_entry_get.hh,v retrieving revision 1.17 diff -u -p -r1.17 fticonfig_entry_get.hh --- fticonfig_entry_get.hh 26 Oct 2004 23:58:29 -0000 1.17 +++ fticonfig_entry_get.hh 5 Nov 2004 22:02:20 -0000 @@ -110,12 +110,12 @@ public: * @param fte the Fte storage to store the parsed information. * @param buf the buffer with the data to parse. * @param buf_bytes buf_bytes the size of the data in the buffer. - * @param is_rtm_get_only if true, consider only the RTM_GET entries. + * @param type the subset of RTM messages that caller is interested in. * @return true on success, otherwise false. * @see FteX. */ bool parse_buffer_rtm(FteX& fte, const uint8_t *buf, size_t buf_bytes, - bool is_rtm_get_only); + FtiRtmParseType type); /** * Parse information about routing entry information received from Index: fticonfig_entry_get_rtsock.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_entry_get_rtsock.cc,v retrieving revision 1.21 diff -u -p -r1.21 fticonfig_entry_get_rtsock.cc --- fticonfig_entry_get_rtsock.cc 26 Oct 2004 23:55:16 -0000 1.21 +++ fticonfig_entry_get_rtsock.cc 5 Nov 2004 22:02:20 -0000 @@ -303,7 +303,7 @@ FtiConfigEntryGetRtsock::lookup_route_by return (false); } if (parse_buffer_rtm(fte, _rs_reader.buffer(), _rs_reader.buffer_size(), - true) + FtiRtmParseType(GET)) != true) { return (false); } @@ -446,7 +446,7 @@ FtiConfigEntryGetRtsock::lookup_route_by return (false); } if (parse_buffer_rtm(fte, _rs_reader.buffer(), _rs_reader.buffer_size(), - true) + FtiRtmParseType(GET)) != true) { return (false); } Index: fticonfig_entry_parse_rtm.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_entry_parse_rtm.cc,v retrieving revision 1.7 diff -u -p -r1.7 fticonfig_entry_parse_rtm.cc --- fticonfig_entry_parse_rtm.cc 5 Nov 2004 01:27:53 -0000 1.7 +++ fticonfig_entry_parse_rtm.cc 5 Nov 2004 22:02:20 -0000 @@ -39,7 +39,8 @@ #ifndef HAVE_ROUTING_SOCKETS bool -FtiConfigEntryGet::parse_buffer_rtm(FteX& , const uint8_t* , size_t , bool ) +FtiConfigEntryGet::parse_buffer_rtm(FteX& , const uint8_t* , size_t , + FtiRtmParseType) { return false; } @@ -48,13 +49,14 @@ FtiConfigEntryGet::parse_buffer_rtm(FteX bool FtiConfigEntryGet::parse_buffer_rtm(FteX& fte, const uint8_t* buf, - size_t buf_bytes, bool is_rtm_get_only) + size_t buf_bytes, FtiRtmParseType type) { - const struct rt_msghdr* rtm = reinterpret_cast(buf); + const struct rt_msghdr* rtm = + reinterpret_cast(buf); const uint8_t* last = buf + buf_bytes; - + for (const uint8_t* ptr = buf; ptr < last; ptr += rtm->rtm_msglen) { - rtm = reinterpret_cast(ptr); + rtm = reinterpret_cast(ptr); if (RTM_VERSION != rtm->rtm_version) { XLOG_ERROR("RTM version mismatch: expected %d got %d", RTM_VERSION, @@ -62,30 +64,35 @@ FtiConfigEntryGet::parse_buffer_rtm(FteX continue; } - if (is_rtm_get_only) { - // - // Consider only the RTM_GET entries. - // - if (rtm->rtm_type != RTM_GET) + // XXX: ignore entries with an error + if (rtm->rtm_errno != 0) + continue; + + if (type == FtiRtmParseType(GET)) { + if ((rtm->rtm_type != RTM_GET) || !(rtm->rtm_flags & RTF_UP)) + continue; + } else if (type == FtiRtmParseType(UPCALLS)) { + // Upcalls may not be supported in some implementations. +#ifdef RTM_MISS + if (rtm->rtm_type != RTM_MISS) + continue; +#endif +#ifdef RTM_RESOLVE + if (rtm->rtm_type != RTM_RESOLVE) continue; - if (! (rtm->rtm_flags & RTF_UP)) +#endif + } else if (type == FtiRtmParseType(CHANGES)) { + if ((rtm->rtm_type != RTM_ADD) && (rtm->rtm_type != RTM_DELETE) && + (rtm->rtm_type != RTM_CHANGE) && (rtm->rtm_type != RTM_GET)) continue; + } else { + return (false); } - if ((rtm->rtm_type != RTM_ADD) - && (rtm->rtm_type != RTM_DELETE) - && (rtm->rtm_type != RTM_CHANGE) - && (rtm->rtm_type != RTM_GET)) { - continue; - } - - if (rtm->rtm_errno != 0) - continue; // XXX: ignore entries with an error - return (RtmUtils::rtm_get_to_fte_cfg(fte, ftic().iftree(), rtm)); } - - return false; + + return (false); } #endif // HAVE_ROUTING_SOCKETS Index: fticonfig_table_get.hh =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_table_get.hh,v retrieving revision 1.14 diff -u -p -r1.14 fticonfig_table_get.hh --- fticonfig_table_get.hh 26 Oct 2004 23:58:29 -0000 1.14 +++ fticonfig_table_get.hh 5 Nov 2004 22:02:20 -0000 @@ -91,12 +91,12 @@ public: * @param fte_list the list with the Fte entries to store the result. * @param buf the buffer with the data to parse. * @param buf_bytes the size of the data in the buffer. - * @param is_rtm_get_only if true, consider only the RTM_GET entries. + * @param type the subset of RTM messages that caller is interested in. * @return true on success, otherwise false. * @see FteX. */ bool parse_buffer_rtm(int family, list& fte_list, const uint8_t *buf, - size_t buf_bytes, bool is_rtm_get_only); + size_t buf_bytes, FtiRtmParseType type); /** * Parse information about routing table information received from Index: fticonfig_table_get_sysctl.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_table_get_sysctl.cc,v retrieving revision 1.13 diff -u -p -r1.13 fticonfig_table_get_sysctl.cc --- fticonfig_table_get_sysctl.cc 1 Sep 2004 18:12:24 -0000 1.13 +++ fticonfig_table_get_sysctl.cc 5 Nov 2004 22:02:20 -0000 @@ -179,7 +179,8 @@ FtiConfigTableGetSysctl::get_table(int f if (sysctl(mib, sizeof(mib) / sizeof(mib[0]), table_data, &sz, NULL, 0) == 0) { // Parse the result - return (parse_buffer_rtm(family, fte_list, table_data, sz, true)); + return (parse_buffer_rtm(family, fte_list, table_data, sz, + FtiRtmParseType(GET))); } // Error Index: fticonfig_table_observer_rtsock.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_table_observer_rtsock.cc,v retrieving revision 1.10 diff -u -p -r1.10 fticonfig_table_observer_rtsock.cc --- fticonfig_table_observer_rtsock.cc 1 Sep 2004 18:12:25 -0000 1.10 +++ fticonfig_table_observer_rtsock.cc 5 Nov 2004 22:02:20 -0000 @@ -93,7 +93,7 @@ FtiConfigTableObserverRtsock::receive_da // if (ftic().have_ipv4()) { ftic().ftic_table_get_primary().parse_buffer_rtm(AF_INET, fte_list, - data, nbytes, false); + data, nbytes, FtiRtmParseType(CHANGES)); if (! fte_list.empty()) { propagate_fib_changes(fte_list); fte_list.clear(); @@ -106,7 +106,7 @@ FtiConfigTableObserverRtsock::receive_da // if (ftic().have_ipv6()) { ftic().ftic_table_get_primary().parse_buffer_rtm(AF_INET6, fte_list, - data, nbytes, false); + data, nbytes, FtiRtmParseType(CHANGES)); if (! fte_list.empty()) { propagate_fib_changes(fte_list); fte_list.clear(); Index: fticonfig_table_parse_rtm.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_table_parse_rtm.cc,v retrieving revision 1.10 diff -u -p -r1.10 fticonfig_table_parse_rtm.cc --- fticonfig_table_parse_rtm.cc 5 Nov 2004 01:27:53 -0000 1.10 +++ fticonfig_table_parse_rtm.cc 5 Nov 2004 22:02:20 -0000 @@ -40,7 +40,7 @@ #ifndef HAVE_ROUTING_SOCKETS bool FtiConfigTableGet::parse_buffer_rtm(int, list& , const uint8_t* , - size_t , bool ) + size_t , FtiRtmParseType type) { return false; } @@ -50,13 +50,14 @@ FtiConfigTableGet::parse_buffer_rtm(int, bool FtiConfigTableGet::parse_buffer_rtm(int family, list& fte_list, const uint8_t* buf, size_t buf_bytes, - bool is_rtm_get_only) + FtiRtmParseType type) { - const struct rt_msghdr* rtm = reinterpret_cast(buf); + const struct rt_msghdr* rtm = + reinterpret_cast(buf); const uint8_t* last = buf + buf_bytes; - + for (const uint8_t* ptr = buf; ptr < last; ptr += rtm->rtm_msglen) { - rtm = reinterpret_cast(ptr); + rtm = reinterpret_cast(ptr); if (RTM_VERSION != rtm->rtm_version) { XLOG_ERROR("RTM version mismatch: expected %d got %d", RTM_VERSION, @@ -64,25 +65,31 @@ FtiConfigTableGet::parse_buffer_rtm(int continue; } - if (is_rtm_get_only) { - // - // Consider only the RTM_GET entries. - // - if (rtm->rtm_type != RTM_GET) - continue; - if (! (rtm->rtm_flags & RTF_UP)) + if (rtm->rtm_errno != 0) + continue; // XXX: ignore entries with an error + + if (type == FtiRtmParseType(GET)) { + if ((rtm->rtm_type != RTM_GET) || (!(rtm->rtm_flags & RTF_UP))) continue; } - if ((rtm->rtm_type != RTM_ADD) - && (rtm->rtm_type != RTM_DELETE) - && (rtm->rtm_type != RTM_CHANGE) - && (rtm->rtm_type != RTM_GET)) { - continue; + // Upcalls may not be supported in some BSD derived implementations. + if (type == FtiRtmParseType(UPCALLS)) { +#ifdef RTM_MISS + if (rtm->rtm_type != RTM_MISS) + continue; +#endif +#ifdef RTM_RESOLVE + if (rtm->rtm_type != RTM_RESOLVE) + continue; +#endif } - if (rtm->rtm_errno != 0) - continue; // XXX: ignore entries with an error + if (type == FtiRtmParseType(UPCALLS)) { + if ((rtm->rtm_type != RTM_ADD) && (rtm->rtm_type != RTM_DELETE) && + (rtm->rtm_type != RTM_CHANGE) && (rtm->rtm_type != RTM_GET)) + continue; + } #ifdef RTF_LLINFO if (rtm->rtm_flags & RTF_LLINFO) @@ -104,12 +111,12 @@ FtiConfigTableGet::parse_buffer_rtm(int if (rtm->rtm_flags & RTF_BROADCAST) continue; // XXX: ignore broadcast entries #endif - + FteX fte(family); if (RtmUtils::rtm_get_to_fte_cfg(fte, ftic().iftree(), rtm) == true) fte_list.push_back(fte); } - + return true; } --WfZ7S8PLGjBY9Voh-- From bms@spc.org Sat Nov 6 00:36:05 2004 From: bms@spc.org (Bruce M Simpson) Date: Fri, 5 Nov 2004 16:36:05 -0800 Subject: [Xorp-hackers] [PATCH REVISED] FIB upcall/event support, In-Reply-To: <20041105220852.GB734@empiric.icir.org> References: <20041105220852.GB734@empiric.icir.org> Message-ID: <20041106003605.GA10185@empiric.icir.org> --azLHFNyN32YCQGCU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Nov 05, 2004 at 02:08:52PM -0800, Bruce M Simpson wrote: > The last part missing is the wiring-up of these upcall notifications to > the XrlFtiTransactionManager, by way of the FibTableObserver and > FtiConfigTableObserver::propagate_fib_changes(); that will come next. I've added this now. Attached. BMS --azLHFNyN32YCQGCU Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="fea-rtm-upcalls2.diff" Revised patch to add the route resolve/miss notifications. Don't much like having to add new state to Fte, but as we use Fte for mass-shipping of forwarding table state to FIB clients, there is no other clean solution. Fixes a typo where I typed UPCALLS when I meant CHANGES. Could probably do with using a bit-set vector instead of an enum where FtiRtmParseType is concerned, as it would save another downcall to the rtm layer. What this doesn't do is add the stuff necessary for an external FIB observer to say 'I'm not interested in route misses'. Index: fte.hh =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fte.hh,v retrieving revision 1.12 diff -u -p -r1.12 fte.hh --- fte.hh 5 Nov 2004 00:47:27 -0000 1.12 +++ fte.hh 6 Nov 2004 00:29:39 -0000 @@ -45,7 +45,7 @@ public: bool xorp_route) : _net(net), _nexthop(nexthop), _ifname(ifname), _vifname(vifname), _metric(metric), _admin_distance(admin_distance), - _xorp_route(xorp_route), _is_deleted(false) {} + _xorp_route(xorp_route), _is_deleted(false), _is_upcall(false) {} Fte(const N& net) : _net(net), _nexthop(A::ZERO(net.af())), _metric(0), _admin_distance(0), _xorp_route(false) {} @@ -59,6 +59,8 @@ public: bool xorp_route() const { return _xorp_route; } bool is_deleted() const { return _is_deleted; } void mark_deleted() { _is_deleted = true; } + bool is_upcall() const { return _is_upcall; } + void mark_upcall() { _is_upcall = true; } /** * Reset all members @@ -72,6 +74,7 @@ public: _admin_distance = 0; _xorp_route = false; _is_deleted = false; + _is_upcall = false; } /** @@ -90,12 +93,13 @@ public: string str() const { return c_format("net = %s nexthop = %s ifname = %s vifname = %s " "metric = %u admin_distance = %u xorp_route = %s " - "is_deleted = %s", + "is_deleted = %s is_upcall = %s", _net.str().c_str(), _nexthop.str().c_str(), _ifname.c_str(), _vifname.c_str(), _metric, _admin_distance, _xorp_route ? "true" : "false", - _is_deleted ? "true" : "false"); + _is_deleted ? "true" : "false", + _is_upcall ? "true" : "false"); } private: @@ -107,6 +111,8 @@ private: uint32_t _admin_distance; // Route admin distance bool _xorp_route; // This route was installed by XORP bool _is_deleted; // True if the entry was deleted + bool _is_upcall; // True if the entry is actually + // an upcall notification }; typedef Fte Fte4; Index: fticonfig.hh =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig.hh,v retrieving revision 1.23 diff -u -p -r1.23 fticonfig.hh --- fticonfig.hh 5 Nov 2004 00:47:43 -0000 1.23 +++ fticonfig.hh 6 Nov 2004 00:29:39 -0000 @@ -25,6 +25,15 @@ #include "libxorp/trie.hh" #include "fte.hh" + +// Knobs for parse_buffer_rtm() in routing socket FtiConfig implementation +// to denote what caller is interested in. +enum FtiRtmParseType { + CHANGES, // RTM_ADD, RTM_CHANGE, RTM_DELETE, RTM_GET. + GET, // RTM_GET only. + UPCALLS // RTM_MISS and RTM_RESOLVE. +}; + #include "fticonfig_entry_get.hh" #include "fticonfig_entry_set.hh" #include "fticonfig_entry_observer.hh" Index: fticonfig_entry_get.hh =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_entry_get.hh,v retrieving revision 1.17 diff -u -p -r1.17 fticonfig_entry_get.hh --- fticonfig_entry_get.hh 26 Oct 2004 23:58:29 -0000 1.17 +++ fticonfig_entry_get.hh 6 Nov 2004 00:29:39 -0000 @@ -110,12 +110,12 @@ public: * @param fte the Fte storage to store the parsed information. * @param buf the buffer with the data to parse. * @param buf_bytes buf_bytes the size of the data in the buffer. - * @param is_rtm_get_only if true, consider only the RTM_GET entries. + * @param type the subset of RTM messages that caller is interested in. * @return true on success, otherwise false. * @see FteX. */ bool parse_buffer_rtm(FteX& fte, const uint8_t *buf, size_t buf_bytes, - bool is_rtm_get_only); + FtiRtmParseType type); /** * Parse information about routing entry information received from Index: fticonfig_entry_get_rtsock.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_entry_get_rtsock.cc,v retrieving revision 1.21 diff -u -p -r1.21 fticonfig_entry_get_rtsock.cc --- fticonfig_entry_get_rtsock.cc 26 Oct 2004 23:55:16 -0000 1.21 +++ fticonfig_entry_get_rtsock.cc 6 Nov 2004 00:29:39 -0000 @@ -303,7 +303,7 @@ FtiConfigEntryGetRtsock::lookup_route_by return (false); } if (parse_buffer_rtm(fte, _rs_reader.buffer(), _rs_reader.buffer_size(), - true) + FtiRtmParseType(GET)) != true) { return (false); } @@ -446,7 +446,7 @@ FtiConfigEntryGetRtsock::lookup_route_by return (false); } if (parse_buffer_rtm(fte, _rs_reader.buffer(), _rs_reader.buffer_size(), - true) + FtiRtmParseType(GET)) != true) { return (false); } Index: fticonfig_entry_parse_rtm.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_entry_parse_rtm.cc,v retrieving revision 1.7 diff -u -p -r1.7 fticonfig_entry_parse_rtm.cc --- fticonfig_entry_parse_rtm.cc 5 Nov 2004 01:27:53 -0000 1.7 +++ fticonfig_entry_parse_rtm.cc 6 Nov 2004 00:29:39 -0000 @@ -39,7 +39,8 @@ #ifndef HAVE_ROUTING_SOCKETS bool -FtiConfigEntryGet::parse_buffer_rtm(FteX& , const uint8_t* , size_t , bool ) +FtiConfigEntryGet::parse_buffer_rtm(FteX& , const uint8_t* , size_t , + FtiRtmParseType) { return false; } @@ -48,13 +49,14 @@ FtiConfigEntryGet::parse_buffer_rtm(FteX bool FtiConfigEntryGet::parse_buffer_rtm(FteX& fte, const uint8_t* buf, - size_t buf_bytes, bool is_rtm_get_only) + size_t buf_bytes, FtiRtmParseType type) { - const struct rt_msghdr* rtm = reinterpret_cast(buf); + const struct rt_msghdr* rtm = + reinterpret_cast(buf); const uint8_t* last = buf + buf_bytes; - + for (const uint8_t* ptr = buf; ptr < last; ptr += rtm->rtm_msglen) { - rtm = reinterpret_cast(ptr); + rtm = reinterpret_cast(ptr); if (RTM_VERSION != rtm->rtm_version) { XLOG_ERROR("RTM version mismatch: expected %d got %d", RTM_VERSION, @@ -62,30 +64,35 @@ FtiConfigEntryGet::parse_buffer_rtm(FteX continue; } - if (is_rtm_get_only) { - // - // Consider only the RTM_GET entries. - // - if (rtm->rtm_type != RTM_GET) + // XXX: ignore entries with an error + if (rtm->rtm_errno != 0) + continue; + + if (type == FtiRtmParseType(GET)) { + if ((rtm->rtm_type != RTM_GET) || !(rtm->rtm_flags & RTF_UP)) + continue; + } else if (type == FtiRtmParseType(UPCALLS)) { + // Upcalls may not be supported in some implementations. +#ifdef RTM_MISS + if (rtm->rtm_type != RTM_MISS) + continue; +#endif +#ifdef RTM_RESOLVE + if (rtm->rtm_type != RTM_RESOLVE) continue; - if (! (rtm->rtm_flags & RTF_UP)) +#endif + } else if (type == FtiRtmParseType(CHANGES)) { + if ((rtm->rtm_type != RTM_ADD) && (rtm->rtm_type != RTM_DELETE) && + (rtm->rtm_type != RTM_CHANGE) && (rtm->rtm_type != RTM_GET)) continue; + } else { + return (false); } - if ((rtm->rtm_type != RTM_ADD) - && (rtm->rtm_type != RTM_DELETE) - && (rtm->rtm_type != RTM_CHANGE) - && (rtm->rtm_type != RTM_GET)) { - continue; - } - - if (rtm->rtm_errno != 0) - continue; // XXX: ignore entries with an error - return (RtmUtils::rtm_get_to_fte_cfg(fte, ftic().iftree(), rtm)); } - - return false; + + return (false); } #endif // HAVE_ROUTING_SOCKETS Index: fticonfig_table_get.hh =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_table_get.hh,v retrieving revision 1.14 diff -u -p -r1.14 fticonfig_table_get.hh --- fticonfig_table_get.hh 26 Oct 2004 23:58:29 -0000 1.14 +++ fticonfig_table_get.hh 6 Nov 2004 00:29:39 -0000 @@ -91,12 +91,12 @@ public: * @param fte_list the list with the Fte entries to store the result. * @param buf the buffer with the data to parse. * @param buf_bytes the size of the data in the buffer. - * @param is_rtm_get_only if true, consider only the RTM_GET entries. + * @param type the subset of RTM messages that caller is interested in. * @return true on success, otherwise false. * @see FteX. */ bool parse_buffer_rtm(int family, list& fte_list, const uint8_t *buf, - size_t buf_bytes, bool is_rtm_get_only); + size_t buf_bytes, FtiRtmParseType type); /** * Parse information about routing table information received from Index: fticonfig_table_get_sysctl.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_table_get_sysctl.cc,v retrieving revision 1.13 diff -u -p -r1.13 fticonfig_table_get_sysctl.cc --- fticonfig_table_get_sysctl.cc 1 Sep 2004 18:12:24 -0000 1.13 +++ fticonfig_table_get_sysctl.cc 6 Nov 2004 00:29:39 -0000 @@ -179,7 +179,8 @@ FtiConfigTableGetSysctl::get_table(int f if (sysctl(mib, sizeof(mib) / sizeof(mib[0]), table_data, &sz, NULL, 0) == 0) { // Parse the result - return (parse_buffer_rtm(family, fte_list, table_data, sz, true)); + return (parse_buffer_rtm(family, fte_list, table_data, sz, + FtiRtmParseType(GET))); } // Error Index: fticonfig_table_observer.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_table_observer.cc,v retrieving revision 1.7 diff -u -p -r1.7 fticonfig_table_observer.cc --- fticonfig_table_observer.cc 21 Oct 2004 00:10:25 -0000 1.7 +++ fticonfig_table_observer.cc 6 Nov 2004 00:29:39 -0000 @@ -126,6 +126,8 @@ FtiConfigTableObserver::propagate_fib_ch ftex.xorp_route()); if (ftex.is_deleted()) fte4.mark_deleted(); + if (ftex.is_upcall()) + fte4.mark_upcall(); fte_list4.push_back(fte4); } @@ -140,6 +142,8 @@ FtiConfigTableObserver::propagate_fib_ch ftex.xorp_route()); if (ftex.is_deleted()) fte6.mark_deleted(); + if (ftex.is_upcall()) + fte6.mark_upcall(); fte_list6.push_back(fte6); } } Index: fticonfig_table_observer_rtsock.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_table_observer_rtsock.cc,v retrieving revision 1.10 diff -u -p -r1.10 fticonfig_table_observer_rtsock.cc --- fticonfig_table_observer_rtsock.cc 1 Sep 2004 18:12:25 -0000 1.10 +++ fticonfig_table_observer_rtsock.cc 6 Nov 2004 00:29:39 -0000 @@ -93,7 +93,12 @@ FtiConfigTableObserverRtsock::receive_da // if (ftic().have_ipv4()) { ftic().ftic_table_get_primary().parse_buffer_rtm(AF_INET, fte_list, - data, nbytes, false); + data, nbytes, FtiRtmParseType(CHANGES)); +#if 1 + // Upcalls too. XXX Change this enum to a set? + ftic().ftic_table_get_primary().parse_buffer_rtm(AF_INET, fte_list, + data, nbytes, FtiRtmParseType(UPCALLS)); +#endif if (! fte_list.empty()) { propagate_fib_changes(fte_list); fte_list.clear(); @@ -106,7 +111,7 @@ FtiConfigTableObserverRtsock::receive_da // if (ftic().have_ipv6()) { ftic().ftic_table_get_primary().parse_buffer_rtm(AF_INET6, fte_list, - data, nbytes, false); + data, nbytes, FtiRtmParseType(CHANGES)); if (! fte_list.empty()) { propagate_fib_changes(fte_list); fte_list.clear(); Index: fticonfig_table_parse_rtm.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/fticonfig_table_parse_rtm.cc,v retrieving revision 1.10 diff -u -p -r1.10 fticonfig_table_parse_rtm.cc --- fticonfig_table_parse_rtm.cc 5 Nov 2004 01:27:53 -0000 1.10 +++ fticonfig_table_parse_rtm.cc 6 Nov 2004 00:29:39 -0000 @@ -40,7 +40,7 @@ #ifndef HAVE_ROUTING_SOCKETS bool FtiConfigTableGet::parse_buffer_rtm(int, list& , const uint8_t* , - size_t , bool ) + size_t , FtiRtmParseType type) { return false; } @@ -50,13 +50,14 @@ FtiConfigTableGet::parse_buffer_rtm(int, bool FtiConfigTableGet::parse_buffer_rtm(int family, list& fte_list, const uint8_t* buf, size_t buf_bytes, - bool is_rtm_get_only) + FtiRtmParseType type) { - const struct rt_msghdr* rtm = reinterpret_cast(buf); + const struct rt_msghdr* rtm = + reinterpret_cast(buf); const uint8_t* last = buf + buf_bytes; - + for (const uint8_t* ptr = buf; ptr < last; ptr += rtm->rtm_msglen) { - rtm = reinterpret_cast(ptr); + rtm = reinterpret_cast(ptr); if (RTM_VERSION != rtm->rtm_version) { XLOG_ERROR("RTM version mismatch: expected %d got %d", RTM_VERSION, @@ -64,25 +65,31 @@ FtiConfigTableGet::parse_buffer_rtm(int continue; } - if (is_rtm_get_only) { - // - // Consider only the RTM_GET entries. - // - if (rtm->rtm_type != RTM_GET) - continue; - if (! (rtm->rtm_flags & RTF_UP)) + if (rtm->rtm_errno != 0) + continue; // XXX: ignore entries with an error + + if (type == FtiRtmParseType(GET)) { + if ((rtm->rtm_type != RTM_GET) || (!(rtm->rtm_flags & RTF_UP))) continue; } - if ((rtm->rtm_type != RTM_ADD) - && (rtm->rtm_type != RTM_DELETE) - && (rtm->rtm_type != RTM_CHANGE) - && (rtm->rtm_type != RTM_GET)) { - continue; + // Upcalls may not be supported in some BSD derived implementations. + if (type == FtiRtmParseType(UPCALLS)) { +#ifdef RTM_MISS + if (rtm->rtm_type != RTM_MISS) + continue; +#endif +#ifdef RTM_RESOLVE + if (rtm->rtm_type != RTM_RESOLVE) + continue; +#endif } - if (rtm->rtm_errno != 0) - continue; // XXX: ignore entries with an error + if (type == FtiRtmParseType(CHANGES)) { + if ((rtm->rtm_type != RTM_ADD) && (rtm->rtm_type != RTM_DELETE) && + (rtm->rtm_type != RTM_CHANGE) && (rtm->rtm_type != RTM_GET)) + continue; + } #ifdef RTF_LLINFO if (rtm->rtm_flags & RTF_LLINFO) @@ -104,12 +111,15 @@ FtiConfigTableGet::parse_buffer_rtm(int if (rtm->rtm_flags & RTF_BROADCAST) continue; // XXX: ignore broadcast entries #endif - + FteX fte(family); - if (RtmUtils::rtm_get_to_fte_cfg(fte, ftic().iftree(), rtm) == true) + if (RtmUtils::rtm_get_to_fte_cfg(fte, ftic().iftree(), rtm) == true) { + if (type == FtiRtmParseType(UPCALLS)) + fte.mark_upcall(); fte_list.push_back(fte); + } } - + return true; } Index: xrl_fti.cc =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/xrl_fti.cc,v retrieving revision 1.10 diff -u -p -r1.10 xrl_fti.cc --- xrl_fti.cc 3 Aug 2004 03:51:48 -0000 1.10 +++ xrl_fti.cc 6 Nov 2004 00:29:39 -0000 @@ -291,6 +291,44 @@ XrlFtiTransactionManager::send_fib_clien return XORP_ERROR; } +int +XrlFtiTransactionManager::send_fib_client_resolve_route(const string& target_name, + const Fte4& fte) +{ + bool success; + + success = _xrl_fea_fib_client.send_resolve_route4( + target_name.c_str(), + fte.net(), + callback(this, + &XrlFtiTransactionManager::send_fib_client_resolve_route4_cb, + target_name)); + + if (success) + return XORP_OK; + else + return XORP_ERROR; +} + +int +XrlFtiTransactionManager::send_fib_client_resolve_route(const string& target_name, + const Fte6& fte) +{ + bool success; + + success = _xrl_fea_fib_client.send_resolve_route6( + target_name.c_str(), + fte.net(), + callback(this, + &XrlFtiTransactionManager::send_fib_client_resolve_route6_cb, + target_name)); + + if (success) + return XORP_OK; + else + return XORP_ERROR; +} + void XrlFtiTransactionManager::send_fib_client_add_route4_cb( const XrlError& xrl_error, @@ -359,6 +397,40 @@ XrlFtiTransactionManager::send_fib_clien fib_client.send_fib_client_route_change_cb(xrl_error); } +void +XrlFtiTransactionManager::send_fib_client_resolve_route4_cb( + const XrlError& xrl_error, + string target_name) +{ + map::iterator iter; + + iter = _fib_clients4.find(target_name); + if (iter == _fib_clients4.end()) { + // The client has probably gone. Silently ignore. + return; + } + + FibClient4& fib_client = iter->second; + fib_client.send_fib_client_route_change_cb(xrl_error); +} + +void +XrlFtiTransactionManager::send_fib_client_resolve_route6_cb( + const XrlError& xrl_error, + string target_name) +{ + map::iterator iter; + + iter = _fib_clients6.find(target_name); + if (iter == _fib_clients6.end()) { + // The client has probably gone. Silently ignore. + return; + } + + FibClient6& fib_client = iter->second; + fib_client.send_fib_client_route_change_cb(xrl_error); +} + template void XrlFtiTransactionManager::FibClient::activate(const list& fte_list) @@ -394,7 +466,9 @@ XrlFtiTransactionManager::FibClient:: // // Send the appropriate XRLs // - if (! fte.is_deleted()) { + if (fte.is_upcall()) { + success = _xftm.send_fib_client_resolve_route(_target_name, fte); + } else if (! fte.is_deleted()) { // Add a route success = _xftm.send_fib_client_add_route(_target_name, fte); } else { Index: xrl_fti.hh =================================================================== RCS file: /usr/local/www/data/cvs/xorp/fea/xrl_fti.hh,v retrieving revision 1.9 diff -u -p -r1.9 xrl_fti.hh --- xrl_fti.hh 17 Aug 2004 02:20:11 -0000 1.9 +++ xrl_fti.hh 6 Nov 2004 00:29:39 -0000 @@ -189,6 +189,28 @@ public: int send_fib_client_delete_route(const string& target_name, const Fte6& fte); + /** + * Send an XRL to a FIB client to inform it of an IPv4 route miss. + * + * @param target_name the target name of the FIB client. + * @param fte the Fte with the destination to resolve. + * @return XORP_OK on success, otherwise XORP_ERROR. + * @see Fte4. + */ + int send_fib_client_resolve_route(const string& target_name, + const Fte4& fte); + + /** + * Send an XRL to a FIB client to inform it of an IPv6 route miss. + * + * @param target_name the target name of the FIB client. + * @param fte the Fte with the destination to resolve. + * @return XORP_OK on success, otherwise XORP_ERROR. + * @see Fte6. + */ + int send_fib_client_resolve_route(const string& target_name, + const Fte6& fte); + protected: FtiTransactionManager _ftm; uint32_t _max_ops; // Maximum operations in a transaction @@ -203,6 +225,10 @@ private: string target_name); void send_fib_client_delete_route6_cb(const XrlError& xrl_error, string target_name); + void send_fib_client_resolve_route4_cb(const XrlError& xrl_error, + string target_name); + void send_fib_client_resolve_route6_cb(const XrlError& xrl_error, + string target_name); /** * A template class for storing FIB client information. --azLHFNyN32YCQGCU-- From bms@spc.org Tue Nov 9 06:44:59 2004 From: bms@spc.org (Bruce M Simpson) Date: Mon, 8 Nov 2004 22:44:59 -0800 Subject: [Xorp-hackers] Re: cvs commit: src/gnu/usr.bin/cc Makefile src/gnu/usr.bin/cc/c++filt Makefile In-Reply-To: <200411090317.iA93H7Fv076384@repoman.freebsd.org> References: <200411090317.iA93H7Fv076384@repoman.freebsd.org> Message-ID: <20041109064459.GB23401@empiric.icir.org> Thank you very much indeed! On Tue, Nov 09, 2004 at 03:17:07AM +0000, Alexander Kabaev wrote: > Modified files: (Branch: RELENG_5) > gnu/usr.bin/cc Makefile > gnu/usr.bin/cc/c++filt Makefile > Log: > MFC: Unbreak c++filt and reconnect it to the build. From bms@spc.org Thu Nov 11 17:12:01 2004 From: bms@spc.org (Bruce M Simpson) Date: Thu, 11 Nov 2004 09:12:01 -0800 Subject: [Xorp-hackers] FIB resolve upcall support committed. In-Reply-To: <20041106003605.GA10185@empiric.icir.org> References: <20041105220852.GB734@empiric.icir.org> <20041106003605.GA10185@empiric.icir.org> Message-ID: <20041111171201.GN723@empiric.icir.org> The support for passing up RTM_MISS and RTM_RESOLVE events from BSD routing sockets (to facilitate on-demand routing) has now been committed to HEAD. The rtnetlink support for Linux currently lacks these hooks, but the infrastructure is there to do it; patches will be gratefully accepted. Regards, BMS From rafael.guimaraes@ac.upc.es Fri Nov 12 12:33:04 2004 From: rafael.guimaraes@ac.upc.es (Rafael Paoliello Guimaraes) Date: Fri, 12 Nov 2004 13:33:04 +0100 Subject: [Xorp-hackers] Multithreading routing process Message-ID: <4194AD80.6050708@ac.upc.es> Hello, I am migrating a multithreading routing process implementation to XORP and, since XORP does not support multithreading, I was thinking about building a communication thread that receives requests from every thread of the routing process and converts this to XRL calls. In order to do this, I though about a FIFO that could store requests from every thread and the communication process simply processes the requests one by one. Ok, this seems to solve my problem (or not?)... However, in order to process these requests, the only thing the communication thread does is to run the eventloop with a periodic timer that calls a function to process any request that is present in the FIFO. This causes a little delay in the processing of requests (such as sending a control packet, for example). If use a small period for this event, this should not be a problem, however I was thinking if there is any way to process the requests as soon as they are available. Can I, for example, schedule an one-off event (with a time-out equal to zero or close to zero) whenever a new request is available in the FIFO? This would be done concurrently to the eventloop execution. Would this work? Or should I forget this and keep on my original solution? Or even the original solution is not good? Cheers, -- =========================================== Rafael Paoliello Guimaraes PhD Student - Computer Networking Group Department of Computer Architecture (DAC) Polytechnic University of Catalonia (UPC) Phone: +34-934017187 Fax: +34-934017055 URL: http://people.ac.upc.es/rpaoliel =========================================== From atanu@ICSI.Berkeley.EDU Sat Nov 13 07:02:06 2004 From: atanu@ICSI.Berkeley.EDU (Atanu Ghosh) Date: Fri, 12 Nov 2004 23:02:06 -0800 Subject: [Xorp-hackers] Multithreading routing process In-Reply-To: Message from Rafael Paoliello Guimaraes of "Fri, 12 Nov 2004 13:33:04 +0100." <4194AD80.6050708@ac.upc.es> Message-ID: <79610.1100329326@tigger.icir.org> The eventloop can wait for timer or file descriptor events. One solution is to have the non XORP threads write their messages (FIFO) on a pipe or socket. In the XORP thread use the EventLoop::add_selector to register interest in the pipe/socket. Therefore whenever a non XORP thread writes a message the XORP thread will receive a notification immediately. Atanu. >>>>> "Rafael" == Rafael Paoliello Guimaraes writes: Rafael> Hello, I am migrating a multithreading routing process Rafael> implementation to XORP and, since XORP does not support Rafael> multithreading, I was thinking about building a Rafael> communication thread that receives requests from every Rafael> thread of the routing process and converts this to XRL Rafael> calls. In order to do this, I though about a FIFO that could Rafael> store requests from every thread and the communication Rafael> process simply processes the requests one by one. Rafael> Ok, this seems to solve my problem (or not?)... However, in Rafael> order to process these requests, the only thing the Rafael> communication thread does is to run the eventloop with a Rafael> periodic timer that calls a function to process any request Rafael> that is present in the FIFO. This causes a little delay in Rafael> the processing of requests (such as sending a control Rafael> packet, for example). If use a small period for this event, Rafael> this should not be a problem, however I was thinking if Rafael> there is any way to process the requests as soon as they are Rafael> available. Can I, for example, schedule an one-off event Rafael> (with a time-out equal to zero or close to zero) whenever a Rafael> new request is available in the FIFO? This would be done Rafael> concurrently to the eventloop execution. Would this work? Or Rafael> should I forget this and keep on my original solution? Or Rafael> even the original solution is not good? Rafael> Cheers, Rafael> -- Rafael> =========================================== Rafael Paoliello Rafael> Guimaraes PhD Student - Computer Networking Group Department Rafael> of Computer Architecture (DAC) Polytechnic University of Rafael> Catalonia (UPC) Phone: +34-934017187 Fax: +34-934017055 URL: Rafael> http://people.ac.upc.es/rpaoliel Rafael> =========================================== Rafael> _______________________________________________ Xorp-hackers Rafael> mailing list Xorp-hackers@icir.org Rafael> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/xorp-hackers From rafael.guimaraes@ac.upc.es Tue Nov 16 13:55:27 2004 From: rafael.guimaraes@ac.upc.es (Rafael Paoliello Guimaraes) Date: Tue, 16 Nov 2004 14:55:27 +0100 Subject: [Xorp-hackers] Doubts on compiling routing process Message-ID: <419A06CF.8000508@ac.upc.es> Hello All, As you know from previous emails, I am trying to migrate a routing process into XORP. I partially integrated it to XORP but I am little lost with respect to its compilation. Let me explain what I have done so far, and what I don't know how to do. 1) I have create a example.tgt file for my routing process and it does implement only the common interface. I used the command line below to generate the example.xrls, example_base.cc and example_base.hh: cpp -I ../interfaces example.tgt |../scripts/tgt-gen (There is a directory under targets called .deps, should there be an example_base.Po file there? How this is done? What should I change in the Makefiles) 2) The only change I have made so far in the original routing process is to add/remove routes through XORP. This means that, in fact, it is something like the static routes, but routes are not received by XRL, instead it is computed by the routing process. In a second step I will migrate the sending and reception of control packets. The whole code is place into xorp-1.0/example. How do I compile it together with XORP? How do I generate its Makefile? Thanks in advance for any help... Cheers, -- =========================================== Rafael Paoliello Guimaraes PhD Student - Computer Networking Group Department of Computer Architecture (DAC) Polytechnic University of Catalonia (UPC) Phone: +34-934017187 Fax: +34-934017055 URL: http://people.ac.upc.es/rpaoliel =========================================== From pavlin@icir.org Tue Nov 16 19:45:32 2004 From: pavlin@icir.org (Pavlin Radoslavov) Date: Tue, 16 Nov 2004 11:45:32 -0800 Subject: [Xorp-hackers] Doubts on compiling routing process In-Reply-To: Message from Rafael Paoliello Guimaraes of "Tue, 16 Nov 2004 14:55:27 +0100." <419A06CF.8000508@ac.upc.es> Message-ID: <200411161945.iAGJjWwK081908@possum.icir.org> Rafael, > As you know from previous emails, I am trying to migrate a routing > process into XORP. I partially integrated it to XORP but I am little > lost with respect to its compilation. Let me explain what I have done so > far, and what I don't know how to do. > > 1) I have create a example.tgt file for my routing process and it does > implement only the common interface. I used the command line below to > generate the example.xrls, example_base.cc and example_base.hh: > cpp -I ../interfaces example.tgt |../scripts/tgt-gen > (There is a directory under targets called .deps, should there be an > example_base.Po file there? How this is done? What should I change in > the Makefiles) In general, I'd recommend that you check the following document: xorp/docs/xorpdev_101/ Currently, it is available only from the CVS repository, so you have to use the anon CVS, and then use the following commands to generate the document (note that you must have Latex installed to do so): cd xorp; ./configure cd docs/xorpdev_101 gmake Alternatively, you can get the document from the following temporary URL: http://nrg.cs.ucl.ac.uk/mjh/tmp/xorpdev101.pdf Section 4.2 describes what to do to generate stub code for the target and what to add to file xorp/xrl/targets/Makefile.am If you add your own foo.xif XRL interface, then you need to edit file xorp/xrl/interfaces/Makefile.am After you have edited the appropriate Makefile.am files, then you need to do the following from the top-level XORP directory: ./bootstrap Note that to run ./bootstrap you must have installed the appropriate autoconf/automake versions as specified in xorp/README file: - autoconf version 2.53 - automake version 1.5 - libtool version 1.3.4 [Slightly older or slightly newer versions may still be working, but we cannot make any guarantees about that] Running the above shell script will invoke the appropriate autoconf/automake commands to generate the Makefile.in files, etc. After that run ./configure as usual to generate the final Makefile files. As part of that process the .deps stuff is generated as necessary so you don't have to do it explicitly. Note that if you use the Makefile.am files, then you don't have to run the "cpp" command above, because all XRL-related auto-generation will happen once you run "gmake". > 2) The only change I have made so far in the original routing process is > to add/remove routes through XORP. This means that, in fact, it is > something like the static routes, but routes are not received by XRL, > instead it is computed by the routing process. In a second step I will > migrate the sending and reception of control packets. The whole code is > place into xorp-1.0/example. How do I compile it together with XORP? How > do I generate its Makefile? Once you have placed your code in your own directory, you have to modify the following files to add your directory to the compilation process: * Edit xorp/configure.in and add the following line as one of the arguments to AC_OUTPUT: example/Makefile * Edit xorp/Makefile.am and add your own "example" directory to SUBDIRS. Note that the sub-directories are visited in the order they are listed, hence don't add your directory before any of the libraries you may use. After you edit those two files, run ./bootstrap and ./configure as I've described previously. In general, every time you edit configure.in or any of the Makefile.am files you must run ./bootstrap Also, every time you run ./bootstrap to generate new ./configure or Makefile.in files, you must run ./configure before you run "gmake". Hope that helps, Pavlin From M.Handley@cs.ucl.ac.uk Thu Nov 25 13:10:21 2004 From: M.Handley@cs.ucl.ac.uk (Mark Handley) Date: Thu, 25 Nov 2004 13:10:21 +0000 Subject: [Xorp-hackers] Businessweek Article on XORP Message-ID: <69188.1101388221@aardvark.cs.ucl.ac.uk> Does XORP Have Cisco's Number? "The open-source project aims to nurture cheap and simple routing software. If it clicks, the networking giant has a fight on its hands." http://www.businessweek.com/technology/content/nov2004/tc20041129_5206_tc024.htm Lots of good quotes from Atanu - well done Atanu! - Mark From edrt@citiz.net Thu Nov 25 14:23:06 2004 From: edrt@citiz.net (edrt) Date: Thu, 25 Nov 2004 22:23:06 +0800 Subject: [Xorp-hackers] XORP on memory stringent embedded platform Message-ID: <200411251420.iAPEKmeU062539@wyvern.icir.org> Dear XORP developers, I managed to port the multicast subsystem of XORP to a embedded system, functionalities including PIM-SM+Mld6igmp+MFEA+lib{xorp,proto,comm,mrt,xrl}. The embedded system is memeory stringent (32M). The orignal system image size is about 5M (os + app, which contains a lot functionalities implemented using C). After add the XORP related functionality, the image grows to about 11M (debugging information stripped) and the system free memory fall down to a low level. So I doubt if I should port other part of XORP system to that embedded platform. I guess the main reason is because XORP is written in C++. Does the big image size related to stl/template ? Is there good ways to reduce XORP image size? (BTW, I have tried -Os option, but the compiler seems too old to support that. May be I should upgrade the compiler.) Thanks Eddy From bms@spc.org Thu Nov 25 18:58:28 2004 From: bms@spc.org (Bruce M Simpson) Date: Thu, 25 Nov 2004 10:58:28 -0800 Subject: [Xorp-hackers] XORP on memory stringent embedded platform In-Reply-To: <200411251420.iAPEKmeU062539@wyvern.icir.org> References: <200411251420.iAPEKmeU062539@wyvern.icir.org> Message-ID: <20041125185828.GZ733@empiric.icir.org> On Thu, Nov 25, 2004 at 10:23:06PM +0800, edrt wrote: > I guess the main reason is because XORP is written in C++. Does the big image size > related to stl/template ? Is there good ways to reduce XORP image size? I am working on implementing dynamic linkage to reduce XORP's size right now. BMS From pavlin@icir.org Thu Nov 25 20:43:59 2004 From: pavlin@icir.org (Pavlin Radoslavov) Date: Thu, 25 Nov 2004 12:43:59 -0800 Subject: [Xorp-hackers] XORP on memory stringent embedded platform In-Reply-To: Message from "edrt" of "Thu, 25 Nov 2004 22:23:06 +0800." <200411251420.iAPEKmeU062539@wyvern.icir.org> Message-ID: <200411252043.iAPKhxNt050456@possum.icir.org> > I managed to port the multicast subsystem of XORP to a embedded system, > functionalities including PIM-SM+Mld6igmp+MFEA+lib{xorp,proto,comm,mrt,xrl}. > > The embedded system is memeory stringent (32M). The orignal system image size > is about 5M (os + app, which contains a lot functionalities implemented using C). > > After add the XORP related functionality, the image grows to about 11M (debugging > information stripped) and the system free memory fall down to a low level. > So I doubt if I should port other part of XORP system to that embedded platform. > > I guess the main reason is because XORP is written in C++. Does the big image size > related to stl/template ? Is there good ways to reduce XORP image size? > > (BTW, I have tried -Os option, but the compiler seems too old to support that. > May be I should upgrade the compiler.) I just did two compilations (using gcc-3.4.2) with the -O2 flag and the -Os flag respectively. The size of the stripped pim/test_pim stand-alone binary (which contains all multicast-related stuff) with the -O2 flag is 6.4MB. The size with the -Os flag is 5.8MB. Of course, there is no guarantee that you will similar size reduction if you upgrade your compiler, but it may be worth to try. Does your system supports dynamic libraries? If yes, then using XORP dynamic libraries can also help to reduce the size (see the previous email on the subject by Bruce who is working right now on enabling dynamic linkage). Regards, Pavlin From bms@spc.org Thu Nov 25 23:00:41 2004 From: bms@spc.org (Bruce M Simpson) Date: Thu, 25 Nov 2004 15:00:41 -0800 Subject: [Xorp-hackers] XORP on memory stringent embedded platform In-Reply-To: <20041125185828.GZ733@empiric.icir.org> References: <200411251420.iAPEKmeU062539@wyvern.icir.org> <20041125185828.GZ733@empiric.icir.org> Message-ID: <20041125230041.GK733@empiric.icir.org> On Thu, Nov 25, 2004 at 10:58:28AM -0800, Bruce M Simpson wrote: > On Thu, Nov 25, 2004 at 10:23:06PM +0800, edrt wrote: > > I guess the main reason is because XORP is written in C++. Does the big image size > > related to stl/template ? Is there good ways to reduce XORP image size? On FreeBSD 4.10-RELEASE the disk footprint is down to 38MB with shared libraries. I haven't benchmarked the memory footprint yet, having problems finding a version of Valgrind for FreeBSD 4.x. FreeBSD's top(1) doesn't have a 'shared' column like Linux's (and such a column is dubious at best). It is possible to get something of a feel for how much of the resident set size is made up by shared pages by inspecting the process map (using procfs directly, and/or procmap from ports). e.g. List all XORP PIDs: ps auxww | grep xorp | awk '{print $2}' Determine shared library code pages resident per vm_map_entry: procmap | grep 'r-x' | awk '{print $3}' then total the returned figures to get an idea of each XORP process's resident shared library page use. I'm making inquiries about a tool which might do this better. The problem is that it's quite invasive - for starters it requires that you know how the FreeBSD VM is organised. Then you need to be able to look at the process's vm_map, make a pass over every vm_map_entry and note what the backing object is. Then pool stats per backing object. In the case of a shared library this is always a vnode. This would give us some idea of the reduced in-core size we get from using shared libraries. Hopefully this stuff should be in CVS soon. Regards, BMS From edrt@citiz.net Fri Nov 26 06:20:22 2004 From: edrt@citiz.net (edrt) Date: Fri, 26 Nov 2004 14:20:22 +0800 Subject: [Xorp-hackers] XORP on memory stringent embedded platform Message-ID: <200411260623.iAQ6NdeU072573@wyvern.icir.org> >> I managed to port the multicast subsystem of XORP to a embedded system, >> functionalities including PIM-SM+Mld6igmp+MFEA+lib{xorp,proto,comm,mrt,xrl}. >> >> The embedded system is memeory stringent (32M). The orignal system image size >> is about 5M (os + app, which contains a lot functionalities implemented using C). >> >> After add the XORP related functionality, the image grows to about 11M (debugging >> information stripped) and the system free memory fall down to a low level. >> So I doubt if I should port other part of XORP system to that embedded platform. >> >> I guess the main reason is because XORP is written in C++. Does the big image size >> related to stl/template ? Is there good ways to reduce XORP image size? >> >> (BTW, I have tried -Os option, but the compiler seems too old to support that. >> May be I should upgrade the compiler.) > >I just did two compilations (using gcc-3.4.2) with the -O2 flag and >the -Os flag respectively. The size of the stripped pim/test_pim >stand-alone binary (which contains all multicast-related stuff) with >the -O2 flag is 6.4MB. The size with the -Os flag is 5.8MB. >Of course, there is no guarantee that you will similar size >reduction if you upgrade your compiler, but it may be worth to try. > >Does your system supports dynamic libraries? If yes, then using >XORP dynamic libraries can also help to reduce the size (see the >previous email on the subject by Bruce who is working right now on >enabling dynamic linkage). > Thanks Bruce and Pavlin for the suggestion. Yes, VxWorks (the embedded os I'm using) supports dynamic library. And I'll try to make the multicast functionality dynamically loadded as you suggested. But on the other hand, even if XORP is dynamically loadded, it still consumes much of the system memeory after it is loadded. Also, some of the embedded platform is Flash stringent (16M), as XORP grows this could also become a problem on embedded platform (though, at present, I can compress the dynamic library stored on the flash), If stl/template is the main factor result in big image size, is there any possiblity XORP can also be build & release with some kind of small memory footprint library? (if XORP is also intended to be deployed on embedded system) I just do a quick google search, and found this uSTL (micro-STL) is a partial implementation of the STL specification intended to reduce code size of the derivative programs. seems worth a try... Anyway, if anyone knows any tricky method that could reduce the XORP program size, pls post it on the thread, I'm really appreciate of that. Thanks Eddy From bms@spc.org Fri Nov 26 08:28:00 2004 From: bms@spc.org (Bruce M Simpson) Date: Fri, 26 Nov 2004 00:28:00 -0800 Subject: [Xorp-hackers] XORP on memory stringent embedded platform In-Reply-To: <200411260623.iAQ6NdeU072573@wyvern.icir.org> References: <200411260623.iAQ6NdeU072573@wyvern.icir.org> Message-ID: <20041126082800.GF61236@empiric.icir.org> On Fri, Nov 26, 2004 at 02:20:22PM +0800, edrt wrote: > But on the other hand, even if XORP is dynamically loadded, it still > consumes much of the system memeory after it is loadded. Also, some of > the embedded platform is Flash stringent (16M), as XORP grows this could > also become a problem on embedded platform (though, at present, I can > compress the dynamic library stored on the flash), Dynamic linkage should reduce the run-time memory footprint further (and potentially reduce data and instruction cache churn). Whilst we have not done any profiling in this area, we'd hope to be able to perform some initial profiling under FreeBSD 5.3 with Valgrind soon. > If stl/template is the main factor result in big image size, is there any > possiblity XORP can also be build & release with some kind of small memory > footprint library? (if XORP is also intended to be deployed on embedded system) It's a bit more complicated than that. Whilst mitigating template expansion by using partial expansion is one technique of many which can be used, it's not the only technique. It is definitely on our list, though. Don't forget that you do not need to install all of the components; the RIB, FEA and Finder are essential. Thank you for the STL suggestion. We may look into this, we are of course always open to patches. Regards, BMS