[From nobody Fri Aug 15 04:26:51 2008
Message-ID: <4660B516.4000902@incunabulum.net>
Date: Sat, 02 Jun 2007 01:08:54 +0100
From: Bruce M Simpson <bms@incunabulum.net>
User-Agent: Thunderbird 1.5.0.10 (X11/20070407)
MIME-Version: 1.0
To: Pavlin Radoslavov <pavlin@icir.org>
CC: xorp-dev@xorp.org
Subject: Shared libraries are a win
References: <200705312235.l4VMZMBN034332@possum.icir.org>
In-Reply-To: <200705312235.l4VMZMBN034332@possum.icir.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Pavlin Radoslavov wrote:
>
> I like the patch, especially if we are moving toward shared
> libraries as we seem are going to (at least with the bottom of the
> FEA).
OK. Be aware though this may not do the right thing for static library 
builds of the tree; I would aim to test with the LIBADD change and 
--enable-static --disable-shared before doing anything.

I just did some basic profiling with exmap on Linux 2.6 of a XORP router 
with a single static route entry.

exmap is able to profile Linux memory map usage by mapped object as well 
as by process; both lists may be browsed using a GTK gui. It uses a 
custom kernel module to build inverted indexes of each process page 
table mapping to loaded VM objects.

Object              Reported RSS [KB]    Actual RSS [KB]   Saving
-----------------------------------------------------------------
libxipc                4110KB            947KB              77%
libxorp                2449KB            601KB              75%  
libfeaclient            772KB            288KB              63%
libcomm                 124KB             44KB              65%

Reported RSS refers to the share of resident set size, as a sum of RSS 
over system processes, occupied by a particular VM object. Actual RSS 
refers to the count of those pages for each object which are shared by 
processes referencing that object.

Saving is the difference between actual RSS and reported RSS expressed 
as an inverted ratio of reported RSS, i.e. 77% less physical memory is 
used across all XORP processes sharing libxipc text and data segments.

libxipc is the biggest win. libxorp less so. The XRL stub libraries are 
less of a win, so I haven't reported them here.

The use of shared libraries here may actually impede XORP performance. 
An initial hit will be taken during lazy relocation through the ELF PLT 
for each process linking to a XORP shared library. Detailed L2 cache 
profiling and timings of XRL latency, e.g. using Valgrind, would be 
needed to observe if this impacts runtime performance to any measurable 
degree.

To summarise, libxipc, libfeaclient, libcomm and libxorp are definite 
candidates for shared linking. Shared libraries are a win for runtime 
XORP memory footprint.

Regards,
BMS

P.S.
The author of exmap has helpfully offered information for porting to 
FreeBSD, although I may not have time to get round to this; the FreeBSD 
KDE folks have expressed an interest in the tool.


]