[Xorp-hackers] Boost use

[Bruce Simpson]

Hi,

Nothing like a good healthy rigorous technical debate on a caffeinated 
Monday morning.

On 04/26/10 04:45, Ben Greear wrote:
>
> The ref-pointer logic might could be done better, but I don't think we 
> need
> to include boost to do that.  We could, for instance, use a 
> 'RefCountable'
> base class that stores a ref count, and a RefPointer objec that stores 
> a pointer
> to RefCountable and manages the object lifetime.  In general, I think we
> should minimize ref-pointer usage where possible, but in some places,
> it is quite useful.

Thoroughly disagree,
the above is more or less suggesting reinventing the Boost smart_ptr 
library.

Why reinvent when you can reuse?

Your position seems to be that the cost of reuse is too high. I'll 
refute this position below

>
> With regard to passing by reference or value, I do not understand the
> confusion.  It's no different than any other object in that regard
> and anyone programming in C++ just needs to understand the difference
> in passing by ref v/s passing by value.

Thoroughly disagree,
operator overloads are what kibosh this argument

Taking the reference of a ref_ptr influences its behaviour,
in a way which is not at all clear from its syntax.

It isn't the first time I've had to do this, and I complained loudly to 
Atanu about it when I had to fix Spt's memory leaks when developing 
xorp_olsr for CenGen back in 2007, because it breaks the intent of the code.

This is a danger of operator overloads in C++ and the semantics aren't 
clear from the syntax --
Boost on the contrary has dealt with this problem well, it is becoming 
part of C++0x, it is already part of C++0x.

Do you honestly expect a developer under pressure to deliver rapidly,
to not make the mistake of misreading a & symbol?

For my part, I find this risk unacceptable, in my best professional 
judgement.

>
> Shared memory would require locking and/or message passing, and allows 
> one
> app to corrupt another.

We already implicitly lock on the socket buffers,
this is in effect pushing the burden of synchronization onto the host OS.

Shared memory allows "one app to corrupt another" if and only if page 
mappings fail a bounds check, this would either be due to software 
failure in the app's mappings, or a serious kernel security regression.

In practice address space is carefully managed and page boundaries are 
enforced
-- this point seems moot


> I'd love to be able to reproduce this 14m v/s 1m BGP issue, but to date,
> I haven't found a way to do that.  If anyone can help set up such a test,
> I am very interested in diagnosing it.

Please try running a production XORP router in the Default-Free Zone, or 
see:
     
http://mailman.icsi.berkeley.edu/pipermail/xorp-users/2009-July/003277.html

PS I remember RIPE were very interested in XORP for collection --
they switched to Quagga probably for performance reasons (pure 
speculation on my part)

[Again see RIPE MRTd collection feeds I sent you off-list in the last 10 
days]

>
> Again, XRL may not be perfect, but it seems relatively fast as is,
> and there are several easy memory new/delete optimizations waiting
> for an hour or two of time to implement.

"Relatively fast" -- have you benchmarked this in a real app?

I have no reason to doubt Sebastien's figures,
and they converge with the experience of other former users such as 
Hasso Tepper.

Hasso was particularly critical of XORP's performance across the board,
however he was always careful to point out his criticism was constructive

If you want to tackle this point head on, feel free,
but I speculate that ye may reach much the same conclusions as I did,
when I was on the XORP, Inc. account.

Again, Mark's silence on this point is inscrutable.

>
> No one has offered lately.  If someone plans to actively work on the 
> public
> XORP SVN tree and wants to use boost, hopefully they can also speak up
> in support of using Boost.
>

Arguments about silent majorities abound.
I certainly plan to vote in the upcoming UK election --
unfortunately that system is largely not representative of my beliefs.

> The default regex works fine (at least on Linux), and I didn't even
> have to link against any external libraries.  From what I can tell,
> this is also the case on BSD.

It seems Boost's regex library is large relative to the size of regex/pcre
(mostly because of the C++ wrapper which we don't use yet)

I responded to Adam Greenhalgh off-list yesterday,
suggesting that moving back to platform regex is a reasonable compromise.

I believe this alone is enough to address boost objections based on 
resource -- please read on

We are in agreement on this point, but little else.

> The downsides of boost for me are:
>
> 1)  It's a large, complex set of libraries that one would have to 
> understand well to
>   make good use of or even read boost-enabled xorp code.  It doesn't 
> seem like
>   it would replace any sizeable amount of Xorp logic, so it's simply 
> more learning
>   curve to overcome for anyone who doesn't already know boost.

On the contrary, it's clearly documented and well presented in "Beyond 
the C++ Template Library" -- Karlsson
I encourage you to actually review the libraries and what they offer.

It's also part of the emerging new C++ standards.

I was reading "the Boost book" at the time when writing OLSR, and I can 
happily relate, that in that time, I spotted a number of opportunities 
for code re-use with Boost.
Indeed, ref_ptr was a garden rake on the ground for message processing.

There wasn't enough time to start from scratch once the project was in 
flight, and I had to deliver something which worked.
If I were to start again, I would use Boost. For me, it's as simple as 
that, I have confidence in it

XORP is already pretty sophisticated stuff for an entry-level C++ 
developer to comprehend.

This is largely qualitative argument, but it seems reasonable to believe 
that professional C++ developers nowadays,
are likely to find the learning curve for the XORP source code *lower*,
if it were to make use of a class/template library seeing uptake in that 
industry.

Currently this is not the case.

FWIW anyone in my day-to-day life outside of this screen and 4 walls,
who seek to learn C++, I tell them:
   "Learn Boost if you want a future in C++"

>
> 2)  It is not installed by default on Fedora Linux (and others).  This 
> means that users
>   must install this to use xorp.  In some ways, this is a minor 
> problem easily fixed, but
>   it is one more impediment that might discourage new users.  Also, 
> that it is not required
>   by the base install of Fedora, indicates to me that it is not that 
> widely used, at least in
>   Linux.

Neither is libncurses-dev, another XORP compile-time dependency,
which surely does not satisfy the definition of 'not that widely used' 
above -- but open to discussion

Quick search on rpm.pbone.net yields: 150 hits for Fedora 11 packages 
requiring boost

It is mostly a template library which tends to be used by servers / 
daemons / frameworks rather than desktop apps.
It is used by last.fm, Facebook, Spotify, and others.
   (Last reality check I did, those were highly successful ventures)

Their dev teams clearly see the value of Boost.
They have a set of requirements not dissimilar to those of XORP.

-- this point seems moot

>
> 2b.  It is a pain to get xorp-svn to compile on Ubuntu because you 
> have to install a specific
>   version of boost, and ubuntu has 2 to choose from.

This is only an issue if you're developing the code, not deploying it

-- this point seems moot
.
>
> 3)  For anyone working with constrained amounts of disk storage, 
> lib-boost is a large
>   amount of extra disk space.

This is only really an issue, if you actually need to deploy all of the 
libraries,
in addition to the headers.

We don't even touch the libraries apart from libboost_regex,
and as pointed out elsewhere in this response,
this issue is easily resolved.

Are you honestly saying,
that 35MB of headers/libs (which don't change apart from on a point 
release of the package),
is a problem for you when hosting a compile at development time?
(16MB of those are headers, and then, mostly comments and whitespace)

It's still substantially less than e.g. SunStudio or Intel C/C++'s disk 
footprint.
For the purposes of utility, it's not an apples-and-oranges comparison
(all are build infrastructure/toolchain)

If you must take that up with someone, take it up with your package vendor.

Ubuntu have split up the Boost packages significantly for this reason,
which helps folk out at deployment time.

Again, as Boost is mostly a C++ *TEMPLATE* library, as soon as we move 
back to platform regex,

-- this point seems moot


>
> I'm more than happy to do the work to back it out, and I will do so in
> a way that allows us to easily start using boost again in the future if
> we so decide (xorp.ct can currently be compiled with or without boost 
> support
> currently).

Backspace is an option for some.

What I'm advocating isn't even new or risky; it's state-of-the-art now,
and your objections seem easily met.

>
> So, as I'm sure you've noticed, my vote is to not use boost.

There seems to be significant difference of opinion on this issue,
and many of the points raised in objection,
don't seem rooted in fact (or are based on poor research)--

Apart from the point about libboost_regex being on the large side,
which I informally discussed with Adam yesterday,
and which is very easy to fix.

best regards
BMS