[Xorp-hackers] Boost use

Ben Greear greearb at candelatech.com
Mon Apr 26 10:12:20 PDT 2010

On 04/26/2010 02:28 AM, Bruce Simpson wrote:
> Hi,
> Nothing like a good healthy rigorous technical debate on a caffeinated
> Monday morning.
> On 04/26/10 04:45, Ben Greear wrote:
>> The ref-pointer logic might could be done better, but I don't think we
>> need
>> to include boost to do that. We could, for instance, use a 'RefCountable'
>> base class that stores a ref count, and a RefPointer objec that stores
>> a pointer
>> to RefCountable and manages the object lifetime. In general, I think we
>> should minimize ref-pointer usage where possible, but in some places,
>> it is quite useful.
> Thoroughly disagree,
> the above is more or less suggesting reinventing the Boost smart_ptr
> library.
> Why reinvent when you can reuse?

If I re-write it, I can optimize it exactly for my needs, including adding
any debugging logic or similar that I see fit.  And, I can read the entirety
of the code and know that it is exactly the same on all platforms and OSs,
regardless of what someone else might have installed on their OS.

> Do you honestly expect a developer under pressure to deliver rapidly,
> to not make the mistake of misreading a & symbol?

There are all sorts of ways to write broken code, and bugs will go into
the system, but we can fix the bugs..and can review the code as it goes
in to check for that sort of thing.

I do expect developers to know the difference in value v/s ref, but if
they don't, I'm more than happy to explain it to them and/or fix the
bugs myself.

>> Shared memory would require locking and/or message passing, and allows
>> one
>> app to corrupt another.
> We already implicitly lock on the socket buffers,
> this is in effect pushing the burden of synchronization onto the host OS.
> Shared memory allows "one app to corrupt another" if and only if page
> mappings fail a bounds check, this would either be due to software
> failure in the app's mappings, or a serious kernel security regression.

The data in the shared memory can be corrupted, and requires locking and/or
message passing to keep that from happening.  I prefer to keep the apps' memory completely
separate, or merge them completely into the same single-threaded binary.

>> I'd love to be able to reproduce this 14m v/s 1m BGP issue, but to date,
>> I haven't found a way to do that. If anyone can help set up such a test,
>> I am very interested in diagnosing it.
> Please try running a production XORP router in the Default-Free Zone, or
> see:
> http://mailman.icsi.berkeley.edu/pipermail/xorp-users/2009-July/003277.html
> PS I remember RIPE were very interested in XORP for collection --
> they switched to Quagga probably for performance reasons (pure
> speculation on my part)
> [Again see RIPE MRTd collection feeds I sent you off-list in the last 10
> days]

I haven't figured out how to feed that MRTD data into xorp to test this,
and I don't have a network where I can put xorp on a BGP network.

The one part of bgp/harness logic that seems like it could feed MRTD data uses
non-batched XRL commands, so it's definitely going to be slow.  But, I might could
load on one xorp, and then have it peer with another and test that peer sync.

>> Again, XRL may not be perfect, but it seems relatively fast as is,
>> and there are several easy memory new/delete optimizations waiting
>> for an hour or two of time to implement.
> "Relatively fast" -- have you benchmarked this in a real app?

I tested with the xorp XRL sender/receiver test.  I also looked
at code in bgp, and it all seems serialized (one XRL request at a
time outstanding).  That means select, send, (process wake on receiver, including
flushing cache), select, receive, process-data, send result for every XRL request.
The original sender does no further work until it gets the response back, and receiver does no
work until it gets the next request.
Batching would be a huge improvement here.

> I have no reason to doubt Sebastien's figures,
> and they converge with the experience of other former users such as
> Hasso Tepper.

Sure, but that doesn't mean it's the IPC transport that is to blame.

> Hasso was particularly critical of XORP's performance across the board,
> however he was always careful to point out his criticism was constructive
> If you want to tackle this point head on, feel free,
> but I speculate that ye may reach much the same conclusions as I did,
> when I was on the XORP, Inc. account.

I need to be able to reproduce the problem first.  I think anyone attempting
to tackle this needs the same..otherwise we may waste months optimizing something
that doesn't actually matter.

>> 1) It's a large, complex set of libraries that one would have to
>> understand well to
>> make good use of or even read boost-enabled xorp code. It doesn't seem
>> like
>> it would replace any sizeable amount of Xorp logic, so it's simply
>> more learning
>> curve to overcome for anyone who doesn't already know boost.
> On the contrary, it's clearly documented and well presented in "Beyond
> the C++ Template Library" -- Karlsson
> I encourage you to actually review the libraries and what they offer.

Clearly documented is nice, but doesn't mean it isn't a learning curve.

> It's also part of the emerging new C++ standards.

When it emerges, and 5 years later when most distributions have supported
it for a few years, then I'd have less complaints, though I still don't like
using large opaque packages unless I really have to.

I vote no, you vote yes.  Hopefully others will chime in as well.


Ben Greear <greearb at candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

More information about the Xorp-hackers mailing list