[Xorp-hackers] FEA performance improvements: only 'pull' active interfaces.

[Pavlin Radoslavov]

Ben,

At the high level (ignoring the extra complexity) I like the idea of
the FEA dealing only with those interfaces that it needs to (i.e.,
only the configured interfaces).

Though, with a large patch like yours that affects the FEA in number
of ways it requires very careful integration.
Hence, please add it to Bugzilla like the previous patch.
I will leave it to you to decide whether to reuse the previous
Bugzilla entry or open a new one.
Also, please add a comment to the Bugzilla entry that the patch
includes other changes like the per-interface socket.

On the technical side, why did you have to merge live_config with
pulled_config? From performance perspective it shouldn't make
difference.

Thanks,
Pavlin


Ben Greear <greearb at candelatech.com> wrote:

> I have completed the first pass at my attempt to speed up
> FEA's handling of interfaces.
> 
> Basically, I removed live_config entirely, using pulled_config
> in it's place.  For pulled_config, I only pull info about configured
> interfaces, and for the observer, I ignore anything not in the
> configured interfaces.  Interfaces are added to the various iftrees
> when they are configured by the user.
> 
> In doing this, I hacked on a bunch of the xrl handler classes.
> Mostly cosmetic, but it makes the patch quite large.  I tried to
> remove or minimize direct access to the ifconfig's iftree objects,
> but some seem necessary and remain.
> 
> The patch also still has a lot of debugging code in it.
> 
> But, it does appear to work.
> 
> I tested my 30-node scenario with ~600 interfaces (about 10-15 of them
> associated with each xorp instance, and others not in any xorp).
> 
> With my patch, the load still goes to around 30 when I'm heavily modifying
> interfaces in the xorps, but the time to make a xorpsh commit was a max of
> about 6 seconds and the system was generally responsive.
> 
> fea also starts up quicker since it doesn't have to read all the interfaces
> in on startup.
> 
> Without this latest addition, the system load went to 30, and xorpsh
> commits were taking 90+ seconds.
> 
> So, it's definitely a winner for xorp on linux in my scenario.  It's
> likely that if netlink is not used, or if there are few interfaces, or
> if there are lots of interfaces and xorp is using all of them, then there
> will NOT be a lot of gain from my patch (maybe even slightly worse performance
> since I'm not batch-reading all interfaces with netlink now.)
> 
> Its also likely I broke the compile on some systems, as some of the netlink
> code was still using live_config(), but evidently was #ifdef'd out on Linux
> since it compiled fine for me.
> 
> Systems that don't use netlink shouldn't be affected much either way, I think,
> but I've no way to really test them.
> 
> The patch is too big for the mailing list, but can be downloaded
> from here:
> 
> http://www.candelatech.com/oss/fea_iftree.patch
> 
> Comments welcome.  If there is something I can do or change to give this more
> of a chance of being accepted, please let me know.
> 
> Thanks,
> Ben
> 
> -- 
> Ben Greear <greearb at candelatech.com>
> Candela Technologies Inc  http://www.candelatech.com
> 
> _______________________________________________
> Xorp-hackers mailing list
> Xorp-hackers at icir.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/xorp-hackers