[Xorp-hackers] testing xorp

[Atanu Ghosh]

Hi,

You are right we don't have anything on the web site about our testing
procedures. We say a little when we announce a new release:
<http://mailman.icsi.berkeley.edu/pipermail/xorp-announce/2007/000006.html>.

The unit/regression tests can invoked by running "gmake check". To run
through all the tests takes about three hours. We have a tinderbox setup
that continuously builds XORP on a number of different operating systems
with different compilers, the output is sent to an internal mailing
list.

Whenever we write code or fix bugs we try and add test cases:
<http://www.xorp.org/bugzilla/show_bug.cgi?id=717>

The tests vary in complexity from simple unit tests to verify that the
class that holds IPv4 addresses works correctly, to starting BGP and
injecting 17000 routes. The test programs have the prefix test_* we have
about 154 (1) of these programs and most of them will run multiple tests
(One of the BGP test scripts has 58 tests). Some of the complex routing
protocols such as BGP have their own test harness
(http://www.xorp.org/releases/1.4/docs/test_harness/test_harness.pdf),
which allow the whole process to be started and tested.

XRL's can easily represented in text form so a number of our tests are
run from scripts.

We have a number of machines permanently running with full BGP feeds
(IPv4 and IPv6). We have one host that receives both feeds and has all
the protocols enabled, we track the resource usage of the XORP processes
on this host, memory usage per process, routes in kernel, swap space and
open files.

We don't currently make the results of our testing available.

	 Atanu.

(1) find . -name 'test_*' -print | egrep '*.cc$|*.py$|*.sh$' | wc -l

>>>>> "Gabor" == Gabor Szabo <szabgab at gmail.com> writes:

    Gabor> Hi,
    Gabor> I am trying to collect information and write articles about
    Gabor> Quality Assurance and Automated Testing in Open Source Projects
    Gabor> http://www.szabgab.com/blog/2007/07/1183825266.html

    Gabor> I have information on many stand-alone tools and applications
    Gabor> (e.g. Perl, PostgreSQL) yet to be posted on my site,
    Gabor> but I would like to include information on how testing is done for
    Gabor> networking applications and especially xorp.

    Gabor> I looked at http://www.xorp.org/ and http://www.vyatta.com/
    Gabor> but the only thing I found was
    Gabor> http://www.vyatta.com/twiki/bin/view/Community/UnhTestResults
    Gabor> This is not what I was looking for.

    Gabor> Are there automatic (unit) tests in the project?
    Gabor> If I want to run them, how can I do it?
    Gabor> Are there smoke testers?
    Gabor> Where are the reports?

    Gabor> I'd appreciate pointers, where should I read about this?

    Gabor> regards
    Gabor> Gabor

    Gabor> -- 
    Gabor> Gabor Szabo
    Gabor> http://www.szabgab.com/
    Gabor> Perl Training in Israel  http://www.pti.co.il/

    Gabor> _______________________________________________
    Gabor> Xorp-hackers mailing list
    Gabor> Xorp-hackers at icir.org
    Gabor> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/xorp-hackers