From ldunn at cisco.com  Mon Mar 10 11:44:01 2008
From: ldunn at cisco.com (Lawrence D. Dunn)
Date: Mon, 10 Mar 2008 12:44:01 -0600
Subject: [Tmrg] Towards a Common TCP Evaluation Suite
In-Reply-To: <aa7d2c6d0803101021t4d150f3dn97633e457d6aff53@mail.gmail.com>
References: <5D977804-59A9-4A06-8E71-4C88AF56FA50@csnet.cs.odu.edu>	
	<56D7B140-63DE-4560-91E6-46C70F7A4583@mac.com>
	<aa7d2c6d0803101021t4d150f3dn97633e457d6aff53@mail.gmail.com>
Message-ID: <p062309a3c3fb314a614d@[10.10.10.59]>

Lachlan,
   Here are my "raw notes" taken during your talk...

Larry
--
<start notes>

  >  4:01p Lachlan A.
      >  Towards a Common TCP Evauation Suite
         Lachlan Andrew1, Cesar Marcondes2, Sally Floyd3, Lawrence
         Dunn4, Romaric Guillier5, Wang Gang6, Lars Eggert7,Sangtae Ha8,
         Injong Rhee8,
         1 Caltech, 2 UCLA, 3 ICSI, 4 Cisco Systems, 5 INRIA, 6 NEC
         China, 7 Nokia, 8 NCSU
      >  He mentioned this was in part, in response to Dunn's "Injong and Doug
         agree on results" goal...
      >  and reduce arguments, need to repeat experiments which may/may
         not match someone else's
      >  want them to be as realistic as possible; not a "benchmark", as
         sally points out people will tune to meet the benchmark
      >  balance reductionism (doug's 1-variable at a time) with realism
      >  design space
          >  start w/ simulator, or testbed
          >  choose topology (delay, bitrate)
          >  flows under test
          >  cross traffic
          >  metrics to track
          >  comments
              >  richard- convergence is the wrong name; please be
                 precise in terminology- like suppressing slowstart is
                 not the same as "a new flow starts..."
              >  is 80% for transients misleading? maybe a spread of
                 points, to help characterize shape a bit?
              >  richard- on impact on reno test- endstations are
                 uncontrollable, and he's afraid things will sync or end
                 up in lockstep due to endstation effects; lachlan- we
                 watch experimental loading; light host links/cpu,
                 tighter bottleneck
              >


<end notes>
At 10:21 AM -0700 3/10/08, Lachlan Andrew wrote:
>Greetings all,
>
>The TCP test suite presentation at PFLDnet received a lot of useful
>feedback.  I took some notes afterwards, but missed many of the questions.
>Could those who raised issues during the talk please repeat them to the list?
>(The slides are at <...>.)
>
>The following are the comments I remember, and my responses.
>
>- For the RTT distribution, why limit it to the nine discrete values,
>   when a modified dummynet can give per-flow delays?
>
>   I said that it was to make it more platform-independent.  However,
>   the point remains:  Why 6 nodes, not 4 or 8?  We'll have to revisit
>   this when we look at whether these link delays give the right RTT
>   distribution when long-RTT flows complete.
>
>- What is "real"?  How do we know that the "realistic" cases we study
>   are better than simpler ones?
>
>   We should probably quote measurement studies to back things up.
>   I would add that we must justify that the benefits of capturing any
>   given real effect is worth the cost, for example in terms of making
>   results potentially misleading due to statistical effects, or
>   difficult to interpret for other reasons.
>
>- It is not "realistic" to assume that the newly arriving flow misses
>   slow start.
>
>   I still think it is a useful case to consider, but we should justify it.
>   If we say it is the "worst case", then we'd have to justify not applying
>   the same logic to losses due to cross traffic.  The worst case is if
>   there are none during the transient, which I think would happen more
>   often than a flow exiting slow start on the first RTT.
>
>   If we have time, it would be interesting to see the sensitivity of
>   convergence time to cross traffic; instead of one test with 10%,
>   we could have one with 0% and one with 30%.
>
>- The "convergence time" section needs a new name
>
>- When we have multiple flows being generated by a single host, we need
>   to avoid them getting into lock-step because of host issues.
>
>   I think that other processing issues will keep us to fairly low rates
>   (<= 622M?)  in testbeds, at least Linux ones, and so this shouldn't
>   be a problem.
>
>- For the 3-hop topology in which each flow gets 60ms delay, we currently
>   specify having seven delay elements.  It is possible to achieve this
>   with only two delay elements (one of the "access" links, and one of
>   the bottleneck links).
>
>   Are these equivalent?  They both result in the same RTTs; most TCP
>   throughput models only care what the RTTs, not which links it occurs on.
>   We can't do the same with shifting buffers around, but I think we can
>   shift propagation delays around freely, can't we?
>
>   If they're not equivalent, could we standardize on the one with
>   two delays instead of seven?  It would mean we only need 4 dummynets
>   instead of 7.
>
>Also, Lars suggested that all authors of the PFLDnet paper become
>acknowledgements, and that we build up a new set of authors based on
>who keeps the suite moving, independent of who was at the round table.
>
>Cheers,
>Lachlan
>
>
>
>--
>Lachlan Andrew  Dept of Computer Science, Caltech
>1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
>Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
>http://netlab.caltech.edu/lachlan


From lachlan.andrew at gmail.com  Mon Mar 17 14:43:11 2008
From: lachlan.andrew at gmail.com (Lachlan Andrew)
Date: Mon, 17 Mar 2008 14:43:11 -0700
Subject: [Tmrg] [Iccrg] TCP evaluation suite round-table
In-Reply-To: <47DE986B.9050501@cis.udel.edu>
References: <aa7d2c6d0708151724o535697a3mf4976152a9fb6a66@mail.gmail.com>
	<aa7d2c6d0711051542y43cbe08ah6ae81ff71ee35cb5@mail.gmail.com>
	<47DE986B.9050501@cis.udel.edu>
Message-ID: <aa7d2c6d0803171443s4382b973j8cc6a4fdaefc31c1@mail.gmail.com>

Greetings Preethi,

On 17/03/2008, Preethi Natarajan <nataraja at cis.udel.edu> wrote:
>
>  Are there any recommendations for cross-traffic generation/Tmix
>  "connection vector" parameters used in "Towards a Common TCP Evaluation
>  Suite"?.
>  Specifically, any pointers to details such as the number of
>  FTP/HTTP/voice sessions to use, "think" time for HTTP sessions etc.

Not yet.  The plan is to produce standard connection vectors, probably
some snippets from the actual traces that the Tmix team have been
using.

We'll keep you informed of any developments.

Cheers,
Lachlan

-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
http://netlab.caltech.edu/lachlan

From sallyfloyd at mac.com  Tue Mar 18 09:48:36 2008
From: sallyfloyd at mac.com (Sally Floyd)
Date: Tue, 18 Mar 2008 09:48:36 -0700
Subject: [Tmrg] Towards a Common TCP Evaluation Suite
In-Reply-To: <aa7d2c6d0803101021t4d150f3dn97633e457d6aff53@mail.gmail.com>
References: <5D977804-59A9-4A06-8E71-4C88AF56FA50@csnet.cs.odu.edu>
	<56D7B140-63DE-4560-91E6-46C70F7A4583@mac.com>
	<aa7d2c6d0803101021t4d150f3dn97633e457d6aff53@mail.gmail.com>
Message-ID: <19B51173-135B-4D82-BF98-086118F2599E@mac.com>

Lachlan -

>
> - For the 3-hop topology in which each flow gets 60ms delay, we  
> currently
>  specify having seven delay elements.  It is possible to achieve this
>  with only two delay elements (one of the "access" links, and one of
>  the bottleneck links).
>
>  Are these equivalent?  They both result in the same RTTs; most TCP
>  throughput models only care what the RTTs, not which links it  
> occurs on.
>  We can't do the same with shifting buffers around, but I think we can
>  shift propagation delays around freely, can't we?
>
>  If they're not equivalent, could we standardize on the one with
>  two delays instead of seven?  It would mean we only need 4 dummynets
>  instead of 7.


I think it makes more sense to specify the topology as it is specified  
now
in Section G ("a ?parking-lot? topology with three (horizontal)  
bottleneck
links and four (vertical) access links"), and then to add that in a  
testbed,
this can be *implemented* with only N delay elements.  Because we want
the scenarios to be useful for simulators as well as testbeds, and some
simulators don't use delay elements.  And it makes it more clear the
real-world topology that corresponds to our simulation or experiment.

- Sally
http://www.icir.org/floyd/



From lachlan.andrew at gmail.com  Tue Mar 18 10:12:33 2008
From: lachlan.andrew at gmail.com (Lachlan Andrew)
Date: Tue, 18 Mar 2008 10:12:33 -0700
Subject: [Tmrg] Towards a Common TCP Evaluation Suite
In-Reply-To: <19B51173-135B-4D82-BF98-086118F2599E@mac.com>
References: <5D977804-59A9-4A06-8E71-4C88AF56FA50@csnet.cs.odu.edu>
	<56D7B140-63DE-4560-91E6-46C70F7A4583@mac.com>
	<aa7d2c6d0803101021t4d150f3dn97633e457d6aff53@mail.gmail.com>
	<19B51173-135B-4D82-BF98-086118F2599E@mac.com>
Message-ID: <aa7d2c6d0803181012w3e6d4959x76344795b639121c@mail.gmail.com>

Greetings Sally,

On 18/03/2008, Sally Floyd <sallyfloyd at mac.com> wrote:
>
> I think it makes more sense to specify the topology as it is specified
>  now in Section G ("a "parking-lot" topology with three (horizontal)
>  bottleneck
>  links and four (vertical) access links"), and then to add that in a
>  testbed, this can be *implemented* with only N delay elements.

I wasn't proposing a change to the arrangements of links, just the
allocation of delays on those links.  The description above describes
both delay allocations equally well.  The only distinction is whether
all links have equal delay, or two have large delays and the rest have
negligible delays.

If the two delay arrangements are equivalent (as I believe they are)
then I agree it is good to describe the symmetric one and note that
the asymmetric one is equivalent.  It would only become awkward if
there is a useful metric for which they give different results.

> Because we want
>  the scenarios to be useful for simulators as well as testbeds, and some
>  simulators don't use delay elements.  And it makes it more clear the
>  real-world topology that corresponds to our simulation or experiment.

I don't quite understand either of these points.
- If a simulator has no delay elements then I don't understand how it
can implement either topology, because the both have delays.  In ns, a
delay element is just a link with non-negligible delay.
- It isn't clear that a completely symmetric case is more real-world
than a highly asymmetric case; both sound like "useful
oversimplifications" to me.

(I'm not meaning to be argumentative; we agree that it is best to
present the symmetric case as the nominal network.)

Cheers,
Lachlan

-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
http://netlab.caltech.edu/lachlan

From sallyfloyd at mac.com  Tue Mar 18 14:33:18 2008
From: sallyfloyd at mac.com (Sally Floyd)
Date: Tue, 18 Mar 2008 14:33:18 -0700
Subject: [Tmrg] Towards a Common TCP Evaluation Suite
In-Reply-To: <aa7d2c6d0803181012w3e6d4959x76344795b639121c@mail.gmail.com>
References: <5D977804-59A9-4A06-8E71-4C88AF56FA50@csnet.cs.odu.edu>
	<56D7B140-63DE-4560-91E6-46C70F7A4583@mac.com>
	<aa7d2c6d0803101021t4d150f3dn97633e457d6aff53@mail.gmail.com>
	<19B51173-135B-4D82-BF98-086118F2599E@mac.com>
	<aa7d2c6d0803181012w3e6d4959x76344795b639121c@mail.gmail.com>
Message-ID: <15CDB731-42C9-430A-832A-B21A1824AE40@mac.com>


On Mar 18, 2008, at 10:12 AM, Lachlan Andrew wrote:

> Greetings Sally,
>
> On 18/03/2008, Sally Floyd <sallyfloyd at mac.com> wrote:
>>
>> I think it makes more sense to specify the topology as it is  
>> specified
>> now in Section G ("a "parking-lot" topology with three (horizontal)
>> bottleneck
>> links and four (vertical) access links"), and then to add that in a
>> testbed, this can be *implemented* with only N delay elements.
>
> I wasn't proposing a change to the arrangements of links, just the
> allocation of delays on those links.  The description above describes
> both delay allocations equally well.  The only distinction is whether
> all links have equal delay, or two have large delays and the rest have
> negligible delays.
>
> If the two delay arrangements are equivalent (as I believe they are)
> then I agree it is good to describe the symmetric one and note that
> the asymmetric one is equivalent.  It would only become awkward if
> there is a useful metric for which they give different results.

My assumption was this topology:

       A ------ B ------ C ------ D

with four access links,  A-E, B-F, C-G, D-H.
(I didn't draw the access links, because I don't trust mail
readers to all present it the same way...)

The flows with multiple bottlenecks go from A to D, and vice versa.
The single bottleneck flows go between E and F, F and G, and G and
H.  So there are three congested links, and four separate paths.
All paths have the same 60 ms round-trip time (in the absence of
queueing delay.)

So I am assuming that you want the links B-F and C-D to each have a
30 ms. one-way delay, and for the other links to have 0 ms. delay.
Or something like that.  But for all links to have the same
queue sizes and such.

I would be happy for the paper to describe the symmetric case,
and to note that the asymmetric case is roughly equivalent.
(I wouldn't expect it to be *exactly* equivalent, because the
timing of packets arriving at forward-path and reverse-path
queues should be slightly different.)

- Sally
http://www.icir.org/floyd/


From lachlan.andrew at gmail.com  Mon Mar 24 07:45:26 2008
From: lachlan.andrew at gmail.com (Lachlan Andrew)
Date: Mon, 24 Mar 2008 07:45:26 -0700
Subject: [Tmrg] Towards a Common TCP Evaluation Suite - traffic
	generator question
In-Reply-To: <02cf01c88d7d$d82c4ef0$c44c1cac@ad.research.nec.com.cn>
References: <aa7d2c6d0708151724o535697a3mf4976152a9fb6a66@mail.gmail.com>
	<02cf01c88d7d$d82c4ef0$c44c1cac@ad.research.nec.com.cn>
Message-ID: <aa7d2c6d0803240745p2f171bacyff3c982a29cee58f@mail.gmail.com>

Greetings,

I've contacted the Tmix team asking for their measured traces, and
hope to generate suitable short traces from them.  If they don't reply
soon, I'll try to generate some synthetic traffic (probably Poisson
arrivals and Pareto file sizes).

However, if we need to resort to that, I'd suggest that we go back to
using  Harpoon  which actually exists for Linux.  The minor increase
in realism it allows will be lost if we have to use synthetic data
anyway.

The Full-TCP issue also applies to the main set of "general tests".
One one hand, I think that the authors of a modification to TCP should
be willing to write it for the Full-TCP module.  On the other hand,
this is another reason to go back to  Harpoon,  especially if people
want to compare their enhancements to the existing enhancements.

Cheers,
Lachlan

On 24/03/2008, Wang gang <wanggang at research.nec.com.cn> wrote:
> Dear all,
>
>  I have a question about using Tmix in the traffic generation in
>  ns2 simulaion.
>
>  If we use Tmix as the traffic generator in the three node pairs
>  of the Dumb-Bell topology,  we need to have the connection
>  vectors for the described  load and packet size distributions
>  listed in the paper, is that right? Or who will provide them?
>
>  And since Tmix is using Full-TCP, if we want to do
>  IV E. Impact on standard TCP traffic and
>  F. Intra-protocol fairness in the paper, we need the TCP variants
>  implemented using Full-TCP (They are not now). Is this
>  a problem? Correct me if I'm wrong.



-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Ph: +1 (626) 395-8820    Fax: +1 (626) 568-3603
http://netlab.caltech.edu/lachlan

From lipeng967 at 163.com  Mon Mar 24 21:46:31 2008
From: lipeng967 at 163.com (lipeng967)
Date: Tue, 25 Mar 2008 12:46:31 +0800 (CST)
Subject: [Tmrg] request to join in the maillist
Message-ID: <11167518.289281206420391567.JavaMail.coremail@bj163app93.163.com>

Dear Professor,
    I am glad to join in the maillist .Thanks.

 
 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/tmrg-interest/attachments/20080325/7d829849/attachment.html 

From sallyfloyd at mac.com  Tue Mar 25 11:00:48 2008
From: sallyfloyd at mac.com (Sally Floyd)
Date: Tue, 25 Mar 2008 11:00:48 -0700
Subject: [Tmrg] RFC 5166: Metrics for the Evaluation of Congestion Control
	Mechanisms
Message-ID: <1408FBE7-0544-4EFE-92EA-E269127AD4F9@mac.com>

Just a note that "Metrics for the Evaluation of Congestion Control
Mechanisms" has, after a slow process, appears as an Informational
RFC.

"http://www.ietf.org/rfc/rfc5166.txt"

The next steps for TMRG are to finish the internet-draft on
"Tools for the Evaluation of Simulation and Testbed Scenarios", and
to see the completion of the TCP Evaluation Suite begun in the
PFLDnet 2008 paper on "Towards a Common TCP Evaluation Suite".

- Sally
http://www.icir.org/floyd/