<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html lang="en"><head><title>Common TCP Evaluation Suite</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="description" content="Common TCP Evaluation Suite">
<meta name="generator" content="xml2rfc v1.33 (http://xml.resource.org/)">
<style type='text/css'><!--
body {
font-family: verdana, charcoal, helvetica, arial, sans-serif;
font-size: small; color: #000; background-color: #FFF;
margin: 2em;
}
h1, h2, h3, h4, h5, h6 {
font-family: helvetica, monaco, "MS Sans Serif", arial, sans-serif;
font-weight: bold; font-style: normal;
}
h1 { color: #900; background-color: transparent; text-align: right; }
h3 { color: #333; background-color: transparent; }
td.RFCbug {
font-size: x-small; text-decoration: none;
width: 30px; height: 30px; padding-top: 2px;
text-align: justify; vertical-align: middle;
background-color: #000;
}
td.RFCbug span.RFC {
font-family: monaco, charcoal, geneva, "MS Sans Serif", helvetica, verdana, sans-serif;
font-weight: bold; color: #666;
}
td.RFCbug span.hotText {
font-family: charcoal, monaco, geneva, "MS Sans Serif", helvetica, verdana, sans-serif;
font-weight: normal; text-align: center; color: #FFF;
}
table.TOCbug { width: 30px; height: 15px; }
td.TOCbug {
text-align: center; width: 30px; height: 15px;
color: #FFF; background-color: #900;
}
td.TOCbug a {
font-family: monaco, charcoal, geneva, "MS Sans Serif", helvetica, sans-serif;
font-weight: bold; font-size: x-small; text-decoration: none;
color: #FFF; background-color: transparent;
}
td.header {
font-family: arial, helvetica, sans-serif; font-size: x-small;
vertical-align: top; width: 33%;
color: #FFF; background-color: #666;
}
td.author { font-weight: bold; font-size: x-small; margin-left: 4em; }
td.author-text { font-size: x-small; }
/* info code from SantaKlauss at http://www.madaboutstyle.com/tooltip2.html */
a.info {
/* This is the key. */
position: relative;
z-index: 24;
text-decoration: none;
}
a.info:hover {
z-index: 25;
color: #FFF; background-color: #900;
}
a.info span { display: none; }
a.info:hover span.info {
/* The span will display just on :hover state. */
display: block;
position: absolute;
font-size: smaller;
top: 2em; left: -5em; width: 15em;
padding: 2px; border: 1px solid #333;
color: #900; background-color: #EEE;
text-align: left;
}
a { font-weight: bold; }
a:link { color: #900; background-color: transparent; }
a:visited { color: #633; background-color: transparent; }
a:active { color: #633; background-color: transparent; }
p { margin-left: 2em; margin-right: 2em; }
p.copyright { font-size: x-small; }
p.toc { font-size: small; font-weight: bold; margin-left: 3em; }
table.toc { margin: 0 0 0 3em; padding: 0; border: 0; vertical-align: text-top; }
td.toc { font-size: small; font-weight: bold; vertical-align: text-top; }
ol.text { margin-left: 2em; margin-right: 2em; }
ul.text { margin-left: 2em; margin-right: 2em; }
li { margin-left: 3em; }
/* RFC-2629 <spanx>s and <artwork>s. */
em { font-style: italic; }
strong { font-weight: bold; }
dfn { font-weight: bold; font-style: normal; }
cite { font-weight: normal; font-style: normal; }
tt { color: #036; }
tt, pre, pre dfn, pre em, pre cite, pre span {
font-family: "Courier New", Courier, monospace; font-size: small;
}
pre {
text-align: left; padding: 4px;
color: #000; background-color: #CCC;
}
pre dfn { color: #900; }
pre em { color: #66F; background-color: #FFC; font-weight: normal; }
pre .key { color: #33C; font-weight: bold; }
pre .id { color: #900; }
pre .str { color: #000; background-color: #CFF; }
pre .val { color: #066; }
pre .rep { color: #909; }
pre .oth { color: #000; background-color: #FCF; }
pre .err { background-color: #FCC; }
/* RFC-2629 <texttable>s. */
table.all, table.full, table.headers, table.none {
font-size: small; text-align: center; border-width: 2px;
vertical-align: top; border-collapse: collapse;
}
table.all, table.full { border-style: solid; border-color: black; }
table.headers, table.none { border-style: none; }
th {
font-weight: bold; border-color: black;
border-width: 2px 2px 3px 2px;
}
table.all th, table.full th { border-style: solid; }
table.headers th { border-style: none none solid none; }
table.none th { border-style: none; }
table.all td {
border-style: solid; border-color: #333;
border-width: 1px 2px;
}
table.full td, table.headers td, table.none td { border-style: none; }
hr { height: 1px; }
hr.insert {
width: 80%; border-style: none; border-width: 0;
color: #CCC; background-color: #CCC;
}
--></style>
</head>
<body>
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<table summary="layout" width="66%" border="0" cellpadding="0" cellspacing="0"><tr><td><table summary="layout" width="100%" border="0" cellpadding="2" cellspacing="1">
<tr><td class="header">Network Working Group</td><td class="header">L. Andrew</td></tr>
<tr><td class="header">Internet-Draft</td><td class="header">Caltech</td></tr>
<tr><td class="header">Intended status: BCP</td><td class="header"> coauthors TBD</td></tr>
<tr><td class="header">Expires: December 29, 2008</td><td class="header">June 27, 2008</td></tr>
</table></td></tr></table>
<h1><br />Common TCP Evaluation Suite<br />draft-irtf-tmrg-tests-00.txt</h1>
<h3>Status of this Memo</h3>
<p>
By submitting this Internet-Draft,
each author represents that any applicable patent or other IPR claims of which
he or she is aware have been or will be disclosed,
and any of which he or she becomes aware will be disclosed,
in accordance with Section 6 of BCP 79.</p>
<p>
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups.
Note that other groups may also distribute working documents as
Internet-Drafts.</p>
<p>
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any time.
It is inappropriate to use Internet-Drafts as reference material or to cite
them other than as “work in progress.”</p>
<p>
The list of current Internet-Drafts can be accessed at
<a href='http://www.ietf.org/ietf/1id-abstracts.txt'>http://www.ietf.org/ietf/1id-abstracts.txt</a>.</p>
<p>
The list of Internet-Draft Shadow Directories can be accessed at
<a href='http://www.ietf.org/shadow.html'>http://www.ietf.org/shadow.html</a>.</p>
<p>
This Internet-Draft will expire on December 29, 2008.</p>
<h3>Abstract</h3>
<p>
This document presents an evaluation test suite for the initial
evaluation of proposed TCP modifications. The goal of the test
suite is to allow researchers to quickly and easily evaluate
their proposed TCP extensions in simulators and testbeds using
a common set of well-defined, standard test cases, in order to
compare and contrast proposals against standard TCP as well as
other proposed modifications. This test suite is not intended to
result in an exhaustive evaluation of a proposed TCP modification
or new congestion control mechanism. Instead, the focus is on
quickly and easily generating an initial evaluation report that
allows the networking community to understand and discuss the
behavioral aspects of a new proposal, in order to guide further
experimentation that will be needed to fully investigate the
specific aspects of a new proposal.
</p><a name="toc"></a><br /><hr />
<h3>Table of Contents</h3>
<p class="toc">
<a href="#anchor1">1.</a>
Requirements notation<br />
<a href="#anchor2">2.</a>
Introduction<br />
<a href="#sec:crossTraffic">3.</a>
Traffic generation<br />
<a href="#anchor3">3.1.</a>
Loads<br />
<a href="#anchor4">3.2.</a>
Equilibrium<br />
<a href="#anchor5">3.3.</a>
Packet size distribution<br />
<a href="#sec:RTTs">3.4.</a>
Round Trip Times<br />
<a href="#anchor6">4.</a>
Scenarios<br />
<a href="#sec:General">4.1.</a>
Basic scenarios<br />
<a href="#anchor7">4.1.1.</a>
Topology and background traffic<br />
<a href="#anchor8">4.1.2.</a>
Flows under test<br />
<a href="#anchor9">4.1.3.</a>
Outputs<br />
<a href="#anchor10">4.2.</a>
Delay/throughput tradeoff as function of queue size<br />
<a href="#anchor11">4.2.1.</a>
Topology and background traffic<br />
<a href="#anchor12">4.2.2.</a>
Flows under test<br />
<a href="#anchor13">4.2.3.</a>
Outputs<br />
<a href="#sec:convergence">4.3.</a>
Ramp up time: completion time of one flow<br />
<a href="#anchor14">4.3.1.</a>
Topology and background traffic<br />
<a href="#anchor15">4.3.2.</a>
Flows under test<br />
<a href="#anchor16">4.3.3.</a>
Outputs<br />
<a href="#sec:transient">4.4.</a>
Transients: release of bandwidth, arrival of many flows<br />
<a href="#anchor17">4.4.1.</a>
Topology and background traffic<br />
<a href="#anchor18">4.4.2.</a>
Flows under test<br />
<a href="#anchor19">4.4.3.</a>
Outputs<br />
<a href="#anchor20">4.5.</a>
Impact on standard TCP traffic<br />
<a href="#anchor21">4.5.1.</a>
Topology and background traffic<br />
<a href="#anchor22">4.5.2.</a>
Flows under test<br />
<a href="#anchor23">4.5.3.</a>
Outputs<br />
<a href="#anchor24">4.5.4.</a>
Suggestions<br />
<a href="#anchor25">4.6.</a>
Intra-protocol and inter-RTT fairness<br />
<a href="#anchor26">4.6.1.</a>
Topology and background traffic<br />
<a href="#anchor27">4.6.2.</a>
Flows under test<br />
<a href="#anchor28">4.6.3.</a>
Outputs<br />
<a href="#anchor29">4.7.</a>
Multiple bottlenecks<br />
<a href="#anchor30">4.7.1.</a>
Topology and background traffic<br />
<a href="#anchor31">4.7.2.</a>
Flows under test<br />
<a href="#anchor32">4.7.3.</a>
Outputs<br />
<a href="#sec:conc">4.8.</a>
Conclusions<br />
<a href="#anchor33">4.9.</a>
Acknowledgements<br />
<a href="#anchor34">5.</a>
IANA Considerations<br />
<a href="#anchor35">6.</a>
Security Considerations<br />
<a href="#rfc.references1">7.</a>
References<br />
<a href="#rfc.references1">7.1.</a>
Normative References<br />
<a href="#rfc.references2">7.2.</a>
Informative References<br />
<a href="#rfc.authors">§</a>
Authors' Addresses<br />
<a href="#rfc.copyright">§</a>
Intellectual Property and Copyright Statements<br />
</p>
<br clear="all" />
<a name="anchor1"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.1"></a><h3>1.
Requirements notation</h3>
<p>(Do we need this for a Best Current Practice RFC?)
</p>
<p>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL",
"SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY",
and "OPTIONAL" in this document are to be interpreted as
described in <a class='info' href='#RFC2119'>[RFC2119]<span> (</span><span class='info'>Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.</span><span>)</span></a>.
</p>
<a name="anchor2"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.2"></a><h3>2.
Introduction</h3>
<p>
This document describes a common test suite for the initial evaluation of
new TCP extensions. It defines a small number of evaluation scenarios,
including traffic and delay distributions, network topologies, and
evaluation parameters and metrics. The motivation for such an evaluation
suite is to help researchers in evaluating their proposed modifications
to TCP. The evaluation suite will also enable independent duplication
and verification of reported results by others, which is an important
aspect of the scientific method that is not often put to use by the
networking community. A specific target is that the evaluations should
be able to be completed in three days of simulations, or in a reasonable
amount of effort in a testbed.
</p>
<p>
This document is an outcome of a ``round-table'' meeting on TCP evaluation,
held at Caltech on
November 8-9, 2007.
This document is the first step in constructing the evaluation suite;
the goal is for the evaluation suite to be adapted in response from
feedback from the networking community.
</p>
<a name="sec:crossTraffic"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.3"></a><h3>3.
Traffic generation</h3>
<p>
Congestion control concerns the response of flows to bandwidth limitations
or to the presence of other flows.
For a realistic testing of a congestion control protocol, we design
scenarios to
use reasonably-typical traffic; most scenarios use traffic generated
from a traffic generator, with a range of start times for user sessions,
connection sizes, and the like,
mimicking the traffic patterns
commonly observed in the Internet. Cross-traffic and reverse-path traffic
have the
desirable
effect of reducing the occurrence of pathological conditions
such as global synchronization
among competing flows that might otherwise be mis-interpreted as normal average behaviours of
those protocols <a class='info' href='#FK03'>[FK03]<span> (</span><span class='info'>Floyd, S. and E. Kohler, “Internet Research Needs Better Models,” .</span><span>)</span></a>, <a class='info' href='#MV06'>[MV06]<span> (</span><span class='info'>Mascolo, S. and F. Vacirca, “The Effect of Reverse Traffic on the Performance of New TCP Congestion Control Algorithms for Gigabit Networks,” .</span><span>)</span></a>.
This traffic must be reasonably realistic for the tests to predict
the behaviour of congestion control protocols in real networks, and also
well-defined
so that statistical noise does not mask important effects.
</p>
<p>
It is important that the same ``amount'' of congestion or cross-traffic be used
for the testing scenarios of different congestion control algorithms.
This is complicated by the fact that packet arrivals and even flow arrivals
are influenced by the behavior of the algorithms.
For this reason, a pure packet-level generation of traffic where
generated traffic does not respond to the
behaviour of other present flows is not suitable. Instead,
emulating application or user behaviours at the end points using reactive
protocols such as TCP in a closed-loop fashion
results in a closer approximation of cross-traffic,
where user behaviours are modeled by well-defined parameters
for source inputs (e.g., request sizes for HTTP),
destination inputs (e.g., response
size), and think times between pairs of source
and destination inputs.
By setting appropriate parameters for the traffic generator,
we can emulate non-greedy user-interactive traffic
(e.g., HTTP 1.1, SMTP and Telnet)
as well as greedy traffic (e.g., P2P and long file downloads).
This approach models protocol reactions to the congestion caused
by other flows in the common paths, although it fails to model the reactions
of users themselves to the presence of the congestion.
</p>
<p>
While the protocols being tested may differ,
it is important that we maintain the same ``load'' or level of congestion
for the experimental scenarios.
To enable this, we use a hybrid of open-loop and close-loop approaches.
For this test suite, network traffic consists of sessions corresponding to individual users.
Because users are independent, these session arrivals are well modeled by an
open-loop Poisson process. A session may consist of a single greedy
TCP flow, multiple greedy flows separated by user ``think'' times, or a single
non-greedy flow with embedded think times.
The session arrival process forms a Poisson process <a class='info' href='#HVA03'>[HVA03]<span> (</span><span class='info'>Hohn, N., Veitch, D., and P. Abry, “The Impact of the Flow Arrival Process in Internet Traffic,” .</span><span>)</span></a>.
Both the think times and burst sizes have heavy-tailed distributions, with
the exact
distribution based on empirical studies. The think times and
burst sizes will be chosen independently. This is unlikely to be the case in
practice, but we have not been able to find any measurements of the joint
distribution. We invite researchers to study this joint distribution, and
future revisions of this test suite will use such statistics when they are
available.
</p>
<p>
There are several traffic generators available that implement a similar approach
to that discussed above. For now, we are planning to use the
Tmix <a class='info' href='#Tmix'>[Tmix]<span> (</span><span class='info'>Weigle, M., Adurthi, P., Hernandez-Campos, F., Jeffay, K., and F. Smith, “Tmix: a tool for generating realistic TCP application workloads in ns-2,” .</span><span>)</span></a> traffic generator.
Tmix represents each TCP connection by a connection vector
consisting of a sequence of (request-size, response-size, think-time) triples,
thus representing bi-directional traffic.
Connection vectors used for traffic generation can be obtained from Internet traffic traces.
By taking measurement from various points of the Internet such as
campus networks, DSL access links, and IPS core backbones, we can obtain sets of
connection vectors for different levels of congested links. We plan to publish these
connection vectors as part of this test suite.
</p>
<a name="anchor3"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.3.1"></a><h3>3.1.
Loads</h3>
<p>
For most current traffic generators, the traffic is specified by an
arrival rate for independent user sessions, along with specifications of
connection sizes, number of connections per sessions, user wait
times within sessions, and the like.
For many of the scenarios, such as the basic scenarios in
<a class='info' href='#sec:General'>Section 4.1<span> (</span><span class='info'>Basic scenarios</span><span>)</span></a>, each scenario is run for a range of loads,
where the load is varied by varying the
rate of session arrivals.
For a given congestion control mechanism,
experiments run with different loads are likely
to have different packet drop rates, and different
levels of statistical multiplexing.
</p>
<p>
Because the session arrival times
are specified independently of the transfer times, one way to specify the
load would be as
A = E[f]/E[t],
where E[f] is the mean session size (in bits transferred),
E[t] is the mean session inter-arrival time in seconds,
and A is the load in bps.
</p>
<p>
It is important to test congestion control in ``overloaded'' conditions.
However, if A>c , where c is the capacity of the bottleneck link,
then the system has no equilibrium. Such cases are studied in
<a class='info' href='#sec:transient'>Section 4.4<span> (</span><span class='info'>Transients: release of bandwidth, arrival of many flows</span><span>)</span></a>. In long-running experiments with A>c ,
the expected
number of flows would
increase without bound. This means that the measured results would be very
sensitive to the duration of the simulation.
</p>
<p>
Instead, for equilibrium experiments, we measure the load as the
``mean number of jobs in an M/G/1 queue using processor sharing,'' where a job
is a user session. This reflects the fact that TCP
aims at processor sharing of variable sized files. Because processor sharing
is a symmetric discipline <a class='info' href='#Kelly79'>[Kelly79]<span> (</span><span class='info'>Kelly, F., “Reversibility and stochastic networks,” .</span><span>)</span></a>, the mean number of flows is equal to
that of an M/M/1 queue, namely rho/(1-rho) , where
rho=lambda S/C , and lambda [flows per second] is the arrival rate of
jobs/flows, S [bits] is the mean job size and C [bits per second]
is the bottleneck capacity.
For small loads, say 10%,
this is essentially equal to the fraction of the capacity. However, for
overloaded systems, the fraction of the bandwidth used will be much less than
this measure of load.
</p>
<p>
In order to improve the traffic generators used in these scenarios,
we invite researchers to explore how the user behavior, as reflected
in the connection sizes, user wait times, and number
of connections per session,
might be affected by the level of congestion experienced
within a session <a class='info' href='#RMC03'>[RMC03]<span> (</span><span class='info'>Rossi, D., Mellia, M., and C. Casetti, “User Patience and the Web: a Hands-on Investigation,” .</span><span>)</span></a>.
</p>
<a name="anchor4"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.3.2"></a><h3>3.2.
Equilibrium</h3>
<p>
In order to minimize the dependence of the results on the experiment durations,
scenarios should be as stationary as possible. To this end, experiments will
start with rho/(1-rho) active cross-traffic flows,
with traffic of the specified load.
<strong>It is still an open issue whether to use tests with rho>1 .
If such tests are used, the initial number of flows will need to be
defined.</strong>
</p>
<p>
Note that the distribution of the durations of the active flows at a given time
is (often significantly) different from the distribution of flow durations,
skewed toward long flows. For simplicity, this will be ignored and the
initial flow sizes will be drawn from the general flow size distribution.
</p>
<a name="anchor5"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.3.3"></a><h3>3.3.
Packet size distribution</h3>
<p>
For flows generated by the traffic generator, 10% use 536-byte packets,
and 90% 1500-byte packets. The packet size of each flow will be
specified along with the start time and duration, to maximize the
repeatability.
</p>
<a name="sec:RTTs"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.3.4"></a><h3>3.4.
Round Trip Times</h3>
<p>
Most tests use a simple dumbbell
topology with a central link that connects two routers, as illustrated in
<a class='info' href='#General-dumbbell'>Figure 1</a>. Each router is connected to three nodes by edge links.
In order to generate a typical range of round trip times,
edge links have different delays.
On one side, the one-way propagation delays are: 0 ms, 12 ms and 25 ms;
on the other: 2 ms, 37 ms, and 75 ms.
Traffic is uniformly shared among the nine source/destination pairs,
giving a distribution of per-flow RTTs in the absence of queueing delay
shown in <a class='info' href='#tab:RTTs'>Figure 2</a>.
These RTTs are computed for a dumbbell topology with a delay of 0 ms
for the central link.
The delay for the central link is given in the specific scenarios
in the next section.
<br /><hr class="insert" />
<a name="General-dumbbell"></a>
</p>
<div style='display: table; width: 0; margin-left: 3em; margin-right: auto'><pre>
Node 1 Node 4
\_ _/
\_ _/
\_ __________ Intermediate __________ _/
| | link | |
Node 2 ------| Router 1 |----------------| Router 2 |------ Node 5
_|__________| |__________|_
_/ \_
_/ \_
Node 3 / \ Node 6
</pre></div><p>
<p>A dumbbell topology
</p><table border="0" cellpadding="0" cellspacing="2" align="center"><tr><td align="center"><font face="monaco, MS Sans Serif" size="1"><b> Figure 1 </b></font><br /></td></tr></table><hr class="insert" />
For dummynet experiments, delays can be obtained by specifying the delay
of each flow.
<br /><hr class="insert" />
<a name="tab:RTTs"></a>
<div style='display: table; width: 0; margin-left: 3em; margin-right: auto'><pre>
------------------------------------------
| Path | RTT || Path | RTT || Path | RTT |
|------+-----++------+-----++------+-----|
| 1-4 | 4 || 1-5 | 74 || 1-6 | 150 |
| 2-4 | 28 || 2-5 | 98 || 2-6 | 174 |
| 3-4 | 54 || 3-5 | 124 || 3-6 | 200 |
------------------------------------------
</pre></div>
<p>RTTs of the paths between two nodes, in milliseconds.
<strong> These RTTs are subject to change, based on comparison between the resulting
packet-weighted RTT distribution and measurements</strong>
<strong> I'd like to change the RTT 1-4 to 3ms or 5ms instead of 4ms... -- LA</strong>
</p><table border="0" cellpadding="0" cellspacing="2" align="center"><tr><td align="center"><font face="monaco, MS Sans Serif" size="1"><b> Figure 2 </b></font><br /></td></tr></table><hr class="insert" />
<a name="anchor6"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4"></a><h3>4.
Scenarios</h3>
<p>
It is not possible to provide TCP researchers with a complete set of scenarios
for
an exhaustive evaluation of a new TCP extension; especially because the
characteristics of a new extension will often require experiments with specific
scenarios that highlight its behavior. On the other hand, an
exhaustive evaluation of a TCP extension will need to include several
standard scenarios, and it is the focus of the test suite described
in this section to define this initial set of test cases.
</p>
<a name="sec:General"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.1"></a><h3>4.1.
Basic scenarios</h3>
<p>
The purpose of the basic scenarios is to explore the behavior of a TCP
extension over different link types. The scenarios use the dumbbell topology of
<a class='info' href='#sec:RTTs'>Section 3.4<span> (</span><span class='info'>Round Trip Times</span><span>)</span></a>, with the link delays modified as specified below.
</p>
<p>
This basic topology is used to instantiate several basic scenarios, by appropriately choosing capacity and delay parameters for the individual links. Depending on the configuration, the bottleneck link may be in one of the edge links or the central link.
</p>
<a name="anchor7"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.1.1"></a><h3>4.1.1.
Topology and background traffic</h3>
<p>
The basic scenarios are for a single topology, with a range of capacities and
RTTs. For each scenario, traffic levels of uncongested, mild congestion, and
moderate congestion are specified; these are explained below.
</p>
<p>
<strong>Data Center:</strong> The data center scenario models a case where bandwidth is plentiful and link delays are generally low. It uses the same configuration for the central link and all of the edge links.
All links have a capacity of either
1 Gbps, 2.5 Gbps or 10 Gbps; links from nodes 1, 2 and 4 have a one-way
propagation delay of 1 ms, while those from nodes 3, 5 and 6 have
10 ms <a class='info' href='#WCL05'>[WCL05]<span> (</span><span class='info'>Wei, D., Cao, P., and S. Low, “Time for a TCP Benchmark Suite?,” .</span><span>)</span></a>, and the common link has 0 ms delay.
</p>
<p>
Uncongested: TBD
Mild congestion: TBD
Moderate congestion: TBD
</p>
<p>
<strong>Access Link:</strong> The access link scenario
models an access link connecting an institution (e.g., a university
or corporation) to an ISP. The central and edge links are all 100 Mbps.
The one-way propagation delay of the central link is 2 ms, while the edge links
have the delays given in <a class='info' href='#sec:RTTs'>Section 3.4<span> (</span><span class='info'>Round Trip Times</span><span>)</span></a>. Our goal in assigning delays
to edge links is only to give a realistic distribution of
round-trip times for traffic on the central link.
</p>
<p>
Uncongested: TBD
Mild congestion: TBD
Moderate congestion: TBD
</p>
<p>
<strong>Trans-Oceanic Link:</strong> The trans-oceanic scenario
models a test case where mostly lower-delay edge links feed into a high-delay
central link.
The central link is 1 Gbps, with a one-way propagation delay of 65 ms.
The edge links have the same bandwidth as the central link, with
the one-way delays given in <a class='info' href='#sec:RTTs'>Section 3.4<span> (</span><span class='info'>Round Trip Times</span><span>)</span></a>.
An alternative would be to use smaller delays for the edge links,
with one-way delays for each set of three edge links of 5, 10, and 25 ms.
<strong>Implementations MAY use a smaller bandwidth for the trans-oceanic link,
for example to run a simulation in a feasible amount of time.
In testbeds, one of the metrics SHOULD be the number of timeouts in
servers, due to implementation issues when running at high speed.</strong>
</p>
<p>
Uncongested: TBD
Mild congestion: TBD
Moderate congestion: TBD
</p>
<p>
<strong>Geostationary Satellite:</strong> The geostationary
satellite scenario models an asymmetric test case with a high-bandwidth
downlink and a low-bandwidth uplink
<a class='info' href='#HK99'>[HK99]<span> (</span><span class='info'>Henderson, T. and R. Katz, “Transport Protocols for Internet-Compatible Satellite Networks,” .</span><span>)</span></a>, <a class='info' href='#GF04'>[GF04]<span> (</span><span class='info'>Gurtov, A. and S. Floyd, “Modeling Wireless Links for Transport Protocols,” .</span><span>)</span></a>. The capacity of the
central link is 40 Mbps with a one-way propagation delay of 300 ms.
The downlink
capacity of the edge links is also 40 Mbps, but their uplink capacity
is only 4 Mbps.
Edge one-way delays are as given in <a class='info' href='#sec:RTTs'>Section 3.4<span> (</span><span class='info'>Round Trip Times</span><span>)</span></a>.
Note that ``downlink'' is towards the router for edge links attached to the
first router, and away from the router for edge links on the other router.
</p>
<p>
Uncongested: TBD
Mild congestion: TBD
Moderate congestion: TBD
</p>
<p>
<strong>Wireless Access:</strong> The wireless access scenario
models wireless access to the wired backbone.
The capacity of the central link is 100 Mbps with 2 ms
of one-way
delay. All links to Router 1 are wired. Router 2 has a shared wireless link of
nominal bit rate 11 Mbps (to
model IEEE 802.11b links) or 54 Mbps (IEEE 802.11a/g) with
a one-way delay of 1us connected to dummy nodes 4', 5' and 6',
which are then connected to nodes 4, 5 and 6 by wired links of delays
2, 37 and
75 ms. This is to achieve the same RTT distribution as the other scenarios,
while allowing a CSMA model to have realistic delay for a WLAN.
</p>
<p>
Note that wireless links have many other unique properties not captured by
delay and bitrate.
In particular, the physical layer might suffer from propagation effects
that result in packet losses, and the MAC layer might add high
jitter under contention or large steps in bandwidth due to adaptive
modulation and coding.
Specifying these properties is beyond the scope of
the current first version of this test suite.
</p>
<p>
Uncongested: TBD
Mild congestion: TBD
Moderate congestion: TBD
</p>
<p>
<strong>Dial-up Link:</strong>
The dial-up link scenario models a network with
a dial-up link of 64 kbps and a one-way delay of 5 ms for the central
link.
<strong>modems are asymmetric, 56k downlink and 33.6k or 48k uplink.
Should we change this?</strong>
This could be thought of as modeling a scenario reported as typical
in Africa, with many users sharing a single low-bandwidth dial-up
link.
</p>
<p>
Uncongested: TBD
Mild congestion: TBD
Moderate congestion: TBD
</p>
<p>
<strong>Traffic:</strong>
For each of the basic scenarios, three cases are tested:
uncongested; mild congestion, and moderate congestion.
All cases will use scaled versions of the traces
available at <http://wil.cs.caltech.edu/suite>.
<strong>The exact traffic loads and run times
for each scenario still need to be agreed upon.
There is ongoing debate about whether rho>1 is needed
to get moderate to high congestion. If rho>1 is used, note that the
results will depend heavily on the run time, because congestion
will progressively build up. In those cases, metrics which consider this
non-stationarity may be more useful than average quantities.</strong>
In the default case, the reverse path has
a low level of traffic (10% load).
The buffer size at the two routers is set to the maximum
bandwidth-delay-product for a 100 ms flow
(i.e., a maximum queueing delay of 100 ms),
with drop-tail queues in units of packets.
Each run will be for at least a hundred seconds, and the metrics will
not cover the initial warm-up times of each run.
(Testbeds might use longer run times, as should simulations with smaller
bandwidth-delay products.)
</p>
<p>
As with all of the scenarios in this document, the basic scenarios
could benefit from more measurement studies about characteristics
of congested links in the current Internet, and about trends
that could help predict the characteristics of congested links in
the future.
This would include more measurements on typical packet drop rates,
and on the range of round-trip times for traffic on congested links.
</p>
<p>
For the access link scenario, more extensive simulations or
experiments will be run,
with both drop-tail and RED queue management, with
drop-tail queues in units of both bytes and packets,
and with RED queue management both in byte mode and in packet mode.
Specific TCP extensions may require the
evaluation of associated AQM mechanisms.
For the access link scenario, simulations or experiments will
also include runs with a reverse-path load equal to the
forward-path load.
For the access link scenario, additional experiments will
use a range of buffer sizes, including 20% and 200% of the
bandwidth-delay product for a 100 ms flow.
</p>
<a name="anchor8"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.1.2"></a><h3>4.1.2.
Flows under test</h3>
<p>
For this basic scenario, there is no differentiation between
``cross-traffic'' and the ``flows under test''. The aggregate traffic is
under test, with the metrics exploring both
aggregate traffic and distributions of flow-specific metrics.
</p>
<a name="anchor9"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.1.3"></a><h3>4.1.3.
Outputs</h3>
<p>
For each run, the following metrics will be collected,
for the central link in each direction: the aggregate link utilization, the average packet drop rate,
and the average queueing delay, all over the second half of the run.
<strong>This metric could be difficult to gather in emulated testbeds since routers
statistics of queue utilization are not always reliable and depend on
time-scale.</strong>
Separate statistics SHOULD be reported for each direction in the satellite and
wireless access scenarios, since those networks are asymmetric.
<strong>Should "over the second half of the run" be "starting after 50s"? Sally used
the second half of the run, for 100s simulations, but we to get non-random
results, we should run for longer. The warm-up time doesn't need to scale up
with the run length.</strong>
</p>
<p>
Other metrics of interest for general
scenarios can be grouped in two sets: flow-centric and stability. The
flow-centric metrics include the sending rate, good-put, cumulative loss and
queueing delay trajectory for each flow, over time,
and the transfer time per flow versus file size.
<strong>Testbeds could use monitors in the TCP layer (e.g., Web100) to estimate
the queueing delay and loss.</strong>
<strong>NS2 flowmon has problems, because it seems not to release memory associated
with terminated flows. </strong>
Stability properties of interest include the
standard deviation of the throughput and the queueing delay for the
bottleneck link and for flows <a class='info' href='#WCL05'>[WCL05]<span> (</span><span class='info'>Wei, D., Cao, P., and S. Low, “Time for a TCP Benchmark Suite?,” .</span><span>)</span></a>. The worst case stability is also considered.
</p>
<a name="anchor10"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.2"></a><h3>4.2.
Delay/throughput tradeoff as function of queue size</h3>
<p>
Different queue management mechanisms
have different delay-throughput tradeoffs. E.g., Adaptive Virtual
Queue <a class='info' href='#KS01'>[KS01]<span> (</span><span class='info'>Kunniyur, S. and R. Srikant, “Analysis and Design of an Adaptive Virtual Queue (AVQ) Algorithm for Active Queue Management,” .</span><span>)</span></a> gives low delay, at the expense of lower throughput.
Different congestion control mechanisms may have different tradeoffs,
which these tests aim to illustrate.
</p>
<a name="anchor11"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.2.1"></a><h3>4.2.1.
Topology and background traffic</h3>
<p>
These tests use the topology of <a class='info' href='#sec:RTTs'>Section 3.4<span> (</span><span class='info'>Round Trip Times</span><span>)</span></a>.
This test is run for the access link scenario in
<a class='info' href='#sec:General'>Section 4.1<span> (</span><span class='info'>Basic scenarios</span><span>)</span></a>.
</p>
<p>
For each Drop-Tail scenario set, five tests are run, with buffer sizes of
10%, 20%, 50%, 100%, and 200% of the Bandwidth Delay Product (BDP)
for a 100 ms flow. For each AQM scenario (if used), five tests are run,
with a target average queue size of 2.5%, 5%, 10%, 20%, and 50%
of the BDP, with a buffer equal to the BDP.
</p>
<a name="anchor12"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.2.2"></a><h3>4.2.2.
Flows under test</h3>
<p>
The level of traffic from the traffic generator
will be specified
so that when a buffer size of 100% of the BDP is used with
Drop Tail queue management, there is a moderate level of congestion
(e.g., 1-2% packet drop rates when Standard TCP is used). Alternately,
a range of traffic levels could be chosen, with a scenario set run for
each traffic level (as in the examples cited below).
</p>
<a name="anchor13"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.2.3"></a><h3>4.2.3.
Outputs</h3>
<p>
For each test, three figures are kept, the average throughput, the average
packet drop rate, and the average queueing delay over the second half
of the test.
</p>
<p>
For each set of scenarios, the output is two graphs. For the
delay/bandwidth graph, the x-axis shows the average queueing delay,
and the y-axis shows the average throughput. For the drop-rate graph,
the x-axis shows the average queueing delay, and the y-axis shows
the average packet drop rate. Each pair of graphs illustrates the
delay/throughput/drop-rate tradeoffs for this congestion control
mechanism. For an AQM mechanism, each pair of graphs also illustrates
how the throughput and average queue size vary (or don't vary) as a
function of the traffic load. Examples of delay/throughput tradeoffs appear in Figures 1-3 of <a class='info' href='#FS01'>[FS01]<span> (</span><span class='info'>Floyd, S., Gummadi, R., and S. Shenker, “Adaptive RED: An Algorithm for Increasing the Robustness of RED,” .</span><span>)</span></a>
and Figures 4-5 of <a class='info' href='#AHM08'>[AHM08]<span> (</span><span class='info'>Andrew, L., Hanly, S., and R. Mukhtar, “Active Queue Management for Fair Resource Allocation in Wireless Networks,” .</span><span>)</span></a>.
</p>
<a name="sec:convergence"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.3"></a><h3>4.3.
Ramp up time: completion time of one flow</h3>
<p>
These tests aim to determine how quickly existing flows make room for new flows.
</p>
<a name="anchor14"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.3.1"></a><h3>4.3.1.
Topology and background traffic</h3>
<p>
Dumbbell. At least three capacities should be used, as close as possible to:
56 kbps, 10 Mbps and 1 Gbps. The 56 kbps case is included to
investigate the performance using mobile handsets.
</p>
<p>
For each capacity, three RTT scenarios should be tested, in which the existing
and newly arriving flow have RTTs of (80 ms, 80 ms), (120 ms, 30 ms) and
(30 ms, 120 ms).
</p>
<p>
Throughout the experiment, there is also 10% bidirectional cross traffic,
as described in <a class='info' href='#sec:crossTraffic'>Section 3<span> (</span><span class='info'>Traffic generation</span><span>)</span></a>, using the mix of RTTs described
in <a class='info' href='#sec:RTTs'>Section 3.4<span> (</span><span class='info'>Round Trip Times</span><span>)</span></a>. All traffic is from the new TCP extension.
</p>
<a name="anchor15"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.3.2"></a><h3>4.3.2.
Flows under test</h3>
<p>
Traffic is dominated by two long lived flows, because we believe that to be the
worst case, in which convergence is slowest.
</p>
<p>
One flow starts in ``equilibrium'' (at least having finished
normal slow-start). A new flow then starts; slow-start is disabled by setting
the initial slow-start threshold to the initial CWND. Slow start is disabled
because this is the worst case, and could happen if
a loss occurred in the first RTT.
</p>
<p>
The experiment ends once the new flow has run for five minutes.
Both of the flows use 1500-byte packets.
</p>
<a name="anchor16"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.3.3"></a><h3>4.3.3.
Outputs</h3>
<p>
The output of these experiments are the time until the 1500(10^n) th byte of
the new flow is received, for n = 1,2,... . This measures how quickly the
existing flow releases capacity to the new flow, without requiring a definition
of when ``fairness'' has been achieved. By leaving the upper limit on n
unspecified, the test remains applicable to very high-speed networks.
</p>
<p>
A single run of this test cannot achieve
statistical reliability by running for a long time. Instead, an average over
at least three runs should be taken. Each run must use different
cross traffic, as specified in <a class='info' href='#sec:crossTraffic'>Section 3<span> (</span><span class='info'>Traffic generation</span><span>)</span></a>.
</p>
<a name="sec:transient"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.4"></a><h3>4.4.
Transients: release of bandwidth, arrival of many flows</h3>
<p>
These tests investigate the impact of a sudden change of congestion level.
They differ from the "Ramp up time" test in that the congestion here is caused
by unresponsive traffic.
</p>
<a name="anchor17"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.4.1"></a><h3>4.4.1.
Topology and background traffic</h3>
<p>
The network is a single bottleneck link, with bit
rate 100 Mbps, with a buffer of 1024 packets (120% BDP at 100 ms).
</p>
<p>
The transient traffic is generated using UDP, to avoid overlap with the
scenario of <a class='info' href='#sec:convergence'>Section 4.3<span> (</span><span class='info'>Ramp up time: completion time of one flow</span><span>)</span></a> and isolate the behavior of the flows
under study.
Three transients are tested:
</p>
<ol class="text">
<li> step decrease from 75 Mbps to 0 Mbps,
</li>
<li> step increase from 0 Mbps to 75 Mbps,
</li>
<li> 30 step increases of 2.5 Mbps at 1 s intervals,
simulating a ``flash crowd'' effect.
</li>
</ol><p>
These transients occur after the flow under test has exited slow-start, and
remain until the end of the experiment.
</p>
<p>
There is no TCP cross traffic
as described in <a class='info' href='#sec:crossTraffic'>Section 3<span> (</span><span class='info'>Traffic generation</span><span>)</span></a>
in this experiment.
because flow arrivals/departures occur on timescales long compared with these effects.
</p>
<a name="anchor18"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.4.2"></a><h3>4.4.2.
Flows under test</h3>
<p>
There is one flow under test: a long-lived flow in the same direction as the
transient traffic, with a 100 ms RTT.
</p>
<a name="anchor19"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.4.3"></a><h3>4.4.3.
Outputs</h3>
<p>
For the decrease in cross traffic, the metrics are
(i) the time taken for the
flow under test to increase its window to 60%, 80% and 90% of its BDP, and
(ii) the maximum
change of the window in a single RTT while the window is increasing to that
value.
</p>
<p>
For cases with an increase in cross traffic, the metric is
the number of packets dropped by the cross traffic from the start of the
transient until 100 s after the transient. This measures the harm caused by
algorithms which reduce their rates too slowly on congestion.
</p>
<a name="anchor20"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.5"></a><h3>4.5.
Impact on standard TCP traffic</h3>
<p>
Many new TCP proposals achieve a gain, G , in their own throughput at the
expense of a loss, L , in the throughput of standard TCP flows sharing a
bottleneck, as well as by increasing the link utilization.
In this context a "standard TCP flow" is defined as a flow using
<a class='info' href='#RFC2883'>SACK TCP<span> (</span><span class='info'>Floyd, S., Mahdavi, J., Mathis, M., and M. Podolsky, “An Extension to the Selective Acknowledgement (SACK) Option for TCP,” July 2000.</span><span>)</span></a> [RFC2883], but without
<a class='info' href='#RFC3168'>ECN<span> (</span><span class='info'>Ramakrishnan, K., Floyd, S., and D. Black, “The Addition of Explicit Congestion Notification (ECN) to IP,” September 2001.</span><span>)</span></a> [RFC3168].
<strong>What about: </strong>
<a class='info' href='#RFC1323'>Window scaling<span> (</span><span class='info'>Jacobson, V., Braden, B., and D. Borman, “TCP Extensions for High Performance,” May 1992.</span><span>)</span></a> [RFC1323] (yes),
<a class='info' href='#RFC4138'>FRTO<span> (</span><span class='info'>Sarolahti, P. and M. Kojo, “Forward RTO-Recovery (F-RTO): An Algorithm for Detecting Spurious Retransmission Timeouts with TCP and the Stream Control Transmission Protocol (SCTP),” August 2005.</span><span>)</span></a> [RFC4138] (yes),
<a class='info' href='#RFC3465'>ABC<span> (</span><span class='info'>Allman, M., “TCP Congestion Control with Appropriate Byte Counting (ABC),” February 2003.</span><span>)</span></a> [RFC3465] (no)?
The intention is for a "standard TCP flow" to correspond to
TCP as commonly deployed in the Internet today (with the notable exception of
CUBIC, which runs by default on the majority of web servers).
This scenario quantifies this tradeoff.
</p>
<a name="anchor21"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.5.1"></a><h3>4.5.1.
Topology and background traffic</h3>
<p>
The dumbbell of <a class='info' href='#sec:RTTs'>Section 3.4<span> (</span><span class='info'>Round Trip Times</span><span>)</span></a> is used with the same capacities as for the
convergence tests (<a class='info' href='#sec:convergence'>Section 4.3<span> (</span><span class='info'>Ramp up time: completion time of one flow</span><span>)</span></a>).
All traffic in this scenario comes from the flows under test.
</p>
<a name="anchor22"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.5.2"></a><h3>4.5.2.
Flows under test</h3>
<p>
The scenario is performed by conducting pairs of experiments, with identical
flow arrival times and flow sizes. Within each experiment, flows are divided
into two camps. For every flow in camp A, there is a flow with the same
size, source and destination in camp B, and vice versa. The start time of the
two flows are within 2 s.
</p>
<p>
The file sizes and start times
are as specified in <a class='info' href='#sec:crossTraffic'>Section 3<span> (</span><span class='info'>Traffic generation</span><span>)</span></a>, with start times scaled to
achieve loads of 50% and 100%. In addition, both camps have a long-lived
flow. The experiments last for 1200 seconds.
</p>
<p>
In the first experiment, called BASELINE, both camp A and camp B use standard
TCP. In the second, called MIX, camp A uses standard TCP and camp B uses the
new TCP extension.
</p>
<p>
The rationale for having paired camps is to remove the statistical uncertainty
which would come from randomly choosing half of the flows to run each
algorithm. This way, camp A and camp B have the same loads.
</p>
<a name="anchor23"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.5.3"></a><h3>4.5.3.
Outputs</h3>
<p>
The gain achieved by the new algorithm and loss incurred by standard TCP
are given by G=T(B)_Mix/T(B)_Baseline and
L=T(A)_Mix/T(A)_Baseline
where T(x) is the throughput obtained by camp x ,
measured as
the amount of data acknowledged by the receivers (that is, ``goodput''),
and taken over the last 8000 seconds of the experiment.
</p>
<p>
The loss, L , is analogous to the ``bandwidth stolen from TCP'' in <a class='info' href='#SA03'>[SA03]<span> (</span><span class='info'>Souza, E. and D. Agarwal, “A HighSpeed TCP Study: Characteristics and Deployment Issues,” .</span><span>)</span></a>
and ``throughput degradation'' in <a class='info' href='#SSM07'>[SSM07]<span> (</span><span class='info'>Shimonishi, H., Sanadidi, M., and T. Murase, “Assessing Interactions among Legacy and High-Speed TCP Protocols,” .</span><span>)</span></a>.
</p>
<p>
A plot of G vs L represents the tradeoff between
efficiency and loss.
</p>
<a name="anchor24"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.5.4"></a><h3>4.5.4.
Suggestions</h3>
<p>
Other statistics of interest are the values of G and L for each quartile of
file sizes. This will reveal whether the new proposal is more aggressive in
starting up or more reluctant to release its share of capacity.
</p>
<p>
As always, testing at other loads and averaging over multiple runs are
encouraged.
</p>
<a name="anchor25"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.6"></a><h3>4.6.
Intra-protocol and inter-RTT fairness</h3>
<p>
These tests aim to measure bottleneck bandwidth sharing
among flows of the same protocol with the same RTT, which represents
the flows going through the same routing path.
The tests also measure inter-RTT fairness, the
bandwidth sharing among flows
of the same protocol where routing paths have a common
bottleneck segment but might have different overall paths
with different RTTs.
</p>
<a name="anchor26"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.6.1"></a><h3>4.6.1.
Topology and background traffic</h3>
<p>
The topology, the capacity and cross traffic
conditions of these tests are the same as in <a class='info' href='#sec:convergence'>Section 4.3<span> (</span><span class='info'>Ramp up time: completion time of one flow</span><span>)</span></a>.
The bottleneck buffer is varied from 25% to 200% BDP for a 100 ms flow,
increasing by factors of 2.
</p>
<a name="anchor27"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.6.2"></a><h3>4.6.2.
Flows under test</h3>
<p>
We use two flows of the same protocol for this experiment.
The RTTs of the flows range from 10 ms to 160 ms
(10 ms, 20 ms, 40 ms, 80 ms, and 160 ms)
such that the ratio of the minimum
RTT over the maximum RTT is at most 1/16.
<strong>In case a testbed doesn't support up to 160 ms RTT, the RTTs MAY
be scaled down in proportion to the maximum RTT supported in that environment.</strong>
</p>
<p>
Intra-protocol fairness: For each run, two flows with the same RTT,
taken from the range of RTTs above start randomly within the first 10% of the experiment.
The order in which these flows start doesn't matter. An additional test of interest,
but not part of this suite, would involve two extreme cases -
two flows with very short or long RTTs
(e.g., the delay less then 1-2 ms represents communication happen in the data-center
and the delay larger than 600 ms considers communication over satellite).
</p>
<p>
Inter-RTT fairness: For each run, one flow with a fixed RTT of 160 ms starts first,
and another flow with a different RTT taken from the range of RTTs above, joins afterward.
The starting times of both two flows are randomly chosen within
the first 10% of the experiment as before.
</p>
<a name="anchor28"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.6.3"></a><h3>4.6.3.
Outputs</h3>
<p>
The output of this experiment is the ratio of the average throughput
values of the two flows. The output also includes the packet drop rate
for the congested link.
</p>
<a name="anchor29"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.7"></a><h3>4.7.
Multiple bottlenecks</h3>
<p>
These experiments explore the relative bandwidth for a flow
that traverses multiple bottlenecks, and flows with the same round-trip time
that each traverse only one of the bottleneck links.
</p>
<a name="anchor30"></a><br /><hr />
<table summary="layout" cellpadding="0" cellspacing="2" class="TOCbug" align="right"><tr><td class="TOCbug"><a href="#toc"> TOC </a></td></tr></table>
<a name="rfc.section.4.7.1"></a><h3>4.7.1.
Topology and background traffic</h3>
<p>
The topology is a ``parking-lot'' topology with
three (horizontal) bottleneck links and four (vertical) access links.
The bottleneck links have a rate of 100 Mbps,
and the access links have a rate of 1 Gbps.
</p>
<p>
All flows have a round-trip time of 60 ms, to enable the effect
of traversing multiple bottlenecks to be distinguished from that
of different round trip times. This can be achieved as in
<a class='info' href='#fig:multihop-asymmetric'>Figure 3</a> by (a) the second access link
having a one-way delay of 30 ms (b) the bottleneck link to which
it does not connect having a one-way delay of 30 ms and (c) all
other links having negligible delay. This can be extended to more than
three bottlenecks as shown in <a class='info' href='#fig:multihop-continued'>Figure 4</a>, by
assigning a delay of 30 ms to every alternate access link, and to
zero or one of the bottleneck links.)
For the special case of three hops, an alternative is for all links to have
a one-way delay of 10 ms, as shown in <a class='info' href='#fig:multihop-symmetric'>Figure 5</a>.
It is not clear whether there are interesting performance differences between
these two topologies, and if so, which is more typical of the actual internet.
</p><br /><hr class="insert" />
<a name="fig:multihop-asymmetric"></a>
<div style='display: table; width: 0; margin-left: 3em; margin-right: auto'><pre>
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >
_____ 100M 0ms ____________ 100M 0ms _____________ 100M 30ms ____
| ................ | ................ | ................ |
1G : : 1G : : 1G : : 1G
| : : | : : | : : |
0ms : : 30ms : : 0ms : : 0ms
| ^ V | ^ V | ^ V |
</pre></div>
<p>Basic multi-hop topology.
</p><table border="0" cellpadding="0" cellspacing="2" align="center"><tr><td align="center"><font face="monaco, MS Sans Serif" size="1"><b> Figure 3 </b></font><br /></td></tr></table><hr class="insert" />
<br /><hr class="insert" />
<a name="fig:multi