From falk at isi.edu  Mon Jul 10 05:54:10 2006
From: falk at isi.edu (Aaron Falk)
Date: Mon, 10 Jul 2006 08:54:10 -0400
Subject: [Tmrg-interest] Fwd: Possible Internet2 awards program for new TCP
	stacks
References: <6.2.0.14.2.20060706140924.03dfc1a0@mail.internet2.edu>
Message-ID: <88676A75-868F-425B-AD3F-832093BC4A59@ISI.EDU>

Of possible interest.  --aaron

Begin forwarded message:

> From: Richard Carlson <rcarlson at internet2.edu>
> Date: July 6, 2006 2:44:48 PM EDT (CA)
> To: Joe Touch <touch at ISI.EDU>, Pascale Primet <pascale.primet at ens- 
> lyon.fr>, Katsushi Kobayashi <ikob at koganei.wide.ad.jp>, Aaron Falk  
> <falk at ISI.EDU>, "Douglas Leith" <doug at eee.strath.ac.uk>, "R. Hughes- 
> Jones" <R.Hughes-Jones at manchester.ac.uk>, "Cottrell, Les"  
> <cottrell at slac.stanford.edu>, Brian Tierney <bltierney at lbl.gov>
> Cc: Injong Rhee <rhee at eos.ncsu.edu>, Sally Floyd <floyd at icir.org>,  
> Jason Leigh <drspiff at mac.com>, Richard Carlson  
> <rcarlson at internet2.edu>
> Subject: Possible Internet2 awards program for new TCP stacks
>
> All;
>
> Hopefully you all know about the Internet2 Land Speed Record (LSR)  
> awards program http://lsr.internet2.edu/  Briefly, the LSR awards  
> program was established in 2000 to promote demonstrations that  
> highlight TCP's potential.  As the rules state, each entry must be  
> a RFC-791/793 stack with 1 or more sockets sending data between 2  
> Internet nodes.
>
> The original record was 751 Mbps over 5600 Kmeters.  The latest  
> record is 8.8 Gpbs over 30,000 Kmeters (the web page is out of  
> date).  Given that the current record crossed some 10 GE WAN-PHY  
> links and the rules call for a 10% increase to set a new record, we  
> may have to wait a while before the next generation 40/100 Gig HW  
> comes out.
>
> While these experiments show that TCP is capable of running at line  
> rates over any distance, we all know that performance can drop  
> dramatically if losses occur.  In fact, if you look at the MRTG  
> graphs the wining teams supply you can see they make multiple runs  
> and discard tests that suffer from any type of packet loss.
>
> Internet2 is looking for some way to promote testing, evaluation,  
> and growth in the field of TCP stack research.  One idea is to  
> create a new awards program, along the lines of the existing LSR  
> program.   I'm looking for advice and guidance on:
>
> 1) is this a good idea?
>
> 2) if so, what rules need to be changes/modified?
>         a) should some type of loss metric be included?
>         b) verification mechanisms?
>         c) is distance an important metric?
>         d) ensure others duplicate/repeat the experiment
>
> 3) if not, is there a better way to promote TCP stack research?
>
> Thanks in advance.  Any thought, comments, or suggestions would be  
> greatly appreciated.  Feel free to share this email with your  
> colleagues and collaborators.
>
> Regards;
> Rich
>
>
>
>
> ------------------------------------
>
>
>
> Richard A. Carlson                              e-mail:  
> RCarlson at internet2.edu
> Network Engineer                                phone:  (734) 352-7043
> Internet2                                       fax:    (734) 913-4255
> 1000 Oakbrook Dr; Suite 300
> Ann Arbor,  MI  48104


From lachlan at caltech.edu  Tue Jul 11 11:46:42 2006
From: lachlan at caltech.edu (lachlan at caltech.edu)
Date: Tue, 11 Jul 2006 11:46:42 -0700 (PDT)
Subject: [Tmrg-interest] Possible Internet2 awards program for new TCP stacks
Message-ID: <9555287327lachlan@caltech.edu>

> From: Richard Carlson <rcarlson at internet2.edu>
> Date: July 6, 2006 2:44:48 PM EDT (CA)
>
> Internet2 is looking for some way to promote testing, evaluation,
> and growth in the field of TCP stack research.  One idea is to
> create a new awards program, along the lines of the existing LSR
> program.   I'm looking for advice and guidance on:
>
> 1) is this a good idea?
>
> 2) if so, what rules need to be changes/modified?
>         a) should some type of loss metric be included?
>         b) verification mechanisms?
>         c) is distance an important metric?
>         d) ensure others duplicate/repeat the experiment
>
> 3) if not, is there a better way to promote TCP stack research?
>
> Thanks in advance.  Any thought, comments, or suggestions would be
> greatly appreciated.  Feel free to share this email with your
> colleagues and collaborators.

This sounds like a very good idea.  If the RFC793 restriction is lifted,
it will be important not simply to favour the most aggressive protocol.

One approach would be to devise a more comprehensive "benchmark" (AKA
"best practice test suite"), which results in a single figure
representing throughput, fairness etc.

One question would be whether it is better to use "live" networks, or
more controllable testbeds.  Just as the LSR keeps the *protocol* fixed,
the competition for protocols should keep the *hardware* as standard as
possible.  Here at Caltech, we're looking at setting our WAN-in-Lab up
as a common platform for multiple users to benchmark their protocols on.
 Another possibility would be to specify a common hardware setup (CPU
speeds, bus types, NICs, link emulators, all intervening switches, ...).

This touches on points (b) and (d):  If there are several "standard"
testbeds set up, then repeatability can be achieved by repeating on all
testbeds, and verifiability can be achieved by having a standard logging
system built in to the testbeds.

It would be nice if everyone compared their stacks on a standard
distance.  If that is not practical (for example, if the award is for
"live" networks) then reporting (achived_bandwidth*delay) would be more
effective.  However, high values can be achieved with "slow"
(computationally intensive) algorithms by using very long paths
(circulating the globe multiple times, for example).

David Wei recently pointed out to me the Google FileSystem (GFS) test of
running 32 independent parallel file transfers of a given size.  The
time-to-finish of the last flow measures the efficiency and fairness of
the scheme.  It can easily be mapped into "Gbps" if that is what people
are familiar with.

This test alone doesn't capture effects like RTT unfairness, or slow
convergence to fairness.  In an emulated environment, they could be
factored in by passing all 32 flows through different RTTs, drawn from a
"realistic" distribution, and  starting them at different times.  For
example, the result could be the maximum time from (starting in 
increasing order or RTT) and (starting in decreasing order of RTT). 
That would protect against unfairness from new flows starting too
aggressively or not aggressively enough.

For this test, it is probably appropriate to neglect factors that are
important for more general-purpose benchmarks (such as cross traffic and
reverse traffic).

$0.02
Lachlan Andrew



From floyd at icir.org  Wed Jul 12 13:22:32 2006
From: floyd at icir.org (Sally Floyd)
Date: Wed, 12 Jul 2006 13:22:32 -0700
Subject: [Tmrg-interest] a report on the Transport Modeling Research Group
Message-ID: <200607122022.k6CKMWlt085907@cougar.icir.org>

The TMRG mailing list has been largely dormant for many months now,
so this is a report of the current status.  The TRMG web page is
at: "http://www.icir.org/tmrg/".

* The draft on "Metrics for the Evaluation of Congestion Control 
Mechanisms" is essentially complete.  It needs a volunteer for
a shephard, to see if it is ready for publication.

* The draft on "Tools for the Evaluation of Simulation and Testbed
Scenarios" needs contributions on the topics of "Distribution
of packet sequence numbers", "Characterization of Congested Links
in Terms of Bandwidth and Typical Levels of Congestion", and
"Characterization of Network Changes Affecting Congestion",
characterizing the current state of our knowledge (if any).

* Evaluating Congestion Control Mechanisms over Challenging Lower
Layers (e.g., wireless).  The Tools draft above has a section
place-holder for a section on "Characterization of Challenging Lower
Layers."  This would probably be best in a separate draft, along
with best current practices for the evaluation of congestion control
mechanisms over these challenging lower layers.

* Yong Xia and others are working on a draft giving best current
practice for the evaluation of congestion control mechanisms in
general scenarios, including single-bottleneck topologies, parking-lot
topologiee, and general topologies.  This will be sent to the
mailing list when it is ready for feedback.

- Sally

From andras.veres at ericsson.com  Thu Jul 13 02:21:48 2006
From: andras.veres at ericsson.com (=?iso-8859-1?Q?Andr=E1s_Veres_=28IJ/ETH=29?=)
Date: Thu, 13 Jul 2006 11:21:48 +0200
Subject: [Tmrg-interest] a report on the Transport Modeling Research
	Group
Message-ID: <C93FBB58FB6B594BA2F4E6CDFF056F12A10C83@esealmw116.eemea.ericsson.se>

Sally,

We are doing an investigation on whether there is any foreeable problem with current/future TCP in future high-speed wireless networks at Ericsson. I was reading the work on this list with great interest, and it has helped me a lot defining the right metrics and tools for this work. I have just a few minor comments:

In "Metrics for the Evaluation of Congestion Control Mechanisms" the response to changes section states that the congestion control should be responsive to changes due to congestion and route changes. I propose to extend this slightly. In the future (maybe very near future) we expect that most hosts will be connected via wireless/mobile networks to the Internet and mobility will be frequent. It this case it will be very common to perform handoffs between very different cells with different levels of congestion and very different capacity. I suggest that the "response to changes" section should be extented with something like this:

"Congestion control mechanisms should respond to sudden bandwidth-delay product changes due to mobility in the future. Such bandwith-delay product changes are expected to be more frequent and expected to have greater impact than path changes today. Due to mobility, both the bandwidth, and the round-trip delay may suddenly change. Due to the heterogenity of wireless access types (802.11b,a,g, WIMAX, WCDMA, HS-WCDMA, E-GPRS, Bluetooth etc), the congestion control protocol has to be able to handle BDP changes (with reasonable efficiency) in the orders of several magnitudes"

/Andras







-----Original Message-----
From: tmrg-interest-bounces at ICSI.Berkeley.EDU [mailto:tmrg-interest-bounces at ICSI.Berkeley.EDU] On Behalf Of Sally Floyd
Sent: Wednesday, July 12, 2006 10:23 PM
To: tmrg-interest at ICSI.Berkeley.EDU
Subject: [Tmrg-interest] a report on the Transport Modeling Research Group

The TMRG mailing list has been largely dormant for many months now, so this is a report of the current status.  The TRMG web page is
at: "http://www.icir.org/tmrg/".

* The draft on "Metrics for the Evaluation of Congestion Control Mechanisms" is essentially complete.  It needs a volunteer for a shephard, to see if it is ready for publication.

* The draft on "Tools for the Evaluation of Simulation and Testbed Scenarios" needs contributions on the topics of "Distribution of packet sequence numbers", "Characterization of Congested Links in Terms of Bandwidth and Typical Levels of Congestion", and "Characterization of Network Changes Affecting Congestion", characterizing the current state of our knowledge (if any).

* Evaluating Congestion Control Mechanisms over Challenging Lower Layers (e.g., wireless).  The Tools draft above has a section place-holder for a section on "Characterization of Challenging Lower Layers."  This would probably be best in a separate draft, along with best current practices for the evaluation of congestion control mechanisms over these challenging lower layers.

* Yong Xia and others are working on a draft giving best current practice for the evaluation of congestion control mechanisms in general scenarios, including single-bottleneck topologies, parking-lot topologiee, and general topologies.  This will be sent to the mailing list when it is ready for feedback.

- Sally
_______________________________________________
Tmrg-interest mailing list
Tmrg-interest at ICSI.Berkeley.EDU
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/tmrg-interest


From mascolo at poliba.it  Fri Jul 14 07:45:08 2006
From: mascolo at poliba.it (Saverio Mascolo)
Date: Fri, 14 Jul 2006 16:45:08 +0200
Subject: [Tmrg-interest] a report on the Transport Modeling Research
	Group
Message-ID: <000e01c6a754$94ab8d00$723bccc1@HPSM>

in section 3.2.2 "minimizing oscillations" i would add a metrci such as coviarance:

cov(x)=stdev(x)/E(x)


saverio mascolo


On 7/12/06, Sally Floyd <floyd at icir.org> wrote: 
  The TMRG mailing list has been largely dormant for many months now,
  so this is a report of the current status.  The TRMG web page is
  at: "http://www.icir.org/tmrg/".

  * The draft on "Metrics for the Evaluation of Congestion Control
  Mechanisms" is essentially complete.  It needs a volunteer for
  a shephard, to see if it is ready for publication.

  * The draft on "Tools for the Evaluation of Simulation and Testbed
  Scenarios" needs contributions on the topics of "Distribution
  of packet sequence numbers", "Characterization of Congested Links
  in Terms of Bandwidth and Typical Levels of Congestion", and
  "Characterization of Network Changes Affecting Congestion",
  characterizing the current state of our knowledge (if any).

  * Evaluating Congestion Control Mechanisms over Challenging Lower
  Layers (e.g., wireless).  The Tools draft above has a section
  place-holder for a section on "Characterization of Challenging Lower
  Layers."  This would probably be best in a separate draft, along
  with best current practices for the evaluation of congestion control
  mechanisms over these challenging lower layers.

  * Yong Xia and others are working on a draft giving best current
  practice for the evaluation of congestion control mechanisms in
  general scenarios, including single-bottleneck topologies, parking-lot
  topologiee, and general topologies.  This will be sent to the
  mailing list when it is ready for feedback.

  - Sally
  _______________________________________________
  Tmrg-interest mailing list
  Tmrg-interest at ICSI.Berkeley.EDU
  http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/tmrg-interest


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/tmrg-interest/attachments/20060714/e26a7349/attachment.html 

From floyd at icir.org  Sun Jul 23 13:07:23 2006
From: floyd at icir.org (Sally Floyd)
Date: Sun, 23 Jul 2006 13:07:23 -0700
Subject: [Tmrg-interest] a report on the Transport Modeling Research
	Group
Message-ID: <200607232007.k6NK7NcT059758@cougar.icir.org>

Andras -

>In "Metrics for the Evaluation of Congestion Control Mechanisms"
>the response to changes section states that the congestion control
>should be responsive to changes due to congestion and route changes.
>I propose to extend this slightly.  In the future (maybe very near
>future) we expect that most hosts will be connected via wireless/mobile
>networks to the Internet and mobility will be frequent.  It this
>case it will be very common to perform handoffs between very different
>cells with different levels of congestion and very different capacity.
>I suggest that the "response to changes" section should be extented
>with something like this:
>
>"Congestion control mechanisms should respond to sudden bandwidth-delay
>product changes due to mobility in the future. Such bandwith-delay
>product changes are expected to be more frequent and expected to
>have greater impact than path changes today. Due to mobility, both
>the bandwidth, and the round-trip delay may suddenly change. Due
>to the heterogenity of wireless access types (802.11b,a,g, WIMAX,
>WCDMA, HS-WCDMA, E-GPRS, Bluetooth etc), the congestion control
>protocol has to be able to handle BDP changes (with reasonable
>efficiency) in the orders of several magnitudes"

Good idea.  And many thanks for the suggested text.

I revised it slightly, and added the following:

 Congestion control mechanisms also have to contend with sudden 
 changes in the bandwidth-delay product due to mobility.  Such
 bandwith-delay product changes are expected to become more frequent
 and to have greater impact than path changes today.  As a result
 of both mobility and of the heterogenity of wireless access types 
 (802.11b,a,g, WIMAX, WCDMA, HS-WCDMA, E-GPRS, Bluetooth, etc.),
 both the bandwidth and the round-trip delay can change suddenly,
 sometimes by several orders of magnitude.

- Sally


From floyd at icir.org  Sun Jul 23 13:30:32 2006
From: floyd at icir.org (Sally Floyd)
Date: Sun, 23 Jul 2006 13:30:32 -0700
Subject: [Tmrg-interest] a report on the Transport Modeling Research
	Group
Message-ID: <200607232030.k6NKUW6v059812@cougar.icir.org>

Saverio -

>in section 3.2.2 "minimizing oscillations" i would add a metric such as 
>coviarance:
>
>cov(x)=3Dstdev(x)/E(x)

Many thanks.  Will do.  

- Sally