[Tmrg] Fwd: Re: TCP evaluation suite round-table

[Lawrence D. Dunn]

tmrg-interest folks,

  Here's Lachlan's reply (with his permission).

Larry
--

>X-from-outside-Cisco: 64.233.184.228
>X-IronPort-Anti-Spam-Filtered: true
>X-IronPort-Anti-Spam-Result: Ao8CANusykZA6bjknmdsb2JhbAASjXsBAQIHBAYPGIkzAiQ
>X-IronPort-AV: i="4.19,290,1183359600";
>    d="scan'208"; a="37870124:sNHT18213240"
>DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed;
>         d=gmail.com; s=beta;
> 
>h=domainkey-signature:received:received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
> 
>b=heDStVgnVKrH8deXPyWpHuWTsBnuo6gOKIYBO3uCwI5yGkQtEIgE6HtmwyMj0SyqCFKSzK+0wBZUbD3QxPxfL8Nofw+ZOdP/2wqic2nvA+oIqqzgPfFv9+iFjUIetiMPl1iTQUliMK7P34DRpZNIN3ZoXUwoBgsq4PbSc19QCWI=
>Date: Tue, 21 Aug 2007 09:15:04 -0700
>From: "Lachlan Andrew" <lachlan.andrew at gmail.com>
>Reply-To: l.andrew at ieee.org
>To: "Lawrence D. Dunn" <ldunn at cisco.com>, "David Hayes" <david.hayes at ieee.org>
>Subject: Re: [Tmrg] TCP evaluation suite round-table
>Authentication-Results: sj-dkim-3; 
>header.From=lachlan.andrew at gmail.com; dkim=pass (
>	sig from gmail.com/beta verified; );
>
>Greetings Larry,
>
>On 21/08/07, Lawrence D. Dunn <ldunn at cisco.com> wrote:
>>  Lachlan,  (unicast),
>>    ( I almost sent this to the list, but figured, since parts sound
>>     like a bit of a tangential ramble,  I'd start with a unicast,
>>     and if I feel the same way in a day or so, maybe send it to the 
>>list... ;-)
>
>As always, you raise some really good points.  The list would benefit
>from all of them.
>
>>     For example, your last sentence (..."useful"...) seems to imply
>>     that giving all capacity to single-hop flows is somehow bad, or wrong.
>>     Though I probably agree, it seems that maybe this connects
>>     "utilization" to some notion of "fairness" (i.e. on what basis
>>      have we concluded that behavior-X is bad/wrong/misleading?)
>>     Are utilization and fairness meant to be coupled, or orthogonal,
>>     or varies-by-scheme-and-that's-OK?
>
>My personal opinion is that Frank Kelly was spot on.  Fairness and
>utilization are intimately coupled, and it is easy to evaluate them
>together if we can agree on what we're trying to achieve.
>
>I agree that we should steer away from value judgements at the moment.
>  However, if we're trying to decide on a small set of measurements
>from which we *can* make value judgements, then I think we should
>avoid introducing   misleading   coupling of quantities like fairness
>and throughput.
>
>A metric which is clearly positively correlated with one quantity
>(increases with throughput) and negatively correlated with another
>(decreases with fairness) is IMO going to be less useful in eventually
>making those value judgements, than one which is positively correlated
>with one quantity and *independent* of the other.
>
>If we're going to introduce a measure which is dependent on both
>throughput and fairness, I'd advocate "aggregate utility", which
>increases with both.
>
>>     Also, it might be worth considering, for link utilization,
>>     whether a single-valued metric is the right choice.
>>     For example, is a "square wave" w/ average utilization
>>     of 50% somehow "better" or "worse" that a somewhat-noisy
>>     utilization that seems to hover near 50%? Do we care?
>>     Maybe we should stay away from better/worse judgements at this
>>     early stage, but a single-valued metric (average of the sums
>>     or sum of averages in your example below? )
>>     probably means that we can't tell the difference.
>>     On one hand, maybe it's best not to complicate things.
>>     On another hand, maybe adding standard-deviation,
>>     or some other metric(s)-of-your-choice, might at least help
>>     capture differentiation between two behaviors
>>     that is useful.
>>
>>     Or not.  Maybe I'm over-thinking it. ;-)
>
>Again, a very good point.  On question to ask would be why variation
>in utilization is bad.  To me, it is only bad if it has a measurable
>effect on some quantity that applications care about, like loss,
>jitter or expected file transfer time.  That quantity may be
>experienced by the "big" TCP flow, or by CBR cross traffic, etc.
>Rather than measuring variation for its own sake, I'd prefer to find
>metrics which measure its harm.
>
>>     Is it correct to assume that we're excluding exotica like some
>>     TCP proxy that "fans-out" a single flow, multicast-fashion,
>>     to multiple downstream receivers? I suspect that if such
>>     a proxy were placed after the bottleneck, it might wreak havoc
>>     with the sum-of-receive-rates approach.
>
>I agree that it would mean that the sum-of-receive-rates could be much
>greater than the link capacity, but I see that as appropriate and in
>fact a big benefit of this approach.
>
>This is very related to my discussions with Michael Welzl on  iccrg
>about rate-given-corruption (did you see them?).  If a flow is giving
>benefit to two users, to me it is twice as useful as one giving
>benefit only to one, and "should" be given a greater share of the
>bottleneck resource.  Again, how much more depends on what "utility"
>we're trying to maximize, and how much it "costs" the network to
>transport the data.
>
>>     (East counter-point: "well, that's not really TCP" - perhaps so,
>>     but it makes me think that the tighter we can bound what
>>     is in/out of the measurement bounds, the fewer disputes
>>     we might have later on. OTOH, tighter-bounds might stifle
>>     some cool/creative approaches, so maybe it's best to not
>>     worry about it).
>
>Yes.  My concern with what is in/out is routing.  Maximizing the sum
>of (link_count*throughput) encourages longer paths.  If we're dealing
>with fixed routing, then I think we should try to come up with metrics
>that are meaningful for all protocols which decide when and how much
>to transmit, whether TCP or not.
>
>Feel free to forward any of this to the list.
>
>Cheers,
>Lachlan
>
>--
>Lachlan Andrew  Dept of Computer Science, Caltech
>1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
>Phone: +1 (626) 395-8820    Fax: +1 (626) 568-3603